NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…
Search
People also ask
What other optimizers does NVIDIA support for research?
In addition to Muon, NVIDIA also supports many other optimizers for the research community to explore, including: The ultimate form of orthogonalized optimizer MOP (Momentum Orthogonalized by Polar decomposition) An advanced SOAP variant that updates eigen basis per step with eigen decomposition plus KL correction in REKLS
Advancing Emerging Optimizers for Accelerated LLM Training with NVIDIA Megatron | NVIDIA Technical BlogRobotics – NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…
Data Science – NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…
Edge Computing – NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…
MLOps – NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…
Content Creation / Rendering – NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…
Trustworthy AI / Cybersecurity – NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…
Data Center / Cloud – NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…
Simulation / Modeling / Design – NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…
Computer Vision / Video Analytics – NVIDIA Technical Blog
…expanding their context windows, with recent models supporting sequences of 128K tokens, 256K tokens, and beyond.... 9 MIN READ Feb 02, 2026 Optimizing Communication for Mixture-of-Experts Training with Hybrid Expert…