In addition to Muon, NVIDIA also supports many other optimizers for the research community to explore, including: The ultimate form of orthogonalized optimizer MOP (Momentum Orthogonalized by Polar decomposition) An advanced SOAP variant that updates eigen basis per step with eigen decomposition plus KL correction in REKLS
Advancing Emerging Optimizers for Accelerated LLM Training with NVIDIA Megatron | NVIDIA Technical Blog…The UST has its roots in sparse compiler technology; see, for example, Compiler Support for Sparse Matrix Computations , Sparse Tensor Algebra Compilation and Compiler Support for Sparse Tensor Computations in MLIR . This…
…As history accumulates, the LLM must process more tokens before generating a response, extending the bottleneck that support models must be masked behind. Consolidating support models like ASR and TTS provides a…
…Multi-Teacher On-Policy Distillation enables continuous improvement and domain specialization by training Nemotron 3 Ultra with dense feedback from over ten domain-specific teacher models, supported by a massive and transparent…
…In the future, when more CUDA Toolkit libraries are supported, CuPy will have a lighter maintenance overhead and have fewer wheels to release. Users will benefit from a faster CUDA runtime. Numba…
…When the VLM supports multi-frame input, batch sampled frames over a configurable duration (e.g. 1 minute) before sending to the model. Each batch must contain frames from a single stream…
…The example targets Ubuntu 24.04 but works on any distribution OpenShell supports. A build.nvidia.com API key for inference. The default model is nvidia/nemotron-3-super-120b-a12b. Hermes…
…Supported Platforms Nsight Systems is distributed through multiple packages. Pick a “Profiling Target” column and learn what hosts may be used to profile (local or remote) as well as view reports. Profiling…
…compute metrics like Trajectory Efficiency (steps/tokens per success), Tool Call Accuracy, and failure mode distribution (plan, tool, environment). Tip #3: Make tool usage a first-class signal Most production agents succeed…
…This new release of AI-Q adds first-class support for connecting to authenticated MCP servers as data sources, so research pipelines can pull from the same enterprise systems agents already use…
…playing a key role in shaping CUDA and distributed computing initiatives, building the team from the ground up and scaling it to support a rapidly growing ecosystem. View all posts by Anshuman…