Enterprise deployments have shown a consistent pattern: when organizations move from static GPU allocation to dynamic scheduling, cluster usage becomes far more dynamic. Over-quota resources (the shared pool beyond guaranteed quotas) become one of the most heavily utilized resource types. Teams regularly exceed their guaranteed allocations, resulting in higher GPU utilization and more compute time for researchers. This makes over-quota fairness critical. When a significant portion of cluster value comes from this shared pool, that pool needs to be divided fairly over time. The classical statel
Ensuring Balanced GPU Allocation in Kubernetes Clusters with Time-Based Fairshare | NVIDIA Technical Blog…annotators, an LLM judge, rule-based verifiers, or synthetically generated preference data, since DPO is agnostic to the source of the preference signal. Preference signals eliminate the need for a separate reward…
…Nemotron 3 Super employs a hybrid Mamba-Transformer MoE architecture with NVFP4 precision on Blackwell GPUs, achieving high throughput and efficiency for multi-agent tasks, while Nemotron 3 Content Safety delivers low…
…language model (LLM) development, NVIDIA Megatron Core has emerged as the foundational framework for training massive transformer models at scale. The open source library offers industry-leading parallelism and GPU-optimized performance…
…High-throughput continuous batching and streaming for Super. SGLang Cookbook : Fast, lightweight inference optimized for multi-agent tool-calling workloads. NVIDIA TensorRT LLM Cookbook : Fully optimized TensorRT LLM engines with latent MoE…
…GPU between graphics and compute. Code agents: Trapping the ghost Andrej Karpathy, a founding member of OpenAI, likens working with large language models (LLMs) to summoning ghosts , an apt metaphor for LLM…
…It details how to build reproducible, structured product Q&A datasets by combining controlled sampling, LLM-based generation, and automated LLM-as-a-judge quality scoring, ensuring datasets are ready for distillation…
…Furthermore, we utilized a ReplicatedLinear block for the router logits. Since the router weights are small, replicating them across GPUs eliminates the need for expensive communication during the gating phase, keeping the…
…View all posts by Oleg Rybakov View all posts by Oleg Rybakov About Wenwen Gao Wenwen Gao is a senior product manager for NeMo at NVIDIA, focusing on LLM training framework and…
…optimized inference across multiple GPU architectures, including NVIDIA Ampere, NVIDIA Hopper, and NVIDIA Blackwell GPU families, and for popular inference engines, including vLLM and NVIDIA TensorRT-LLM. It supports FP8 and NVFP4…
…Learn more In LLM training, Expert Parallel (EP) communication for hyperscale mixture-of-experts (MoE) models is challenging. EP communication is essentially all-to-all, but due to its dynamics and sparseness…