Tracked topic
QAD uses the original full-precision model (teacher) to teach the quantized model (student). First, create a quantized model by running PTQ on the full-precision model. Then distill the frozen BF16 model into the quantized model using a KL divergence loss comparing the teacher’s and student’s logits. Figure 1 shows the two-stage QAD process used to build the Nemotron 3.5 Lightning NVFP4 checkpoint. The full-precision BF16 model serves as the frozen teacher and is also the starting point for Stage 1, a PTQ pass that quantizes weights to W4A16 to produce the quantized student. In Stage 2, the st
Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer | NVIDIA Technical Blog… To enable it, set the quantization flag in MaxText to switch into the NVFP4 path. Two modes are exposed: quantization=te nvfp4 : NVFP4 with Random Hadamard Transform. Recommended when the convergence under te nvfp4 no rht is not satisfactory. quantization=te nvfp4 no rht : NVFP4 without RHT. …
… The built-in NVFP4 configs range from NVFP4 DEFAULT CFG , which quantizes broadly, to more selective presets like NVFP4 MLP ONLY CFG , NVFP4 EXPERTS ONLY CFG , and NVFP4 OMLP ONLY CFG that restrict FP4 to the MLP and expert layers while keeping the sensitive attention projections in higher precisio… …
… The Hugging Face PTQ example walks through quantizing a Hugging Face model to NVFP4 using the different PTQ recipes previously explained. import modelopt.torch.quantization as mtq define forward loop with dataset def forward loop model : for batch in calib dataloader: model batch Quantize base mode… …
… After adding Wgrad times, non-GEMM overhead and NVFP4-specific quantization costs, the end-to-end gap between NVFP4 and MXFP8 in training is consistent with these kernel-level numbers. …
… It also ships with a quantized version to NVFP4 where users can run this model on a consumer gaming card or NVIDIA DGX Spark with only a small cost in accuracy. …
… BF16 and NVFP4 The model is available today on Hugging Face with BF16 checkpoints, and an NVFP4 quantized checkpoint for DiffusionGemma is also available using NVIDIA Model Optimizer . …
… NVIDIA는 또한 DFlash 드래프트 모델도 출시하며, 이는 다른 모델과 비교 측정하여 워크로드에 가장 적합한 것을 선택할 수 있습니다. 양자화 Nemotron 3.5 Lightning은 BF16과 함께 NVFP4 체크포인트를 제공하며, NVIDIA Blackwell, NVIDIA Hopper, NVIDIA Ampere GPU 전반에서 Nemotron 3 Ultra를 구동하는 동일한 특화 NVFP4 커널을 사용합니다. 동일한 파일이 데이터 센터에서와 마찬가지로 데스크톱 DGX Spark에서도 동일하게 작동합니다. …
… Quantization Nemotron 3.5 Lightning ships with an NVFP4 checkpoint alongside BF16, using the same specialized NVFP4 kernels that power Nemotron 3 Ultra across NVIDIA Blackwell, NVIDIA Hopper, and NVIDIA Ampere GPUs. …
… NVFP4 정밀도 동일한 NVFP4 체크포인트가 NVIDIA Hopper, NVIDIA Blackwell, Ampere GPU에서 실행됩니다. 개발자들은 특수 NVFP4 양자화 커널 덕분에 모든 NVIDIA GPU 아키텍처에서 하나의 체크포인트를 사용할 수 있습니다. NVFP4는 또한 Blackwell에서 BF16 대비 동일한 상호작용성으로 GPU당 최대 5배 높은 처리량을 제공합니다. …
… As a result, most matrix-multiplication-heavy operations, such as the linear layers, can fully exploit NVFP4 for higher throughput while keeping model fidelity. To learn more about training with lower bit-width, check out Pretraining Large Language Models with NVFP4 NVIDIA Research, 2025 . …