Tracked topic
QAD uses the original full-precision model (teacher) to teach the quantized model (student). First, create a quantized model by running PTQ on the full-precision model. Then distill the frozen BF16 model into the quantized model using a KL divergence loss comparing the teacher’s and student’s logits. Figure 1 shows the two-stage QAD process used to build the Nemotron 3.5 Lightning NVFP4 checkpoint. The full-precision BF16 model serves as the frozen teacher and is also the starting point for Stage 1, a PTQ pass that quantizes weights to W4A16 to produce the quantized student. In Stage 2, the st
Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer | NVIDIA Technical Blog… Looking ahead, the NVIDIA Rubin platform delivers large leaps in NVFP4 capability for training and inference, offering 35 petaFLOPS of NVFP4 training compute, and 50 petaFLOPs of NVFP4 Transformer Engine inference compute. …
… Selective BF16 layers are essential for NVFP4: Ablation studies show that fully NVFP4 models diverge. Stable training requires keeping some layers in BF16, particularly near the end of the network, to mitigate NVFP4 quantization error. …
… To enable it, set the quantization flag in MaxText to switch into the NVFP4 path. Two modes are exposed: quantization=te nvfp4 : NVFP4 with Random Hadamard Transform. Recommended when the convergence under te nvfp4 no rht is not satisfactory. quantization=te nvfp4 no rht : NVFP4 without RHT. …
… The built-in NVFP4 configs range from NVFP4 DEFAULT CFG , which quantizes broadly, to more selective presets like NVFP4 MLP ONLY CFG , NVFP4 EXPERTS ONLY CFG , and NVFP4 OMLP ONLY CFG that restrict FP4 to the MLP and expert layers while keeping the sensitive attention projections in higher precisio… …
… The Hugging Face PTQ example walks through quantizing a Hugging Face model to NVFP4 using the different PTQ recipes previously explained. import modelopt.torch.quantization as mtq define forward loop with dataset def forward loop model : for batch in calib dataloader: model batch Quantize base mode… …
… After adding Wgrad times, non-GEMM overhead and NVFP4-specific quantization costs, the end-to-end gap between NVFP4 and MXFP8 in training is consistent with these kernel-level numbers. …
… It also ships with a quantized version to NVFP4 where users can run this model on a consumer gaming card or NVIDIA DGX Spark with only a small cost in accuracy. …
… BF16 and NVFP4 The model is available today on Hugging Face with BF16 checkpoints, and an NVFP4 quantized checkpoint for DiffusionGemma is also available using NVIDIA Model Optimizer . …
… NVIDIA는 또한 DFlash 드래프트 모델도 출시하며, 이는 다른 모델과 비교 측정하여 워크로드에 가장 적합한 것을 선택할 수 있습니다. 양자화 Nemotron 3.5 Lightning은 BF16과 함께 NVFP4 체크포인트를 제공하며, NVIDIA Blackwell, NVIDIA Hopper, NVIDIA Ampere GPU 전반에서 Nemotron 3 Ultra를 구동하는 동일한 특화 NVFP4 커널을 사용합니다. 동일한 파일이 데이터 센터에서와 마찬가지로 데스크톱 DGX Spark에서도 동일하게 작동합니다. …
…これにより、パラメーターのオーバーヘッドを最小限に抑えながら、トレーニングの安定性を向上させます。ヘッドは、オフセット固有のショートカットに分裂するのではなく、一貫した継続に合意できるようになります。 同様の重み共有により、独立してトレーニングされたヘッドが通常劣化するような長いドラフト長においても、投機的ドラフトの一貫性が向上します。 ネイティブ NVFP4 事前トレーニング ほとんどの量子化モデルは、全精度で計算を開始し、トレーニング後に圧縮されるため、精度の低下は避けられないものです。 Super では別のアプローチを採用しています。事前トレーニング中の浮動小数点乗算/累積演算の大部分は NVFP4 、すなわち NVIDIA 4 ビット浮動小数点形式で実行されています。 Blackwell 向けに最適化されたこの手法は、精度を維持しながら、FP8 と比較してメモリ要件を大幅に削減しながらも、推論を高速化します。 低精度でネイティブにトレーニングを行うことは…