Tracked topic
QAD uses the original full-precision model (teacher) to teach the quantized model (student). First, create a quantized model by running PTQ on the full-precision model. Then distill the frozen BF16 model into the quantized model using a KL divergence loss comparing the teacher’s and student’s logits. Figure 1 shows the two-stage QAD process used to build the Nemotron 3.5 Lightning NVFP4 checkpoint. The full-precision BF16 model serves as the frozen teacher and is also the starting point for Stage 1, a PTQ pass that quantizes weights to W4A16 to produce the quantized student. In Stage 2, the st
Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer | NVIDIA Technical Blog…Native NVFP4 pretraining optimized for NVIDIA Blackwell, significantly cutting memory requirements and speeding up inference by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining accuracy. Multi-environment reinforcement…
…Use Optimized Models Run NVIDIA-optimized open-weight models locally with day-0 support, NVFP4 quantization, and tuned Tensor Core performance. Run a Model Efficiently Run open-weight models with accelerated runtimes…
…Harness-facing Dynamo settings Our experiments used the newly released nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 model, though the same issues apply across models, reasoning parsers, and tool-call parsers…
…エージェント型推論向けのオープン ハイブリッド Mamba-Transformer MoE Nemotron 3 Super は、高容量の推論モデルにおける典型的な効率と精度のトレードオフを軽減するアーキテクチャ革新を導入しています。 3 MIN READ 2026 年 2 月 6 日 NVFP4 が AI のトレーニングと推論を加速する 3 つの方法 NVIDIA による徹底的な共同設計によって、モデルのトレーニングと推論の両方において、優れた精度で大幅なパフォーマンスの向上が達成が見込めるようになりました。 2 MIN READ…
…또한 FP8과 NVFP4 양자화 , 효율적인 비디오 샘플링, NVIDIA 최적화 커널을 지원해 예측 가능하고 지연 시간이 낮은 추론을 제공합니다. 여기에 3D 컨볼루션 기반 시공간 처리가 결합되면 워크스테이션부터 데이터센터, 클라우드 배포 환경까지 GPU 전반에서…
…NVIDIA GB300 NVL72 demonstrates up to 20x higher agentic coding performance. The NVIDIA Vera Rubin platform is expected to extend these gains by leveraging 50 PFLOPs of NVFP4 compute and leveraging the…
…重要なポイントが 1 つあるとすれば、適切な量子化の精度を使用することです。 NVFP4、INT4、W4A16 などのフォーマットは、多くの LLM ワークロードで高い精度を維持しながら、メモリとストレージの要件を大幅に削減します。 実際のユース ケース: Reachy Mini Jetson Mini Assistant これらのメモリ最適化の効果を示すために、Jetson Orin Nano 上で実行されるオンデバイス対話型 AI ロボットである Reachy Mini Jetson Assistant を考えてみましょう。これは…
…Cosmos 3 NIM은 BF16, FP8, NVFP4 양자화 체크포인트 선택을 지원합니다. NVFP4 양자화는 모델의 수치 정밀도를 BF16에서 4비트 부동소수점으로 낮춰 최대 2배의 추론 속도 향상을 달성합니다. vLLM: 연속 배칭(continuous batching), 페이지드 어텐션…
…That was combined with the powerful compute capabilities of Blackwell, along with NVFP4 weight quantization, for an additional 2x speedup, with an even bigger performance gain of 2.8x seen at higher…
…This is enabled by deep co-design across NVIDIA Blackwell, NVLink™, and NVLink Switch for scale-out; NVFP4 for low-precision accuracy; and NVIDIA Dynamo and TensorRT™ LLM for speed and flexibility…