Tracked topic
…Built on the NVIDIA Rubin architecture, the Rubin CPX GPU uses a cost‑efficient, monolithic die design packed with powerful NVFP4 computing resources and is optimized to deliver extremely high performance and…
…Is nvfp4 RTF and vram on a 24gb consumer card. fp8 sits ~30gb, bf16 ~60, so nvfp4 is the only realistic path on a 4090. is megatron-bridge the inference route you…
…これにより、パラメーターのオーバーヘッドを最小限に抑えながら、トレーニングの安定性を向上させます。ヘッドは、オフセット固有のショートカットに分裂するのではなく、一貫した継続に合意できるようになります。 同様の重み共有により、独立してトレーニングされたヘッドが通常劣化するような長いドラフト長においても、投機的ドラフトの一貫性が向上します。 ネイティブ NVFP4 事前トレーニング ほとんどの量子化モデルは、全精度で計算を開始し、トレーニング後に圧縮されるため、精度の低下は避けられないものです。 Super では別のアプローチを採用しています。事前トレーニング中の浮動小数点乗算/累積演算の大部分は NVFP4 、すなわち NVIDIA 4 ビット浮動小数点形式で実行されています。 Blackwell 向けに最適化されたこの手法は、精度を維持しながら、FP8 と比較してメモリ要件を大幅に削減しながらも、推論を高速化します。 低精度でネイティブにトレーニングを行うことは…
…The model includes features like speculative decoding, harness-optimized training, and quantization (NVFP4 and BF16 checkpoints) to deliver strong accuracy with up to 4x output speed compared to similar-sized models, making…
Here's a brand new Blackwell-native, prefill-optimized 4-bit quant that runs 50% faster on compatible hardware than a Q4 quant of the same memory footprint. And it runs 4-7% faster than other NVFP4 quants as benchmarked …
These are single stream numbers Following on from my previous post about v100s (here) and inspired by this comment (here) I decided to work on kernels that allow for an extremely fast path for Nvfp4 weights on sm70 and a…
Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It ran NInfer, a specia…
Safetensors, llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved: https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved GGUFs, llmfan46/Qwen3.5-35B-A3B-uncensored-here…
Poolside have updated the FP8 and NVFP4 checkpoints for Laguna S 2.1, increasing the default context size to 1 million, and updating the configs. Here's hoping they fixed the looping issue, this model has been great in m…
…NVFP4 정밀도 동일한 NVFP4 체크포인트가 NVIDIA Hopper, NVIDIA Blackwell, Ampere GPU에서 실행됩니다. 개발자들은 특수 NVFP4 양자화 커널 덕분에 모든 NVIDIA GPU 아키텍처에서 하나의 체크포인트를 사용할 수 있습니다. NVFP4는 또한 Blackwell에서 BF16 대비…
…Blackwell systems support NVFP4 along with other bit-width formats such as FP8 and FP16/BF16 (see Figure 6, below). NVFP4 is designed specifically for balancing model accuracy and speed; it applies…
…Fusing MXFP8 and NVFP4 quantization to reduce exposed memory overhead The popularity of lower precision recipes such as MXFP8 and NVFP4 for pretraining is rising, with these precisions providing significant speedup with…
…각 모델은 BF16 체크포인트로 Hugging Face에서 지금 바로 사용할 수 있으며, NVIDIA Blackwell 개발자를 위한 Gemma-4-31B NVFP4 양자화 체크포인트 도 NVIDIA Model Optimizer 와 vLLM을 통해 제공됩니다. NVFP4는 8비트 정밀도와…
…데이터센터의 NVIDIA Blackwell부터 에지의 Jetson까지 전 플랫폼을 지원하며, NVFP4 양자화·NIM 마이크로서비스·NeMo 파인튜닝으로 지금 바로 시작해 보세요. 4 MIN READ 2026년 4월 3일 CUDA Tile 프로그래밍, 이제 BASIC에서도! CUDA 13.1의…
…To leverage the most performant kernels available for each deployment, quantization was performed to FP8 on NVIDIA Hopper and to NVFP4 on NVIDIA Blackwell. To achieve the best performance for both Hopper…