Tracked topic
…NVIDIA developed the GB10 Superchip, which combines innovations from datacenters, such as NVFP4, CUDA, SLANG, TensorRT, vLLM, CX-7 NIC, NVLINK C2C, TMEM, and more, down to client platforms that utilize a…
…July 13, 2026 Serving NVFP4 Models on AMD Instinct™ MI355 Accelerators — ROCm Blogs Learn how to serve NVFP4 models on AMD Instinct™ MI355 using an emulation pipeline in vLLM — no format conversion…
…Pair the NVFP4 checkpoints with Google’s new Multi-Token Prediction drafters to get up to 3x faster inference at identical output quality, enabling frontier-class reasoning to run locally on NVIDIA…
…エージェント型推論向けのオープン ハイブリッド Mamba-Transformer MoE Nemotron 3 Super は、高容量の推論モデルにおける典型的な効率と精度のトレードオフを軽減するアーキテクチャ革新を導入しています。 3 MIN READ 2026 年 2 月 6 日 NVFP4 が AI のトレーニングと推論を加速する 3 つの方法 NVIDIA による徹底的な共同設計によって、モデルのトレーニングと推論の両方において、優れた精度で大幅なパフォーマンスの向上が達成が見込めるようになりました。 2 MIN READ…
Here's a brand new Blackwell-native, prefill-optimized 4-bit quant that runs 50% faster on compatible hardware than a Q4 quant of the same memory footprint. And it runs 4-7% faster than other NVFP4 quants as benchmarked …
These are single stream numbers Following on from my previous post about v100s (here) and inspired by this comment (here) I decided to work on kernels that allow for an extremely fast path for Nvfp4 weights on sm70 and a…
Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It ran NInfer, a specia…
Safetensors, llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved: https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved GGUFs, llmfan46/Qwen3.5-35B-A3B-uncensored-here…
Poolside have updated the FP8 and NVFP4 checkpoints for Laguna S 2.1, increasing the default context size to 1 million, and updating the configs. Here's hoping they fixed the looping issue, this model has been great in m…
…Looking forward, NVIDIA's Rubin is just on the horizon and is expected to extend these leads through a supercharged AI architecture, which will offer 50 PFLOPs of compute from NVFP4, and…
…또한 FP8과 NVFP4 양자화 , 효율적인 비디오 샘플링, NVIDIA 최적화 커널을 지원해 예측 가능하고 지연 시간이 낮은 추론을 제공합니다. 여기에 3D 컨볼루션 기반 시공간 처리가 결합되면 워크스테이션부터 데이터센터, 클라우드 배포 환경까지 GPU 전반에서…
…NVIDIA GB300 NVL72 demonstrates up to 20x higher agentic coding performance. The NVIDIA Vera Rubin platform is expected to extend these gains by leveraging 50 PFLOPs of NVFP4 compute and leveraging the…
…a single GPU; its distilled variant can be deployed on a single RTX 5090 with NVFP4 quantization to denoise a 60s 720p clip in 34s. On our one-minute world-model benchmark…
…Baseten used the low-precision NVFP4 data format, the NVIDIA TensorRT-LLM library — an open source C++/Python framework for optimizing large language model inference on NVIDIA GPUs that includes tensor parallelism…
…重要なポイントが 1 つあるとすれば、適切な量子化の精度を使用することです。 NVFP4、INT4、W4A16 などのフォーマットは、多くの LLM ワークロードで高い精度を維持しながら、メモリとストレージの要件を大幅に削減します。 実際のユース ケース: Reachy Mini Jetson Mini Assistant これらのメモリ最適化の効果を示すために、Jetson Orin Nano 上で実行されるオンデバイス対話型 AI ロボットである Reachy Mini Jetson Assistant を考えてみましょう。これは…