고급 융합 커널로 MoE 학습 처리량 높이기
…MXFP8·NVFP4 양자화를 융합해 노출된 메모리 오버헤드 줄이기 사전 학습을 위한 MXFP8, NVFP4 같은 저정밀도 레시피의 인기가 높아지고 있으며, 이러한 정밀도는 정확도에 미치는 영향을 최소화하면서도 상당한 속도 향상을 제공합니다. 이러한 저정밀도 레시피에서는…
Search
Related topics:
NVFP4
All sources
developer.nvidia.com
56
blogs.nvidia.com
27
wccftech.com
16
huggingface.co
7
tomshardware.com
4
igorslab.de
3
techpowerup.com
3
guru3d.com
2
xda-developers.com
2
amd.com
2
hardwareluxx.de
2
intel.com
1
Tracked topic
NVFP4
Live now
A Qwen3.8-27B NVFP4 (with vision) setup is reported to run on one RTX 5090 using a 451K token KV-cache at ~120 tokens/s under a 400W limit.
Live sources:
r/LocalLLaMA
Data Center Deep Learning Product Performance Hub
…This is enabled by deep co-design across NVIDIA Blackwell, NVLink™, and NVLink Switch for scale-out; NVFP4 for low-precision accuracy; and NVIDIA Dynamo and TensorRT™ LLM for speed and flexibility…
How NVIDIA’s Inference Software Stack Powers the Lowest Token Cost
…Disaggregated serving, large expert parallelism over NVIDIA NVLink interconnect technology, NVFP4 precision and multi-token prediction each deliver meaningful gains on their own. Combined, they increase throughput by up to 20x. The…
Paper page - Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
…Nemotron 3 Ultra is our most capable model yet, employing multiple key technologies - LatentMoE , Multi Token Prediction (MTP), NVFP4 pre-training, multi-environment RLVR , MOPD, and reasoning budget control . Nemotron 3 Ultra…
Discussions and forums
Fastest NVFP4 quant of Qwen3.8 27B out there
Here's a brand new Blackwell-native, prefill-optimized 4-bit quant that runs 50% faster on compatible hardware than a Q4 quant of the same memory footprint. And it runs 4-7% faster than other NVFP4 quants as benchmarked …
366 t/s Qwen3.6 27B NVFP4 on v100s
These are single stream numbers Following on from my previous post about v100s (here) and inspired by this comment (here) I decided to work on kernels that allow for an extremely fast path for Nvfp4 weights on sm70 and a…
Qwen3.8-27B NVFP4 with vision + 451K token KV-cache on one RTX 5090 (power limited to 400W) at 120 tokens/s average
Hello, So I've been trying lots of combinations in that never-ending landscape of options and settings. I wanted a proper quant of 3.8 27B running as fast as possible on my 5090 at 400W, with vision and with as much KV-c…
NVFP4 on VOLTA! Despite being built for Blackwell, I made four 2017 V100s run Qwen 3.8 NVFP4 natively and match my $6000 RTX 5090.
Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It ran NInfer, a specia…
Qwen3.5 35B A3B uncensored heretic Native MTP Preserved is Out Now With the Full 785 MTPs Preserved and Retained, Available in Safetensors, GGUFs. NVFP4, NVFP4 GGUFs and GPTQ-Int4 Formats
Safetensors, llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved: https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved GGUFs, llmfan46/Qwen3.5-35B-A3B-uncensored-here…
Introducing Nemotron 3 Super: An Open Hybrid Mamba-Transformer MoE for Agentic Reasoning | NVIDIA Technical Blog
…Native NVFP4 pretraining optimized for NVIDIA Blackwell, significantly cutting memory requirements and speeding up inference by 4x on NVIDIA B200 compared to FP8 on NVIDIA H100, while maintaining accuracy. Multi-environment reinforcement…
NVIDIA Local AI: Build and Run AI on Your GPU
…Use Optimized Models Run NVIDIA-optimized open-weight models locally with day-0 support, NVFP4 quantization, and tuned Tensor Core performance. Run a Model Efficiently Run open-weight models with accelerated runtimes…
Streaming Tokens and Tools: Multi-Turn Agentic Harness Support in NVIDIA Dynamo | NVIDIA Technical Blog
…Harness-facing Dynamo settings Our experiments used the newly released nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 model, though the same issues apply across models, reasoning parsers, and tool-call parsers…
I switched my local LLM setup to Ollama's new MLX engine, and my Mac suddenly feels twice as fast
…Ollama's updated MLX engine now supports NVIDIA's model-optimized NVFP4 quantization format. Quantization reduces the memory required to run a model, but it also removes some information from the original…
GIGABYTE Demonstrates AI TOP ATOM Four-Node Clustering for Scientific Computing
…Powered by NVIDIA NemoClaw blueprints, the workflow orchestrates NVIDIA Nemotron-3-Nano-30B-NVFP4 open-source models for research hypothesis generation and dispatches GROMACS to execute simulations across the cluster. By connecting…
Announcing the AMD ROCm Certification Program — Your Pathway to AMD GPU Excellence
…July 13, 2026 Serving NVFP4 Models on AMD Instinct™ MI355 Accelerators — ROCm Blogs Learn how to serve NVFP4 models on AMD Instinct™ MI355 using an emulation pipeline in vLLM — no format conversion…