Tracked topic
…Q (IGX T7000), NVIDIA RTX 6000 ADA (IGX Orin 700); ISL/OSL: 2028/128; Quantization: NVFP4 (IGX T7000), W4A16 (IGX Orin 700); Framework: VLLM Configuration: dGPU: NVIDIA RTX PRO 6000 Blackwell Max…
…NVIDIA collaborated with vLLM, Ollama, llama.cpp and LM Studio to provide the best local deployment experience for Nemotron 3.5 Lightning models — offering developers choice of NVFP4 and GGUF format of…
…This advance shifts inference performance limits from matrix math to non-linear SFU operations, making hardware-software co-design techniquesincluding LDTM.STAT offloading, CUDNN optimization, and NVFP4 KVCache managementcritical for maximizing attention…
…An NVFP4 Parallel Infrastructure for Long Video Generation (2026) Spike-Aware C++ INT8 Inference for Sparse Spiking Language Models on Commodity CPUs (2026) Still: Amortized KV Cache Compaction in a Single Forward…
Here's a brand new Blackwell-native, prefill-optimized 4-bit quant that runs 50% faster on compatible hardware than a Q4 quant of the same memory footprint. And it runs 4-7% faster than other NVFP4 quants as benchmarked …
These are single stream numbers Following on from my previous post about v100s (here) and inspired by this comment (here) I decided to work on kernels that allow for an extremely fast path for Nvfp4 weights on sm70 and a…
Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It ran NInfer, a specia…
Safetensors, llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved: https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved GGUFs, llmfan46/Qwen3.5-35B-A3B-uncensored-here…
Poolside have updated the FP8 and NVFP4 checkpoints for Laguna S 2.1, increasing the default context size to 1 million, and updating the configs. Here's hoping they fixed the looping issue, this model has been great in m…
…Nvidia behauptet, damit eine deutlich schnellere Inferenz zu erreichen, was möglicherweise an dem im Modell verwendeten Datentyp NVFP4 liegt. Auch der optimierte Attention-Mechanismus mit vielen Mamba-Layern trägt dazu bei, der…
…das von NVIDIA optimierte NVFP4 sowie der Standard MXFP4 wurden von den eingereichten Partnern teilweise bereits verwendet. Erstmals wird in den Ergebnissen genannt, welche Genauigkeit verwendet wurde. Für zukünftige Runden des MLPerf…
…NVIDIA pioneered the RTX exclusive acceleration of NVFP4 – a floating point format that reduces VRAM consumption by up to 60% on NVIDIA GPUs based on the Blackwell-architecture. When powered…
…The NVIDIA Vera Rubin platform employs extreme co-design across multiple specialized chips (NVL72, Vera CPU, Groq 3 LPX, NVLink 6, ConnectX-9, BlueField-4, Spectrum-X) and software optimizations (Dynamo, NVFP4…
…memory, providing up to 22 TB/s of total bandwidth, along with 50 PFLOPS of NVFP4 compute performance. At the transistor level, each NVIDIA Rubin GPU has 336 billion transistors, with an…
…For example, NVFP4 , a new data format that uses just four bits per parameter during large language model ( LLM ) training, was discovered with Nemotron. This advancement — which dramatically reduces energy use — is…