Tracked topic
…concurrent coding agents, and other high-throughput multimodal use cases. Developers can use StepFun’s NVFP4-quantized checkpoint available through Hugging Face for boosted inference due to reduced memory bandwidth and storage…
…NVIDIA Vera Rubin 플랫폼 은 50 PFLOPs의 NVFP4 연산을 활용하고 Vera CPU를 활용해 LLM 도구 호출을 가속함으로써, 에이전트 워크플로의 종단 간 성능과 경제성, 효율을 개선하며 이러한 이득을 한층 확장할 것으로 기대됩니다. 에이전트…
…On the NVIDIA Blackwell platform, the model runs in NVFP4 precision. That cuts memory requirements and pushes inference up to 4x faster than FP8 on NVIDIA Hopper, with no loss in accuracy…
…With Rubin CPX, NVIDIA has essentially covered up the prefill stages through its attention-acceleration engines and massive NVFP4 compute. For decoding, NVIDIA will leverage Groq, as Jensen mentioned. LPUs leverage on…
Here's a brand new Blackwell-native, prefill-optimized 4-bit quant that runs 50% faster on compatible hardware than a Q4 quant of the same memory footprint. And it runs 4-7% faster than other NVFP4 quants as benchmarked …
These are single stream numbers Following on from my previous post about v100s (here) and inspired by this comment (here) I decided to work on kernels that allow for an extremely fast path for Nvfp4 weights on sm70 and a…
Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It ran NInfer, a specia…
Safetensors, llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved: https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved GGUFs, llmfan46/Qwen3.5-35B-A3B-uncensored-here…
Poolside have updated the FP8 and NVFP4 checkpoints for Laguna S 2.1, increasing the default context size to 1 million, and updating the configs. Here's hoping they fixed the looping issue, this model has been great in m…
…Further optimizations, including NVFP4 precision, are expected to deliver enhanced performance gains over time. Architectural innovations for long-context inference Qwen3.8-2.4T-A95B is built for the hardest agentic workloads…
…Users can also leverage NVIDIA's NVFP4, FastVideo, and ComfyUI for enhanced text, image, and video-to-video generation. Meta is also releasing Muse Glimmer, its latest 30-billion open-weight model…
…NVLink switch and fabric (for the NVL72 rack-scale design and NVIDIA HGX architecture), and NVFP4 -enabled Tensor Cores, increasing throughput per watt. Recent SemiAnalysis InferenceX data shows that NVIDIA software optimizations…
…Recently, Nvidia introduced a new 4-bit number format, NVFP4 , to accelerate AI workloads, which generally tolerate low-precision math. But some tasks require precision. Singh cited geographical information systems, such as…
…At just 2.6 billion parameters, its distilled version generates a full 60-second clip in 34 seconds on a single NVIDIA GeForce RTX 5090 GPU using the NVFP4 format — delivering up…
…However, we are supply constrained." "One of the best ways to improve memory use is to use extremely, extremely low precision," Huang said, pointing to NVFP4, Nvidia's 4-bit floating-point…