I'm running a 284-billion-parameter model across two machines, and it finally matches the cloud
… This is because software dequantization saturates a 273 GB/s memory bus, while NVFP4 doesn't have to deal with that. …
Search
Tracked topic
NVFP4
Live now
An NVFP4 quant of Qwen3.8 27B is being touted as the fastest available right now, based on user-perceived performance.
Live sources:
r/LocalLLaMA