Sipeed Crams 32GB LPDDR5 and a 60 TOPS NPU Into a Compact RISC-V Board That Hits 15 Tokens/s on Qwen-3.5 35B AI LLMs
… If you want a small AI machine that can handle up to 30B LLMs, then these definitely look enticing. …
Search
Tracked topic
LLMs
Large language models are machine learning models trained to predict and generate text and other language-based outputs.
Top stories
This PCIe AI Accelerator Card Can Run 700B LLMs Locally With 384 GB Memory at Just 240W, Less Than Half The Power of RTX PRO 6000 Blackwell
… The platform is purpose-built for LLMs with optimized performance and power efficiency in mind. …
AMD Pushes Ryzen AI MAX 400 ‘Gorgon Halo’ to 192GB Memory, Letting a Single Chip Run 300B+ Parameter LLMs Locally
… With up to 192 GB of unified memory, the Ryzen AI MAX 400 chips will be able to support massive AI LLMs locally. …
AMD's vLLM-ATOM Plugin Supercharges DeepSeek-R1, Kimi-K2, and gpt-oss-120B AI LLM Inference on Instinct MI350 and MI400 Accelerators
… AMD Offers Big Boost To AI LLMs With Its vLLM-ATOM Plugin That Works Seamlessly With vLLM & Accelerates AI Inference Performance The vLLM-ATOM is a purpose-built plugin that aims to improve inference performance across various AI LLMs. …
Turn Your Vacant M.2 Slot Into a 20B LLM Cruncher With This Dedicated AI Module: Packing 32 GB Memory & 60 TOPs
… Now, in terms of performance, the 32 GB memory capacity enables the module to easily run AI LLMs with up to 20B parameters. …
Intel's Latest Drivers Let's Users Allocate Up To 93% of System Memory To Arc iGPUs For Wider AI LLM Support
… Trying To Run AI LLMs That Don't Fit On Your Arc iGPU? …
NVIDIA's V100, An 8-Year Old GPU, Now Sells for $100 and Crushes Modern Consumer Cards in AI LLM Workloads
… While this proves that older GPUs are still viable for AI LLMs, offering great value and efficiency, they do require extra modding, which isn't up to everyone to perform. The 32 GB model does cost around $400 - $500 US, but the extra memory capacity can further help in bigger AI LLMs. …
Apple’s Local LLM Revenue To Suffer Major Hit As M5 Ultra Mac Studio Not Expected Until Q4, Older Configurations Also Sold Out
… Currently, there are no alternatives to powerful hardware that can run local LLMs with increased memory, with NVIDIA’s RTX PRO 6000 limited to 96GB of GDDR7 VRAM and priced between $6,500 and $9,500. …
NVIDIA Boosts RTX AI PCs With 35% Faster LLM & 3x Faster Creative AI Performance, NVFP4 To Reduce VRAM Usage
… There are two parts of this update, first is faster LLM performance, offering up to 40% higher performance in LLMs such as GPT-OSS, Nemotron Nano V2, and Sque 3 308. …
Intel Confirms Arc Pro B70 Workstation GPU Via LLM-Scaler vLLM AI Release Notes
…However, today we are seeing the GPU model name mentioned in the open-source AI-inference framework developed by Intel called LLM-Scaler vLLM. The release notes for the latest LLM-Scaler…