Edge Computing – NVIDIA Technical Blog
…You can optimize for specific GPU configurations and achieve... 9 MIN READ Jan 08, 2026 Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM Large language models…
Search
Related topics:
LLMs
All sources
xda-developers.com
40
developer.nvidia.com
33
blogs.nvidia.com
14
amd.com
8
theregister.com
3
phoronix.com
3
wccftech.com
2
press.asus.com
2
huggingface.co
2
extremetech.com
1
cnx-software.com
1
nextplatform.com
1
Videos
More videosNetworking / Communications – NVIDIA Technical Blog
…You can optimize for specific GPU configurations and achieve... 9 MIN READ Jan 08, 2026 Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM Large language models…
Content Creation / Rendering – NVIDIA Technical Blog
…You can optimize for specific GPU configurations and achieve... 9 MIN READ Jan 08, 2026 Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM Large language models…
Trustworthy AI / Cybersecurity – NVIDIA Technical Blog
…You can optimize for specific GPU configurations and achieve... 9 MIN READ Jan 08, 2026 Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM Large language models…
Top stories
I built my own Googlebook with a Raspberry Pi, local LLMs, and old hardware
1w agoI added a second GPU just for local AI workloads, and it cost less than upgrading my main one
1w ago13 years later, the GTX Titan is still the most important GPU Nvidia ever made
1w agoMy RTX 5090 can't keep up with Apple Silicon on the biggest local LLMs, and I hate to admit it
1w agoData Center / Cloud – NVIDIA Technical Blog
…You can optimize for specific GPU configurations and achieve... 9 MIN READ Jan 08, 2026 Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM Large language models…
Simulation / Modeling / Design – NVIDIA Technical Blog
…You can optimize for specific GPU configurations and achieve... 9 MIN READ Jan 08, 2026 Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM Large language models…
Computer Vision / Video Analytics – NVIDIA Technical Blog
…You can optimize for specific GPU configurations and achieve... 9 MIN READ Jan 08, 2026 Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM Large language models…
Discussions and forums
My first 10 inch rack with local LLM! No more Spotify, Google Home, Netflix, ChatGPT...
I'm pretty new to homelabbing and this is my first mini rack! Started with the Beelink ME Mini and then just kinda grew from there (it's always the way hey haha). It idles at 70 watts (not too shabby for how much is goin…
Computer build using Intel Optane Persistent Memory - Can run 1 trillion parameter model at over 4 tokens/sec
As the title states, my build is indeed able to run a 1 trillion parameter model (in this case Kimi K2.5) locally at ~4 tokens/second. I thought r/LocalLLaMA would be interested in the build due to that stat line, and al…
80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP
Just wanted to share my config in hopes of helping other 12GB GPU owners achieve what I see as very respectable token generation speeds with modest VRAM. Using the latest llama.cpp build + MTP PR, I got over 80 tok/sec w…
2.5x faster inference with Qwen 3.6 27B using MTP - Finally a viable option for local agentic coding - 262k context on 48GB - Fixed chat template - Drop-in OpenAI and Anthropic API endpoints
2026-05-07 edit: I have updated the hardware based recommendations with more focus on quality. I do not recommend q4_0 KV cache anymore beyond 64k context. After multiple rounds of testing with the different size quants,…
services with actually generous free tiers for open-source projects. my list, what would you add?
Been in the weeds shipping an OSS side project for the past few weeks (social media publishing API). Real launch post is coming, this isn't that. Along the way I kept a list of services that actually have usable free tie…
Agentic AI / Generative AI – NVIDIA Technical Blog
…You can optimize for specific GPU configurations and achieve... 9 MIN READ Jan 08, 2026 Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM Large language models…
Claude Code with a local LLM running offline is the hybrid setup I didn't know I needed
…between multiple local LLMs, but honestly, the only one right now that's capable enough for coding is Qwen3-Coder-Next, and until that changes, I don't need another running. That…
Unpacking the deceptively simple science of tokenomics
…For the same amount of power, InferenceX data shows that TensorRT LLM running on Nvidia's B200 GPUs is significantly more efficient at serving models like DeepSeek R1 than something like SGLang…