I built a local LLM server I can access from anywhere, and it uses a Raspberry Pi
…Related I ran local LLMs on a "dead" GPU, and the results surprised me My Pascal card may not be ideal for intensive workloads, but it's more than enough for light…
Search
Related topics:
LLMs
All sources
xda-developers.com
37
developer.nvidia.com
29
blogs.nvidia.com
14
amd.com
7
theregister.com
3
intel.com
3
huggingface.co
3
phoronix.com
2
wccftech.com
2
press.asus.com
2
extremetech.com
1
cnx-software.com
1
Videos
More videosTech Mahindra Bridges India's Language Gap with AI
…We needed to ensure GPUs were not a requirement, and that it could run on a typical PC as well as a server.” Solution: Open-Source AI for India One of the…
LLM Inference Benchmarking: How Much Does Your LLM Inference Cost? | NVIDIA Technical Blog
…How do you calculate required server capacity for peak LLM request volumes? To calculate the required infrastructure for a given LLM application, we need to identify the following constraints: Latency type and…
LLM From Scratch is a hands-on workshop where you write every piece of an AI from nothing
…Related After a year of self-hosting LLMs, I realized the real bottleneck isn’t the GPU Hardware is just the entry fee for local intelligence.
Top stories
Trying to self-host LLMs made me realize local AI has a friction problem, not a quality problem
5d agoAMD just dropped a compact AI workstation that makes discrete GPUs look outdated for running LLMs
1w agoI added a second GPU just for local AI workloads, and it cost less than upgrading my main one
1w ago13 years later, the GTX Titan is still the most important GPU Nvidia ever made
1w agoMLOps – NVIDIA Technical Blog
…You can optimize for specific GPU configurations and achieve... 9 MIN READ Jan 08, 2026 Accelerating LLM and VLM Inference for Automotive and Robotics with NVIDIA TensorRT Edge-LLM Large language models…
I ditched my dedicated GPU for integrated graphics and cut my power bill in half
…for my media center Using a GPU to handle specific workloads is a must-have, depending on what you wish to achieve. For running a large language model (LLM), you absolutely need…
Nemotron-Nano-9B-v2-Japanese の推論チュートリアル
…The user is asking which prefecture is famous for \"Kusatsu Senbei,\" which is a type of cracker. Wait, the user wrote \"草加せんべい\" which is \"Kusatsu Senbei.\" But I need to check if…
Discussions and forums
My first 10 inch rack with local LLM! No more Spotify, Google Home, Netflix, ChatGPT...
I'm pretty new to homelabbing and this is my first mini rack! Started with the Beelink ME Mini and then just kinda grew from there (it's always the way hey haha). It idles at 70 watts (not too shabby for how much is goin…
Computer build using Intel Optane Persistent Memory - Can run 1 trillion parameter model at over 4 tokens/sec
As the title states, my build is indeed able to run a 1 trillion parameter model (in this case Kimi K2.5) locally at ~4 tokens/second. I thought r/LocalLLaMA would be interested in the build due to that stat line, and al…
80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP
Just wanted to share my config in hopes of helping other 12GB GPU owners achieve what I see as very respectable token generation speeds with modest VRAM. Using the latest llama.cpp build + MTP PR, I got over 80 tok/sec w…
2.5x faster inference with Qwen 3.6 27B using MTP - Finally a viable option for local agentic coding - 262k context on 48GB - Fixed chat template - Drop-in OpenAI and Anthropic API endpoints
2026-05-07 edit: I have updated the hardware based recommendations with more focus on quality. I do not recommend q4_0 KV cache anymore beyond 64k context. After multiple rounds of testing with the different size quants,…
services with actually generous free tiers for open-source projects. my list, what would you add?
Been in the weeds shipping an OSS side project for the past few weeks (social media publishing API). Real launch post is coming, this isn't that. Along the way I kept a list of services that actually have usable free tie…
Peak Training: Blackwell Delivers Next-Level MLPerf Training Performance
…The same benchmark run using Hopper needed 256 GPUs. The Blackwell training results follow an earlier submission to MLPerf Inference 4.1, where Blackwell delivered up to 4x more LLM inference performance…
I ditched Copilot on VS Code for this free extension, and it's miles ahead
…AI for help, you can see why I don’t want to use cloud-based models in my workflow. Related Your old GPU can still run big LLMs – you just need the…
How Small Language Models Are Key to Scalable Agentic AI | NVIDIA Technical Blog
…adding a new skill or fixing a behavior can be done in a few GPU hours on an SLM, compared to days or weeks of fine-tuning for LLMs. With edge deployments…