… For Qwen3 on my compact system, the 4B model managed around 4 tok/s with a simple question, and when asked what XDA Developers is. Not brilliant, but more than sufficient for loading queries while doing something else. The Intel Core i5-10210U was never designed with local LLMs in mind. …
… Once I ran the command llama-cli -hf bartowski/Qwen3.5-0.8B:Q4 K M -p "Tell me about XDA Developers" -n 128 , the LLM provider pulled the model, loaded it on the Raspberry Pi, and began processing the prompt. …
… Sign in to your XDA account We, at XDA, absolutely love local LLMs . …
… It's not been six months since orchestrators for Claude and other LLMs started taking over GitHub, but that's a long time in frontier model time, and the LLMs have gotten exponentially better since. …
… Sign in to your XDA account Like many, my first dive into using local LLMs was with LM Studio . …
… If you’ve read my articles at XDA, you’re probably aware that I use LLMs with certain LXCs and Docker containers. …
… Sign in to your XDA account LM Studio has been my default runner for as long as I've been running local LLMs , which is more than long enough now to call it part of my daily flow rather than just something I'm experimenting with anymore. …
… Fair enough, LLMs are somewhat digital representations of toddlers, and this is expected behavior for that age. …
… Sign in to your XDA account The idea of local LLMs is fascinating. …
… But I do want to briefly explain how local LLMs actually work, because it makes everything else in this piece make a lot more sense. …