Running Ollama on a 15W CPU sounded ridiculous until I got it working with decent results
… The Intel Core i5-10210U was never designed with local LLMs in mind. It's a mobile chip slapped onto a compact mini PC motherboard. …
Search
Related topics:
LLMs
ChatGPT, Claude, and Gemini still have a critical UI flaw, and it's not an easy one to fix
… Canvas, Artifacts, Codex, and others are transitional designs We're not there yet OpenAI, Anthropic, and Google knew that the chatbot mode wasn't going to suffice for coding, designing, and other hands-on projects. …
I turned my phone into a local LLM server, and it handles vision, voice, and tool calls
… They've got multimodal input text, image, and audio , a 128K context window, and a hybrid attention design that keeps memory use low. On a modern phone with enough RAM and a modern chipset, both of these models can run at surprising speed, complete with tool calling. …
You don't need an expensive GPU to run a local LLM that actually works
… General models can largely handle most tasks, but specific LLMs designed for use in these areas can outshine larger general counterparts, unlocking more performance without touching hardware or making a single tweak. …
After a year of self-hosting LLMs, I realized the real bottleneck isn’t the GPU
… When a GPU isn't available, LLMs run entirely in system RAM. …
Your old GPU can still run big LLMs – you just need the right tweaks
… Related 7 things I wish I knew when I started self-hosting LLMs I've been self-hosting LLMs for quite a while now, and these are all of the things I learned over time that I wish I knew at the start. …
I don't need Claude, this is the local LLM I run on my NAS that powers my smart home
… Most NAS devices have an older-generation chip. I have a Ugreen DH4300+ NAS, which runs an 8-core rocket chip processor. …
My RTX 5090 can't keep up with Apple Silicon on the biggest local LLMs, and I hate to admit it
… However, for the specific job of running large local LLMs, the architecture Apple landed on almost by accident as a power-efficiency play for laptops turned out to be the right shape for a workload nobody was thinking about when it was designed. …
I manage my ESP32 boards with this free self-hosted tool
… So, it shouldn’t sound weird when I say I’ve got a bunch of ESP32 MCUs in my arsenal, including tiny system-on-chip boards. …
AMD shipped Nvidia's new AI laptop over a year ago, and the software is finally catching up
… It already powers the DGX Spark desktop, which is something that company CEO Jensen Huang himself confirmed when he tied the consumer N1 and N1X chips to the same design. …