Search

Ollama if you are comfortable in a terminal, LM Studio if you would rather have a window with controls, and Jan if you want something that works out of the box and never asks you to install anything else. All three are free, all three run entirely offline, and all three use llama.cpp underneath, so the model behavior is the same. The difference is the interface, not the speed.

Usually not. llama.cpp has no NPU backend, so Ollama and LM Studio will leave the neural engine at zero percent on a Snapdragon or Ryzen AI machine and run on CPU or GPU instead. Reaching the NPU currently requires a vendor path: Qualcomm GenieX, AMD Lemonade with its NPU runtime, Intel OpenVINO, or Apple’s Neural Engine through the OS frameworks. It is worth knowing what you gain, which is battery life and low-power always-on operation rather than speed. NPUs also top out around 7B models today, and a discrete GPU will beat one on throughput every time.

For chat, 8GB of system RAM runs a 3B model, and 16GB runs a 7 to 8B model acceptably on CPU alone. For useful speed, you want the model in VRAM or unified memory: 16GB handles the 7 to 14B class, 24GB reaches the 30B class at Q4, and 32GB or more is where agentic coding work stops being frustrating. Above that, capacity matters more than bandwidth, which is why 96 to 128GB unified-memory machines run models that no consumer graphics card can hold.

Less than the marketing suggests, today. Most local LLM runtimes lean on the GPU, and every tokens-per-second number on this page came from GPU inference except the Snapdragon EliteBook, where the platform’s AI stack is the point. NPUs currently earn their keep on efficiency and on INT8 image generation paths, and they are why several of these machines qualify as Copilot+ PCs. Buy for the GPU and memory first.