LM Studio

Ollama if you are comfortable in a terminal, LM Studio if you would rather have a window with controls, and Jan if you want something that works out of the box and never asks you to install anything else. All three are free, all three run entirely offline, and all three use llama.cpp underneath, so the model behavior is the same. The difference is the interface, not the speed.

Usually not. llama.cpp has no NPU backend, so Ollama and LM Studio will leave the neural engine at zero percent on a Snapdragon or Ryzen AI machine and run on CPU or GPU instead. Reaching the NPU currently requires a vendor path: Qualcomm GenieX, AMD Lemonade with its NPU runtime, Intel OpenVINO, or Apple’s Neural Engine through the OS frameworks. It is worth knowing what you gain, which is battery life and low-power always-on operation rather than speed. NPUs also top out around 7B models today, and a discrete GPU will beat one on throughput every time.

For chat, 8GB of system RAM runs a 3B model, and 16GB runs a 7 to 8B model acceptably on CPU alone. For useful speed, you want the model in VRAM or unified memory: 16GB handles the 7 to 14B class, 24GB reaches the 30B class at Q4, and 32GB or more is where agentic coding work stops being frustrating. Above that, capacity matters more than bandwidth, which is why 96 to 128GB unified-memory machines run models that no consumer graphics card can hold.

For the desktop apps on this page, Windows and macOS are the smoother ride. LM Studio, Jan, AnythingLLM, and Ollama all install natively, GPU drivers come from the usual places, and nothing requires a terminal. Linux earns its keep one level down: the production serving engines, vLLM and SGLang, are Linux-first, multi-GPU serving effectively assumes it, and some of the newest acceleration paths land there before anywhere else, including AMD’s Ryzen AI NPU support, which arrived on Linux and wants a very recent kernel. The gap is narrower than it used to be. AMD unified its ROCm releases across