Docker Model Runner does everything Ollama does, but I'm still not switching
…It gets the job done, and I don’t have to tweak a lot of things. Recently, Docker announced its own model runner , which offers many of the same features you get…
Search
All sources
xda-developers.com
23
tweaktown.com
11
developer.nvidia.com
4
wccftech.com
3
tomshardware.com
3
tweakers.net
2
androidauthority.com
1
pcworld.com
1
cnx-software.com
1
neowin.net
1
amd.com
1
cnet.com
1
iPhone 17 Pro Successfully Demonstrated Running A 400B Large Language Model, A Feat That Requires Minimum Of 200GB Memory Even When Compressed
Large Language Models with 400 billion parameters can only be run on capable hardware with heaps of memory, as even a quantized or compressed version requires a minimum of 200GB RAM. Looking…
These 5 small tweaks made my self-hosted LLM setup way more productive
…Sign in to your XDA account Most discussions about self-hosted LLMs focus heavily on GPUs , benchmarks, quantization, and running bigger models locally. I used to think the same way. But after…
I added a second GPU just for local AI workloads, and it cost less than upgrading my main one
…To further ensure your local model fits inside the VRAM, use tweaks like quantized models, offloading lesser-used layers to the RAM, and slightly reduced context windows. That's how I'm…
Discussions and forums
Laguna-S-2.1 "thinking forever" loops seem to be a quantization artifact
If you're running Laguna S 2.1 on llama.cpp and hitting thinking loops because it won't close its tags, you might want to look at your quant before you spend too much time tweaking settings. I spent a day debugging this,…
Gemma 4 12B Q3: +8.55% Coding Performance From Tensor-Level Quantization Allocation
Ive been experimenting with task-aware GGUF quants for months, taking inspiration from TASA and TAQO but pushing the allocation lower to to the tensor level. The basic idea is to generate a custom imatrix from a category…
Google's free Gemma 4 model runs on hardware you probably already own
…The latest from Google is Gemma 4 , and while there are four models in the family, each is tweaked for different tasks. That makes them interesting to use: you can choose the…
The iPhone 17 Pro can run a 400B parameter Large Language Model on-device by streaming weights from the SSD
…The specific model in question is Qwen3.5-397B-A17B (2-bit quantized), which is a 397B model with 17B active parameters. Per Daniel's published paper , only 5.5 GB of…
I finally ditched Ollama after using llama.cpp's WebUI, and I'm not going back anytime soon
…It often outperforms Ollama—faster tokens/sec and lower latency—while exposing deep tweakable inference settings. Ollama keeps convenience king—easy installs, one-click model swaps—so neither tool needs to be…
A Modder’s RTX 4080 Was Enough To Play AAA Games, But Not For Running LLMs, So He Integrated NVIDIA’s Tesla V100 At A Throwaway Price To Run 27B AI Models
…With a little tweaking here and there to reduce the fan noise by using a 9V battery and a PWM jumper, the V100’s modded cooler was now operating at 10 percent…
GIGABYTE Pushes Into The AI Assistant Race With GiMATE, Powering Three New AORUS And AERO Laptops Built Around RTX 50 Series GPUs
…The Customize Mode includes various tweaks for controlling the CPU operation on the AMD Ryzen CPUs such as SPL for long-term CPU power, SPPOT for managing boost powre, and FPPT for…
I tested every local LLM tweak people recommend, and only these ones actually mattered
…For example, the same temperature setting will behave differently depending on your model, its size, quantization, and whatever the other parameters are doing at the same time. Changing two things at once…