I replaced ChatGPT's free tier with a tiny local model running on my old laptop
… Otherwise, the inference engine would refuse to detect my GPU and instead switch to CPU for the processing tasks. …
Although the HDMI dummy plug is a fairly generic device that's quite easy to make at home, if you're good with the soldering iron. Before we get into all that, you might wonder why I'd want to use my PC in a headless setup, lacking a display. There are at least two use cases I can think of immediately. First, when I remote into a PC using a low-powered laptop, I don't need this PC to have any I/O accessible on location so long as it is powered on. However, when the GPU cannot detect a connected display or monitor, the remote client defaults to a rather low default resolution of 1024x768 pixels
An HDMI dummy plug might be the smallest cheap gadget that's actually useful… Otherwise, the inference engine would refuse to detect my GPU and instead switch to CPU for the processing tasks. …
… This sets the stage for constant context-switching between my main PC and my laptop with different interfaces—one reinforcing habits I'm trying to break and one pushing me to try new ones. …
… For instance, my GPU likes to nap if there's no display connected and switched on. …
… Related Can you upgrade a laptop graphics card GPU ? Can you make a graphics switch in your favorite laptop? …
… Heck, I occasionally use it to power my remote gaming VM So far, I’ve only covered the use cases where my GPU pairs with my containerized apps. …
… Likewise, video-editing tools like Davinci Resolve run without any issues, and since I’ve got it permanently hardwired to the same switch as my main NAS server, logging into my Windows 11 VM from my laptops and editing videos is a lot faster than manually transferring the files onto my portable dev… …
… Apple Silicon's unified memory architecture means the CPU and GPU share the same high-bandwidth memory pool. A MacBook with 16GB or 32GB of unified RAM can load and run LLMs at speeds that rival or exceed systems with discrete GPUs, making Apple laptops surprisingly competitive for local AI. …
… Related Your old GPU can still run big LLMs – you just need the right tweaks There's a lot you can do with these models Proxmox LXCs are incredible for hosting llama.cpp With some GPU passthrough wizardry, I can put my old graphics cards to good use Like most LLM-hosting enthusiasts, I started my j… …
… Even Immich can harness your GPU for machine learning-aided facial recognition, object detection, and smart search tools. But leaving these standalone tools aside, your gaming GPU’s biggest utility lies in its ability to run LLMs. …
… Desktop PCs come with a wide range of ports, and modern GPUs support both HDMI and DisplayPort. The case for laptops and mobile devices, however, is different. …