I switched to Linux for local LLMs and setup that took hours on Windows took minutes
I switched to Linux for local LLMs and setup that took hours on Windows took minutes
https://www.makeuseof.com/switched-to-linux-for-local-llms-setup-took-minutes/
Publish Date: 2026-04-30 15:30:00
Source Domain: www.makeuseof.com
I never thought it would be so challenging to run a local LLM on Windows. Even when it seemed fine, I later realized that the instance was running entirely on my CPU. Configuring drivers, environment variables, and eventually setting up WSL2 after pulling an Ollama model felt like a separate project. And after that, I found it very exhausting to maintain the stack.
I was pleasantly surprised when I tried it on Linux. The process was more direct, easier, and honestly, I was no longer expecting things to break. It felt similar to when I turned my Linux terminal into a local AI assistant.
The setup that should have taken ten minutes
Why Ollama on Windows keeps pulling you back in
The setup process for Ollama on Windows may seem straightforward, but in reality, it goes far beyond downloading and running the installer. On the first run, you may be convinced that everything is perfect because you’ll get text and responses. However, checking with the command: ollama ps, reveals an idle GPU. This is because it’s running on the CPU, and a mere 3 to 5 tokens per second is substantially slower than it should be.
Ollama’s default silent fallback is common and hard to spot because it emits no error. This behavior is possible on other OSes if the VRAM available is too small for the model. However, on Windows, several other factors, such as driver/installer race conditions, environment…
