I Tried This Open Source ChatGPT Alternative on Linux, But Went Back to Ollama
I Tried This Open Source ChatGPT Alternative on Linux, But Went Back to Ollama
Publish Date: 2026-06-02 08:27:00
Source Domain: itsfoss.com
I may hate AI slop but I am not a AI hater. I have found decent use of the AI tools and I try to include these tools in my workflow wherever it makes sense.
While mainstream LLMs like ChatGPT and Perplexity have decent free offering, they leach on the user data. “If you are not paying for the product, you are the product”.
That’s why I am loving the idea of exploring local AI and I have spend my fair share of time experimenting with LLMs that can be run on normal systems.
Recently, I discovered Jan AI. It is a polished, genuinely usable desktop app that runs entirely on my machine. In fact, I once tried replacing Ollama and llama cpp with Jan AI, but later changed my mind.
I’ll explain why I switched back to Ollama in the later sections. First, let’s learn about Jan AI.
What is Jan AI?
Jan is a free and open-source desktop application that lets you run various large language models directly on your own hardware. You can think of it as a self-hosted, offline-capable ChatGPT, except the model runs on your CPU or GPU, and no data ever leaves your machine.
The project is developed by the Jan.ai company, and the source code is available on GitHub under the AGPL-3.0 license. It’s built on top of llama.cpp under the hood, which means it can run quantized GGUF models efficiently even without a dedicated GPU.
What I found impressive is that Jan’s desktop application is built using the Tauri framework instead of Electron JS, which gives it a good performance boost and I think eats less RAM, too.
A quick catch, you still need plenty of RAM for running your local LLM.
The app supports Linux, macOS, and Windows. I used it on my Linux machine.
System requirements
Running a local LLM with Jan AI does require decent hardware. Here’s what to realistically expect:
- 8 GB RAM minimum. It’s enough for 7B parameter models at 4-bit quantization (Q4_K_M). You’ll notice slowdowns with other applications open. 16 GB RAM is the sweet spot. It can comfortably run…
