I used to be that guy about Ollama. You know the one. "Just run ollama run mistral, it's that easy." I said that to anyone with a laptop and a pulse. And for a while, it kinda was.
But six months in, the cracks got harder to ignore. Models felt slow for no obvious reason. Weird behavior I couldn't explain. Too many moments where I realized I had no clue what was actually happening under the hood. I went digging. What I found made me delete Ollama off my machine entirely.
Genuinely never going back.
What I Thought I Was Running
Give credit where it's due: Ollama solved a real problem. Back in 2023, running a local LLM meant cloning llama.cpp, compiling from C++, figuring out arcane flags, and hoping. That process alone filtered out most people. Ollama wrapped it in ollama run and suddenly everyone could do it. I was grateful at the time.
What took me months to realize? I wasn't really using Ollama. I was using llama.cpp with an Ollama-colored wrapper.
Every single model I ran through Ollama was actually being executed by Georgi Gerganov's inference engine. Every token generated on my laptop was generated by code he and hundreds of other open-source contributors wrote. Ollama was the waiter bringing the food, but llama.cpp was the kitchen.
How I Found Out the Kitchen Hated Being Credited
When I started reading through GitHub issues and community discussions, I found something that pissed me off. Ollama had shipped llama.cpp's code in their binaries for over a year without including the MIT license notice. That license has basically one rule: include the copyright notice. And they just didn't do it.
Someone opened an issue in early 2024. It sat for 400 days without a reply from the maintainers. Not a single response. Four hundred days. That's not an oversight. You don't miss something for 400 days unless you're actively ignoring it.
A separate issue, opened in April 2024, specifically asked for llama.cpp attribution in Ollama's README. A community member submitted a pull request adding the credit line within hours. After enough external pressure, Ollama's co-founder finally added a single line at the bottom of the README: "llama.cpp project founded by Georgi Gerganov." That was it. The bare minimum.
The team's response to that PR told me everything. They argued they spend lots of time fixing and patching things for Ollama users, and that they planned to transition to their own engine anyway. Translation: we don't want to credit llama.cpp because it makes us look like we depend on it.
When They Actually Forked, It Got Worse
Mid-2025, Ollama finally followed through. They forked away from llama.cpp, built a custom backend on top of ggml. Official reason was "stability." llama.cpp moves fast and occasionally breaks things; Ollama's enterprise customers needed reliability. Fair enough on paper.
First time I hit a crash after that fork: I was trying to run a vision model and it hard-failed with a GGML assertion error. Spent an hour debugging before checking upstream, where the exact same model ran perfectly in raw llama.cpp. Ollama had reintroduced bugs the llama.cpp team had fixed years ago.
Georgi Gerganov himself spotted it. He pointed out that Ollama had forked GGML and made bad changes. More bug reports followed. Broken structured output. Vision models failing. Missing tensor type support for newer models. When GPT-OSS 20B launched, it worked fine in llama.cpp and failed in Ollama, because Ollama's backend didn't support the tensor types it needed.
The irony was so thick I could've served it at brunch. They'd spent years downplaying their dependence on llama.cpp, and the moment they tried to go it alone, they made a worse version of it.
The Numbers Were Real, Just Smaller Than Advertised
I'm not usually a benchmark person. But after noticing Ollama felt sluggish compared to screenshots people were posting from llama.cpp, curiosity got me. So I went looking for someone who had measured it properly.
The gap is real, and it is not the 2x you see quoted around. The one comparison I found that publishes its harness and raw data ran Qwen2.5-Coder-7B at Q4 over a 12-prompt suite: llama.cpp came out about 10% ahead on an RTX 5060 Ti (77.0 against 69.1 tokens per second) and about 14% ahead on an M3 Max (53.5 against 46.2). Anything claiming a doubling without naming hardware, quantization and flags is reporting a misconfiguration, not a benchmark.
Single digits to low double digits is roughly what you'd expect, because it's the same math underneath. What you're paying for is the wrapper: a Go server, its scheduling, and defaults that don't match the flags you'd pass llama.cpp yourself. The genuinely ugly gaps come from somewhere else, which is the vendored engine drifting behind upstream. There's an open issue where Ollama holds at roughly 34 t/s on AMD via Vulkan while a current llama.cpp build does 52-56, entirely because the vendored copy predates two merged patches. That's version lag, and version lag is what a fork costs you.
The Modelfile Situation Made Me Want to Scream
Few months in, I wanted to change the system prompt and temperature on a model I used regularly. Figured there'd be a config file somewhere, maybe a startup flag. Nope.
In Ollama, changing one parameter means:
- Run
ollama show --modelfileto export the current config - Edit the Modelfile
- Run
ollama createto build a new model entry
The problem with step 3? It forces a full rebuild: reconvert and rehash the weights, twenty-odd seconds, plus a transient temp copy if you're importing from a GGUF file. Ollama dedupes the blobs afterward, so you don't end up permanently storing the model twice, but you're re-running the whole import pipeline to change a temperature setting. That's like having to pull your car's engine apart because you wanted to adjust the seat position.
In llama.cpp, I just pass --temp 0.7 as a command-line flag. Done. Nothing gets rebuilt, and there's no proprietary config format to learn.
And the chat template situation was even worse. GGUF files already have the model's chat template embedded in them. That's one of the design goals of the GGUF format: self-contained model files with all the metadata you need. But Ollama doesn't just read the embedded template. It maintains a hardcoded list of templates it recognizes. If your model's template isn't on that list (which happens all the time with newer or less popular models), Ollama silently falls back to a bare prompt format that breaks the model's instruction following. You then have to manually extract the Jinja template from the GGUF, translate it into Go template syntax (different language), and write it into a Modelfile.
Meanwhile, llama.cpp just reads the embedded template and uses it. Same for LM Studio. Only Ollama turns it into a whole production.
The Names Were a Lie (Accidentally or Otherwise)
When DeepSeek R1 dropped in January 2025, I ran ollama run deepseek-r1 and started chatting. It was fine, I guess. Underwhelming for all the hype.
Turns out I wasn't running DeepSeek R1 at all. I was running a distilled 7-billion-parameter version based on Qwen. The real R1 is a 671-billion-parameter monster that needs a datacenter. Hugging Face listed the distilled versions with the "Distill" prefix. Everyone listed them correctly, actually. Except Ollama, who dropped the "Distill" from their library and CLI.
This wasn't minor. People flooded social media claiming they ran "DeepSeek R1 on their laptop" and then wondering why it was so mediocre. It created real confusion, and probably pissed off DeepSeek's team.
GitHub issues were opened asking to fix the naming. Both closed as duplicates. No fix. As of today, ollama run deepseek-r1 still pulls a tiny distilled model. Ollama knew the difference. They chose to obscure it. My guess: "DeepSeek-R1" sounds more impressive than "DeepSeek-R1-Distill-Qwen-32B" and drives more downloads.
The Desktop App Debacle
In July 2025, Ollama announced a desktop GUI app. Finally, I thought. A proper UI. Then I looked into it.
The app was developed in a private repository. No license. No public source code. This from a project that had spent years building its reputation on being open-source. Community members immediately flagged concerns about AGPL-licensed dependencies bundled without proper compliance. The website placed the download button right next to a GitHub link, making it look like you were getting the MIT-licensed open-source tool when you were actually getting a closed-source application.
The code did get open-sourced eventually, months later, after enough people complained. But the rollout told me what I needed to know about where the project's instincts lie.
The Bottleneck Nobody Talks About
I started noticing a pattern: a cool new model drops on Hugging Face. Within hours, people like Unsloth and Bartowski have uploaded GGUF quantizations. With llama.cpp, I can run it immediately:
llama-server -hf unsloth/Qwen3.5-35B-A3B-GGUF:Q4_K_M
With Ollama, I wait. Someone at Ollama has to package the model for their registry, choose which quantizations to offer (typically just Q4_K_M and Q8_0, skipping Q5, Q6, and all the IQ quants), convert the chat template to Go format, and push it. Until then, that model doesn't exist in Ollama's world unless I manually do the Modelfile dance.
The quantization limitation alone is frustrating. Ollama only supports creating Q4_K_S, Q4_K_M, Q8_0, F16, and F32 quantizations. If you need Q5_K_M, Q6_K, or practically any IQ format (things llama.cpp has supported for years), you get told to "use a different tool." For a project that sells itself as the easy option, that's an odd line to draw.
The Cloud Pivot Felt Like a Betrayal
I got into local LLMs for privacy. The whole point was my prompts never left my machine.
Late 2025, Ollama introduced cloud-hosted models alongside their local library. Proprietary models like MiniMax appeared in the model list. Selecting them would route your prompts to third-party cloud providers. The disclosure was subtle.
Users asked what happens to their data when it goes through "Ollama Cloud" to an external provider. Ollama's docs say they don't store or log prompt content, but say nothing about what the third-party provider does. For models hosted on Alibaba Cloud, there's no zero-data-retention guarantee at all.
Then CVE-2025-51471 dropped. A token exfiltration vulnerability where a malicious registry server could trick Ollama into sending its authentication token to an attacker-controlled endpoint during a model pull. The fix took months. For a tool that positions itself on privacy, a vulnerability that leaks credentials during routine operations is... not great.
What I Use Now (And Why It's Not Harder)
The tools that Ollama wraps are directly accessible, and setting them up takes a few minutes. Here's what I run:
My daily driver is just llama.cpp. The llama-server binary gives me an OpenAI-compatible API, the same endpoint format Open WebUI, SillyTavern, and Continue.dev all speak. I can tune context windows, sampling parameters, batch processing, whatever I need. It's faster, more reliable, and I actually understand what it's doing.
I pair it with llama-swap for multi-model orchestration. Load, unload, hot-swap models behind a single API endpoint. Add LiteLLM on top and you get a unified proxy that routes across backends with model aliasing.
For quick demos, Mozilla's llamafile is my go-to. One executable, one model, six operating systems, zero installation. Download, double-click, done. That's the ease of use Ollama promised, except nothing is hidden.
If I want a desktop GUI, I reach for Jan (fully open-source, AGPLv3) or koboldcpp (AGPL, a llama.cpp fork with a built-in web UI). Real FOSS projects. Auditable code.
And yeah, I'll even recommend LM Studio even though it's proprietary. Because the developers act in good faith. They maintain an acknowledgements page crediting llama.cpp. They don't hide what's under the hood. Any GGUF file works out of the box. The difference between a good-faith wrapper and a bad-faith one isn't whether they charge money. It's whether they respect the work they stand on. LM Studio does.
The Pattern I Wish I'd Seen Sooner
Looking back, the trajectory was predictable. It's a playbook by now: build trust on open source, minimize attribution to look independent to investors, create lock-in with hashed filenames and a proprietary registry, ship closed-source components, add cloud monetization. They hit every beat.
Ollama stores downloaded models with hashed filenames in its own directory format. If you've been pulling models for months, you can't just point llama.cpp or LM Studio at those files without extra work. You can bring your own models into Ollama, but taking them out is deliberately painful. Vendor lock-in, open-source costume.
At every decision point where Ollama could have played fair, they chose whatever helped the fundraising narrative.
The Bottom Line
Georgi Gerganov hacked together the first version of llama.cpp in a single evening in March 2023. That one night pretty much kicked off the local LLM movement. Since then, he and hundreds of contributors have made it possible to run increasingly powerful models on consumer hardware. All MIT-licensed, community-driven, no gatekeepers.
Ollama wrapped that work. Raised VC money on it. Refused to credit it for over a year. Forked it badly. Shipped proprietary software. Pivoted to cloud. Take your pick.
The local LLM ecosystem doesn't need Ollama. It needs llama.cpp. Everything else is just packaging. And there's better packaging out there.
My laptop is faster without Ollama on it. My models actually work. And I sleep better knowing what's actually running.