Ollama vs llama.cpp on an RTX 5080: Measured (2026)

The argument shows up in every local-AI thread: Ollama is “just a wrapper” and real performance lives in bare llama.cpp. Nobody arguing it ever posts numbers from the same machine. So I ran both servers on the same RTX 5080, loading the same GGUF bytes, with generation pinned to temperature 0, seed 42, and fresh prompts every run.

The answer surprised me, because I went in expecting to confirm the forum consensus and instead measured something smaller, stranger, and more useful. The wrapper costs almost nothing, and each side wins one event.

Decode goes to llama.cpp. Prefill goes to Ollama. Nobody wins big.

The measured table

Ollama 0.32.1 versus llama.cpp b10507 (llama-server), full GPU offload, 4096 context, 256-token generations, decode medians of five runs. Prefill measured at real prompt length (~420 tokens), median of three fresh runs with a discarded warmup. The short version of why the prompt length matters is below.

² Upstream GGUF (the Ollama blob refuses to load; see the portability catch below). gpt-oss is a mixture-of-experts model and its decode rate on upstream llama.cpp is prompt-dependent: a fixed prompt held a tight 232-242 t/s (median 238.5), while varied prompts were bimodal, mostly ~230-243 with a ~176 slow path — honest range 175-243, varied-prompt median 235.3. The published 222 median also carries a cold first run at 186. Ollama’s fork ran a flat 195.7-196.3 under the same nonce scheme. The gpt-oss prefill cells are dashed because the long-prompt pass was not run on this model.

Desktop baseline was 1,642 MiB of GPU memory; subtract it for the models’ own footprints. Wall-clock recomputation agreed with each server’s self-reported rates within 2% on every run but one — the first gpt-oss run on llama-server landed 3.6% off, a cold-start artifact. Wherever both servers loaded the same file, VRAM matched within 7 MiB. Same weights, same memory. The gpt-oss row is the exception, and it is carrying two different conversions of the model, not a wrapper difference. These figures also run 2-7% above our earlier RTX 5080 throughput dataset for the same models, because that capture used 500-1,650-token prompts where this one uses short ones; both pages state their regime.

llama.cpp decodes 2-6% faster on the dense models, more on the MoE below. Ollama prefills 4-7% faster at real prompt sizes. Neither gap changes what a model feels like to use, and neither survives a driver update, an engine release, or a quantization change as the deciding factor in a purchase or a migration.

The overhead ladder

One more instrument makes the picture honest. llama.cpp ships llama-bench, which times the raw engine loop with no server in front of it. On Llama 3.2 3B:

Each layer costs single-digit percent, and that is the entire “wrapper tax”: 13% from the bare engine loop to the most convenient tool in the stack, most of it spent on serving machinery that any usable setup needs anyway, whichever side of the argument you started on. Forum threads promise dramatic slowdowns. None reproduced here.

That is the whole controversy, measured.

How my first measurement lied to me

The first pass of this bench used a short, ~40-token prompt, and it showed Ollama’s prefill rates landing 32-55% above llama-server’s — a gap that turned out to be an artifact, because at tiny prompt sizes “prefill tokens per second” is dominated by fixed per-request overhead rather than compute, which means it measures which server shuffles a request faster rather than which one ingests your context faster, and re-measured at ~420 prompt tokens, llama-server’s prefill jumped from ~5,300 to ~16,800 t/s on the 3B while the gap collapsed to single digits. The flags people credit for prefill speed did nothing here either: flash attention on or off, micro-batch 512 or 2048, all within noise at real prompt length on this card. Flags were not the story. Prompt length was.

So treat one-line-prompt prefill numbers as request plumbing, not throughput.

The portability catch

Ollama stores weights as standard GGUF blobs, and ollama show <model> --modelfile prints the path. Point llama-server at it and both tools share one copy of the weights. That worked perfectly for Llama 3.2 3B and Qwen 2.5 14B.

It failed for gpt-oss:20b, and the failure is worth knowing about. llama.cpp refused the blob with unknown model architecture: 'gptoss', and the server died on the spot. Ollama’s conversion tags the architecture with its own name, while upstream llama.cpp expects gpt-oss — so the one model Ollama co-launched is the one whose Ollama download you cannot carry to the upstream engine, so its download does not travel. The fix is downloading the upstream GGUF separately, at the cost of a second ~12GB copy on disk. The upstream copy then decoded 13-22% faster than Ollama’s fork on most prompts and allocated 1.1GB less GPU memory at the same context, so the annoyance bought something real, about 1.09 GiB of it.

Which one should you run?

Run Ollama when you want model management to be someone else’s problem: pulls, updates, memory eviction, an OpenAI-style API on 11434, and every tutorial on the internet assuming you have it. Our RTX 5080 throughput dataset and the RTX 5060 local LLM guide both ride on it for exactly that reason.

Run llama-server when you want the newest engine the day it ships, exact control over batching, context, and sampling, or the last few percent of decode speed. For these models the engines share llama.cpp lineage; Ollama’s README lists llama.cpp among its supported backends, and the gpt-oss arch tag shows it also ships its own engine paths. You are choosing how much of the engine’s surface to touch. Throughput is the wrong reason to switch in either direction — 2-6% is less than the gain from one quantization step or one driver update.

Method and limits

Both servers ran the same sha256-identical GGUF blobs from Ollama’s own store (except the gpt-oss case above), one server resident at a time, fresh process per model, with ollama stop and process-tree kills between runs. Every request used a unique prompt nonce; llama-server additionally got cache_prompt: false. Generation pinned to temperature 0, seed 42, 256 tokens (64 for the long-prefill pass), context 4096 on both. Rates come from each server’s own timing counters, cross-checked by wall-clock recomputation (within 2% throughout) and by llama-bench as a third instrument.

The limits, named. One GPU, one box, three models, servers on default flags. Token counts for identical prompt text differed slightly between servers (433 vs 407 on the long prompt) because Ollama applies its chat template to /api/generate while llama-server’s /completion takes the prompt raw — rates are per-token, so the comparison holds, but byte-identical inputs produce slightly different token streams. Ollama’s engine is a vendored llama.cpp build, so some of the decode gap is simply engine version lag, and it will move with every release on both sides. Date everything; this page says 2026-08-20.