Quick answer

Measured 2026-08-20 on one RTX 5080: llama.cpp b10507 decodes 2-6% faster than Ollama 0.32.1 on dense models and 13-22% faster on the gpt-oss MoE (via different conversions of that model); Ollama prefills 4-7% faster at ~420-token prompts. VRAM matches within 7 MiB on shared GGUF files. Published as CC BY 4.0 CSV with per-run arrays.

By LK Wood IV · Published 2026-08-20 · ~3 min read · St. Louis County, MO

The short version

I ran both servers on one GPU across three models. Same box. Same settings. llama.cpp b10507 wins decode by 2-6% on dense models and by 13-22% on the gpt-oss MoE; Ollama 0.32.1 wins real-prompt prefill by 4-7%. VRAM matches within 7 MiB wherever the two load the same file. The raw engine (llama-bench) runs 367 t/s on the 3B where the full Ollama stack runs 319, so the entire serving stack costs about 13%.

Download: llm-server-compare-2026-08.csv. CC BY 4.0, per-run decode arrays included.

The measured configurations

ModelServerDecodePrefill (long)GPU mem (MiB)Load
Llama 3.2 3BOllama 0.32.1319 t/s17,444 t/s4,3433.2s
Llama 3.2 3Bllama.cpp b10507338 t/s16,809 t/s4,3392.5s
Qwen 2.5 14BOllama 0.32.199.4 t/s5,004 t/s10,9415.6s
Qwen 2.5 14Bllama.cpp b10507101.4 t/s4,697 t/s10,9428.1s
gpt-oss 20BOllama 0.32.1195.8 t/s—14,1528.6s
gpt-oss 20Bllama.cpp (upstream GGUF)222-238 t/s²—13,03719.8s

² Prompt-dependent on this MoE: a fixed prompt held 232-242 t/s (median 238.5); varied prompts were bimodal with a ~176 slow path (median 235.3), and the published 222 median includes a cold first run. Ollama’s fork ran a flat 195.7-196.3 under the same nonce scheme. Load times are cold-start to first ready response: llama-server was faster on the 3B, Ollama faster on both larger models.

The desktop baseline was 1,642 MiB of GPU memory before any model loaded (nvidia-smi reports MiB; the CSV column is vram_total_mib_median). These figures run 2-7% above our RTX 5080 throughput dataset for the same models because that capture used 500-1,650-token prompts; both pages state their regime.

Method

I kept one server resident at a time, with a fresh process per model and ollama stop plus process-tree kills between configurations. Every request carried a unique nonce prefix, llama-server additionally ran with prompt caching disabled, and generation was pinned to temperature 0, seed 42, 256 tokens (64 on the long-prefill pass), context 4096, full offload. Rates are each server’s own counters; a wall-clock recomputation agreed within 2% on every run (one cold-start exception at 3.6%, disclosed above), and llama-bench served as a third instrument on the 3B (pp512 19,589 ± 2,071 t/s, tg256 367.1 ± 0.7 t/s, raw engine loop). The capture scripts are available on request via hello@techfuelhq.com.

The limits, named. One GPU, one box, three models, default server flags, and a short-prompt prefill column retained only to document its own overhead artifact. Token counts for identical text differ slightly between servers because Ollama applies its chat template to /api/generate while llama-server’s /completion takes prompts raw. The decision-layer writeup is the companion article; this page is the citable record.

Frequently asked questions

Which is faster, Ollama or llama.cpp?
In this dataset, llama.cpp b10507 decodes 2-6% faster than Ollama 0.32.1 on dense models (338 vs 319 t/s on Llama 3.2 3B, 101.4 vs 99.4 on Qwen 2.5 14B) and 13-22% faster on the gpt-oss 20B mixture-of-experts, where the two servers load different conversions of the model. Ollama prefills 4-7% faster at real prompt lengths. Both run the same engine lineage, so the gaps are wrapper and version effects rather than different math.
How was the comparison kept fair?
Same machine, same GPU, and for the dense models the same sha256-identical GGUF files loaded by both servers, one server resident at a time. Generation pinned to temperature 0, seed 42, 256 tokens, context 4096, full offload, with a unique prompt nonce per run and prompt caching disabled. Rates come from each server’s own timing counters, and a wall-clock recomputation agreed within 2% on every run but one (the first gpt-oss run on llama-server, 3.6% off — a cold-start artifact).
Why does the gpt-oss row use a different model file?
Ollama’s gpt-oss:20b blob declares the architecture as ‘gptoss’, which upstream llama.cpp rejects with ‘unknown model architecture’. The llama.cpp row therefore uses ggml-org’s upstream MXFP4 GGUF of the same model. That build also allocated 1.1GB less GPU memory than Ollama’s fork at the same context — a real difference between the two conversions, disclosed rather than hidden.
What does the short-prompt prefill column measure?
Mostly per-request overhead, and it is kept in the CSV precisely to demonstrate that. At a ~40-token prompt, prefill tokens-per-second is dominated by fixed request costs; at ~420 tokens the same servers report 1.6-3.2x higher prefill rates (config-dependent) and the between-server gap collapses to single digits. Use prefill_long_tps for throughput claims.

Evidence ledger

Last updated
Methodology
This guide was written and edited by Lowell K. Wood IV in St. Louis County, MO. Specs and prices verified against vendor and project documentation current on the date above. Full editorial standard: methodology.
Update log
  • 2026-08-20 — v1.0.0 first publication. Six measured configurations (3 models x 2 servers), temperature 0, seed 42, per-iteration prompt nonce, fresh server per model, wall-clock cross-check within 2% on all runs but one cold start, llama-bench as a third instrument on the 3B. gpt-oss on llama.cpp uses the upstream GGUF because Ollama’s blob carries its own ‘gptoss’ architecture tag.
Corrections
Spotted an error or a stale number? Email hello@techfuelhq.com. Confirmed corrections are added to the update log above.

About the author

Written by Lowell K. Wood IV, who builds and runs TechFuelHQ from St. Louis, Missouri.