Two captures on one RTX 5080. 2026-08-20: llama.cpp b10507 decodes 2-6% faster than Ollama 0.32.1 on dense models. 2026-08-26: Ollama 0.32.15 decodes 5-11% faster than LM Studio 0.4.21 (351 vs 316 t/s on Llama 3.2 3B), while LM Studio prefills faster and its upstream gpt-oss conversion decodes 14% faster. CC BY 4.0 CSVs.
- Dataset ID
techfuelhq-llm-server-compare- Version
- 1.1.0
- Format
- CSV
- Published
- Page updated
Provenance
- Aug 20, 2026Published
- Nov 18, 2026Last substantive update
The short version
Versions tested: Ollama 0.32.1 and llama.cpp b10507 on August 20, 2026; Ollama 0.32.15 (its vendored llama.cpp at commit 9d77fa172) and LM Studio 0.4.21 (llama.cpp runtime 2.29.1, CUDA 12) on August 26, 2026, all on one RTX 5080. These figures describe those builds. A newer release of any of them can decode at a different speed.
This page holds two captures on the same retail RTX 5080. On 2026-08-20 I ran Ollama against bare llama.cpp; on 2026-08-26 I ran Ollama against LM Studio. The two dense models used identical GGUF files; gpt-oss needed different conversions. The two captures used different Ollama versions six days apart, so llama.cpp and LM Studio were never measured against each other, and the two results cannot be chained. Across both captures the pattern holds: all three tools run llama.cpp code underneath (Ollama ships its own vendored copy and its own gpt-oss conversion), decode differences on dense models stay inside 11%, and the settings each app ships with move throughput far more than the app choice does.
Downloads: llm-server-compare-2026-08.csv (Ollama vs llama.cpp) and llm-server-compare-2026-08-26-lmstudio.csv (Ollama vs LM Studio, both conditions). CC BY 4.0, per-run decode arrays included. If you use the second file to compare startup, see where each timer starts.
Which server decoded the same model faster?
Each row compares the same dense model within one August 2026 capture at 4,096 context. Ollama is the denominator. Rates are median output tokens per second.
| Capture | Model | Ollama | Other server | Other rate | Difference |
|---|---|---|---|---|---|
| Aug 20 | Llama 3.2 3B | 318.51 | llama.cpp b10507 | 338.22 | +6.2% |
| Aug 26 | Llama 3.2 3B | 350.52 | LM Studio 0.4.21 | 316.00 | -9.8% |
| Aug 20 | Qwen 2.5 14B | 99.43 | llama.cpp b10507 | 101.38 | +2.0% |
| Aug 26 | Qwen 2.5 14B | 102.21 | LM Studio 0.4.21 | 97.63 | -4.5% |
The August 20 Ollama version was 0.32.1; August 26 used 0.32.15. Do not combine different capture dates into an upgrade result. The gpt-oss comparison is excluded because the August 20 upstream GGUF differed.
Reproduce this comparison
To recalculate the published table offline, save the Python script as reproduce.py in an empty folder. Inside that folder, create any subfolders shown below and save each input at its displayed path.
From that folder, run this command with Python 3.10 or later. It prints the rows and input hashes; it does not run the hardware or start a server.
python reproduce.py --data-root . --product llm-server-compareCompare the output with this product’s entry in the published JSON and input hashes.
To collect a new measurement: Match the complete GGUF hash, runtime and backend versions, offload, context, prompt set, seed and cache policy. Retain a wall-clock check beside engine counters.
Capture 2: Ollama 0.32.15 vs LM Studio 0.4.21 (2026-08-26)
Settings normalized on both sides: context 4096, full GPU offload, temperature 0, seed 42, one serving slot, per-run prompt nonces throughout. Decode figures are medians of five runs; long-prefill figures, measured at ~450 prompt tokens, are medians of three with a warmup discarded. The decision-layer writeup is the LM Studio comparison article.
| Model | App | Decode | Prefill (long) | GPU mem (MiB) | Model load only |
|---|---|---|---|---|---|
| Llama 3.2 3B | Ollama 0.32.15 | 350.5 t/s | 17,862 t/s | 3,760 | 2.3s |
| Llama 3.2 3B | LM Studio 0.4.21 | 316.0 t/s | 20,337 t/s | 3,808 | 2.8s |
| Qwen 2.5 14B | Ollama 0.32.15 | 102.2 t/s | 5,038 t/s | 10,356 | 5.5s |
| Qwen 2.5 14B | LM Studio 0.4.21 | 97.6 t/s | 5,567 t/s | 10,405 | 7.7s |
| gpt-oss 20B | Ollama 0.32.15 (own fork) | 213.7 t/s | 9,187 t/s | 13,570 | 8.1s |
| gpt-oss 20B | LM Studio 0.4.21 (upstream MXFP4) | 242.8 t/s | 9,618 t/s | 12,533 | 8.4s |
Where the August 26 timers start
In the August 26 CSV, load_s is inside cold_first_request_s for Ollama. For LM Studio, it times the CLI load before the first request starts. On Llama 3.2 3B, LM Studio spent 2.76 s loading and 0.07 s on the first request. The two rounded readings add to about 2.83 s. Ollama’s 2.38 s request already includes 2.27 s of loading.
Each startup row is one completed nonstreaming request from an unloaded model and warm filesystem cache. The separate ttft_ms_median is a resident server measurement: five requests in LM Studio, unmeasured in Ollama. Neither is client-visible time to first token. See the worked comparison for both dense models and the offline replay.
Timing export 1.0.0. Download the timing CSV and JSON. The correction record links the exact revision of both files and identifies the source measurements. The change register lists other versioned corrections.
At stock as-installed settings the dense rows barely move, but two defaults bite: LM Studio’s automatic GPU offload held gpt-oss to 186.3 t/s, 23% below its own tuned figure — and a single-variable re-run (stock config, only the offload slider moved to max) recovered the whole gap at 244.9 t/s.
Ollama’s app-level context slider set to 256k (clamped to this model’s 131,072 maximum) dropped Llama 3.2 3B from 351 to 102 t/s with the KV cache spilled to CPU. Both are recorded as their own rows in the CSV and raw JSON, except the 256k-slider run, which is in the raw JSON only: /data/vram-cliff-raw/ollama-256k-slider-3b.json.
Idle app overhead: the Ollama service tree held 78-132 MB of RAM; LM Studio’s GUI plus server held 709-928 MB across quiet-window observations (one 3.1 GB transitional reading mid-unload excluded, kept in the raw JSON). Ollama’s engine is its vendored llama.cpp at commit 9d77fa172; LM Studio’s is its packaged llama.cpp runtime 2.29.1 (CUDA 12) — and both run the model in a llama-server process.
Ollama alone gained 3-10% decode over its own 08-20 numbers here (319→351 on the 3B; 195.8→213.5 on gpt-oss, defaults condition) across six days and fourteen patch-version increments — with the caveat that the 08-20 capture predates this page’s quiet-window discipline, so part of that spread may be ambient. Either way, single-digit decode gaps between these tools do not survive a version bump.
Capture 1: Ollama 0.32.1 vs llama.cpp b10507 (2026-08-20)
| Model | Server | Decode | Prefill (long) | GPU mem (MiB) | Load |
|---|---|---|---|---|---|
| Llama 3.2 3B | Ollama 0.32.1 | 319 t/s | 17,444 t/s | 4,343 | 3.2s |
| Llama 3.2 3B | llama.cpp b10507 | 338 t/s | 16,809 t/s | 4,339 | 2.5s |
| Qwen 2.5 14B | Ollama 0.32.1 | 99.4 t/s | 5,004 t/s | 10,941 | 5.6s |
| Qwen 2.5 14B | llama.cpp b10507 | 101.4 t/s | 4,697 t/s | 10,942 | 8.1s |
| gpt-oss 20B | Ollama 0.32.1 | 195.8 t/s | — | 14,152 | 8.6s |
| gpt-oss 20B | llama.cpp (upstream GGUF) | 222-238 t/s² | — | 13,037 | 19.8s |
² Prompt-dependent on this MoE: a fixed prompt held 232-242 t/s (median 238.5); varied prompts were bimodal with a ~176 slow path (median 235.3), and the published 222 median includes a cold first run. Ollama’s fork ran a flat 195.7-196.3 under the same nonce scheme. Load times are cold-start to first ready response: llama-server was faster on the 3B, Ollama faster on both larger models.
The desktop baseline was 1,642 MiB of GPU memory before any model loaded (nvidia-smi reports MiB; the CSV column is vram_total_mib_median). These figures run 2-7% above our RTX 5080 throughput dataset for the same models because that capture used 500-1,650-token prompts; both pages state their regime.
Method
For the August 20 Ollama/llama.cpp capture I kept one server resident at a time, with a fresh process per model and ollama stop
plus process-tree kills between configurations. Every request carried a unique nonce prefix, llama-server
additionally ran with prompt caching disabled, and generation was pinned to temperature
0, seed 42, 256 tokens (64 on the long-prefill pass), context 4096, full offload.
Rates are each server’s own counters; a wall-clock recomputation agreed within 2% on every run (one cold-start exception at 3.6%, disclosed above), and llama-bench served as a third instrument on the 3B (pp512 19,589 ± 2,071 t/s, tg256 367.1 ± 0.7 t/s, raw engine loop). The capture scripts are available on request via contact@techfuelhq.com.
The limits, named. One GPU, one box, three models per capture, and a short-prompt prefill column retained only to document its own overhead artifact. Token counts for identical text differ slightly between tools because each applies its own chat template handling. The decision-layer writeups are Ollama vs llama.cpp and Ollama vs LM Studio; this page is the citable record for both.
Frequently asked questions
Which is faster, Ollama or LM Studio?
Which is faster, Ollama or llama.cpp?
How was the comparison kept fair?
Why does the gpt-oss row use a different model file?
What does the short-prompt prefill column measure?
Updates and corrections
- Last updated
- Methodology
- See our methodology for research and review standards.
- Update log
- 2026-11-18 — Added the versions tested near the top, said plainly that the two captures cannot be chained, and named the page as two pairwise comparisons. No measurement changed.
- 2026-11-05 — Defined where each app’s August 26 load and first-request timers start, with a timing download. The original measurements are unchanged.
- 2026-11-04 — Added a dense-model decode comparison within each August capture. The original files and capture dates are unchanged.
- 2026-08-26 — v1.1.0 added the Ollama 0.32.15 vs LM Studio 0.4.21 capture: three models, both apps, stock and normalized settings, with RAM, first-request and load times.
- 2026-08-20 — v1.0.0 first publication: Ollama 0.32.1 vs llama.cpp b10507 on three models, temperature 0, seed 42, a fresh server per model.
- Corrections
- Spotted an error? Email a correction with the affected line and what you found. The draft includes this page’s address. Verified changes are recorded here. Contact details.