RTX 5080 Local LLM Throughput: Measured Tokens/Second Dataset

By LK Wood IV · Published 2026-08-12 · Updated 2026-08-24 · ~4 min read · St. Louis County, MO

The short version

This is a measured dataset, not a spec-sheet estimate: real decode and prefill tokens/second for local LLMs on the same retail RTX 5080 that runs this site’s open bench dataset. Four models spanning 3B to 20.9B parameters, two context windows, two prompt sizes, three repetitions per cell, ollama’s native counters, temperature 0.

Per-GPU pages exist elsewhere for the RTX 5090, 4090, and 5060 — none for the 5080 when this dataset was first captured (checked 2026-08-12). This page is the 5080 entry, from a card someone actually owns.

The numbers (capture 2026-08-12)

Paired values are the ~500-token / ~1,650-token prompt cells. Every value is the median of 3 runs.

*The starred Llama VRAM reading includes gpt-oss:20b still resident from the previous bench — see the FAQ. The 5,495 MiB row is the model alone.

Three readings worth naming:

• Decode speed is flat across context at these prompt sizes. 4k to 16k num_ctx moves gpt-oss:20b by less than 2 tok/s. Context capacity costs VRAM (Qwen: 10.6 → 12.8 GiB), not decode speed, until the prompt itself gets long.
• The 20B model is the surprise. gpt-oss:20b’s MXFP4 quantization decodes twice as fast as the dense 14B at Q4_K_M and matches the dense 7B — architecture and quantization beat parameter count.
• Prefill is not the bottleneck at these sizes. Even the slowest model ingests ~5,000 tok/s; a 1,650-token prompt costs a third of a second.

The standout result has its own explainer now: why the 21B MoE doubles the dense 14B — the bandwidth arithmetic, shown in full against these rows. And the same card’s ollama rates now have a bare-llama.cpp counterpart: Ollama vs llama.cpp, measured head-to-head, which also explains why its short-prompt numbers run 2-7% above this page’s long-prompt rows.

Coding models (added in 1.1.0, capture 2026-08-26)

Four coding-focused models joined the matrix for the tested coding-picks article, captured with the same method and pinned settings in an exclusive GPU window (no other model server or GPU process live, ~20GB system RAM free, baseline VRAM recorded in the raw JSON):

Paired values are the ~500 / ~1,650-token prompt cells, medians of 3 fresh-prefill runs each, same pins as every other row (temperature 0, seed 42, 256-token output, per-rep nonce).

² Same-model re-capture for continuity: 3-5% under its 08-12 rows on a card six hours into continuous bench work — see the 1.1.0 changelog note. ³ Partial offload (the 19GB file exceeds the card); see the note below. ⁴ devstral’s prompt cells landed at 1,694 / 2,597 actual tokens (its tokenizer expands the filler text); 4k context only — see the note below.

Notes that belong next to those rows:

• qwen3-coder:30b is the matrix’s first partial-offload row. Its 19GB Q4_K_M exceeds the card, so ollama ran it 74% GPU-resident (measured via /api/ps at 8k context; the split itself moves with load-time state); the decode rate is real but carries a dependency the resident rows do not have: free system RAM. The same model on the same card decoded at 83 tok/s with ~20GB free and under 1 tok/s while the box was paging at 1.2GB free — measured during the eval runs, raw logs preserved. Treat any offloaded-model tok/s figure (ours included) as conditional on RAM headroom.
• devstral:24b appears at 4096 context only. At 8k context its 14GB file plus KV cache spilled and decode collapsed below 1 tok/s on this box; the honest row is the configuration that actually fits. The 16k cells were not captured for it.

The coding eval (v1, run 2026-08-26)

The same five models (the four above plus the already-tabled gpt-oss:20b) went through techfuelhq-coding-eval-v1: eight tasks — bugfix, implement-from-spec, test-writing against a planted mutant, behavior-preserving refactor, legacy-code explanation, SQL row-compared on seeded SQLite, regex against fullmatch lists, and two-turn agentic tool-calling — two repetitions each at temperature 0, seed 42, scored only by executed checkers. No judge model anywhere. The exact prompts are published as coding-eval-tasks-v1.json and every repetition’s outcome is in rtx-5080-coding-eval-v1.csv.

A task counts as stable only when both repetitions pass; “1 of 2” is reported as flaky, never rounded up. Interpretation, picks, and the failure stories live in the article; this page is the canonical home of the numbers.

Method, exactly

The capture script (in the site repository) drives ollama’s /api/generate and records its native eval_count/eval_duration and prompt_eval_count/prompt_eval_duration — the timers the runtime itself keeps — never wall-clock division. Each repetition prepends a unique nonce so every run pays a full prefill; without it, warm repetitions hit ollama’s prompt cache and report ~60,000 tok/s cache-hit rates that measure nothing (we caught exactly that in the first capture attempt and rejected those rows). Generation is pinned: temperature 0, seed 42, 256-token target output. A cell enters the CSV only when all 3 repetitions pass mechanical checks (enough tokens generated for a stable rate; a genuine full-prompt prefill observed).

Bench environment, recorded per row: NVIDIA GeForce RTX 5080 (16 GiB), driver 610.88, ollama 0.32.1, Windows 11, Ryzen 7 7800X3D. Model identity is pinned by digest in the CSV, because a tag like qwen2.5:14b can silently move.

Limitations, honestly

One retail card, one driver, one runtime. These are this unit’s numbers — silicon lottery, thermals, and runtime version all move results; that is why each row carries its digest, driver, and date instead of claiming to be a universal constant. The VRAM column is whole-GPU nvidia-smi at measurement time (see FAQ). llama.cpp direct, vLLM, and image-generation throughput are planned lanes, not covered yet.

Use the data (CC BY 4.0)

Download the CSV: rtx-5080-llm-throughput.csv. License: Creative Commons Attribution 4.0. Attribute as “TechFuelHQ RTX 5080 LLM Throughput Dataset” linking to this page. Sizing a build for local AI? Pair these measured rows with the LLM VRAM calculator and the LLM speed calculator — the estimators tell you what should fit; this table shows what one real card actually did.

The dataset is also mirrored on Kaggle and Hugging Face for researchers who discover data through those catalogs. The page here is canonical for corrections, limitations, and the current method.

The full matrix, rendered from the raw CSV

Every measured cell, straight from the downloadable file at build time.

Rendered at build time from the raw CSV — the table and the download can never disagree.

Change log

• 1.1.0 (2026-08-26) — coding models join the matrix: qwen2.5-coder 7B and 14B, devstral 24B (4k context only — at 8k its spill collapsed decode below 1 tok/s on this box), and qwen3-coder:30b, the dataset’s first partial-offload row. Same method, same pins, captured in an exclusive GPU window after a night of multi-session contention taught us to verify the window at the process level (the full incident trail lives in the repo’s receipts file). Two honest notes attached to this capture: offloaded-model decode is conditional on load-time memory state (we measured the same 30B at 0.7, 62, and 83 tok/s under three host states), and this capture’s gpt-oss rows run ~5% below the 08-12 rows on a card six hours into continuous bench work — both captures are real, both carry their dates. Also new: the coding-eval grid (techfuelhq-coding-eval-v1), its task-prompt JSON, and the per-rep results CSV as second and third distributions.
• 1.0.2 (2026-08-14) — VRAM correction, no re-measurement. The prose and FAQ were converting the CSV’s vram_resident_mib column by 1000 and calling the result GiB; every narrative footprint was ~2.4% high. Recomputed at 1024 (gpt-oss:20b is 13.7–14.0 GiB resident, not 14.0–14.4). The measured table was already correct.
• 1.0.1 (2026-08-12) — added Qwen 2.5 7B (Q4_K_M): 176.6-180.0 tok/s decode, ~10k tok/s prefill, 6.0-6.9 GiB resident measured alone — four models now span 3B-20.9B.
• 1.0.0 (2026-08-12) — first capture: gpt-oss:20b, Qwen 2.5 14B, Llama 3.2 3B × 4k/16k context × ~500/~1,650-token prompts, 3 reps per cell, ollama 0.32.1, driver 610.88. First-capture defect worth recording: warm-run prompt-cache contamination was detected and eliminated by per-rep nonces before any row entered the dataset.