Measured on my RTX 5080, which reports 16,303 MiB: gpt-oss:20b held 14,067 MiB and decoded at 190 tokens per second on Ollama 0.32.1 (2026-08-12). Qwen3-Coder 30B does not fit whole and ran at 62. Llama 3.2 3B fell from 351 to 103 tokens per second when a 131,072-token context spilled it. Each pin has its date and versions.
- Dataset ID
techfuelhq-rtx-5080-16gb-vram-cliff- Version
- 1.0.0
- Format
- CSV, JSON
- Published
- Page updated
Contents
At a glance
- Class
- Data
Provenance
- Nov 17, 2026Published
The map
Every pin is a local language model I ran on my own retail RTX 5080, placed by the memory in use on the whole card and the decode speed that came with it. Nothing here is estimated. For a card I do not own or a setup I have not run, the LLM VRAM calculator and the LLM speed calculator estimate, and the VRAM tiers guide covers other card sizes. This page only reports.
- Measured. Copied from a capture file, shown with its capture date.
- Computed. Arithmetic on measured values, shown with its formula.
Showing 36 of 36 measured configurations
The map
- A measured configuration
- Ran partly off the card
- Reading includes another model
- Same setup, context raised
Every pin is one row from a capture file. Nothing between two pins was measured, and nothing here is an estimate.
Readout
Click or tap near a pin, or pick a row in the list, and its measured record appears here.
- Configurations
- 36
- Rows pinned on the map
- Captured
- 2026-08-12
to 2026-08-26 - First and last capture date
- Model tags
- 8
- Quantizations
- 2
- Context lengths
- 4
- Runtime versions
- 4
For a card or a setup that is not on this map, the LLM VRAM calculator estimates. This page does not.
Every measured configuration
- llama3.2:3b
Q4_K_M · 4,096 ctx
Ollama 0.32.15 defaults
3,760 MiB
351.22 tok/s
Full measured record
llama3.2:3b Q4_K_M · 4,096 context · Ollama 0.32.15 · defaults
Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 3,760 MiB
- Measured
- Decode
- 351.22 tok/s
- Measured
- Left on the card
- 12,543 MiB
- Computed: 16,303 − 3,760
- Share of the card
- 23.1%
- Computed: 3,760 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- ollama 0.32.15
- Engine
- vendored llama.cpp 9d77fa172
- Condition
- defaults
- Model name in the file
- llama3.2:3b
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096
- Runs
- 5
- Decode, median (server counters)
- 351.22 tok/s
- Decode, median (wall clock)
- 345.19 tok/s
- Each run
- 351.15 · 350.9 · 351.86 · 351.32 · 351.22 tok/s
- Prefill, long prompt, median
- 17802.7 tok/s at 434 prompt tokens
- Time to first token
- Empty in this file
- Whole-card memory, median
- 3760 MiB
- Whole-card memory before the load
- 1053 MiB
- Added by loading the model
- 2,707 MiB Computed: 3,760 − 1,053
- Load
- 2.67 s
- Cold first request
- 2.79 s
- App RAM while loaded
- 1234.0 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 3.67 GiB Computed: 3,760 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - llama3.2:3b
Q4_K_M · 4,096 ctx
Ollama 0.32.15 normalized
3,760 MiB
350.52 tok/s
Full measured record
llama3.2:3b Q4_K_M · 4,096 context · Ollama 0.32.15 · normalized
Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 3,760 MiB
- Measured
- Decode
- 350.52 tok/s
- Measured
- Left on the card
- 12,543 MiB
- Computed: 16,303 − 3,760
- Share of the card
- 23.1%
- Computed: 3,760 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- ollama 0.32.15
- Engine
- vendored llama.cpp 9d77fa172
- Condition
- normalized
- Model name in the file
- llama3.2:3b
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096
- Runs
- 5
- Decode, median (server counters)
- 350.52 tok/s
- Decode, median (wall clock)
- 338.11 tok/s
- Each run
- 349.35 · 350.38 · 350.52 · 350.63 · 350.96 tok/s
- Prefill, long prompt, median
- 17862.3 tok/s at 432 prompt tokens
- Time to first token
- Empty in this file
- Whole-card memory, median
- 3760 MiB
- Whole-card memory before the load
- 1057 MiB
- Added by loading the model
- 2,703 MiB Computed: 3,760 − 1,057
- Load
- 2.27 s
- Cold first request
- 2.38 s
- App RAM while loaded
- 1239.6 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 3.67 GiB Computed: 3,760 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - llama3.2:3b
Q4_K_M · 4,096 ctx
LM Studio 0.4.21 normalized
3,808 MiB
316.00 tok/s
Full measured record
llama3.2:3b Q4_K_M · 4,096 context · LM Studio 0.4.21 · normalized
Measured at 3,808 MiB resident on with LM Studio 0.4.21. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 3,808 MiB
- Measured
- Decode
- 316.00 tok/s
- Measured
- Left on the card
- 12,495 MiB
- Computed: 16,303 − 3,808
- Share of the card
- 23.4%
- Computed: 3,808 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- LM Studio 0.4.21
- Engine
- llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
- Condition
- normalized
- Model name in the file
- llama-3.2-3b-q4km
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096
- Runs
- 5
- Decode, median (server counters)
- 316.0 tok/s
- Decode, median (wall clock)
- 309.91 tok/s
- Each run
- 310.55 · 316.0 · 315.86 · 316.23 · 316.6 tok/s
- Prefill, long prompt, median
- 20336.7 tok/s at 442 prompt tokens
- Time to first token, median
- 9.2 ms
- Whole-card memory, median
- 3808 MiB
- Whole-card memory before the load
- 1103 MiB
- Added by loading the model
- 2,705 MiB Computed: 3,808 − 1,103
- Load
- 2.76 s
- Cold first request
- 0.07 s
- App RAM while loaded
- 3469.6 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 3.72 GiB Computed: 3,808 ÷ 1,024
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - llama3.2:3b
Q4_K_M · 8,192 ctx
LM Studio 0.4.21 defaults
4,260 MiB
314.86 tok/s
Full measured record
llama3.2:3b Q4_K_M · 8,192 context · LM Studio 0.4.21 · defaults
Measured at 4,260 MiB resident on with LM Studio 0.4.21. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 4,260 MiB
- Measured
- Decode
- 314.86 tok/s
- Measured
- Left on the card
- 12,043 MiB
- Computed: 16,303 − 4,260
- Share of the card
- 26.1%
- Computed: 4,260 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- LM Studio 0.4.21
- Engine
- llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
- Condition
- defaults
- Model name in the file
- llama-3.2-3b-q4km
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 8192
- Runs
- 5
- Decode, median (server counters)
- 314.86 tok/s
- Decode, median (wall clock)
- 308.68 tok/s
- Each run
- 273.28 · 312.54 · 316.75 · 315.22 · 314.86 tok/s
- Prefill, long prompt, median
- 20341.8 tok/s at 440 prompt tokens
- Time to first token, median
- 9.4 ms
- Whole-card memory, median
- 4260 MiB
- Whole-card memory before the load
- 1122 MiB
- Added by loading the model
- 3,138 MiB Computed: 4,260 − 1,122
- Load
- 2.75 s
- Cold first request
- 0.07 s
- App RAM while loaded
- 3471.9 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 4.16 GiB Computed: 4,260 ÷ 1,024
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - llama3.2:3b
Q4_K_M · 4,096 ctx
llama.cpp b10507
4,339 MiB
338.22 tok/s
Full measured record
llama3.2:3b Q4_K_M · 4,096 context · llama.cpp b10507
Measured at 4,339 MiB resident on with llama.cpp b10507. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 4,339 MiB
- Measured
- Decode
- 338.22 tok/s
- Measured
- Left on the card
- 11,964 MiB
- Computed: 16,303 − 4,339
- Share of the card
- 26.6%
- Computed: 4,339 ÷ 16,303
Full measured row
- Captured
- from the file's footer notes
- Runtime
- llama.cpp b10507
- Model name in the file
- llama3.2:3b
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096 from the file's settings note
- Offload
- Full offload from the file's settings note
- Runs
- 5
- Decode, median (server counters)
- 338.22 tok/s
- Decode, median (wall clock)
- 336.01 tok/s
- Each run
- 339.2 · 338.2 · 335.0 · 338.2 · 339.7 tok/s
- Prefill, short prompt, median
- 5321.0 tok/s
- Prefill, long prompt
- 16809 tok/s at 407 prompt tokens
- Whole-card memory, median
- 4339 MiB
- Load
- 2.5 s
- Desktop baseline at bench start
- 1642 MiB one reading for the whole capture
- Driver
- Not recorded in this file
- Resident in GiB
- 4.24 GiB Computed: 4,339 ÷ 1,024
Source file: llm-server-compare-2026-08.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - llama3.2:3b
Q4_K_M · 4,096 ctx
Ollama 0.32.1
4,343 MiB
318.51 tok/s
Full measured record
llama3.2:3b Q4_K_M · 4,096 context · Ollama 0.32.1
Measured at 4,343 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 4,343 MiB
- Measured
- Decode
- 318.51 tok/s
- Measured
- Left on the card
- 11,960 MiB
- Computed: 16,303 − 4,343
- Share of the card
- 26.6%
- Computed: 4,343 ÷ 16,303
Full measured row
- Captured
- from the file's footer notes
- Runtime
- ollama 0.32.1
- Model name in the file
- llama3.2:3b
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096 from the file's settings note
- Offload
- Full offload from the file's settings note
- Runs
- 5
- Decode, median (server counters)
- 318.51 tok/s
- Decode, median (wall clock)
- 313.25 tok/s
- Each run
- 319.0 · 318.7 · 318.2 · 318.5 · 317.5 tok/s
- Prefill, short prompt, median
- 7038.1 tok/s
- Prefill, long prompt
- 17444 tok/s at 433 prompt tokens
- Whole-card memory, median
- 4343 MiB
- Load
- 3.2 s
- Desktop baseline at bench start
- 1642 MiB one reading for the whole capture
- Driver
- Not recorded in this file
- Resident in GiB
- 4.24 GiB Computed: 4,343 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: llm-server-compare-2026-08.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - llama3.2:3b
Q4_K_M · 16,384 ctx
Ollama 0.32.1
5,495 MiB
312.89 tok/s
Full measured record
llama3.2:3b Q4_K_M · 16,384 context · Ollama 0.32.1
Measured at 5,495 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 5,495 MiB
- Measured
- Decode
- 312.89 tok/s
- Measured
- Left on the card
- 10,808 MiB
- Computed: 16,303 − 5,495
- Share of the card
- 33.7%
- Computed: 5,495 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- llama3.2:3b digest a80c4f17acd5
- Parameters
- 3.2B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 16384
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 471
- Repetitions
- 3
- Decode, median
- 312.89 tok/s
- Prefill, median
- 17885.62 tok/s
- Load, median
- 162.9 ms
- Whole-card memory after the cell
- 5495 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1657
- Repetitions
- 3
- Decode, median
- 298.59 tok/s
- Prefill, median
- 19397.36 tok/s
- Load, median
- 158.0 ms
- Whole-card memory after the cell
- 5501 MiB
- Resident in GiB
- 5.37 GiB Computed: 5,495 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - qwen2.5-coder:7b
Q4_K_M · 4,096 ctx
Ollama 0.32.1
5,671 MiB
173.73 tok/s
Full measured record
qwen2.5-coder:7b Q4_K_M · 4,096 context · Ollama 0.32.1
Measured at 5,671 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 5,671 MiB
- Measured
- Decode
- 173.73 tok/s
- Measured
- Left on the card
- 10,632 MiB
- Computed: 16,303 − 5,671
- Share of the card
- 34.8%
- Computed: 5,671 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen2.5-coder:7b digest dae161e27b0e
- Parameters
- 7.6B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 4096
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 488
- Repetitions
- 3
- Decode, median
- 173.73 tok/s
- Prefill, median
- 9651.9 tok/s
- Load, median
- 159.1 ms
- Whole-card memory after the cell
- 5671 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1368
- Repetitions
- 3
- Decode, median
- 171.73 tok/s
- Prefill, median
- 9506.54 tok/s
- Load, median
- 159.6 ms
- Whole-card memory after the cell
- 5671 MiB
- Resident in GiB
- 5.54 GiB Computed: 5,671 ÷ 1,024
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - qwen2.5:7b
Q4_K_M · 4,096 ctx
Ollama 0.32.1
6,190 MiB
180.02 tok/s
Full measured record
qwen2.5:7b Q4_K_M · 4,096 context · Ollama 0.32.1
Measured at 6,190 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 6,190 MiB
- Measured
- Decode
- 180.02 tok/s
- Measured
- Left on the card
- 10,113 MiB
- Computed: 16,303 − 6,190
- Share of the card
- 38.0%
- Computed: 6,190 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen2.5:7b digest 845dbda0ea48
- Parameters
- 7.6B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 4096
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 488
- Repetitions
- 3
- Decode, median
- 180.02 tok/s
- Prefill, median
- 10152.28 tok/s
- Load, median
- 129.8 ms
- Whole-card memory after the cell
- 6190 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1674
- Repetitions
- 3
- Decode, median
- 176.79 tok/s
- Prefill, median
- 9672.39 tok/s
- Load, median
- 123.4 ms
- Whole-card memory after the cell
- 6190 MiB
- Resident in GiB
- 6.04 GiB Computed: 6,190 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:7b on (UTC), stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokens Computed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - qwen2.5-coder:7b
Q4_K_M · 16,384 ctx
Ollama 0.32.1
6,503 MiB
174.58 tok/s
Full measured record
qwen2.5-coder:7b Q4_K_M · 16,384 context · Ollama 0.32.1
Measured at 6,503 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 6,503 MiB
- Measured
- Decode
- 174.58 tok/s
- Measured
- Left on the card
- 9,800 MiB
- Computed: 16,303 − 6,503
- Share of the card
- 39.9%
- Computed: 6,503 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen2.5-coder:7b digest dae161e27b0e
- Parameters
- 7.6B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 16384
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 488
- Repetitions
- 3
- Decode, median
- 174.58 tok/s
- Prefill, median
- 9471.68 tok/s
- Load, median
- 145.9 ms
- Whole-card memory after the cell
- 6503 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1368
- Repetitions
- 3
- Decode, median
- 171.41 tok/s
- Prefill, median
- 9825.47 tok/s
- Load, median
- 178.3 ms
- Whole-card memory after the cell
- 6515 MiB
- Resident in GiB
- 6.35 GiB Computed: 6,503 ÷ 1,024
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - qwen2.5:7b
Q4_K_M · 16,384 ctx
Ollama 0.32.1
7,022 MiB
179.66 tok/s
Full measured record
qwen2.5:7b Q4_K_M · 16,384 context · Ollama 0.32.1
Measured at 7,022 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 7,022 MiB
- Measured
- Decode
- 179.66 tok/s
- Measured
- Left on the card
- 9,281 MiB
- Computed: 16,303 − 7,022
- Share of the card
- 43.1%
- Computed: 7,022 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen2.5:7b digest 845dbda0ea48
- Parameters
- 7.6B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 16384
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 488
- Repetitions
- 3
- Decode, median
- 179.66 tok/s
- Prefill, median
- 10133.1 tok/s
- Load, median
- 127.9 ms
- Whole-card memory after the cell
- 7022 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1674
- Repetitions
- 3
- Decode, median
- 176.61 tok/s
- Prefill, median
- 9984.37 tok/s
- Load, median
- 127.7 ms
- Whole-card memory after the cell
- 7034 MiB
- Resident in GiB
- 6.86 GiB Computed: 7,022 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:7b on (UTC), stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokens Computed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - qwen2.5-coder:14b
Q4_K_M · 4,096 ctx
Ollama 0.32.1
10,169 MiB
94.89 tok/s
Full measured record
qwen2.5-coder:14b Q4_K_M · 4,096 context · Ollama 0.32.1
Measured at 10,169 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 10,169 MiB
- Measured
- Decode
- 94.89 tok/s
- Measured
- Left on the card
- 6,134 MiB
- Computed: 16,303 − 10,169
- Share of the card
- 62.4%
- Computed: 10,169 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen2.5-coder:14b digest 9ec8897f747e
- Parameters
- 14.8B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 4096
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 488
- Repetitions
- 3
- Decode, median
- 94.89 tok/s
- Prefill, median
- 5271.63 tok/s
- Load, median
- 219.8 ms
- Whole-card memory after the cell
- 10169 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1368
- Repetitions
- 3
- Decode, median
- 93.8 tok/s
- Prefill, median
- 4942.3 tok/s
- Load, median
- 159.4 ms
- Whole-card memory after the cell
- 10171 MiB
- Resident in GiB
- 9.93 GiB Computed: 10,169 ÷ 1,024
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - qwen2.5:14b
Q4_K_M · 4,096 ctx
Ollama 0.32.15 defaults
10,356 MiB
102.25 tok/s
Full measured record
qwen2.5:14b Q4_K_M · 4,096 context · Ollama 0.32.15 · defaults
Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 10,356 MiB
- Measured
- Decode
- 102.25 tok/s
- Measured
- Left on the card
- 5,947 MiB
- Computed: 16,303 − 10,356
- Share of the card
- 63.5%
- Computed: 10,356 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- ollama 0.32.15
- Engine
- vendored llama.cpp 9d77fa172
- Condition
- defaults
- Model name in the file
- qwen2.5:14b
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096
- Runs
- 5
- Decode, median (server counters)
- 102.25 tok/s
- Decode, median (wall clock)
- 101.06 tok/s
- Each run
- 102.13 · 102.34 · 102.25 · 102.3 · 102.25 tok/s
- Prefill, long prompt, median
- 4983.8 tok/s at 453 prompt tokens
- Time to first token
- Empty in this file
- Whole-card memory, median
- 10356 MiB
- Whole-card memory before the load
- 1053 MiB
- Added by loading the model
- 9,303 MiB Computed: 10,356 − 1,053
- Load
- 5.51 s
- Cold first request
- 5.67 s
- App RAM while loaded
- 1604.1 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 10.11 GiB Computed: 10,356 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - qwen2.5:14b
Q4_K_M · 4,096 ctx
Ollama 0.32.15 normalized
10,356 MiB
102.21 tok/s
Full measured record
qwen2.5:14b Q4_K_M · 4,096 context · Ollama 0.32.15 · normalized
Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 10,356 MiB
- Measured
- Decode
- 102.21 tok/s
- Measured
- Left on the card
- 5,947 MiB
- Computed: 16,303 − 10,356
- Share of the card
- 63.5%
- Computed: 10,356 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- ollama 0.32.15
- Engine
- vendored llama.cpp 9d77fa172
- Condition
- normalized
- Model name in the file
- qwen2.5:14b
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096
- Runs
- 5
- Decode, median (server counters)
- 102.21 tok/s
- Decode, median (wall clock)
- 101.23 tok/s
- Each run
- 102.21 · 102.21 · 102.17 · 102.21 · 102.21 tok/s
- Prefill, long prompt, median
- 5038.3 tok/s at 452 prompt tokens
- Time to first token
- Empty in this file
- Whole-card memory, median
- 10356 MiB
- Whole-card memory before the load
- 1053 MiB
- Added by loading the model
- 9,303 MiB Computed: 10,356 − 1,053
- Load
- 5.52 s
- Cold first request
- 5.68 s
- App RAM while loaded
- 1617.6 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 10.11 GiB Computed: 10,356 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - qwen2.5:14b
Q4_K_M · 4,096 ctx
LM Studio 0.4.21 normalized
10,405 MiB
97.63 tok/s
Full measured record
qwen2.5:14b Q4_K_M · 4,096 context · LM Studio 0.4.21 · normalized
Measured at 10,405 MiB resident on with LM Studio 0.4.21. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 10,405 MiB
- Measured
- Decode
- 97.63 tok/s
- Measured
- Left on the card
- 5,898 MiB
- Computed: 16,303 − 10,405
- Share of the card
- 63.8%
- Computed: 10,405 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- LM Studio 0.4.21
- Engine
- llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
- Condition
- normalized
- Model name in the file
- qwen2.5-14b-q4km
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096
- Runs
- 5
- Decode, median (server counters)
- 97.63 tok/s
- Decode, median (wall clock)
- 96.62 tok/s
- Each run
- 96.89 · 97.75 · 97.46 · 97.79 · 97.63 tok/s
- Prefill, long prompt, median
- 5566.5 tok/s at 453 prompt tokens
- Time to first token, median
- 24.7 ms
- Whole-card memory, median
- 10405 MiB
- Whole-card memory before the load
- 1106 MiB
- Added by loading the model
- 9,299 MiB Computed: 10,405 − 1,106
- Load
- 7.68 s
- Cold first request
- 0.16 s
- App RAM while loaded
- 10411.1 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 10.16 GiB Computed: 10,405 ÷ 1,024
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - qwen2.5:14b
Q4_K_M · 4,096 ctx
Ollama 0.32.1
10,841 MiB
96.65 tok/s
Full measured record
qwen2.5:14b Q4_K_M · 4,096 context · Ollama 0.32.1
Measured at 10,841 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 10,841 MiB
- Measured
- Decode
- 96.65 tok/s
- Measured
- Left on the card
- 5,462 MiB
- Computed: 16,303 − 10,841
- Share of the card
- 66.5%
- Computed: 10,841 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen2.5:14b digest 7cdf5a0187d5
- Parameters
- 14.8B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 4096
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 488
- Repetitions
- 3
- Decode, median
- 96.65 tok/s
- Prefill, median
- 5202.23 tok/s
- Load, median
- 182.5 ms
- Whole-card memory after the cell
- 10841 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1674
- Repetitions
- 3
- Decode, median
- 94.48 tok/s
- Prefill, median
- 4876.41 tok/s
- Load, median
- 125.8 ms
- Whole-card memory after the cell
- 10855 MiB
- Resident in GiB
- 10.59 GiB Computed: 10,841 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - qwen2.5:14b
Q4_K_M · 4,096 ctx
Ollama 0.32.1
10,941 MiB
99.43 tok/s
Full measured record
qwen2.5:14b Q4_K_M · 4,096 context · Ollama 0.32.1
Measured at 10,941 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 10,941 MiB
- Measured
- Decode
- 99.43 tok/s
- Measured
- Left on the card
- 5,362 MiB
- Computed: 16,303 − 10,941
- Share of the card
- 67.1%
- Computed: 10,941 ÷ 16,303
Full measured row
- Captured
- from the file's footer notes
- Runtime
- ollama 0.32.1
- Model name in the file
- qwen2.5:14b
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096 from the file's settings note
- Offload
- Full offload from the file's settings note
- Runs
- 5
- Decode, median (server counters)
- 99.43 tok/s
- Decode, median (wall clock)
- 98.59 tok/s
- Each run
- 99.5 · 99.5 · 99.4 · 99.4 · 99.3 tok/s
- Prefill, short prompt, median
- 3151.4 tok/s
- Prefill, long prompt
- 5004 tok/s at 451 prompt tokens
- Whole-card memory, median
- 10941 MiB
- Load
- 5.6 s
- Desktop baseline at bench start
- 1642 MiB one reading for the whole capture
- Driver
- Not recorded in this file
- Resident in GiB
- 10.68 GiB Computed: 10,941 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: llm-server-compare-2026-08.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - qwen2.5:14b
Q4_K_M · 4,096 ctx
llama.cpp b10507
10,942 MiB
101.38 tok/s
Full measured record
qwen2.5:14b Q4_K_M · 4,096 context · llama.cpp b10507
Measured at 10,942 MiB resident on with llama.cpp b10507. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 10,942 MiB
- Measured
- Decode
- 101.38 tok/s
- Measured
- Left on the card
- 5,361 MiB
- Computed: 16,303 − 10,942
- Share of the card
- 67.1%
- Computed: 10,942 ÷ 16,303
Full measured row
- Captured
- from the file's footer notes
- Runtime
- llama.cpp b10507
- Model name in the file
- qwen2.5:14b
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096 from the file's settings note
- Offload
- Full offload from the file's settings note
- Runs
- 5
- Decode, median (server counters)
- 101.38 tok/s
- Decode, median (wall clock)
- 101.29 tok/s
- Each run
- 101.3 · 101.2 · 101.4 · 101.4 · 101.4 tok/s
- Prefill, short prompt, median
- 2037.8 tok/s
- Prefill, long prompt
- 4697 tok/s at 423 prompt tokens
- Whole-card memory, median
- 10942 MiB
- Load
- 8.1 s
- Desktop baseline at bench start
- 1642 MiB one reading for the whole capture
- Driver
- Not recorded in this file
- Resident in GiB
- 10.69 GiB Computed: 10,942 ÷ 1,024
Source file: llm-server-compare-2026-08.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - qwen2.5:14b
Q4_K_M · 8,192 ctx
LM Studio 0.4.21 defaults
11,194 MiB
98.15 tok/s
Full measured record
qwen2.5:14b Q4_K_M · 8,192 context · LM Studio 0.4.21 · defaults
Measured at 11,194 MiB resident on with LM Studio 0.4.21. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 11,194 MiB
- Measured
- Decode
- 98.15 tok/s
- Measured
- Left on the card
- 5,109 MiB
- Computed: 16,303 − 11,194
- Share of the card
- 68.7%
- Computed: 11,194 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- LM Studio 0.4.21
- Engine
- llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
- Condition
- defaults
- Model name in the file
- qwen2.5-14b-q4km
- Quantization
- Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 8192
- Runs
- 5
- Decode, median (server counters)
- 98.15 tok/s
- Decode, median (wall clock)
- 97.31 tok/s
- Each run
- 97.36 · 98.14 · 98.17 · 98.15 · 98.17 tok/s
- Prefill, long prompt, median
- 5195.8 tok/s at 454 prompt tokens
- Time to first token, median
- 24.6 ms
- Whole-card memory, median
- 11194 MiB
- Whole-card memory before the load
- 1103 MiB
- Added by loading the model
- 10,091 MiB Computed: 11,194 − 1,103
- Load
- 7.38 s
- Cold first request
- 0.15 s
- App RAM while loaded
- 10406.8 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 10.93 GiB Computed: 11,194 ÷ 1,024
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - gpt-oss:20bRan partly off the card
MXFP4 · 8,192 ctx
LM Studio 0.4.21 defaults
12,310 MiB
186.26 tok/s
Full measured record
gpt-oss:20b MXFP4 · 8,192 context · LM Studio 0.4.21 · defaults
Measured at 12,310 MiB resident on with LM Studio 0.4.21. That figure is the whole card as
nvidia-smireported it, Windows desktop included.Ran partly off the card
LM Studio's default automatic GPU offload was in effect, at its default 8,192 context. The dataset attributes the lower decode rate to that setting. The normalized row is the same model with full offload. Runtime comparison dataset.
- Resident, whole card
- 12,310 MiB
- Measured
- Decode
- 186.26 tok/s
- Measured
- Left on the card
- 3,993 MiB
- Computed: 16,303 − 12,310
- Share of the card
- 75.5%
- Computed: 12,310 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- LM Studio 0.4.21
- Engine
- llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
- Condition
- defaults
- Model name in the file
- gpt-oss-20b
- Quantization
- MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 8192
- Runs
- 5
- Decode, median (server counters)
- 186.26 tok/s
- Decode, median (wall clock)
- 184.34 tok/s
- Each run
- 183.42 · 186.26 · 186.84 · 186.98 · 185.4 tok/s
- Prefill, long prompt, median
- 5241.3 tok/s at 474 prompt tokens
- Time to first token, median
- 78.9 ms
- Whole-card memory, median
- 12310 MiB
- Whole-card memory before the load
- 1119 MiB
- Added by loading the model
- 11,191 MiB Computed: 12,310 − 1,119
- Load
- 8.43 s
- Cold first request
- 0.36 s
- App RAM while loaded
- 12917.9 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 12.02 GiB Computed: 12,310 ÷ 1,024
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - qwen2.5-coder:14b
Q4_K_M · 16,384 ctx
Ollama 0.32.1
12,485 MiB
94.49 tok/s
Full measured record
qwen2.5-coder:14b Q4_K_M · 16,384 context · Ollama 0.32.1
Measured at 12,485 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 12,485 MiB
- Measured
- Decode
- 94.49 tok/s
- Measured
- Left on the card
- 3,818 MiB
- Computed: 16,303 − 12,485
- Share of the card
- 76.6%
- Computed: 12,485 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen2.5-coder:14b digest 9ec8897f747e
- Parameters
- 14.8B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 16384
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 488
- Repetitions
- 3
- Decode, median
- 94.49 tok/s
- Prefill, median
- 4909.36 tok/s
- Load, median
- 159.5 ms
- Whole-card memory after the cell
- 12485 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1368
- Repetitions
- 3
- Decode, median
- 93.41 tok/s
- Prefill, median
- 4842.15 tok/s
- Load, median
- 153.2 ms
- Whole-card memory after the cell
- 12487 MiB
- Resident in GiB
- 12.19 GiB Computed: 12,485 ÷ 1,024
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - gpt-oss:20b
MXFP4 · 4,096 ctx
LM Studio 0.4.21 normalized
12,533 MiB
242.83 tok/s
Full measured record
gpt-oss:20b MXFP4 · 4,096 context · LM Studio 0.4.21 · normalized
Measured at 12,533 MiB resident on with LM Studio 0.4.21. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 12,533 MiB
- Measured
- Decode
- 242.83 tok/s
- Measured
- Left on the card
- 3,770 MiB
- Computed: 16,303 − 12,533
- Share of the card
- 76.9%
- Computed: 12,533 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- LM Studio 0.4.21
- Engine
- llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
- Condition
- normalized
- Model name in the file
- gpt-oss-20b
- Quantization
- MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096
- Runs
- 5
- Decode, median (server counters)
- 242.83 tok/s
- Decode, median (wall clock)
- 240.13 tok/s
- Each run
- 242.83 · 242.82 · 243.14 · 241.74 · 244.35 tok/s
- Prefill, long prompt, median
- 9617.7 tok/s at 474 prompt tokens
- Time to first token, median
- 38.6 ms
- Whole-card memory, median
- 12533 MiB
- Whole-card memory before the load
- 1101 MiB
- Added by loading the model
- 11,432 MiB Computed: 12,533 − 1,101
- Load
- 8.43 s
- Cold first request
- 0.27 s
- App RAM while loaded
- 13176.6 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 12.24 GiB Computed: 12,533 ÷ 1,024
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - gpt-oss:20b
MXFP4 · 4,096 ctx
llama.cpp b10507
13,037 MiB
221.96 tok/s
Full measured record
gpt-oss:20b MXFP4 · 4,096 context · llama.cpp b10507
Measured at 13,037 MiB resident on with llama.cpp b10507. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 13,037 MiB
- Measured
- Decode
- 221.96 tok/s
- Measured
- Left on the card
- 3,266 MiB
- Computed: 16,303 − 13,037
- Share of the card
- 80.0%
- Computed: 13,037 ÷ 16,303
Full measured row
- Captured
- from the file's footer notes
- Runtime
- llama.cpp b10507 (upstream GGUF)
- Model name in the file
- gpt-oss:20b
- Quantization
- MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096 from the file's settings note
- Offload
- Full offload from the file's settings note
- Runs
- 5
- Decode, median (server counters)
- 221.96 tok/s
- Decode, median (wall clock)
- 220.53 tok/s
- Each run
- 185.8 · 242.6 · 222.0 · 231.9 · 175.3 tok/s
- Prefill, short prompt, median
- 1078.7 tok/s
- Prefill, long prompt
- Empty in this file
- Whole-card memory, median
- 13037 MiB
- Load
- 19.8 s
- Desktop baseline at bench start
- 1642 MiB one reading for the whole capture
- Driver
- Not recorded in this file
- Resident in GiB
- 12.73 GiB Computed: 13,037 ÷ 1,024
Source file: llm-server-compare-2026-08.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - qwen2.5:14b
Q4_K_M · 16,384 ctx
Ollama 0.32.1
13,157 MiB
97.02 tok/s
Full measured record
qwen2.5:14b Q4_K_M · 16,384 context · Ollama 0.32.1
Measured at 13,157 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 13,157 MiB
- Measured
- Decode
- 97.02 tok/s
- Measured
- Left on the card
- 3,146 MiB
- Computed: 16,303 − 13,157
- Share of the card
- 80.7%
- Computed: 13,157 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen2.5:14b digest 7cdf5a0187d5
- Parameters
- 14.8B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 16384
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 488
- Repetitions
- 3
- Decode, median
- 97.02 tok/s
- Prefill, median
- 5225.96 tok/s
- Load, median
- 131.3 ms
- Whole-card memory after the cell
- 13157 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1674
- Repetitions
- 3
- Decode, median
- 94.58 tok/s
- Prefill, median
- 4864.67 tok/s
- Load, median
- 128.9 ms
- Whole-card memory after the cell
- 13159 MiB
- Resident in GiB
- 12.85 GiB Computed: 13,157 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - gpt-oss:20b
MXFP4 · 4,096 ctx
Ollama 0.32.1
13,381 MiB
181.50 tok/s
Full measured record
gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.1
Measured at 13,381 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log.
- Resident, whole card
- 13,381 MiB
- Measured
- Decode
- 181.50 tok/s
- Measured
- Left on the card
- 2,922 MiB
- Computed: 16,303 − 13,381
- Share of the card
- 82.1%
- Computed: 13,381 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- gpt-oss:20b digest 17052f91a42e
- Parameters
- 20.9B
- Quantization
- MXFP4
- Context (num_ctx)
- 4096
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 503
- Repetitions
- 3
- Decode, median
- 181.5 tok/s
- Prefill, median
- 9607.49 tok/s
- Load, median
- 383.7 ms
- Whole-card memory after the cell
- 13381 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1360
- Repetitions
- 3
- Decode, median
- 178.37 tok/s
- Prefill, median
- 11001.37 tok/s
- Load, median
- 375.9 ms
- Whole-card memory after the cell
- 13385 MiB
- Resident in GiB
- 13.07 GiB Computed: 13,381 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - gpt-oss:20b
MXFP4 · 4,096 ctx
Ollama 0.32.15 defaults
13,570 MiB
213.54 tok/s
Full measured record
gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.15 · defaults
Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 13,570 MiB
- Measured
- Decode
- 213.54 tok/s
- Measured
- Left on the card
- 2,733 MiB
- Computed: 16,303 − 13,570
- Share of the card
- 83.2%
- Computed: 13,570 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- ollama 0.32.15
- Engine
- vendored llama.cpp 9d77fa172
- Condition
- defaults
- Model name in the file
- gpt-oss:20b
- Quantization
- MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096
- Runs
- 5
- Decode, median (server counters)
- 213.54 tok/s
- Decode, median (wall clock)
- 212.03 tok/s
- Each run
- 213.22 · 213.64 · 213.48 · 213.54 · 213.76 tok/s
- Prefill, long prompt, median
- 9239.4 tok/s at 475 prompt tokens
- Time to first token
- Empty in this file
- Whole-card memory, median
- 13570 MiB
- Whole-card memory before the load
- 1053 MiB
- Added by loading the model
- 12,517 MiB Computed: 13,570 − 1,053
- Load
- 18.56 s
- Cold first request
- 23.67 s
- App RAM while loaded
- 2138.8 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 13.25 GiB Computed: 13,570 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - gpt-oss:20b
MXFP4 · 4,096 ctx
Ollama 0.32.15 normalized
13,570 MiB
213.65 tok/s
Full measured record
gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.15 · normalized
Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 13,570 MiB
- Measured
- Decode
- 213.65 tok/s
- Measured
- Left on the card
- 2,733 MiB
- Computed: 16,303 − 13,570
- Share of the card
- 83.2%
- Computed: 13,570 ÷ 16,303
Full measured row
- Captured
- from the file's name
- App
- ollama 0.32.15
- Engine
- vendored llama.cpp 9d77fa172
- Condition
- normalized
- Model name in the file
- gpt-oss:20b
- Quantization
- MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096
- Runs
- 5
- Decode, median (server counters)
- 213.65 tok/s
- Decode, median (wall clock)
- 211.8 tok/s
- Each run
- 213.62 · 213.65 · 213.68 · 213.25 · 213.73 tok/s
- Prefill, long prompt, median
- 9187.1 tok/s at 474 prompt tokens
- Time to first token
- Empty in this file
- Whole-card memory, median
- 13570 MiB
- Whole-card memory before the load
- 1053 MiB
- Added by loading the model
- 12,517 MiB Computed: 13,570 − 1,053
- Load
- 8.05 s
- Cold first request
- 8.36 s
- App RAM while loaded
- 2152.1 MB
- Driver
- Not recorded in this file
- Resident in GiB
- 13.25 GiB Computed: 13,570 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: llm-server-compare-2026-08-26-lmstudio.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - gpt-oss:20b
MXFP4 · 16,384 ctx
Ollama 0.32.1
13,683 MiB
183.00 tok/s
Full measured record
gpt-oss:20b MXFP4 · 16,384 context · Ollama 0.32.1
Measured at 13,683 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log.
- Resident, whole card
- 13,683 MiB
- Measured
- Decode
- 183.00 tok/s
- Measured
- Left on the card
- 2,620 MiB
- Computed: 16,303 − 13,683
- Share of the card
- 83.9%
- Computed: 13,683 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- gpt-oss:20b digest 17052f91a42e
- Parameters
- 20.9B
- Quantization
- MXFP4
- Context (num_ctx)
- 16384
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 503
- Repetitions
- 3
- Decode, median
- 183.0 tok/s
- Prefill, median
- 9092.22 tok/s
- Load, median
- 347.7 ms
- Whole-card memory after the cell
- 13683 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1360
- Repetitions
- 3
- Decode, median
- 178.34 tok/s
- Prefill, median
- 10328.62 tok/s
- Load, median
- 345.3 ms
- Whole-card memory after the cell
- 13683 MiB
- Resident in GiB
- 13.36 GiB Computed: 13,683 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - gpt-oss:20b
MXFP4 · 4,096 ctx
Ollama 0.32.1
14,067 MiB
190.14 tok/s
Full measured record
gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.1
Measured at 14,067 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 14,067 MiB
- Measured
- Decode
- 190.14 tok/s
- Measured
- Left on the card
- 2,236 MiB
- Computed: 16,303 − 14,067
- Share of the card
- 86.3%
- Computed: 14,067 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- gpt-oss:20b digest 17052f91a42e
- Parameters
- 20.9B
- Quantization
- MXFP4
- Context (num_ctx)
- 4096
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 503
- Repetitions
- 3
- Decode, median
- 190.14 tok/s
- Prefill, median
- 9078.28 tok/s
- Load, median
- 297.7 ms
- Whole-card memory after the cell
- 14067 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1658
- Repetitions
- 3
- Decode, median
- 187.27 tok/s
- Prefill, median
- 10464.66 tok/s
- Load, median
- 279.5 ms
- Whole-card memory after the cell
- 14060 MiB
- Resident in GiB
- 13.74 GiB Computed: 14,067 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - gpt-oss:20b
MXFP4 · 4,096 ctx
Ollama 0.32.1
14,152 MiB
195.79 tok/s
Full measured record
gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.1
Measured at 14,152 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 14,152 MiB
- Measured
- Decode
- 195.79 tok/s
- Measured
- Left on the card
- 2,151 MiB
- Computed: 16,303 − 14,152
- Share of the card
- 86.8%
- Computed: 14,152 ÷ 16,303
Full measured row
- Captured
- from the file's footer notes
- Runtime
- ollama 0.32.1
- Model name in the file
- gpt-oss:20b
- Quantization
- MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
- Context
- 4096 from the file's settings note
- Offload
- Full offload from the file's settings note
- Runs
- 5
- Decode, median (server counters)
- 195.79 tok/s
- Decode, median (wall clock)
- 194.59 tok/s
- Each run
- 195.8 · 195.7 · 196.3 · 196.1 · 195.7 tok/s
- Prefill, short prompt, median
- 3744.3 tok/s
- Prefill, long prompt
- Empty in this file
- Whole-card memory, median
- 14152 MiB
- Load
- 8.6 s
- Desktop baseline at bench start
- 1642 MiB one reading for the whole capture
- Driver
- Not recorded in this file
- Resident in GiB
- 13.82 GiB Computed: 14,152 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: llm-server-compare-2026-08.csv (column
vram_total_mib_median). Method and caveats: runtime comparison dataset. - gpt-oss:20b
MXFP4 · 16,384 ctx
Ollama 0.32.1
14,355 MiB
188.63 tok/s
Full measured record
gpt-oss:20b MXFP4 · 16,384 context · Ollama 0.32.1
Measured at 14,355 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.- Resident, whole card
- 14,355 MiB
- Measured
- Decode
- 188.63 tok/s
- Measured
- Left on the card
- 1,948 MiB
- Computed: 16,303 − 14,355
- Share of the card
- 88.1%
- Computed: 14,355 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- gpt-oss:20b digest 17052f91a42e
- Parameters
- 20.9B
- Quantization
- MXFP4
- Context (num_ctx)
- 16384
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 503
- Repetitions
- 3
- Decode, median
- 188.63 tok/s
- Prefill, median
- 9286.61 tok/s
- Load, median
- 299.2 ms
- Whole-card memory after the cell
- 14355 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1658
- Repetitions
- 3
- Decode, median
- 187.73 tok/s
- Prefill, median
- 10554.32 tok/s
- Load, median
- 280.2 ms
- Whole-card memory after the cell
- 14355 MiB
- Resident in GiB
- 14.02 GiB Computed: 14,355 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - devstral:24b
Q4_K_M · 4,096 ctx
Ollama 0.32.1
14,997 MiB
36.32 tok/s
Full measured record
devstral:24b Q4_K_M · 4,096 context · Ollama 0.32.1
Measured at 14,997 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.Measured at a 4,096 context only. The dataset reports that an 8K attempt spilled and decode collapsed, but no row for that attempt was published, so it has no pin here. Throughput dataset, coding models note.
- Resident, whole card
- 14,997 MiB
- Measured
- Decode
- 36.32 tok/s
- Measured
- Left on the card
- 1,306 MiB
- Computed: 16,303 − 14,997
- Share of the card
- 92.0%
- Computed: 14,997 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- devstral:24b digest 9bd74193e939
- Parameters
- 23.6B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 4096
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 1694
- Repetitions
- 3
- Decode, median
- 36.32 tok/s
- Prefill, median
- 9377.3 tok/s
- Load, median
- 238.2 ms
- Whole-card memory after the cell
- 14997 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 2597
- Repetitions
- 3
- Decode, median
- 34.9 tok/s
- Prefill, median
- 5334.83 tok/s
- Load, median
- 241.7 ms
- Whole-card memory after the cell
- 14997 MiB
- Resident in GiB
- 14.65 GiB Computed: 14,997 ÷ 1,024
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - qwen3-coder:30bRan partly off the card
Q4_K_M · 16,384 ctx
Ollama 0.32.1
15,003 MiB
51.62 tok/s
Full measured record
qwen3-coder:30b Q4_K_M · 16,384 context · Ollama 0.32.1
Measured at 15,003 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.Ran partly off the card
The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note.
- Resident, whole card
- 15,003 MiB
- Measured
- Decode
- 51.62 tok/s
- Measured
- Left on the card
- 1,300 MiB
- Computed: 16,303 − 15,003
- Share of the card
- 92.0%
- Computed: 15,003 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen3-coder:30b digest 06c1097efce0
- Parameters
- 30.5B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 16384
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 467
- Repetitions
- 3
- Decode, median
- 51.62 tok/s
- Prefill, median
- 1524.77 tok/s
- Load, median
- 219.9 ms
- Whole-card memory after the cell
- 15003 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1347
- Repetitions
- 3
- Decode, median
- 51.65 tok/s
- Prefill, median
- 1928.14 tok/s
- Load, median
- 188.9 ms
- Whole-card memory after the cell
- 15003 MiB
- Resident in GiB
- 14.65 GiB Computed: 15,003 ÷ 1,024
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - llama3.2:3bRan partly off the card
Q4_K_M · 131,072 ctx
Ollama 0.32.15 defaults
15,084 MiB
102.68 tok/s
Full measured record
llama3.2:3b Q4_K_M · 131,072 context · Ollama 0.32.15 · defaults
Measured at 15,084 MiB resident on with Ollama 0.32.15. That figure is the whole card as
nvidia-smireported it, Windows desktop included.Ran partly off the card
The capture's own ollama ps line reads 22%/78% CPU/GPU at context 131,072, so part of this model ran from system RAM.
Ollama's app-level context slider was set to 256k, which this model clamps to 131,072. The dataset describes the result as the KV cache spilling to the CPU. Runtime comparison dataset.
- Resident, whole card
- 15,084 MiB
- Measured
- Decode
- 102.68 tok/s
- Measured
- Left on the card
- 1,219 MiB
- Computed: 16,303 − 15,084
- Share of the card
- 92.5%
- Computed: 15,084 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.15
- Condition
- defaults
- Schema
- techfuelhq:lmstudio-vs-ollama:1
- Model
- llama3.2:3b
- Quantization
- Q4_K_M. Not a field in this file. Taken from the same model in the throughput CSV.
- Context
- 131072 from the capture's ollama ps line
- Processor split
- 22% CPU / 78% GPU from the capture's ollama ps line
- Size reported by ollama ps
- 18 GB
- Runs
- 5
- Decode, median (server counters)
- 102.68 tok/s
- Decode, median (wall clock)
- 101.58 tok/s
- Each run
- 97.57 · 99.85 · 103.59 · 102.68 · 103.21 tok/s
- Prefill, long prompt, median
- 9998.8 tok/s at 434 prompt tokens
- Whole-card memory, median
- 15084 MiB
- Whole-card memory before the load
- 1067 MiB
- Added by loading the model
- 14,017 MiB Computed: 15,084 − 1,067
- Load
- 4.02 s
- Cold first request
- 4.2 s
- App RAM while loaded
- 4745.7 MB
- Resident in GiB
- 14.73 GiB Computed: 15,084 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: ollama-256k-slider-3b.json (column
vram_mb_median). Method and caveats: runtime comparison dataset. - qwen3-coder:30bRan partly off the card
Q4_K_M · 4,096 ctx
Ollama 0.32.1
15,101 MiB
61.89 tok/s
Full measured record
qwen3-coder:30b Q4_K_M · 4,096 context · Ollama 0.32.1
Measured at 15,101 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.Ran partly off the card
The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note.
- Resident, whole card
- 15,101 MiB
- Measured
- Decode
- 61.89 tok/s
- Measured
- Left on the card
- 1,202 MiB
- Computed: 16,303 − 15,101
- Share of the card
- 92.6%
- Computed: 15,101 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- qwen3-coder:30b digest 06c1097efce0
- Parameters
- 30.5B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 4096
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 467
- Repetitions
- 3
- Decode, median
- 61.89 tok/s
- Prefill, median
- 1744.18 tok/s
- Load, median
- 197.7 ms
- Whole-card memory after the cell
- 15101 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1347
- Repetitions
- 3
- Decode, median
- 61.55 tok/s
- Prefill, median
- 2368.71 tok/s
- Load, median
- 177.2 ms
- Whole-card memory after the cell
- 15101 MiB
- Resident in GiB
- 14.75 GiB Computed: 15,101 ÷ 1,024
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset. - llama3.2:3bReading includes another model
Q4_K_M · 4,096 ctx
Ollama 0.32.1
15,682 MiB
311.96 tok/s
Full measured record
llama3.2:3b Q4_K_M · 4,096 context · Ollama 0.32.1
Measured at 15,682 MiB resident on with Ollama 0.32.1. That figure is the whole card as
nvidia-smireported it, Windows desktop included.Reading includes another model
gpt-oss:20b was still loaded from the cell before this one, so this reading is the whole card with two models on it, not Llama 3.2 3B alone. The 16,384-context row from the same capture was read after the larger model had been evicted. Throughput dataset, VRAM note.
- Resident, whole card
- 15,682 MiB
- Measured
- Decode
- 311.96 tok/s
- Measured
- Left on the card
- 621 MiB
- Computed: 16,303 − 15,682
- Share of the card
- 96.2%
- Computed: 15,682 ÷ 16,303
Full measured row
- Captured (UTC)
- GPU
- NVIDIA GeForce RTX 5080
- Driver
- 610.88
- Runtime
- Ollama 0.32.1
- Method
- llm-throughput-ollama-v1
- Model
- llama3.2:3b digest a80c4f17acd5
- Parameters
- 3.2B
- Quantization
- Q4_K_M
- Context (num_ctx)
- 4096
- Prompt cell 1
- The row pinned on the map
- Prompt tokens
- 471
- Repetitions
- 3
- Decode, median
- 311.96 tok/s
- Prefill, median
- 18196.57 tok/s
- Load, median
- 162.8 ms
- Whole-card memory after the cell
- 15682 MiB
- Prompt cell 2
- Same configuration, longer prompt
- Prompt tokens
- 1657
- Repetitions
- 3
- Decode, median
- 298.21 tok/s
- Prefill, median
- 17681.08 tok/s
- Load, median
- 156.2 ms
- Whole-card memory after the cell
- 15682 MiB
- Resident in GiB
- 15.31 GiB Computed: 15,682 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Source file: rtx-5080-llm-throughput.csv (column
vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.
No measured configuration matches those filters. That combination was not on the bench, and I do not estimate here. For an estimate, use the LLM VRAM calculator.
How to read it
The map answers a narrow question: what did this model, at this context length, on this runtime version, take on a 16 GB card, and what speed came with it. Each record reads “measured at so many MiB resident on a date with a runtime version” because that is all a measurement can say. It is not a promise that the same model fits on your card today.
The line along the bottom is the card, from zero to the total it reports to nvidia-smi. A pin’s position across is the memory in use on the whole card while that model was loaded, Windows desktop included, and its height is the decode rate. Select a pin and the line fills to it in the measured colour. The hatched remainder is headroom, which I compute by subtracting.
A dashed connector joins the same model, runtime and capture at two context lengths. Flat means memory grew and speed held; a drop means the model stopped fitting. A ring is a run recorded as partly off the card. A diamond is a reading taken while another model was still loaded, so it overstates that model.
Where the cliff shows up
The cliff is not a memory number. In these files, a model that sits whole on the card decodes at the same speed whether it uses a third of the memory or nearly all of it. Speed falls when part of the model no longer fits and runs from system RAM instead.
Each row is the same model, runtime version and capture at two context lengths, and both ends are measured rows. Memory change is the second reading minus the first. Decode change is (second − first) ÷ first.
The table has two rows where raising the context cost speed, and both are runs that left the card. Llama 3.2 3B is the small model in this set, and a large enough context still pushed it over. Qwen3-Coder 30B never fit whole, so its memory reading barely moved and its speed did.
Every other pair held its decode rate within one percent while the context quadrupled. What they paid was memory, and how much depended on the model: the two 14B Qwen models paid the most, and gpt-oss:20b, the largest of them by parameter count, paid the least.
What the files do not hold is a dense model caught in the act of falling off. The throughput dataset says Devstral 24B collapsed when I tried it at an 8K context, but that attempt was never written to the CSV, so there is no pin for it and I have not drawn one.
Energy per thousand tokens
llama3.2:3b Q4_K_M · named “Llama 3.2 3B” in the file · run logged UTC
833 J per 1,000 tokens Computed: 0.833 J per token × 1,000
- Decode, median of 3
- 318.3 tok/s
- Board power, median while loaded
- 265.2 W
- Tokens per joule, as filed
- 1.2
- Joules per token, as filed
- 0.833
More from the power file
- Watt-hours per 1,000 tokens
- 0.231 Wh Computed: 231.0 Wh per million ÷ 1,000
- Decode, slowest and fastest run
- 315.5 to 318.5 tok/s
- Board power, peak
- 307.78 W
- Memory in use, median
- 3,590 MiB
- GPU temperature, median
- 49.0 °C
- GPU load, median
- 82.0%
- Telemetry samples while loaded
- 9
qwen2.5:7b Q4_K_M · named “Qwen 2.5 7B” in the file · run logged UTC
1,886 J per 1,000 tokens Computed: 1.886 J per token × 1,000
- Decode, median of 3
- 182.2 tok/s
- Board power, median while loaded
- 343.6 W
- Tokens per joule, as filed
- 0.53
- Joules per token, as filed
- 1.886
More from the power file
- Watt-hours per 1,000 tokens
- 0.524 Wh Computed: 524.0 Wh per million ÷ 1,000
- Decode, slowest and fastest run
- 182.1 to 182.3 tok/s
- Board power, peak
- 346.08 W
- Memory in use, median
- 5,665 MiB
- GPU temperature, median
- 52.0 °C
- GPU load, median
- 88.0%
- Telemetry samples while loaded
- 14
qwen2.5:14b Q4_K_M · named “Qwen 2.5 14B” in the file · run logged UTC
3,154 J per 1,000 tokens Computed: 3.154 J per token × 1,000
- Decode, median of 3
- 98.8 tok/s
- Board power, median while loaded
- 311.6 W
- Tokens per joule, as filed
- 0.317
- Joules per token, as filed
- 3.154
More from the power file
- Watt-hours per 1,000 tokens
- 0.876 Wh Computed: 876.0 Wh per million ÷ 1,000
- Decode, slowest and fastest run
- 98.6 to 98.8 tok/s
- Board power, peak
- 319.69 W
- Memory in use, median
- 10,171 MiB
- GPU temperature, median
- 50.0 °C
- GPU load, median
- 93.0%
- Telemetry samples while loaded
- 27
gpt-oss:20b MXFP4 · named “gpt-oss 20B” in the file · run logged UTC
1,504 J per 1,000 tokens Computed: 1.504 J per token × 1,000
- Decode, median of 3
- 194.4 tok/s
- Board power, median while loaded
- 292.4 W
- Tokens per joule, as filed
- 0.665
- Joules per token, as filed
- 1.504
More from the power file
- Watt-hours per 1,000 tokens
- 0.418 Wh Computed: 418.0 Wh per million ÷ 1,000
- Decode, slowest and fastest run
- 194.3 to 194.5 tok/s
- Board power, peak
- 294.86 W
- Memory in use, median
- 13,381 MiB
- GPU temperature, median
- 51.0 °C
- GPU load, median
- 84.0%
- Telemetry samples while loaded
- 18
These four runs are a separate capture from the map: one short prompt, three iterations per model, board power sampled once a second and counted only while the GPU was at least half loaded. They were taken at the card’s stock 448 W power limit, which none of the runs reached. Board power is the graphics card alone, not the wall.
The power file records no context length, driver or runtime version, which is why these runs are not pinned on the map. Its dataset page gives the runtime as Ollama 0.32.1.
There is no undervolt or power-limit comparison here because I have not measured one for language models. My undervolt database holds one row for this card and it is a stock 3DMark baseline. Method and the idle-power finding are on the power efficiency dataset.
Versions on the bench, and how stale they are
Driver and runtime versions per capture, exactly as each file records them. Where a file is silent, the cell says so.
The same model at the same context on two Ollama versions. Decode change is (later − earlier) ÷ earlier.
Those are the versions that were installed on the capture dates. Drivers and runtimes have both moved since, and I have not re-run this matrix on newer ones. Read every pin as true for its date and versions, and expect a current install to differ.
The second table is why I say that plainly. Two captures six days apart put the same three models on two Ollama versions, and all three decoded faster on the later one. The runtime comparison dataset notes that the earlier capture had less control over background load, so not all of that gap is the version. Either way, a few percent between runtimes does not survive an update.
What the data covers
- Model tags (8)
- llama3.2:3b 3.2B · Q4_K_M 9 configurations
- qwen2.5-coder:7b 7.6B · Q4_K_M 2 configurations
- qwen2.5:7b 7.6B · Q4_K_M 2 configurations
- qwen2.5-coder:14b 14.8B · Q4_K_M 2 configurations
- qwen2.5:14b 14.8B · Q4_K_M 8 configurations
- gpt-oss:20b 20.9B · MXFP4 10 configurations
- devstral:24b 23.6B · Q4_K_M 1 configuration
- qwen3-coder:30b 30.5B · Q4_K_M 2 configurations
- Quantizations (2)
- MXFP4 10
- Q4_K_M 26
No model was measured at more than one quantization.
- Context lengths (4)
- 4,096 24
- 8,192 3
- 16,384 8
- 131,072 1
- Runtimes (4)
- llama.cpp b10507 3
- LM Studio 0.4.21 6
- Ollama 0.32.1 20
- Ollama 0.32.15 7
- Ran partly off the card (4)
- Power runs (4)
- llama3.2:3b stock limit
- qwen2.5:7b stock limit
- qwen2.5:14b stock limit
- gpt-oss:20b stock limit
What it does not cover yet
- One quantization per model. I cannot show the same model at three quantizations stepping down the card, because each model was measured at one.
- No dense model past the cliff. The runs that left the card are a 3B model at an extreme context, a mixture-of-experts model that never fit, and one LM Studio default that offloaded part of a model that otherwise fits. A dense model that fits at one context and spills at the next is the row this map most needs.
- A gap between 16,384 and 131,072 context. One run sits at the far end. Nothing was measured at 32K or 64K.
- Energy at stock only. No undervolt, no reduced power limit, and no energy figure for a run that spilled.
- One card. Another 16 GB card, or this one with a different desktop load, will read differently.
Method, exactly
I did not run anything new for this page. It reads files I had already published and puts them on one axis.
The throughput rows come from my capture script. For each model, context and prompt size it calls Ollama’s /api/generate three times at temperature 0, seed 42 and a 256-token output, with a unique prefix on every prompt so each run pays a full prefill. Decode and prefill rates are Ollama’s own eval_count over eval_duration and prompt_eval_count over prompt_eval_duration, and the CSV keeps the median of the three. After each cell the script asks nvidia-smi for memory.used and stores it as vram_resident_mib. That one reading is the pin’s position.
That reading is the whole card. It includes the desktop, and it includes any model Ollama had not yet unloaded, which is how one Llama 3.2 3B row came to read 15,682 MiB. The runtime comparison captures used a different script that took the median memory reading across runs and, in the later capture, also read the card before each load.
Each throughput configuration was measured with two prompt sizes. The pin is the first of the two, the shorter prompt, and both are in the record. The model names differ between files, so I join them on the name with punctuation and a trailing quantization tag removed. Only the throughput CSV has a quantization column; where another file’s row shows a quantization, the record says it was taken from the same model there.
The card total, 16,303 MiB, is the nvidia-smi total written into the raw capture file from 2026-08-26. Headroom, share of the card, GiB, the memory a load added, the changes between two rows and the per-thousand-token energy are computed when the site is built, from the measured values beside them, and each one is shown with its formula. Your browser does no arithmetic on them.
Download the files
- rtx-5080-llm-throughput.csv34 rows. Decode, prefill, load time and whole-card memory for each model, context and prompt size on Ollama. Raw capture JSON and the capture script are listed in its manifest.
- llm-server-compare-2026-08.csv6 rows. Ollama beside llama.cpp on three models, with its settings and capture date in the footer notes.
- llm-server-compare-2026-08-26-lmstudio.csv12 rows. Ollama beside LM Studio on three models, at default and normalized settings, with the memory reading before each load.
- ollama-256k-slider-3b.jsonOne raw capture: the 131,072-context run, its five decode runs and the card's reported total. Described in its manifest.
- rtx-5080-llm-power-efficiency.csv4 rows. Board power, decode rate and joules per token at the stock power limit. The one-per-second telemetry is listed in its manifest.
All of it is CC BY 4.0. Attribute as “TechFuelHQ RTX 5080 16 GB VRAM cliff map” with a link to this page. To cite one configuration, open its row in the list and copy its link; the address ends in that row’s own id.
Correction log
No corrections so far. The datasets this page reads keep their own logs: the throughput dataset, the runtime comparison dataset and the power efficiency dataset. If a row here is wrong, write to contact@techfuelhq.com and I will log the change in this section with its date.
Frequently asked questions
Does a 20B model fit in 16 GB of VRAM?
How much VRAM does a longer context cost?
What happens when a model does not fit in 16 GB?
Is the VRAM figure the model alone?
Do these numbers hold for other 16 GB cards or current drivers?
How much energy does a thousand tokens take on this card?
Corrections
- Last updated
- Methodology
- See our methodology for research and review standards.
- Corrections
- Spotted an error? Email a correction with the affected line and what you found. The draft includes this page’s address. Verified changes are recorded here. Contact details.