Quick answer

Measured on my RTX 5080, which reports 16,303 MiB: gpt-oss:20b held 14,067 MiB and decoded at 190 tokens per second on Ollama 0.32.1 (2026-08-12). Qwen3-Coder 30B does not fit whole and ran at 62. Llama 3.2 3B fell from 351 to 103 tokens per second when a 131,072-token context spilled it. Each pin has its date and versions.

Dataset ID
techfuelhq-rtx-5080-16gb-vram-cliff
Version
1.0.0
Format
CSV, JSON
Published
Page updated
Contents

At a glance

Class
Data

Provenance

  • Nov 17, 2026Published

The map

Every pin is a local language model I ran on my own retail RTX 5080, placed by the memory in use on the whole card and the decode speed that came with it. Nothing here is estimated. For a card I do not own or a setup I have not run, the LLM VRAM calculator and the LLM speed calculator estimate, and the VRAM tiers guide covers other card sizes. This page only reports.

  • Measured. Copied from a capture file, shown with its capture date.
  • Computed. Arithmetic on measured values, shown with its formula.

The map

Across: memory in use on the whole card. Up: decode speed.

  • A measured configuration
  • Ran partly off the card
  • Reading includes another model
  • Same setup, context raised

Every pin is one row from a capture file. Nothing between two pins was measured, and nothing here is an estimate.

Readout

Nothing selected

Card total
16,303 MiB
Measured , nvidia-smi total in the raw capture
Configurations
36
Rows pinned on the map
Captured
2026-08-12
to 2026-08-26
First and last capture date
Model tags
8
Quantizations
2
Context lengths
4
Runtime versions
4

For a card or a setup that is not on this map, the LLM VRAM calculator estimates. This page does not.

Every measured configuration

  1. llama3.2:3b Q4_K_M · 4,096 ctx Ollama 0.32.15 defaults 3,760 MiB 351.22 tok/s
    Full measured record

    llama3.2:3b Q4_K_M · 4,096 context · Ollama 0.32.15 · defaults

    Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    3,760 MiB
    Measured
    Decode
    351.22 tok/s
    Measured
    Left on the card
    12,543 MiB
    Computed: 16,303 − 3,760
    Share of the card
    23.1%
    Computed: 3,760 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    ollama 0.32.15
    Engine
    vendored llama.cpp 9d77fa172
    Condition
    defaults
    Model name in the file
    llama3.2:3b
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096
    Runs
    5
    Decode, median (server counters)
    351.22 tok/s
    Decode, median (wall clock)
    345.19 tok/s
    Each run
    351.15 · 350.9 · 351.86 · 351.32 · 351.22 tok/s
    Prefill, long prompt, median
    17802.7 tok/s at 434 prompt tokens
    Time to first token
    Empty in this file
    Whole-card memory, median
    3760 MiB
    Whole-card memory before the load
    1053 MiB
    Added by loading the model
    2,707 MiB Computed: 3,760 − 1,053
    Load
    2.67 s
    Cold first request
    2.79 s
    App RAM while loaded
    1234.0 MB
    Driver
    Not recorded in this file
    Resident in GiB
    3.67 GiB Computed: 3,760 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  2. llama3.2:3b Q4_K_M · 4,096 ctx Ollama 0.32.15 normalized 3,760 MiB 350.52 tok/s
    Full measured record

    llama3.2:3b Q4_K_M · 4,096 context · Ollama 0.32.15 · normalized

    Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    3,760 MiB
    Measured
    Decode
    350.52 tok/s
    Measured
    Left on the card
    12,543 MiB
    Computed: 16,303 − 3,760
    Share of the card
    23.1%
    Computed: 3,760 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    ollama 0.32.15
    Engine
    vendored llama.cpp 9d77fa172
    Condition
    normalized
    Model name in the file
    llama3.2:3b
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096
    Runs
    5
    Decode, median (server counters)
    350.52 tok/s
    Decode, median (wall clock)
    338.11 tok/s
    Each run
    349.35 · 350.38 · 350.52 · 350.63 · 350.96 tok/s
    Prefill, long prompt, median
    17862.3 tok/s at 432 prompt tokens
    Time to first token
    Empty in this file
    Whole-card memory, median
    3760 MiB
    Whole-card memory before the load
    1057 MiB
    Added by loading the model
    2,703 MiB Computed: 3,760 − 1,057
    Load
    2.27 s
    Cold first request
    2.38 s
    App RAM while loaded
    1239.6 MB
    Driver
    Not recorded in this file
    Resident in GiB
    3.67 GiB Computed: 3,760 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  3. llama3.2:3b Q4_K_M · 4,096 ctx LM Studio 0.4.21 normalized 3,808 MiB 316.00 tok/s
    Full measured record

    llama3.2:3b Q4_K_M · 4,096 context · LM Studio 0.4.21 · normalized

    Measured at 3,808 MiB resident on with LM Studio 0.4.21. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    3,808 MiB
    Measured
    Decode
    316.00 tok/s
    Measured
    Left on the card
    12,495 MiB
    Computed: 16,303 − 3,808
    Share of the card
    23.4%
    Computed: 3,808 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    LM Studio 0.4.21
    Engine
    llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
    Condition
    normalized
    Model name in the file
    llama-3.2-3b-q4km
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096
    Runs
    5
    Decode, median (server counters)
    316.0 tok/s
    Decode, median (wall clock)
    309.91 tok/s
    Each run
    310.55 · 316.0 · 315.86 · 316.23 · 316.6 tok/s
    Prefill, long prompt, median
    20336.7 tok/s at 442 prompt tokens
    Time to first token, median
    9.2 ms
    Whole-card memory, median
    3808 MiB
    Whole-card memory before the load
    1103 MiB
    Added by loading the model
    2,705 MiB Computed: 3,808 − 1,103
    Load
    2.76 s
    Cold first request
    0.07 s
    App RAM while loaded
    3469.6 MB
    Driver
    Not recorded in this file
    Resident in GiB
    3.72 GiB Computed: 3,808 ÷ 1,024

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  4. llama3.2:3b Q4_K_M · 8,192 ctx LM Studio 0.4.21 defaults 4,260 MiB 314.86 tok/s
    Full measured record

    llama3.2:3b Q4_K_M · 8,192 context · LM Studio 0.4.21 · defaults

    Measured at 4,260 MiB resident on with LM Studio 0.4.21. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    4,260 MiB
    Measured
    Decode
    314.86 tok/s
    Measured
    Left on the card
    12,043 MiB
    Computed: 16,303 − 4,260
    Share of the card
    26.1%
    Computed: 4,260 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    LM Studio 0.4.21
    Engine
    llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
    Condition
    defaults
    Model name in the file
    llama-3.2-3b-q4km
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    8192
    Runs
    5
    Decode, median (server counters)
    314.86 tok/s
    Decode, median (wall clock)
    308.68 tok/s
    Each run
    273.28 · 312.54 · 316.75 · 315.22 · 314.86 tok/s
    Prefill, long prompt, median
    20341.8 tok/s at 440 prompt tokens
    Time to first token, median
    9.4 ms
    Whole-card memory, median
    4260 MiB
    Whole-card memory before the load
    1122 MiB
    Added by loading the model
    3,138 MiB Computed: 4,260 − 1,122
    Load
    2.75 s
    Cold first request
    0.07 s
    App RAM while loaded
    3471.9 MB
    Driver
    Not recorded in this file
    Resident in GiB
    4.16 GiB Computed: 4,260 ÷ 1,024

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  5. llama3.2:3b Q4_K_M · 4,096 ctx llama.cpp b10507 4,339 MiB 338.22 tok/s
    Full measured record

    llama3.2:3b Q4_K_M · 4,096 context · llama.cpp b10507

    Measured at 4,339 MiB resident on with llama.cpp b10507. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    4,339 MiB
    Measured
    Decode
    338.22 tok/s
    Measured
    Left on the card
    11,964 MiB
    Computed: 16,303 − 4,339
    Share of the card
    26.6%
    Computed: 4,339 ÷ 16,303

    Full measured row

    Captured
    from the file's footer notes
    Runtime
    llama.cpp b10507
    Model name in the file
    llama3.2:3b
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096 from the file's settings note
    Offload
    Full offload from the file's settings note
    Runs
    5
    Decode, median (server counters)
    338.22 tok/s
    Decode, median (wall clock)
    336.01 tok/s
    Each run
    339.2 · 338.2 · 335.0 · 338.2 · 339.7 tok/s
    Prefill, short prompt, median
    5321.0 tok/s
    Prefill, long prompt
    16809 tok/s at 407 prompt tokens
    Whole-card memory, median
    4339 MiB
    Load
    2.5 s
    Desktop baseline at bench start
    1642 MiB one reading for the whole capture
    Driver
    Not recorded in this file
    Resident in GiB
    4.24 GiB Computed: 4,339 ÷ 1,024

    Source file: llm-server-compare-2026-08.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  6. llama3.2:3b Q4_K_M · 4,096 ctx Ollama 0.32.1 4,343 MiB 318.51 tok/s
    Full measured record

    llama3.2:3b Q4_K_M · 4,096 context · Ollama 0.32.1

    Measured at 4,343 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    4,343 MiB
    Measured
    Decode
    318.51 tok/s
    Measured
    Left on the card
    11,960 MiB
    Computed: 16,303 − 4,343
    Share of the card
    26.6%
    Computed: 4,343 ÷ 16,303

    Full measured row

    Captured
    from the file's footer notes
    Runtime
    ollama 0.32.1
    Model name in the file
    llama3.2:3b
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096 from the file's settings note
    Offload
    Full offload from the file's settings note
    Runs
    5
    Decode, median (server counters)
    318.51 tok/s
    Decode, median (wall clock)
    313.25 tok/s
    Each run
    319.0 · 318.7 · 318.2 · 318.5 · 317.5 tok/s
    Prefill, short prompt, median
    7038.1 tok/s
    Prefill, long prompt
    17444 tok/s at 433 prompt tokens
    Whole-card memory, median
    4343 MiB
    Load
    3.2 s
    Desktop baseline at bench start
    1642 MiB one reading for the whole capture
    Driver
    Not recorded in this file
    Resident in GiB
    4.24 GiB Computed: 4,343 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: llm-server-compare-2026-08.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  7. llama3.2:3b Q4_K_M · 16,384 ctx Ollama 0.32.1 5,495 MiB 312.89 tok/s
    Full measured record

    llama3.2:3b Q4_K_M · 16,384 context · Ollama 0.32.1

    Measured at 5,495 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    5,495 MiB
    Measured
    Decode
    312.89 tok/s
    Measured
    Left on the card
    10,808 MiB
    Computed: 16,303 − 5,495
    Share of the card
    33.7%
    Computed: 5,495 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    llama3.2:3b digest a80c4f17acd5
    Parameters
    3.2B
    Quantization
    Q4_K_M
    Context (num_ctx)
    16384
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    471
    Repetitions
    3
    Decode, median
    312.89 tok/s
    Prefill, median
    17885.62 tok/s
    Load, median
    162.9 ms
    Whole-card memory after the cell
    5495 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1657
    Repetitions
    3
    Decode, median
    298.59 tok/s
    Prefill, median
    19397.36 tok/s
    Load, median
    158.0 ms
    Whole-card memory after the cell
    5501 MiB
    Resident in GiB
    5.37 GiB Computed: 5,495 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  8. qwen2.5-coder:7b Q4_K_M · 4,096 ctx Ollama 0.32.1 5,671 MiB 173.73 tok/s
    Full measured record

    qwen2.5-coder:7b Q4_K_M · 4,096 context · Ollama 0.32.1

    Measured at 5,671 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    5,671 MiB
    Measured
    Decode
    173.73 tok/s
    Measured
    Left on the card
    10,632 MiB
    Computed: 16,303 − 5,671
    Share of the card
    34.8%
    Computed: 5,671 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen2.5-coder:7b digest dae161e27b0e
    Parameters
    7.6B
    Quantization
    Q4_K_M
    Context (num_ctx)
    4096
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    488
    Repetitions
    3
    Decode, median
    173.73 tok/s
    Prefill, median
    9651.9 tok/s
    Load, median
    159.1 ms
    Whole-card memory after the cell
    5671 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1368
    Repetitions
    3
    Decode, median
    171.73 tok/s
    Prefill, median
    9506.54 tok/s
    Load, median
    159.6 ms
    Whole-card memory after the cell
    5671 MiB
    Resident in GiB
    5.54 GiB Computed: 5,671 ÷ 1,024

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  9. qwen2.5:7b Q4_K_M · 4,096 ctx Ollama 0.32.1 6,190 MiB 180.02 tok/s
    Full measured record

    qwen2.5:7b Q4_K_M · 4,096 context · Ollama 0.32.1

    Measured at 6,190 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    6,190 MiB
    Measured
    Decode
    180.02 tok/s
    Measured
    Left on the card
    10,113 MiB
    Computed: 16,303 − 6,190
    Share of the card
    38.0%
    Computed: 6,190 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen2.5:7b digest 845dbda0ea48
    Parameters
    7.6B
    Quantization
    Q4_K_M
    Context (num_ctx)
    4096
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    488
    Repetitions
    3
    Decode, median
    180.02 tok/s
    Prefill, median
    10152.28 tok/s
    Load, median
    129.8 ms
    Whole-card memory after the cell
    6190 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1674
    Repetitions
    3
    Decode, median
    176.79 tok/s
    Prefill, median
    9672.39 tok/s
    Load, median
    123.4 ms
    Whole-card memory after the cell
    6190 MiB
    Resident in GiB
    6.04 GiB Computed: 6,190 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of qwen2.5:7b on (UTC), stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokens Computed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  10. qwen2.5-coder:7b Q4_K_M · 16,384 ctx Ollama 0.32.1 6,503 MiB 174.58 tok/s
    Full measured record

    qwen2.5-coder:7b Q4_K_M · 16,384 context · Ollama 0.32.1

    Measured at 6,503 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    6,503 MiB
    Measured
    Decode
    174.58 tok/s
    Measured
    Left on the card
    9,800 MiB
    Computed: 16,303 − 6,503
    Share of the card
    39.9%
    Computed: 6,503 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen2.5-coder:7b digest dae161e27b0e
    Parameters
    7.6B
    Quantization
    Q4_K_M
    Context (num_ctx)
    16384
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    488
    Repetitions
    3
    Decode, median
    174.58 tok/s
    Prefill, median
    9471.68 tok/s
    Load, median
    145.9 ms
    Whole-card memory after the cell
    6503 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1368
    Repetitions
    3
    Decode, median
    171.41 tok/s
    Prefill, median
    9825.47 tok/s
    Load, median
    178.3 ms
    Whole-card memory after the cell
    6515 MiB
    Resident in GiB
    6.35 GiB Computed: 6,503 ÷ 1,024

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  11. qwen2.5:7b Q4_K_M · 16,384 ctx Ollama 0.32.1 7,022 MiB 179.66 tok/s
    Full measured record

    qwen2.5:7b Q4_K_M · 16,384 context · Ollama 0.32.1

    Measured at 7,022 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    7,022 MiB
    Measured
    Decode
    179.66 tok/s
    Measured
    Left on the card
    9,281 MiB
    Computed: 16,303 − 7,022
    Share of the card
    43.1%
    Computed: 7,022 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen2.5:7b digest 845dbda0ea48
    Parameters
    7.6B
    Quantization
    Q4_K_M
    Context (num_ctx)
    16384
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    488
    Repetitions
    3
    Decode, median
    179.66 tok/s
    Prefill, median
    10133.1 tok/s
    Load, median
    127.9 ms
    Whole-card memory after the cell
    7022 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1674
    Repetitions
    3
    Decode, median
    176.61 tok/s
    Prefill, median
    9984.37 tok/s
    Load, median
    127.7 ms
    Whole-card memory after the cell
    7034 MiB
    Resident in GiB
    6.86 GiB Computed: 7,022 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of qwen2.5:7b on (UTC), stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokens Computed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  12. qwen2.5-coder:14b Q4_K_M · 4,096 ctx Ollama 0.32.1 10,169 MiB 94.89 tok/s
    Full measured record

    qwen2.5-coder:14b Q4_K_M · 4,096 context · Ollama 0.32.1

    Measured at 10,169 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    10,169 MiB
    Measured
    Decode
    94.89 tok/s
    Measured
    Left on the card
    6,134 MiB
    Computed: 16,303 − 10,169
    Share of the card
    62.4%
    Computed: 10,169 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen2.5-coder:14b digest 9ec8897f747e
    Parameters
    14.8B
    Quantization
    Q4_K_M
    Context (num_ctx)
    4096
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    488
    Repetitions
    3
    Decode, median
    94.89 tok/s
    Prefill, median
    5271.63 tok/s
    Load, median
    219.8 ms
    Whole-card memory after the cell
    10169 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1368
    Repetitions
    3
    Decode, median
    93.8 tok/s
    Prefill, median
    4942.3 tok/s
    Load, median
    159.4 ms
    Whole-card memory after the cell
    10171 MiB
    Resident in GiB
    9.93 GiB Computed: 10,169 ÷ 1,024

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  13. qwen2.5:14b Q4_K_M · 4,096 ctx Ollama 0.32.15 defaults 10,356 MiB 102.25 tok/s
    Full measured record

    qwen2.5:14b Q4_K_M · 4,096 context · Ollama 0.32.15 · defaults

    Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    10,356 MiB
    Measured
    Decode
    102.25 tok/s
    Measured
    Left on the card
    5,947 MiB
    Computed: 16,303 − 10,356
    Share of the card
    63.5%
    Computed: 10,356 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    ollama 0.32.15
    Engine
    vendored llama.cpp 9d77fa172
    Condition
    defaults
    Model name in the file
    qwen2.5:14b
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096
    Runs
    5
    Decode, median (server counters)
    102.25 tok/s
    Decode, median (wall clock)
    101.06 tok/s
    Each run
    102.13 · 102.34 · 102.25 · 102.3 · 102.25 tok/s
    Prefill, long prompt, median
    4983.8 tok/s at 453 prompt tokens
    Time to first token
    Empty in this file
    Whole-card memory, median
    10356 MiB
    Whole-card memory before the load
    1053 MiB
    Added by loading the model
    9,303 MiB Computed: 10,356 − 1,053
    Load
    5.51 s
    Cold first request
    5.67 s
    App RAM while loaded
    1604.1 MB
    Driver
    Not recorded in this file
    Resident in GiB
    10.11 GiB Computed: 10,356 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  14. qwen2.5:14b Q4_K_M · 4,096 ctx Ollama 0.32.15 normalized 10,356 MiB 102.21 tok/s
    Full measured record

    qwen2.5:14b Q4_K_M · 4,096 context · Ollama 0.32.15 · normalized

    Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    10,356 MiB
    Measured
    Decode
    102.21 tok/s
    Measured
    Left on the card
    5,947 MiB
    Computed: 16,303 − 10,356
    Share of the card
    63.5%
    Computed: 10,356 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    ollama 0.32.15
    Engine
    vendored llama.cpp 9d77fa172
    Condition
    normalized
    Model name in the file
    qwen2.5:14b
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096
    Runs
    5
    Decode, median (server counters)
    102.21 tok/s
    Decode, median (wall clock)
    101.23 tok/s
    Each run
    102.21 · 102.21 · 102.17 · 102.21 · 102.21 tok/s
    Prefill, long prompt, median
    5038.3 tok/s at 452 prompt tokens
    Time to first token
    Empty in this file
    Whole-card memory, median
    10356 MiB
    Whole-card memory before the load
    1053 MiB
    Added by loading the model
    9,303 MiB Computed: 10,356 − 1,053
    Load
    5.52 s
    Cold first request
    5.68 s
    App RAM while loaded
    1617.6 MB
    Driver
    Not recorded in this file
    Resident in GiB
    10.11 GiB Computed: 10,356 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  15. qwen2.5:14b Q4_K_M · 4,096 ctx LM Studio 0.4.21 normalized 10,405 MiB 97.63 tok/s
    Full measured record

    qwen2.5:14b Q4_K_M · 4,096 context · LM Studio 0.4.21 · normalized

    Measured at 10,405 MiB resident on with LM Studio 0.4.21. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    10,405 MiB
    Measured
    Decode
    97.63 tok/s
    Measured
    Left on the card
    5,898 MiB
    Computed: 16,303 − 10,405
    Share of the card
    63.8%
    Computed: 10,405 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    LM Studio 0.4.21
    Engine
    llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
    Condition
    normalized
    Model name in the file
    qwen2.5-14b-q4km
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096
    Runs
    5
    Decode, median (server counters)
    97.63 tok/s
    Decode, median (wall clock)
    96.62 tok/s
    Each run
    96.89 · 97.75 · 97.46 · 97.79 · 97.63 tok/s
    Prefill, long prompt, median
    5566.5 tok/s at 453 prompt tokens
    Time to first token, median
    24.7 ms
    Whole-card memory, median
    10405 MiB
    Whole-card memory before the load
    1106 MiB
    Added by loading the model
    9,299 MiB Computed: 10,405 − 1,106
    Load
    7.68 s
    Cold first request
    0.16 s
    App RAM while loaded
    10411.1 MB
    Driver
    Not recorded in this file
    Resident in GiB
    10.16 GiB Computed: 10,405 ÷ 1,024

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  16. qwen2.5:14b Q4_K_M · 4,096 ctx Ollama 0.32.1 10,841 MiB 96.65 tok/s
    Full measured record

    qwen2.5:14b Q4_K_M · 4,096 context · Ollama 0.32.1

    Measured at 10,841 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    10,841 MiB
    Measured
    Decode
    96.65 tok/s
    Measured
    Left on the card
    5,462 MiB
    Computed: 16,303 − 10,841
    Share of the card
    66.5%
    Computed: 10,841 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen2.5:14b digest 7cdf5a0187d5
    Parameters
    14.8B
    Quantization
    Q4_K_M
    Context (num_ctx)
    4096
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    488
    Repetitions
    3
    Decode, median
    96.65 tok/s
    Prefill, median
    5202.23 tok/s
    Load, median
    182.5 ms
    Whole-card memory after the cell
    10841 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1674
    Repetitions
    3
    Decode, median
    94.48 tok/s
    Prefill, median
    4876.41 tok/s
    Load, median
    125.8 ms
    Whole-card memory after the cell
    10855 MiB
    Resident in GiB
    10.59 GiB Computed: 10,841 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  17. qwen2.5:14b Q4_K_M · 4,096 ctx Ollama 0.32.1 10,941 MiB 99.43 tok/s
    Full measured record

    qwen2.5:14b Q4_K_M · 4,096 context · Ollama 0.32.1

    Measured at 10,941 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    10,941 MiB
    Measured
    Decode
    99.43 tok/s
    Measured
    Left on the card
    5,362 MiB
    Computed: 16,303 − 10,941
    Share of the card
    67.1%
    Computed: 10,941 ÷ 16,303

    Full measured row

    Captured
    from the file's footer notes
    Runtime
    ollama 0.32.1
    Model name in the file
    qwen2.5:14b
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096 from the file's settings note
    Offload
    Full offload from the file's settings note
    Runs
    5
    Decode, median (server counters)
    99.43 tok/s
    Decode, median (wall clock)
    98.59 tok/s
    Each run
    99.5 · 99.5 · 99.4 · 99.4 · 99.3 tok/s
    Prefill, short prompt, median
    3151.4 tok/s
    Prefill, long prompt
    5004 tok/s at 451 prompt tokens
    Whole-card memory, median
    10941 MiB
    Load
    5.6 s
    Desktop baseline at bench start
    1642 MiB one reading for the whole capture
    Driver
    Not recorded in this file
    Resident in GiB
    10.68 GiB Computed: 10,941 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: llm-server-compare-2026-08.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  18. qwen2.5:14b Q4_K_M · 4,096 ctx llama.cpp b10507 10,942 MiB 101.38 tok/s
    Full measured record

    qwen2.5:14b Q4_K_M · 4,096 context · llama.cpp b10507

    Measured at 10,942 MiB resident on with llama.cpp b10507. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    10,942 MiB
    Measured
    Decode
    101.38 tok/s
    Measured
    Left on the card
    5,361 MiB
    Computed: 16,303 − 10,942
    Share of the card
    67.1%
    Computed: 10,942 ÷ 16,303

    Full measured row

    Captured
    from the file's footer notes
    Runtime
    llama.cpp b10507
    Model name in the file
    qwen2.5:14b
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096 from the file's settings note
    Offload
    Full offload from the file's settings note
    Runs
    5
    Decode, median (server counters)
    101.38 tok/s
    Decode, median (wall clock)
    101.29 tok/s
    Each run
    101.3 · 101.2 · 101.4 · 101.4 · 101.4 tok/s
    Prefill, short prompt, median
    2037.8 tok/s
    Prefill, long prompt
    4697 tok/s at 423 prompt tokens
    Whole-card memory, median
    10942 MiB
    Load
    8.1 s
    Desktop baseline at bench start
    1642 MiB one reading for the whole capture
    Driver
    Not recorded in this file
    Resident in GiB
    10.69 GiB Computed: 10,942 ÷ 1,024

    Source file: llm-server-compare-2026-08.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  19. qwen2.5:14b Q4_K_M · 8,192 ctx LM Studio 0.4.21 defaults 11,194 MiB 98.15 tok/s
    Full measured record

    qwen2.5:14b Q4_K_M · 8,192 context · LM Studio 0.4.21 · defaults

    Measured at 11,194 MiB resident on with LM Studio 0.4.21. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    11,194 MiB
    Measured
    Decode
    98.15 tok/s
    Measured
    Left on the card
    5,109 MiB
    Computed: 16,303 − 11,194
    Share of the card
    68.7%
    Computed: 11,194 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    LM Studio 0.4.21
    Engine
    llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
    Condition
    defaults
    Model name in the file
    qwen2.5-14b-q4km
    Quantization
    Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    8192
    Runs
    5
    Decode, median (server counters)
    98.15 tok/s
    Decode, median (wall clock)
    97.31 tok/s
    Each run
    97.36 · 98.14 · 98.17 · 98.15 · 98.17 tok/s
    Prefill, long prompt, median
    5195.8 tok/s at 454 prompt tokens
    Time to first token, median
    24.6 ms
    Whole-card memory, median
    11194 MiB
    Whole-card memory before the load
    1103 MiB
    Added by loading the model
    10,091 MiB Computed: 11,194 − 1,103
    Load
    7.38 s
    Cold first request
    0.15 s
    App RAM while loaded
    10406.8 MB
    Driver
    Not recorded in this file
    Resident in GiB
    10.93 GiB Computed: 11,194 ÷ 1,024

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  20. gpt-oss:20bRan partly off the card MXFP4 · 8,192 ctx LM Studio 0.4.21 defaults 12,310 MiB 186.26 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 8,192 context · LM Studio 0.4.21 · defaults

    Measured at 12,310 MiB resident on with LM Studio 0.4.21. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Ran partly off the card

    LM Studio's default automatic GPU offload was in effect, at its default 8,192 context. The dataset attributes the lower decode rate to that setting. The normalized row is the same model with full offload. Runtime comparison dataset.

    Resident, whole card
    12,310 MiB
    Measured
    Decode
    186.26 tok/s
    Measured
    Left on the card
    3,993 MiB
    Computed: 16,303 − 12,310
    Share of the card
    75.5%
    Computed: 12,310 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    LM Studio 0.4.21
    Engine
    llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
    Condition
    defaults
    Model name in the file
    gpt-oss-20b
    Quantization
    MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    8192
    Runs
    5
    Decode, median (server counters)
    186.26 tok/s
    Decode, median (wall clock)
    184.34 tok/s
    Each run
    183.42 · 186.26 · 186.84 · 186.98 · 185.4 tok/s
    Prefill, long prompt, median
    5241.3 tok/s at 474 prompt tokens
    Time to first token, median
    78.9 ms
    Whole-card memory, median
    12310 MiB
    Whole-card memory before the load
    1119 MiB
    Added by loading the model
    11,191 MiB Computed: 12,310 − 1,119
    Load
    8.43 s
    Cold first request
    0.36 s
    App RAM while loaded
    12917.9 MB
    Driver
    Not recorded in this file
    Resident in GiB
    12.02 GiB Computed: 12,310 ÷ 1,024

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  21. qwen2.5-coder:14b Q4_K_M · 16,384 ctx Ollama 0.32.1 12,485 MiB 94.49 tok/s
    Full measured record

    qwen2.5-coder:14b Q4_K_M · 16,384 context · Ollama 0.32.1

    Measured at 12,485 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    12,485 MiB
    Measured
    Decode
    94.49 tok/s
    Measured
    Left on the card
    3,818 MiB
    Computed: 16,303 − 12,485
    Share of the card
    76.6%
    Computed: 12,485 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen2.5-coder:14b digest 9ec8897f747e
    Parameters
    14.8B
    Quantization
    Q4_K_M
    Context (num_ctx)
    16384
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    488
    Repetitions
    3
    Decode, median
    94.49 tok/s
    Prefill, median
    4909.36 tok/s
    Load, median
    159.5 ms
    Whole-card memory after the cell
    12485 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1368
    Repetitions
    3
    Decode, median
    93.41 tok/s
    Prefill, median
    4842.15 tok/s
    Load, median
    153.2 ms
    Whole-card memory after the cell
    12487 MiB
    Resident in GiB
    12.19 GiB Computed: 12,485 ÷ 1,024

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  22. gpt-oss:20b MXFP4 · 4,096 ctx LM Studio 0.4.21 normalized 12,533 MiB 242.83 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 4,096 context · LM Studio 0.4.21 · normalized

    Measured at 12,533 MiB resident on with LM Studio 0.4.21. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    12,533 MiB
    Measured
    Decode
    242.83 tok/s
    Measured
    Left on the card
    3,770 MiB
    Computed: 16,303 − 12,533
    Share of the card
    76.9%
    Computed: 12,533 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    LM Studio 0.4.21
    Engine
    llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1
    Condition
    normalized
    Model name in the file
    gpt-oss-20b
    Quantization
    MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096
    Runs
    5
    Decode, median (server counters)
    242.83 tok/s
    Decode, median (wall clock)
    240.13 tok/s
    Each run
    242.83 · 242.82 · 243.14 · 241.74 · 244.35 tok/s
    Prefill, long prompt, median
    9617.7 tok/s at 474 prompt tokens
    Time to first token, median
    38.6 ms
    Whole-card memory, median
    12533 MiB
    Whole-card memory before the load
    1101 MiB
    Added by loading the model
    11,432 MiB Computed: 12,533 − 1,101
    Load
    8.43 s
    Cold first request
    0.27 s
    App RAM while loaded
    13176.6 MB
    Driver
    Not recorded in this file
    Resident in GiB
    12.24 GiB Computed: 12,533 ÷ 1,024

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  23. gpt-oss:20b MXFP4 · 4,096 ctx llama.cpp b10507 13,037 MiB 221.96 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 4,096 context · llama.cpp b10507

    Measured at 13,037 MiB resident on with llama.cpp b10507. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    13,037 MiB
    Measured
    Decode
    221.96 tok/s
    Measured
    Left on the card
    3,266 MiB
    Computed: 16,303 − 13,037
    Share of the card
    80.0%
    Computed: 13,037 ÷ 16,303

    Full measured row

    Captured
    from the file's footer notes
    Runtime
    llama.cpp b10507 (upstream GGUF)
    Model name in the file
    gpt-oss:20b
    Quantization
    MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096 from the file's settings note
    Offload
    Full offload from the file's settings note
    Runs
    5
    Decode, median (server counters)
    221.96 tok/s
    Decode, median (wall clock)
    220.53 tok/s
    Each run
    185.8 · 242.6 · 222.0 · 231.9 · 175.3 tok/s
    Prefill, short prompt, median
    1078.7 tok/s
    Prefill, long prompt
    Empty in this file
    Whole-card memory, median
    13037 MiB
    Load
    19.8 s
    Desktop baseline at bench start
    1642 MiB one reading for the whole capture
    Driver
    Not recorded in this file
    Resident in GiB
    12.73 GiB Computed: 13,037 ÷ 1,024

    Source file: llm-server-compare-2026-08.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  24. qwen2.5:14b Q4_K_M · 16,384 ctx Ollama 0.32.1 13,157 MiB 97.02 tok/s
    Full measured record

    qwen2.5:14b Q4_K_M · 16,384 context · Ollama 0.32.1

    Measured at 13,157 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    13,157 MiB
    Measured
    Decode
    97.02 tok/s
    Measured
    Left on the card
    3,146 MiB
    Computed: 16,303 − 13,157
    Share of the card
    80.7%
    Computed: 13,157 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen2.5:14b digest 7cdf5a0187d5
    Parameters
    14.8B
    Quantization
    Q4_K_M
    Context (num_ctx)
    16384
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    488
    Repetitions
    3
    Decode, median
    97.02 tok/s
    Prefill, median
    5225.96 tok/s
    Load, median
    131.3 ms
    Whole-card memory after the cell
    13157 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1674
    Repetitions
    3
    Decode, median
    94.58 tok/s
    Prefill, median
    4864.67 tok/s
    Load, median
    128.9 ms
    Whole-card memory after the cell
    13159 MiB
    Resident in GiB
    12.85 GiB Computed: 13,157 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokens Computed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  25. gpt-oss:20b MXFP4 · 4,096 ctx Ollama 0.32.1 13,381 MiB 181.50 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.1

    Measured at 13,381 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log.

    Resident, whole card
    13,381 MiB
    Measured
    Decode
    181.50 tok/s
    Measured
    Left on the card
    2,922 MiB
    Computed: 16,303 − 13,381
    Share of the card
    82.1%
    Computed: 13,381 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    gpt-oss:20b digest 17052f91a42e
    Parameters
    20.9B
    Quantization
    MXFP4
    Context (num_ctx)
    4096
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    503
    Repetitions
    3
    Decode, median
    181.5 tok/s
    Prefill, median
    9607.49 tok/s
    Load, median
    383.7 ms
    Whole-card memory after the cell
    13381 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1360
    Repetitions
    3
    Decode, median
    178.37 tok/s
    Prefill, median
    11001.37 tok/s
    Load, median
    375.9 ms
    Whole-card memory after the cell
    13385 MiB
    Resident in GiB
    13.07 GiB Computed: 13,381 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  26. gpt-oss:20b MXFP4 · 4,096 ctx Ollama 0.32.15 defaults 13,570 MiB 213.54 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.15 · defaults

    Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    13,570 MiB
    Measured
    Decode
    213.54 tok/s
    Measured
    Left on the card
    2,733 MiB
    Computed: 16,303 − 13,570
    Share of the card
    83.2%
    Computed: 13,570 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    ollama 0.32.15
    Engine
    vendored llama.cpp 9d77fa172
    Condition
    defaults
    Model name in the file
    gpt-oss:20b
    Quantization
    MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096
    Runs
    5
    Decode, median (server counters)
    213.54 tok/s
    Decode, median (wall clock)
    212.03 tok/s
    Each run
    213.22 · 213.64 · 213.48 · 213.54 · 213.76 tok/s
    Prefill, long prompt, median
    9239.4 tok/s at 475 prompt tokens
    Time to first token
    Empty in this file
    Whole-card memory, median
    13570 MiB
    Whole-card memory before the load
    1053 MiB
    Added by loading the model
    12,517 MiB Computed: 13,570 − 1,053
    Load
    18.56 s
    Cold first request
    23.67 s
    App RAM while loaded
    2138.8 MB
    Driver
    Not recorded in this file
    Resident in GiB
    13.25 GiB Computed: 13,570 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  27. gpt-oss:20b MXFP4 · 4,096 ctx Ollama 0.32.15 normalized 13,570 MiB 213.65 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.15 · normalized

    Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    13,570 MiB
    Measured
    Decode
    213.65 tok/s
    Measured
    Left on the card
    2,733 MiB
    Computed: 16,303 − 13,570
    Share of the card
    83.2%
    Computed: 13,570 ÷ 16,303

    Full measured row

    Captured
    from the file's name
    App
    ollama 0.32.15
    Engine
    vendored llama.cpp 9d77fa172
    Condition
    normalized
    Model name in the file
    gpt-oss:20b
    Quantization
    MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096
    Runs
    5
    Decode, median (server counters)
    213.65 tok/s
    Decode, median (wall clock)
    211.8 tok/s
    Each run
    213.62 · 213.65 · 213.68 · 213.25 · 213.73 tok/s
    Prefill, long prompt, median
    9187.1 tok/s at 474 prompt tokens
    Time to first token
    Empty in this file
    Whole-card memory, median
    13570 MiB
    Whole-card memory before the load
    1053 MiB
    Added by loading the model
    12,517 MiB Computed: 13,570 − 1,053
    Load
    8.05 s
    Cold first request
    8.36 s
    App RAM while loaded
    2152.1 MB
    Driver
    Not recorded in this file
    Resident in GiB
    13.25 GiB Computed: 13,570 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: llm-server-compare-2026-08-26-lmstudio.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  28. gpt-oss:20b MXFP4 · 16,384 ctx Ollama 0.32.1 13,683 MiB 183.00 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 16,384 context · Ollama 0.32.1

    Measured at 13,683 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log.

    Resident, whole card
    13,683 MiB
    Measured
    Decode
    183.00 tok/s
    Measured
    Left on the card
    2,620 MiB
    Computed: 16,303 − 13,683
    Share of the card
    83.9%
    Computed: 13,683 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    gpt-oss:20b digest 17052f91a42e
    Parameters
    20.9B
    Quantization
    MXFP4
    Context (num_ctx)
    16384
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    503
    Repetitions
    3
    Decode, median
    183.0 tok/s
    Prefill, median
    9092.22 tok/s
    Load, median
    347.7 ms
    Whole-card memory after the cell
    13683 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1360
    Repetitions
    3
    Decode, median
    178.34 tok/s
    Prefill, median
    10328.62 tok/s
    Load, median
    345.3 ms
    Whole-card memory after the cell
    13683 MiB
    Resident in GiB
    13.36 GiB Computed: 13,683 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  29. gpt-oss:20b MXFP4 · 4,096 ctx Ollama 0.32.1 14,067 MiB 190.14 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.1

    Measured at 14,067 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    14,067 MiB
    Measured
    Decode
    190.14 tok/s
    Measured
    Left on the card
    2,236 MiB
    Computed: 16,303 − 14,067
    Share of the card
    86.3%
    Computed: 14,067 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    gpt-oss:20b digest 17052f91a42e
    Parameters
    20.9B
    Quantization
    MXFP4
    Context (num_ctx)
    4096
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    503
    Repetitions
    3
    Decode, median
    190.14 tok/s
    Prefill, median
    9078.28 tok/s
    Load, median
    297.7 ms
    Whole-card memory after the cell
    14067 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1658
    Repetitions
    3
    Decode, median
    187.27 tok/s
    Prefill, median
    10464.66 tok/s
    Load, median
    279.5 ms
    Whole-card memory after the cell
    14060 MiB
    Resident in GiB
    13.74 GiB Computed: 14,067 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  30. gpt-oss:20b MXFP4 · 4,096 ctx Ollama 0.32.1 14,152 MiB 195.79 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 4,096 context · Ollama 0.32.1

    Measured at 14,152 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    14,152 MiB
    Measured
    Decode
    195.79 tok/s
    Measured
    Left on the card
    2,151 MiB
    Computed: 16,303 − 14,152
    Share of the card
    86.8%
    Computed: 14,152 ÷ 16,303

    Full measured row

    Captured
    from the file's footer notes
    Runtime
    ollama 0.32.1
    Model name in the file
    gpt-oss:20b
    Quantization
    MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
    Context
    4096 from the file's settings note
    Offload
    Full offload from the file's settings note
    Runs
    5
    Decode, median (server counters)
    195.79 tok/s
    Decode, median (wall clock)
    194.59 tok/s
    Each run
    195.8 · 195.7 · 196.3 · 196.1 · 195.7 tok/s
    Prefill, short prompt, median
    3744.3 tok/s
    Prefill, long prompt
    Empty in this file
    Whole-card memory, median
    14152 MiB
    Load
    8.6 s
    Desktop baseline at bench start
    1642 MiB one reading for the whole capture
    Driver
    Not recorded in this file
    Resident in GiB
    13.82 GiB Computed: 14,152 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: llm-server-compare-2026-08.csv (column vram_total_mib_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  31. gpt-oss:20b MXFP4 · 16,384 ctx Ollama 0.32.1 14,355 MiB 188.63 tok/s
    Full measured record

    gpt-oss:20b MXFP4 · 16,384 context · Ollama 0.32.1

    Measured at 14,355 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Resident, whole card
    14,355 MiB
    Measured
    Decode
    188.63 tok/s
    Measured
    Left on the card
    1,948 MiB
    Computed: 16,303 − 14,355
    Share of the card
    88.1%
    Computed: 14,355 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    gpt-oss:20b digest 17052f91a42e
    Parameters
    20.9B
    Quantization
    MXFP4
    Context (num_ctx)
    16384
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    503
    Repetitions
    3
    Decode, median
    188.63 tok/s
    Prefill, median
    9286.61 tok/s
    Load, median
    299.2 ms
    Whole-card memory after the cell
    14355 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1658
    Repetitions
    3
    Decode, median
    187.73 tok/s
    Prefill, median
    10554.32 tok/s
    Load, median
    280.2 ms
    Whole-card memory after the cell
    14355 MiB
    Resident in GiB
    14.02 GiB Computed: 14,355 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokens Computed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  32. devstral:24b Q4_K_M · 4,096 ctx Ollama 0.32.1 14,997 MiB 36.32 tok/s
    Full measured record

    devstral:24b Q4_K_M · 4,096 context · Ollama 0.32.1

    Measured at 14,997 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Measured at a 4,096 context only. The dataset reports that an 8K attempt spilled and decode collapsed, but no row for that attempt was published, so it has no pin here. Throughput dataset, coding models note.

    Resident, whole card
    14,997 MiB
    Measured
    Decode
    36.32 tok/s
    Measured
    Left on the card
    1,306 MiB
    Computed: 16,303 − 14,997
    Share of the card
    92.0%
    Computed: 14,997 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    devstral:24b digest 9bd74193e939
    Parameters
    23.6B
    Quantization
    Q4_K_M
    Context (num_ctx)
    4096
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    1694
    Repetitions
    3
    Decode, median
    36.32 tok/s
    Prefill, median
    9377.3 tok/s
    Load, median
    238.2 ms
    Whole-card memory after the cell
    14997 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    2597
    Repetitions
    3
    Decode, median
    34.9 tok/s
    Prefill, median
    5334.83 tok/s
    Load, median
    241.7 ms
    Whole-card memory after the cell
    14997 MiB
    Resident in GiB
    14.65 GiB Computed: 14,997 ÷ 1,024

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  33. qwen3-coder:30bRan partly off the card Q4_K_M · 16,384 ctx Ollama 0.32.1 15,003 MiB 51.62 tok/s
    Full measured record

    qwen3-coder:30b Q4_K_M · 16,384 context · Ollama 0.32.1

    Measured at 15,003 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Ran partly off the card

    The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note.

    Resident, whole card
    15,003 MiB
    Measured
    Decode
    51.62 tok/s
    Measured
    Left on the card
    1,300 MiB
    Computed: 16,303 − 15,003
    Share of the card
    92.0%
    Computed: 15,003 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen3-coder:30b digest 06c1097efce0
    Parameters
    30.5B
    Quantization
    Q4_K_M
    Context (num_ctx)
    16384
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    467
    Repetitions
    3
    Decode, median
    51.62 tok/s
    Prefill, median
    1524.77 tok/s
    Load, median
    219.9 ms
    Whole-card memory after the cell
    15003 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1347
    Repetitions
    3
    Decode, median
    51.65 tok/s
    Prefill, median
    1928.14 tok/s
    Load, median
    188.9 ms
    Whole-card memory after the cell
    15003 MiB
    Resident in GiB
    14.65 GiB Computed: 15,003 ÷ 1,024

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  34. llama3.2:3bRan partly off the card Q4_K_M · 131,072 ctx Ollama 0.32.15 defaults 15,084 MiB 102.68 tok/s
    Full measured record

    llama3.2:3b Q4_K_M · 131,072 context · Ollama 0.32.15 · defaults

    Measured at 15,084 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Ran partly off the card

    The capture's own ollama ps line reads 22%/78% CPU/GPU at context 131,072, so part of this model ran from system RAM.

    Ollama's app-level context slider was set to 256k, which this model clamps to 131,072. The dataset describes the result as the KV cache spilling to the CPU. Runtime comparison dataset.

    Resident, whole card
    15,084 MiB
    Measured
    Decode
    102.68 tok/s
    Measured
    Left on the card
    1,219 MiB
    Computed: 16,303 − 15,084
    Share of the card
    92.5%
    Computed: 15,084 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.15
    Condition
    defaults
    Schema
    techfuelhq:lmstudio-vs-ollama:1
    Model
    llama3.2:3b
    Quantization
    Q4_K_M. Not a field in this file. Taken from the same model in the throughput CSV.
    Context
    131072 from the capture's ollama ps line
    Processor split
    22% CPU / 78% GPU from the capture's ollama ps line
    Size reported by ollama ps
    18 GB
    Runs
    5
    Decode, median (server counters)
    102.68 tok/s
    Decode, median (wall clock)
    101.58 tok/s
    Each run
    97.57 · 99.85 · 103.59 · 102.68 · 103.21 tok/s
    Prefill, long prompt, median
    9998.8 tok/s at 434 prompt tokens
    Whole-card memory, median
    15084 MiB
    Whole-card memory before the load
    1067 MiB
    Added by loading the model
    14,017 MiB Computed: 15,084 − 1,067
    Load
    4.02 s
    Cold first request
    4.2 s
    App RAM while loaded
    4745.7 MB
    Resident in GiB
    14.73 GiB Computed: 15,084 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: ollama-256k-slider-3b.json (column vram_mb_median). Method and caveats: runtime comparison dataset.

    Full record in the list

  35. qwen3-coder:30bRan partly off the card Q4_K_M · 4,096 ctx Ollama 0.32.1 15,101 MiB 61.89 tok/s
    Full measured record

    qwen3-coder:30b Q4_K_M · 4,096 context · Ollama 0.32.1

    Measured at 15,101 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Ran partly off the card

    The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note.

    Resident, whole card
    15,101 MiB
    Measured
    Decode
    61.89 tok/s
    Measured
    Left on the card
    1,202 MiB
    Computed: 16,303 − 15,101
    Share of the card
    92.6%
    Computed: 15,101 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    qwen3-coder:30b digest 06c1097efce0
    Parameters
    30.5B
    Quantization
    Q4_K_M
    Context (num_ctx)
    4096
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    467
    Repetitions
    3
    Decode, median
    61.89 tok/s
    Prefill, median
    1744.18 tok/s
    Load, median
    197.7 ms
    Whole-card memory after the cell
    15101 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1347
    Repetitions
    3
    Decode, median
    61.55 tok/s
    Prefill, median
    2368.71 tok/s
    Load, median
    177.2 ms
    Whole-card memory after the cell
    15101 MiB
    Resident in GiB
    14.75 GiB Computed: 15,101 ÷ 1,024

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

  36. llama3.2:3bReading includes another model Q4_K_M · 4,096 ctx Ollama 0.32.1 15,682 MiB 311.96 tok/s
    Full measured record

    llama3.2:3b Q4_K_M · 4,096 context · Ollama 0.32.1

    Measured at 15,682 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

    Reading includes another model

    gpt-oss:20b was still loaded from the cell before this one, so this reading is the whole card with two models on it, not Llama 3.2 3B alone. The 16,384-context row from the same capture was read after the larger model had been evicted. Throughput dataset, VRAM note.

    Resident, whole card
    15,682 MiB
    Measured
    Decode
    311.96 tok/s
    Measured
    Left on the card
    621 MiB
    Computed: 16,303 − 15,682
    Share of the card
    96.2%
    Computed: 15,682 ÷ 16,303

    Full measured row

    Captured (UTC)
    GPU
    NVIDIA GeForce RTX 5080
    Driver
    610.88
    Runtime
    Ollama 0.32.1
    Method
    llm-throughput-ollama-v1
    Model
    llama3.2:3b digest a80c4f17acd5
    Parameters
    3.2B
    Quantization
    Q4_K_M
    Context (num_ctx)
    4096
    Prompt cell 1
    The row pinned on the map
    Prompt tokens
    471
    Repetitions
    3
    Decode, median
    311.96 tok/s
    Prefill, median
    18196.57 tok/s
    Load, median
    162.8 ms
    Whole-card memory after the cell
    15682 MiB
    Prompt cell 2
    Same configuration, longer prompt
    Prompt tokens
    1657
    Repetitions
    3
    Decode, median
    298.21 tok/s
    Prefill, median
    17681.08 tok/s
    Load, median
    156.2 ms
    Whole-card memory after the cell
    15682 MiB
    Resident in GiB
    15.31 GiB Computed: 15,682 ÷ 1,024

    Energy for this model, from its own power run

    A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokens Computed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

    Source file: rtx-5080-llm-throughput.csv (column vram_resident_mib). Method and caveats: RTX 5080 LLM throughput dataset.

    Full record in the list

How to read it

The map answers a narrow question: what did this model, at this context length, on this runtime version, take on a 16 GB card, and what speed came with it. Each record reads “measured at so many MiB resident on a date with a runtime version” because that is all a measurement can say. It is not a promise that the same model fits on your card today.

The line along the bottom is the card, from zero to the total it reports to nvidia-smi. A pin’s position across is the memory in use on the whole card while that model was loaded, Windows desktop included, and its height is the decode rate. Select a pin and the line fills to it in the measured colour. The hatched remainder is headroom, which I compute by subtracting.

A dashed connector joins the same model, runtime and capture at two context lengths. Flat means memory grew and speed held; a drop means the model stopped fitting. A ring is a run recorded as partly off the card. A diamond is a reading taken while another model was still loaded, so it overstates that model.

Where the cliff shows up

The cliff is not a memory number. In these files, a model that sits whole on the card decodes at the same speed whether it uses a third of the memory or nearly all of it. Speed falls when part of the model no longer fits and runs from system RAM instead.

Each row is the same model, runtime version and capture at two context lengths, and both ends are measured rows. Memory change is the second reading minus the first. Decode change is (second − first) ÷ first.

Model and runtime Context Resident MiB, measured Memory change, computed Decode tok/s, measured Decode change, computed
llama3.2:3b Ollama 0.32.15, defaults · 131,072: ran partly off the card 4,096 → 131,072 3,760 → 15,084 +11,324 351.22 → 102.68 −70.8%
qwen3-coder:30b Ollama 0.32.1 · 16,384: ran partly off the card 4,096 → 16,384 15,101 → 15,003 −98 61.89 → 51.62 −16.6%
gpt-oss:20b Ollama 0.32.1 · 4,096 → 16,384 14,067 → 14,355 +288 190.14 → 188.63 −0.8%
qwen2.5-coder:14b Ollama 0.32.1 · 4,096 → 16,384 10,169 → 12,485 +2,316 94.89 → 94.49 −0.4%
qwen2.5:7b Ollama 0.32.1 · 4,096 → 16,384 6,190 → 7,022 +832 180.02 → 179.66 −0.2%
llama3.2:3b Ollama 0.32.1 · 4,096: reading includes another model 4,096 → 16,384 15,682 → 5,495 Not comparable 311.96 → 312.89 +0.3%
qwen2.5:14b Ollama 0.32.1 · 4,096 → 16,384 10,841 → 13,157 +2,316 96.65 → 97.02 +0.4%
qwen2.5-coder:7b Ollama 0.32.1 · 4,096 → 16,384 5,671 → 6,503 +832 173.73 → 174.58 +0.5%
gpt-oss:20b Ollama 0.32.1 · 4,096 → 16,384 13,381 → 13,683 +302 181.50 → 183.00 +0.8%

The table has two rows where raising the context cost speed, and both are runs that left the card. Llama 3.2 3B is the small model in this set, and a large enough context still pushed it over. Qwen3-Coder 30B never fit whole, so its memory reading barely moved and its speed did.

Every other pair held its decode rate within one percent while the context quadrupled. What they paid was memory, and how much depended on the model: the two 14B Qwen models paid the most, and gpt-oss:20b, the largest of them by parameter count, paid the least.

What the files do not hold is a dense model caught in the act of falling off. The throughput dataset says Devstral 24B collapsed when I tried it at an 8K context, but that attempt was never written to the CSV, so there is no pin for it and I have not drawn one.

Energy per thousand tokens

  1. llama3.2:3b Q4_K_M · named “Llama 3.2 3B” in the file · run logged UTC

    833 J per 1,000 tokens Computed: 0.833 J per token × 1,000

    Decode, median of 3
    318.3 tok/s
    Board power, median while loaded
    265.2 W
    Tokens per joule, as filed
    1.2
    Joules per token, as filed
    0.833
    More from the power file
    Watt-hours per 1,000 tokens
    0.231 Wh Computed: 231.0 Wh per million ÷ 1,000
    Decode, slowest and fastest run
    315.5 to 318.5 tok/s
    Board power, peak
    307.78 W
    Memory in use, median
    3,590 MiB
    GPU temperature, median
    49.0 °C
    GPU load, median
    82.0%
    Telemetry samples while loaded
    9
  2. qwen2.5:7b Q4_K_M · named “Qwen 2.5 7B” in the file · run logged UTC

    1,886 J per 1,000 tokens Computed: 1.886 J per token × 1,000

    Decode, median of 3
    182.2 tok/s
    Board power, median while loaded
    343.6 W
    Tokens per joule, as filed
    0.53
    Joules per token, as filed
    1.886
    More from the power file
    Watt-hours per 1,000 tokens
    0.524 Wh Computed: 524.0 Wh per million ÷ 1,000
    Decode, slowest and fastest run
    182.1 to 182.3 tok/s
    Board power, peak
    346.08 W
    Memory in use, median
    5,665 MiB
    GPU temperature, median
    52.0 °C
    GPU load, median
    88.0%
    Telemetry samples while loaded
    14
  3. qwen2.5:14b Q4_K_M · named “Qwen 2.5 14B” in the file · run logged UTC

    3,154 J per 1,000 tokens Computed: 3.154 J per token × 1,000

    Decode, median of 3
    98.8 tok/s
    Board power, median while loaded
    311.6 W
    Tokens per joule, as filed
    0.317
    Joules per token, as filed
    3.154
    More from the power file
    Watt-hours per 1,000 tokens
    0.876 Wh Computed: 876.0 Wh per million ÷ 1,000
    Decode, slowest and fastest run
    98.6 to 98.8 tok/s
    Board power, peak
    319.69 W
    Memory in use, median
    10,171 MiB
    GPU temperature, median
    50.0 °C
    GPU load, median
    93.0%
    Telemetry samples while loaded
    27
  4. gpt-oss:20b MXFP4 · named “gpt-oss 20B” in the file · run logged UTC

    1,504 J per 1,000 tokens Computed: 1.504 J per token × 1,000

    Decode, median of 3
    194.4 tok/s
    Board power, median while loaded
    292.4 W
    Tokens per joule, as filed
    0.665
    Joules per token, as filed
    1.504
    More from the power file
    Watt-hours per 1,000 tokens
    0.418 Wh Computed: 418.0 Wh per million ÷ 1,000
    Decode, slowest and fastest run
    194.3 to 194.5 tok/s
    Board power, peak
    294.86 W
    Memory in use, median
    13,381 MiB
    GPU temperature, median
    51.0 °C
    GPU load, median
    84.0%
    Telemetry samples while loaded
    18

These four runs are a separate capture from the map: one short prompt, three iterations per model, board power sampled once a second and counted only while the GPU was at least half loaded. They were taken at the card’s stock 448 W power limit, which none of the runs reached. Board power is the graphics card alone, not the wall.

The power file records no context length, driver or runtime version, which is why these runs are not pinned on the map. Its dataset page gives the runtime as Ollama 0.32.1.

There is no undervolt or power-limit comparison here because I have not measured one for language models. My undervolt database holds one row for this card and it is a stock 3DMark baseline. Method and the idle-power finding are on the power efficiency dataset.

Versions on the bench, and how stale they are

Driver and runtime versions per capture, exactly as each file records them. Where a file is silent, the cell says so.

Captured File GPU driver Runtime versions Rows used here
rtx-5080-llm-throughput.csv 610.88 Ollama 0.32.1 8 configurations
llm-server-compare-2026-08.csv Not recorded in this file llama.cpp b10507, Ollama 0.32.1llama.cpp b10507: (upstream GGUF) 6 configurations
rtx-5080-llm-power-efficiency.csv Not recorded in this file Not recorded in this file 4 power rows
llm-server-compare-2026-08-26-lmstudio.csv Not recorded in this file LM Studio 0.4.21, Ollama 0.32.15Ollama 0.32.15: vendored llama.cpp 9d77fa172LM Studio 0.4.21: llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1 12 configurations
rtx-5080-llm-throughput.csv 610.88 Ollama 0.32.1 9 configurations
ollama-256k-slider-3b.json 610.88 Ollama 0.32.15 1 configuration

The same model at the same context on two Ollama versions. Decode change is (later − earlier) ÷ earlier.

Model Earlier row, measured Later row, measured Decode change, computed
llama3.2:3b4,096 context Ollama 0.32.1, : 318.51 tok/s Ollama 0.32.15, : 351.22 tok/s +10.3%
qwen2.5:14b4,096 context Ollama 0.32.1, : 99.43 tok/s Ollama 0.32.15, : 102.25 tok/s +2.8%
gpt-oss:20b4,096 context Ollama 0.32.1, : 195.79 tok/s Ollama 0.32.15, : 213.54 tok/s +9.1%

Those are the versions that were installed on the capture dates. Drivers and runtimes have both moved since, and I have not re-run this matrix on newer ones. Read every pin as true for its date and versions, and expect a current install to differ.

The second table is why I say that plainly. Two captures six days apart put the same three models on two Ollama versions, and all three decoded faster on the later one. The runtime comparison dataset notes that the earlier capture had less control over background load, so not all of that gap is the version. Either way, a few percent between runtimes does not survive an update.

What the data covers

Model tags (8)
  • llama3.2:3b 3.2B · Q4_K_M 9 configurations
  • qwen2.5-coder:7b 7.6B · Q4_K_M 2 configurations
  • qwen2.5:7b 7.6B · Q4_K_M 2 configurations
  • qwen2.5-coder:14b 14.8B · Q4_K_M 2 configurations
  • qwen2.5:14b 14.8B · Q4_K_M 8 configurations
  • gpt-oss:20b 20.9B · MXFP4 10 configurations
  • devstral:24b 23.6B · Q4_K_M 1 configuration
  • qwen3-coder:30b 30.5B · Q4_K_M 2 configurations
Quantizations (2)
  • MXFP4 10
  • Q4_K_M 26

No model was measured at more than one quantization.

Context lengths (4)
  • 4,096 24
  • 8,192 3
  • 16,384 8
  • 131,072 1
Runtimes (4)
  • llama.cpp b10507 3
  • LM Studio 0.4.21 6
  • Ollama 0.32.1 20
  • Ollama 0.32.15 7
Power runs (4)
  • llama3.2:3b stock limit
  • qwen2.5:7b stock limit
  • qwen2.5:14b stock limit
  • gpt-oss:20b stock limit

What it does not cover yet

  • One quantization per model. I cannot show the same model at three quantizations stepping down the card, because each model was measured at one.
  • No dense model past the cliff. The runs that left the card are a 3B model at an extreme context, a mixture-of-experts model that never fit, and one LM Studio default that offloaded part of a model that otherwise fits. A dense model that fits at one context and spills at the next is the row this map most needs.
  • A gap between 16,384 and 131,072 context. One run sits at the far end. Nothing was measured at 32K or 64K.
  • Energy at stock only. No undervolt, no reduced power limit, and no energy figure for a run that spilled.
  • One card. Another 16 GB card, or this one with a different desktop load, will read differently.

Method, exactly

I did not run anything new for this page. It reads files I had already published and puts them on one axis.

The throughput rows come from my capture script. For each model, context and prompt size it calls Ollama’s /api/generate three times at temperature 0, seed 42 and a 256-token output, with a unique prefix on every prompt so each run pays a full prefill. Decode and prefill rates are Ollama’s own eval_count over eval_duration and prompt_eval_count over prompt_eval_duration, and the CSV keeps the median of the three. After each cell the script asks nvidia-smi for memory.used and stores it as vram_resident_mib. That one reading is the pin’s position.

That reading is the whole card. It includes the desktop, and it includes any model Ollama had not yet unloaded, which is how one Llama 3.2 3B row came to read 15,682 MiB. The runtime comparison captures used a different script that took the median memory reading across runs and, in the later capture, also read the card before each load.

Each throughput configuration was measured with two prompt sizes. The pin is the first of the two, the shorter prompt, and both are in the record. The model names differ between files, so I join them on the name with punctuation and a trailing quantization tag removed. Only the throughput CSV has a quantization column; where another file’s row shows a quantization, the record says it was taken from the same model there.

The card total, 16,303 MiB, is the nvidia-smi total written into the raw capture file from 2026-08-26. Headroom, share of the card, GiB, the memory a load added, the changes between two rows and the per-thousand-token energy are computed when the site is built, from the measured values beside them, and each one is shown with its formula. Your browser does no arithmetic on them.

Download the files

All of it is CC BY 4.0. Attribute as “TechFuelHQ RTX 5080 16 GB VRAM cliff map” with a link to this page. To cite one configuration, open its row in the list and copy its link; the address ends in that row’s own id.

Correction log

No corrections so far. The datasets this page reads keep their own logs: the throughput dataset, the runtime comparison dataset and the power efficiency dataset. If a row here is wrong, write to contact@techfuelhq.com and I will log the change in this section with its date.

Frequently asked questions

Does a 20B model fit in 16 GB of VRAM?
On my card, gpt-oss:20b in MXFP4 did. I measured the whole card at 14,067 MiB with a 4,096-token context and 14,355 MiB at 16,384 on 2026-08-12 with Ollama 0.32.1, against a reported total of 16,303 MiB. Those readings include the Windows desktop. They describe that model file on that date and runtime version, not every 20B model or every build of this one.
How much VRAM does a longer context cost?
In these rows, going from a 4,096 to a 16,384-token context added 288 to 302 MiB on gpt-oss:20b, 832 MiB on both 7B Qwen models and 2,316 MiB on both 14B Qwen models. Decode speed moved by less than 1% in each of those pairs. Within one context setting, a prompt 1.5 to 3.5 times as long changed the memory reading by 14 MiB or less, so it is the context you allocate that costs memory, not the prompt you send.
What happens when a model does not fit in 16 GB?
It still runs, slower, with part of it in system RAM. The clearest row here is Llama 3.2 3B on Ollama 0.32.15: at a 4,096-token context it held 3,760 MiB and decoded at 351.2 tokens per second. With the context at 131,072 the capture recorded a 22%/78% CPU/GPU split, 15,084 MiB and 102.7 tokens per second. Qwen3-Coder 30B, which the throughput dataset records as partially offloaded, decoded at 61.9 tokens per second at 4,096 context and 51.6 at 16,384.
Is the VRAM figure the model alone?
No. Every memory figure on this page is nvidia-smi’s reading for the whole card, which includes the Windows desktop and anything else loaded. The Ollama and LM Studio capture also read the card before each load, between 1,053 and 1,122 MiB, so those rows show what loading the model added. One Llama 3.2 3B row is flagged because another model was still loaded when it was read.
Do these numbers hold for other 16 GB cards or current drivers?
They are one retail RTX 5080 on the driver and runtime versions each row lists, captured between 2026-08-12 and 2026-08-26. Another card, driver or runtime release can land differently. In this data the same three models decoded 2.8% to 10.3% faster in the Ollama 0.32.15 capture than in the 0.32.1 capture six days earlier, and the dataset notes that part of that gap may be background load. For another card or an unmeasured setup, the LLM VRAM calculator gives an estimate.
How much energy does a thousand tokens take on this card?
At the stock power limit, from the power dataset’s joules per token: 833 J per 1,000 tokens for Llama 3.2 3B, 1,504 J for gpt-oss 20B, 1,886 J for Qwen 2.5 7B and 3,154 J for Qwen 2.5 14B. That is board power only, from one short prompt. I have no LLM measurement at an undervolt or a reduced power limit yet, so the page shows none.

Corrections

Last updated
Methodology
See our methodology for research and review standards.
Corrections
Spotted an error? Email a correction with the affected line and what you found. The draft includes this page’s address. Verified changes are recorded here. Contact details.

About the author

I am Lowell K. Wood IV. I research, write and maintain everything on TechFuelHQ from St. Louis County, Missouri.