LLM VRAM Calculator

By .

Find which local AI models fit your GPU: pick card, model size, quantization, and context length to estimate VRAM (weights + KV cache) and which quant fits.

Layer and KV-head math checked 2026-08-24 against model configs; byte-unit definitions checked 2026-10-23.

Nominal GPU capacity

Preset numbers are treated as decimal GB planning budgets here, not measurements of free or allocatable GPU memory. Results estimate memory needs; they do not guarantee a successful model load.

The model

—
estimated requirement, shown in decimal GB and binary GiB
Weights
—
KV cache
—
Labeled capacity
—
Memory breakdown
Estimated total —
This model at each quantization (your GPU + context)
QuantbpwEstimated totalVerdict

Fits, Tight, and Won't fit compare the estimate with the labeled capacity entered above. Real usable memory varies by card, driver, model, runtime, and batch size; keep headroom and verify the load on the target system.

Pick your GPU, the model size you want to run, a quantization level, and a context length. The calculator estimates total VRAM as three parts: the model weights, the KV cache that holds your context, and a small runtime overhead. It converts the declared source units to bytes before comparing them, then shows decimal GB and binary GiB together.

The GPU menu carries nominal labeled capacity, treated here as a decimal GB planning budget. It does not read physical bytes from the card or measure memory that is free or allocatable after the driver, display, and runtime take their shares. Treat Fits, Tight, and Won’t fit as planning verdicts to verify on the target system.

Weights are the big, predictable cost. The selected weight payload is its parameter count times the bits per weight, divided by eight. An 8B selection at FP16 is 16.0 GB (14.9 GiB); at Q4_K_M it is 4.8 GB (4.5 GiB). Model files and measured residency can include other data, so this arithmetic remains an estimate of the weight payload.

The KV cache scales with context, not parameters. Every token in your context window is cached per layer, so a long context can quietly cost more memory than the weights on a small model. The tool estimates the cache from the attention shape of modern GQA models and lets you quantize it (Q8 or Q4) to claw back room.

Headroom matters. The verdict bands leave a margin: an estimate above 92 percent of the entered capacity is marked tight because runtimes, the desktop, and transient spikes all need memory. The green band leaves more room, but it cannot guarantee an out-of-memory error will not occur.

These are planning estimates that assume a llama.cpp / GGUF-style setup. Your exact numbers depend on the specific model, the runtime, and how much you batch.

Worked examples

These are this calculator’s own outputs, computed with the same formula the tool runs in your browser — so you can see a real answer without touching a single input.

Mainstream 16 GB gaming/AI desktop — the RTX 5080 (16 GB label) with an 8B selection at Q4_K_M, an 8K context, and an unquantized FP16 KV cache. This is the calculator’s default state

  • Inputs: Labeled capacity = 16 GB; GPUs = 1; parameter selection = 8B; quantization = Q4_K_M at 4.83 bpw; context = 8192 tokens; KV precision multiplier = 1.
  • Result: The estimate is 6.7 GB (6.2 GiB): 4.8 GB (4.5 GiB) of weights, 1.1 GB (1.0 GiB) of KV cache, and 0.8 GB (0.7 GiB) of runtime allowance. The entered capacity is 16.0 GB (14.9 GiB), leaving an estimated 9.3 GB (8.7 GiB). The per-quant totals are FP16 17.9 GB (16.6 GiB), Q8_0 10.4 GB (9.7 GiB), Q6_K 8.4 GB (7.9 GiB), Q5_K_M 7.5 GB (7.0 GiB), Q4_K_M 6.7 GB (6.2 GiB), Q3_K_M 5.8 GB (5.4 GiB), and Q2_K 5.2 GB (4.9 GiB). Only FP16 is marked Won’t fit.

Single 24 GB labeled card — an RTX 3090 or RTX 4090 with the exact 27B selection at Q4_K_M, a 32K context, and FP16 KV cache

  • Inputs: Labeled capacity = 24 GB; GPUs = 1; parameter selection = 27B; quantization = Q4_K_M at 4.83 bpw; context = 32768 tokens; KV precision multiplier = 1.
  • Result: The estimate is 25.1 GB (23.4 GiB): 16.3 GB (15.2 GiB) of weights, 8.0 GB (7.5 GiB) of KV cache, and 0.8 GB (0.7 GiB) of runtime allowance. That is 1.1 GB (1.0 GiB) above the entered 24.0 GB (22.4 GiB), so the planning verdict is Won’t fit. Q3_K_M lowers the estimate to 22.0 GB (20.5 GiB). Selecting Q8 KV instead halves the cache to 4.0 GB (3.7 GiB) and produces 21.1 GB (19.7 GiB).

Dual-GPU planning case — two RTX 3090 entries at 24 GB each with the 70B selection at Q4_K_M, an 8K context, and FP16 KV cache

  • Inputs: Labeled capacity = 24 GB per card; GPUs = 2; parameter selection = 70B; quantization = Q4_K_M at 4.83 bpw; context = 8192 tokens; KV precision multiplier = 1.
  • Result: The estimate is 45.7 GB (42.6 GiB): 42.3 GB (39.4 GiB) of weights, 2.6 GB (2.5 GiB) of KV cache, and 0.8 GB (0.7 GiB) of runtime allowance. Against the aggregate 48.0 GB (44.7 GiB) input, the planning verdict is Tight with 2.3 GB (2.1 GiB) left. This arithmetic does not establish per-device layer placement or runtime allocation.

Assumptions and sources

  • Bits-per-weight for each quantization level (Q4_K_M at 4.83 down to Q2_K, plus Q8_0 and FP16) follow the GGUF quantization sizes implemented in llama.cpp, the runtime family this calculator assumes; weights memory is then exact arithmetic (parameters x bits / 8).
  • The KV-cache estimate uses the grouped-query-attention shape of modern models, as described for the Llama 3 family in Meta’s Llama 3 paper; the exact cache size varies with each model’s layer count, head count, and head dimension, which is why the FAQ calls it an estimate.
  • NIST’s binary-prefix definitions distinguish a decimal gigabyte (10^9 bytes) from a binary gibibyte (2^30 bytes). The calculator preserves each declared source unit, converts to bytes, and rounds only the displayed GB and GiB values. An explicitly decimal 16 GB quantity is about 14.901161 GiB.
  • The 0.8 GB runtime overhead and the fits/tight/won’t-fit verdict bands are TechFuelHQ planning allowances, stated so you can judge the margin yourself - the FAQ’s “plan for 10 to 15 percent on top” advice carries the same intent.

Related guides and tools

Data

RTX 5080 Local LLM Throughput: Measured Tokens/Second Dataset

Measured decode and prefill tokens/second for gpt-oss 20B, Qwen 2.5 14B and 7B, and Llama 3.2 3B on a retail RTX 5080, …

Articles

Which Local LLM Fits Your GPU in 2026? VRAM Tiers

Which local LLM fits your GPU in 2026: models for 8, 12, 16, 24, and 48GB+ cards at Q4_K_M, with real GGUF file sizes …

Tutorials

How to Self-Host a Local LLM on a Single RTX 5060 in 2026

Run a private LLM on a $300 RTX 5060. VRAM tradeoffs, software stack picks, Windows + Linux setup, real tokens/sec …

GPU

RTX 5080 Performance-per-Watt as a 24/7 AI Inference Card (2026)

Reviews quote 8-15W idle; my retail RTX 5080 measures 52-71W at the desktop. Measured board draw for 24/7 AI inference, …

Frequently asked questions

How accurate is this VRAM estimate?
The weights figure is arithmetic from the selected parameter count and bits per weight; it is not measured residency. The KV cache is an estimate because it depends on a model’s exact layer count, attention heads, and head dimension. The GQA anchor for an 8B model at 32K context in FP16 is 4.0 GiB, or about 4.3 GB. Plan for the estimate plus roughly 10 to 15 percent, since your runtime, batch size, and any draft model add more.
What quantization should I use for a local LLM?
Q4_K_M is the sweet spot for most people: it cuts a model to about a third of its FP16 size while keeping quality close to the original. If you have VRAM to spare, Q5_K_M or Q6_K give a small quality bump. Q8_0 is near-lossless and worth it only when memory is no object. Q3_K_M and Q2_K trade visible quality for size and are best reserved for fitting a model that otherwise would not load at all.
Why does context length use so much memory?
Every token you feed the model is stored in the KV cache for each layer, so memory grows in a straight line with context length. Doubling the context roughly doubles the KV cache. A long 128K context can cost more VRAM than the model weights themselves on smaller models. Quantizing the KV cache to Q8 halves that cost and Q4 quarters it, usually with little quality loss for chat workloads.
Can I run a model that is bigger than my VRAM?
Yes, with tradeoffs. Tools like llama.cpp and Ollama can offload some layers to system RAM and run them on the CPU; the model loads but generation slows down in proportion to how much sits off the GPU. Splitting the model across two GPUs is faster than CPU offload if you have a second card. Otherwise, drop to a smaller quantization or a smaller model size.
Does the calculator handle multiple GPUs?
It adds the labeled capacity entered for each card. That aggregate is a planning budget, not proof that a runtime can divide this model across the cards. Layer placement, per-device buffers, communication overhead, and the memory actually available on each GPU still need a check on the target system.