Local LLM VRAM Calculator
Estimate how much GPU memory a local large language model needs from its parameter count, quantization, context length and number of simultaneous requests, and see whether it fits the GPU you have.
| Quantization | Recommended | Verdict |
|---|
About this tool
Running a language model on your own GPU is mostly a memory question. The weights have to fit in video memory, plus working space and a key-value cache that grows with the context length and with each simultaneous request. This calculator turns those factors into a minimum and a recommended amount of VRAM and tells you whether your GPU is enough.
The numbers are estimates from a rule of thumb, not measurements. Real usage depends on the runtime (llama.cpp, vLLM, Ollama, …), the exact quantization format, the model architecture and the batch settings. Everything is calculated in your browser.
How it works
- Weights = parameters (billions) × bytes per parameter: 2 for FP16/BF16, 1 for 8-bit, 0.5 for 4-bit, 0.375 for 3-bit and 0.25 for 2-bit. Real quantized files are a little larger because some tensors stay at higher precision.
- Overhead = 20% of the weights (runtime buffers, activations, fragmentation).
- KV cache per request = (context length ÷ 2,048) × (parameters ÷ 7) × 0.5 GB. This is a simplification: the true size depends on the number of layers, the attention head layout (models with grouped-query attention need much less) and the cache precision. Total KV cache = per request × simultaneous requests.
- Minimum VRAM = weights + half the overhead + the KV cache for one request. Recommended VRAM = weights + overhead + 1.3 × the total KV cache. System RAM for CPU inference = 1.5 × weights + 4 GB.
- Verdict: green if your memory is at least the recommended amount, yellow if it is between the minimum and the recommended amount, red if it is below the minimum. The table repeats the check for every quantization level with your other settings unchanged.
- For mixture-of-experts models such as Mixtral, enter the total parameter count, not the active count, because all experts must be in memory. The Apple Silicon presets use roughly three quarters of the unified memory, since macOS limits how much the GPU can use by default.
- The estimate ignores the memory used by your display or other programs, multi-GPU splitting overhead, and speed (tokens per second).
FAQ
Q. Why does my real memory use differ from the estimate?
A. Runtimes reserve memory differently, the KV cache depends on the model architecture, and some layers can be kept on the CPU. Treat the result as a first filter and confirm with a real run.
Q. Can I run a model that is larger than my VRAM?
A. Yes, with runtimes that offload some layers to the CPU, at a large cost in speed. The system RAM figure gives a rough idea of what that needs.
Q. Which quantization should I pick?
A. 4-bit is the usual compromise between size and quality. Use 8-bit or FP16 when you have the memory and quality matters; 3-bit and 2-bit save space but degrade output noticeably, especially on small models.