# LLM VRAM Calculator: Which GPU Do I Need?

> Work out GPU memory for running or fine-tuning an LLM by parameters, precision, context and batch, then see the cheapest GPU setup that fits.

- Canonical URL: https://computesuppliers.com/llm-vram-calculator/
- Source: [Hugging Face, model memory anatomy](https://huggingface.co/docs/transformers/model_memory_anatomy)
- Checked: 2026-10-26
- Structured data: https://computesuppliers.com/data/llm-vram-calculator.json

This page includes a free interactive tool that runs in the browser and sends no data. The figures and rules it uses are the ones stated on this page, with their sources.

## The formula

GPU memory for an LLM has four parts:

| Part | Rule of thumb |
|---|---|
| Weights | parameters × bytes per parameter: 4 (FP32), 2 (BF16/FP16), 1 (FP8/INT8), 0.5 (INT4) |
| KV cache | 2 × layers × context × batch × KV width × 2 bytes |
| Training state | full fine-tune with Adam in mixed precision is about 16 bytes per parameter before activations ([Hugging Face](https://huggingface.co/docs/transformers/model_memory_anatomy)) |
| Overhead | CUDA context, fragmentation and framework buffers; we add 10% |

## Quick answers

| Model and task | Approximate VRAM | Smallest common fit |
|---|---|---|
| 8B inference, BF16, 8k context | about 19 GB | one 24 GB card (L4, RTX 4090) |
| 70B inference, BF16, 8k context | about 158 GB | 1 × B200, 2 × H200 or 4 × H100 |
| 70B inference, INT4 | about 42 GB | one 48 GB L40S |
| 70B QLoRA fine-tune | about 50–60 GB | one A100 or H100 80 GB |
| 70B full fine-tune | over 1 TB | multiple 8-GPU nodes |

These are estimates. Long contexts and large batches change the KV cache a lot, so set yours above.

## Why the KV width matters

Modern models share key and value heads (grouped-query attention). Llama 3 70B has 64 query heads but only 8 KV heads of size 128, so its KV width is 1,024, not 8,192. That cuts the cache by 8 times compared with older models.

---
Compute Suppliers · Contact: contact@computesuppliers.com