H100 vs H200: When 141 GB of Memory Pays Off
H200 has 141 GB of HBM3e and 4.8 TB/s against 80 GB on the H100. Which models fit on one card, rental prices per GPU-hour, and when H200 is cheaper per job.
The difference
| H100 SXM | H200 | |
|---|---|---|
| Memory | 80 GB HBM3 | 141 GB HBM3e |
| Bandwidth | about 3.35 TB/s | about 4.8 TB/s |
| Lowest on-demand we found | $3.20 (Hyperstack) | $3.99 (Hyperstack) |
| Next | $3.49 (Jarvislabs) | $4.47 (DigitalOcean) |
Specs: NVIDIA H200. Prices read 26 October 2026.
When the H200 is worth it
- A 70B model in 8-bit (about 81 GB with an 8k context) fits on one H200 with room for longer context; on H100s it needs two.
- A 70B model in 16-bit needs about 158 GB: two H200s instead of three or four H100s.
- Long context or big batches: the KV cache grows with every token; the extra 61 GB holds much more.
- Memory-bound inference: 43% more bandwidth means more tokens per second.
If your model and batch fit comfortably in 80 GB, the H100 is usually cheaper per job.
Per-job example
Serving a 70B model in 16-bit: two H200s at $3.99 = $7.98 an hour, against four H100s at $3.20 = $12.80. The H200 setup is cheaper and avoids splitting the model four ways.
Check your own model with the VRAM calculator.
Spotted an error or a figure that has changed? Let us know