VRAM for local LLM
Params × quantization + context → VRAM
Input
VRAM needed
weights: 3,5 ГБ + KV ~0,9 ГБ
4,9 ГБ
- Min. GPU
- 6 ГБ
Weights = params × bytes per param. Context (KV-cache) adds ~0.1–0.5 GB per 1k tokens by architecture. Guides: 7B Q4 ≈ 4.5 GB, 13B Q4 ≈ 8 GB, 70B Q4 ≈ 40 GB. Quality drops noticeably below Q4.