E—DOCEO / Tools / VRAM Calculator
Tool · Local AI

Local LLM VRAM Calculator

Build a transparent memory budget from model weights, KV cache and a reserve for runtime overhead. This is a planning estimate—not a promise that a model will fully fit on your GPU.

1. Model weights

Best when you already know the GGUF/Ollama model size. Runtime weight allocation can still differ.
This computes the theoretical parameter payload. Real quantization formats include metadata and may use an effective bits-per-weight value different from the nominal label.

2. KV cache optional

Generic transformer estimate: 2 (K + V) × layers × KV heads × head dimension × context × bytes × sequences. Architectures using MLA, sliding-window attention or other cache layouts can behave differently.

3. Planning margin

User-controlled allowance for compute buffers and other runtime overhead.
Used only to show headroom against the planning estimate.
Planning estimate
— GB
Model weights—
KV cache—
Reserve—
Enter your GPU VRAM to compare.

A result below your VRAM is not a guarantee of full GPU residency. Runtimes may allocate additional buffers, reserve memory differently, or offload part of a model to system RAM.

Method

What this calculator actually estimates

Local inference memory is not just the downloaded model file. A useful planning budget separates model weights, the KV cache used for context, and additional runtime buffers. The calculator keeps those components visible instead of hiding them behind a single unexplained number.

Weights

If you know the model's weight or GGUF file size, use that value. The alternative parameters × bits mode is deliberately labelled theoretical: quantization formats add metadata and their effective bits per weight can differ from a simple 4-bit or 8-bit label.

KV cache

For a conventional transformer cache, the estimate scales with layer count, KV-head count, head dimension, context length and cache precision. Longer context therefore increases memory even when the model weights do not change.

Runtime reserve

The reserve is not presented as a measured universal constant. It is an explicit planning margin that you control. Compute buffers and backend allocations vary by model architecture, runtime, GPU backend and settings.

Measured example: in DOCEO LAB #002, the downloaded Qwen3 4B model was listed at 2.5 GB while Ollama reported a 3.5 GB loaded size at a 4096-token context. On that 3 GB GTX 1060 system, Ollama used partial CPU/GPU placement rather than full GPU residency.

Why this is not a “will it fit?” oracle

Exact memory depends on architecture and implementation. Some models use grouped-query attention, sliding-window attention, MLA or other layouts; runtimes can quantize or offload KV cache, allocate compute buffers, and split weights across CPU and GPU. Use this result as a budget, then verify the real runtime with tools such as ollama ps and your GPU memory monitor.

Continue with VRAM explained, context length, quantization, and CPU vs GPU inference.