LLM Model Size Explained
Parameter count describes the scale of a model, but it does not tell you by itself how much disk space, RAM or VRAM a local configuration will need.
What does 1B, 8B or 70B mean?
The B means billion parameters. Parameters are learned numerical values in the model. So a model described as 8B has roughly eight billion learned parameters; 70B means roughly seventy billion.
Parameter count is useful because it gives you a first indication of the amount of weight data and computation involved. It is not, however, the same thing as the model's file size or the memory required by a particular runtime.
Models with the same headline parameter count can also have different architectures, vocabularies, layer structures and capabilities. Treat the number as a scale descriptor, not a complete specification.
Precision changes weight memory
Each parameter has to be represented somehow. A simple first-order calculation is:
weight memory ≈ parameter count × bits per parameter ÷ 8
Ignoring format overhead, an 8B model would therefore need approximately:
| Representation | Idealized raw weight size |
|---|---|
| 32-bit | 32 GB |
| 16-bit | 16 GB |
| 8-bit | 8 GB |
| 4-bit | 4 GB |
This table is arithmetic, not a promise about actual files or VRAM usage. Real formats add metadata and may represent different tensors differently.
Hugging Face gives the same useful rule of thumb for unquantized inference: loading X billion parameters in 16-bit precision requires roughly 2 × X GB just for the weights.
Why local models are often quantized
Quantization stores model weights at lower precision so they require less storage and memory. This is what makes many larger models practical on consumer hardware.
Names such as Q4_K_M identify a particular quantization scheme rather than a universal “exactly four bits for every parameter” representation. Different schemes make different size, speed and quality trade-offs.
This is why you should use the actual model artifact when checking disk requirements and record the exact quantization when publishing a benchmark.
Downloaded file size is not runtime memory
A common local-AI mistake is to compare a downloaded model file directly with GPU VRAM: “the file is 2.5 GB and my GPU has 3 GB, so it fits.”
Runtime inference needs more than the stored weight file. Context state, buffers and backend overhead also consume memory, and some of that memory changes with configuration.
We measured this directly with Qwen3 4B Q4_K_M. ollama ls reported the downloaded model at 2.5 GB, but at active context 4096 ollama ps reported a 3.5 GB runtime. On the 3 GB GTX 1060 test system, Ollama therefore used partial CPU/GPU placement rather than reporting the model as 100% GPU.
See the Qwen3 4B measurements →
Parameter count is not a quality score
A larger parameter count usually means more weight data and more work per generated token, but it does not establish that one model is better for your task.
Training data, architecture, post-training, specialization, context handling and the task itself all matter. A smaller model can be more useful when it is fast enough for an interactive workflow, fits entirely on the available accelerator, or is specialized for the problem.
For the same reason, benchmark results should not be turned into an overall model ranking unless the evaluation actually measures the quality dimension being claimed.
How to choose a model size for local use
Work backwards from your machine and workload:
- Record available system RAM and VRAM.
- Choose the runtime and verify that it supports your hardware backend.
- Shortlist model families suited to the task.
- Compare the actual quantized artifacts, not only parameter counts.
- Reserve memory for context and runtime overhead.
- Check whether the model is fully GPU-resident or partially offloaded.
- Measure generation speed with the context size you really intend to use.
The goal is not to load the largest model that can technically start. It is to find a configuration that fits the task, memory budget and acceptable speed.