What Happens When a Local Model Outgrows a 3GB GPU?
Three model sizes, the same 3 GB GPU and the same 4096 context setting. We measured the transition from a runtime reported fully in VRAM to increasingly partial placement — and recorded what happened to generation throughput.
Reproduced by E—DOCEO on September 25, 2026 on the documented Windows test machine. Three local model configurations were measured with the same prompt and num_ctx 4096, using one cold run and three warm runs per model.
Result
On this Windows system with a GeForce GTX 1060 3GB, the three tested configurations produced three clearly different runtime states in Ollama's /api/ps data.
Gemma 3 1B reported an 877 MB runtime with 877 MB in size_vram and a median warm generation rate of 64.46 tok/s. Qwen3 1.7B reported a 1.900 GB runtime with 1.462 GB in size_vram and 48.75 tok/s. Qwen3 4B reported a 3.525 GB runtime with 1.501 GB in size_vram and 9.44 tok/s.
| Model | Runtime size | size_vram |
size_vram / size |
Warm median |
|---|---|---|---|---|
| Gemma 3 1B | 0.877 GB | 0.877 GB | 100% | 64.46 tok/s |
| Qwen3 1.7B | 1.900 GB | 1.462 GB | 76.9% | 48.75 tok/s |
| Qwen3 4B | 3.525 GB | 1.501 GB | 42.6% | 9.44 tok/s |
A 3 GB GPU did not create a binary “runs / does not run” boundary in this test. Ollama continued to run the larger configurations with only part of the reported runtime represented by
size_vram. The 4B configuration still generated output, but at substantially lower measured throughput.Test environment
| Component | Recorded configuration |
|---|---|
| Test date | September 25, 2026 |
| OS | Windows 10 Pro, build 19045 |
| CPU | AMD Ryzen 5 2600, 6 cores / 12 threads |
| RAM | 34,281,566,208 bytes (~31.9 GiB) |
| GPU | NVIDIA GeForce GTX 1060 3GB — documented test machine |
| Ollama models | gemma3:1b, qwen3:1.7b, qwen3:4b |
| Active context | 4096 for every run |
The v1.5 runner's environment JSON did not capture nvidia-smi: its gpu field is null and the per-run gpu_used_mib_before/after fields are empty. For that reason, this page does not present those missing snapshots as measured LAB #005 data. Runtime memory figures below come from Ollama /api/ps fields recorded after generation.
Method
Every model received the same prompt:
Explain in about 150 words what a local large language model is and give three practical reasons for running one locally.The runner used Ollama's local HTTP API with stream: false, requested think: false, temperature: 0 and num_ctx: 4096. Before each model, the runner requested an unload and checked /api/ps. It then performed one cold run followed by three warm runs without unloading between the warm measurements.
Generation throughput is calculated from Ollama's API counters:
generation tok/s = eval_count / (eval_duration / 1e9)The warm median is the primary throughput summary. Cold load time is retained separately because model loading is not the same operation as token generation.
Runtime size and runtime reported in VRAM
size_vram / size; it is not a claim about the percentage of compute executed by the GPU.The intermediate Qwen3 1.7B point matters. Its runtime was about 1.90 GB, while size_vram was about 1.46 GB. Qwen3 4B increased total runtime to about 3.53 GB, while size_vram remained near 1.50 GB. In other words, the larger runtime did not produce a comparable increase in the VRAM-resident figure reported by the API on this system.
Do not translate these ratios into “GPU utilization” or “percent of tokens generated on GPU.” They describe two runtime memory fields exposed by Ollama. They are useful evidence of changing placement, not a complete performance model.
Warm generation throughput
The two smaller configurations were also much more stable across their three warm runs. Gemma measured 64.61, 64.36 and 64.46 tok/s. Qwen3 1.7B measured 48.75, 48.68 and 49.31 tok/s. Qwen3 4B measured 9.44, 8.16 and 10.10 tok/s.
That variability is part of the result rather than something to hide with an average. With only three warm runs, however, it is not enough to estimate a long-term performance distribution.
All 12 recorded runs
| Model | State | Output tokens | Generation | Load | API total |
|---|---|---|---|---|---|
| Gemma 3 1B | cold | 195 | 61.40 tok/s | 14.883 s | 18.289 s |
| Gemma 3 1B | warm | 192 | 64.61 tok/s | 0.002 s | 3.014 s |
| Gemma 3 1B | warm | 192 | 64.36 tok/s | 0.002 s | 3.025 s |
| Gemma 3 1B | warm | 192 | 64.46 tok/s | 0.002 s | 3.017 s |
| Qwen3 1.7B | cold | 152 | 46.97 tok/s | 12.567 s | 16.643 s |
| Qwen3 1.7B | warm | 167 | 48.75 tok/s | 0.003 s | 3.462 s |
| Qwen3 1.7B | warm | 167 | 48.68 tok/s | 0.003 s | 3.465 s |
| Qwen3 1.7B | warm | 167 | 49.31 tok/s | 0.003 s | 3.414 s |
| Qwen3 4B | cold | 608 | 10.37 tok/s | 18.085 s | 77.078 s |
| Qwen3 4B | warm | 567 | 9.44 tok/s | 0.002 s | 60.254 s |
| Qwen3 4B | warm | 567 | 8.16 tok/s | 0.002 s | 69.627 s |
| Qwen3 4B | warm | 567 | 10.10 tok/s | 0.003 s | 56.286 s |
The models did not generate the same number of output tokens. This is why total request duration is not used as the cross-model speed metric. eval_count / eval_duration is the more appropriate measurement of generation throughput for this dataset.
There is another important anomaly in the raw data: Qwen3 4B returned a visible reasoning block even though the runner requested think: false. Its eval_count therefore covers the tokens the runtime actually generated, not only the final answer visible after that reasoning. LAB #005 preserves this behavior rather than rewriting the run after the fact.
What the measurements do — and do not — show
The measurements support a narrow conclusion: on this machine and Ollama configuration, increasingly large tested runtimes were accompanied by a smaller size_vram / size ratio and lower generation throughput. The 3 GB GPU still participated in the larger runs; the 4B model was not rejected simply because its reported runtime exceeded 3 GB.
They do not establish a universal threshold for 3 GB GPUs. They also do not prove that offloading alone caused the throughput difference. Gemma 3 1B, Qwen3 1.7B and Qwen3 4B differ in model family or size, architecture and compute requirements. The Qwen3 1.7B → 4B comparison is cleaner because both belong to the Qwen3 family, but parameter count and workload still change together with placement.
For model selection, the practical lesson is to test the loaded runtime rather than comparing only the downloaded file size with nominal VRAM. A configuration can run partially offloaded — but whether its resulting speed is useful depends on the workload.
Raw data and reproducibility
The unedited runner outputs used for this page are available here:
Each archive includes the CSV output, environment snapshot and per-run records produced by the benchmark runner. The Qwen3 1.7B run was added later using the same frozen prompt and generation settings; the two already-valid models were not rerun.
Limitations
This is one Windows machine, one GPU, one local runtime and one short prompt. Each model has only one cold and three warm measurements. The test configures a 4096 context capacity but does not fill that context with a long prompt, so it is not a long-context benchmark.
The runner failed to capture direct nvidia-smi snapshots in LAB #005. The memory comparison therefore uses Ollama's recorded size and size_vram fields and labels them as such. Output lengths also differ between models, and Qwen3 4B exposed reasoning text despite the requested thinking setting.
For a controlled study of context allocation on one unchanged model, see LAB #003: Gemma 3 1B from 1K to 32K context. For the earlier Qwen3 4B placement experiment using ollama ps, see LAB #002.