Can Qwen3 4B Run on a 3GB GPU?

Result

Yes, Qwen3 4B Q4_K_M ran on this GTX 1060 3GB system, but Ollama did not place the complete runtime on the GPU. At an active context of 4096, ollama ps reported a 3.5 GB runtime size and 57%/43% CPU/GPU placement. Three generation runs measured 11.97, 12.16 and 12.26 tok/s, for a 12.16 tok/s median.

Reducing active context changed the placement. At 2048, Ollama reported 3.2 GB and 55%/45% CPU/GPU. At 1024 it reported 3.1 GB and 53%/47% CPU/GPU. In this environment, smaller context therefore coincided with a smaller reported runtime and a modestly larger GPU share.

Test environment

Model configuration

The tested local tag was qwen3:4b, ID 359d7dd4bcda. ollama show qwen3:4b reported architecture qwen3, 4.0B parameters, Q4_K_M, embedding length 2560 and model context length 262144. ollama ls reported a 2.5 GB downloaded model.

The model also exposed completion, tools and thinking capabilities. For this benchmark, thinking was explicitly disabled with think: false so the request measured ordinary generation under the same prompt structure used in Lab #001.

Method

The same prompt was used throughout:

Explain in about 150 words what a local large language model is and give three practical reasons for running one locally.

Requests used Ollama's local POST /api/generate endpoint with stream: false, think: false and temperature: 0. The only intentional variable in the context experiment was num_ctx: 4096, 2048 or 1024.

Generation throughput was calculated from the API timing fields:

generation tok/s = eval_count / (eval_duration / 1e9)

We recorded ollama ps after loading each context configuration to capture the runtime size, processor split and active context. nvidia-smi snapshots recorded total system GPU memory, not isolated per-model allocation.

Context size changed runtime size and GPU placement

Across these measurements, reducing active context from 4096 to 1024 changed reported GPU placement from 43% to 47%. The median generation rate was about 10.9% higher at 1024 than at 4096 in this small test set.

That does not mean “smaller context is always 10.9% faster.” Context capacity itself is useful, the sample is small, output lengths varied, and other systems may choose different placement. The measured result is narrower: on this machine and Ollama version, context configuration affected reported runtime size and offload.

Recorded generation runs

The load column is retained because the test sequence included both cold/reload and warm requests. Generation tok/s is the primary throughput metric; total request time should not be read as a pure generation benchmark when load overhead differs.

Freeing Windows VRAM: an A/B observation

At context 4096, an initial idle snapshot showed 921 MiB / 3072 MiB in use before loading Qwen. After loading/inference, snapshots showed approximately 2381–2393 MiB total GPU memory in use, while ollama ps reported 57%/43% CPU/GPU.

We then stopped the model, closed several ordinary desktop applications and reduced idle GPU memory to 525 MiB / 3072 MiB — 396 MiB lower. Reloading the same model at context 4096 produced:

Freeing 396 MiB of idle GPU memory did not change the processor split in this single A/B reload. This does not prove that free VRAM never affects Ollama placement; it only records that this particular reduction was not enough to change the reported 57%/43% split.

What this test tells us

First, download size and runtime memory are different quantities. The 2.5 GB local model produced a 3.5 GB runtime size at context 4096 according to ollama ps.

Second, “runs on a 3 GB GPU” needs qualification. Qwen3 4B did run, but it used partial CPU/GPU offload, unlike the smaller Gemma 3 1B in Lab #001, which Ollama reported at 100% GPU.

Third, active context belongs in VRAM planning. On the same machine and model, changing only num_ctx from 4096 to 1024 reduced reported runtime size from 3.5 GB to 3.1 GB and increased the reported GPU share from 43% to 47%.

Reproduce the 4096-context test

$body = @{
    model  = "qwen3:4b"
    prompt = "Explain in about 150 words what a local large language model is and give three practical reasons for running one locally."
    stream = $false
    think  = $false
    options = @{
        temperature = 0
        num_ctx = 4096
    }
} | ConvertTo-Json
$r = Invoke-RestMethod     -Method Post
-Uri http://localhost:11434/api/generate     -ContentType "application/json"
-Body $body
ollama ps
nvidia-smi
[math]::Round($r.eval_count / ($r.eval_duration / 1e9), 2)

Change only num_ctx to 2048 or 1024 to reproduce the context comparison. Record ollama ps after each reload rather than assuming the processor split remains unchanged.

Limitations

This is a single-machine, single-day test with Ollama 0.34.2, one Qwen3 tag, one quantization and a GTX 1060 3GB under Windows/WDDM. Three generation measurements per context are enough to expose the observed pattern, but not enough to characterize all performance variance.

The output token counts were not identical between every run despite temperature: 0, so the medians should be treated as practical measurements rather than laboratory-grade deterministic benchmarks. The VRAM A/B comparison has only one reduced-desktop generation run. nvidia-smi memory values are aggregate system snapshots.

Documentation and model references

• Ollama — Generate a response
• Ollama — Context length
• Ollama — Qwen3 model library