Gemma 3 1B on a GTX 1060 3GB

Result

In this test, Gemma 3 1B Q4_K_M ran entirely on the NVIDIA GeForce GTX 1060 3GB according to ollama ps, which reported 100% GPU. With an active context of 4096, three API generation runs measured 66.99, 67.14 and 66.13 tokens/second. The median generation throughput was 66.99 tok/s.

This result applies to the exact environment below. It is not a claim that every GTX 1060 3GB system, driver, context size or Ollama release will produce the same speed.

Test environment

CUDA Version: 12.9 above is the compatibility version reported by nvidia-smi; this test did not independently establish that a CUDA Toolkit was installed.

Model configuration

ollama show gemma3:1b reported:

The distinction between the model's reported context capability and the 4096 active context matters: memory and performance can change when context settings change.

Method

The Windows installation was first checked layer by layer: Ollama version, empty local model list, local /api/tags response, model download, CLI inference, ollama ps, and nvidia-smi. The benchmark then used Ollama's local POST /api/generate endpoint with stream: false, temperature: 0 and num_ctx: 4096.

The prompt was kept identical:

Explain in about 150 words what a local large language model is and give three practical reasons for running one locally.

Generation throughput is calculated from Ollama's returned eval_count and eval_duration:

generation tok/s = eval_count / (eval_duration / 1e9)

Ollama documents eval_count as the number of generated output tokens and eval_duration as token-generation time in nanoseconds. This makes the measurement independent of a hand-operated stopwatch.

Three generation runs

The maximum-to-minimum spread in generation throughput was about 1.5%. That is useful evidence of short-run consistency, but three runs are still a small sample.

Prompt-processing rates were 281.87, 989.09 and 1089.90 tok/s. We do not use those values as the headline benchmark: the prompt contained only 35 tokens and the first request occurred under a different runtime/cache state, making that comparison less useful.

Observed GPU memory

nvidia-smi snapshots showed:

This is deliberately labelled observed GPU memory, not “the model requires 998 MiB of VRAM.” Under Windows/WDDM, nvidia-smi was reporting aggregate GPU memory while the desktop and other applications were also active. The snapshots establish system-level memory use around this test, not an isolated per-model allocation.

The post-benchmark nvidia-smi process list did show llama-server.exe as a compute process, while ollama ps independently reported 100% GPU.

Reproduce the benchmark

After installing Ollama and downloading gemma3:1b, this PowerShell request reproduces the benchmark configuration:

$body = @{
    model  = "gemma3:1b"
    prompt = "Explain in about 150 words what a local large language model is and give three practical reasons for running one locally."
    stream = $false
    options = @{
        temperature = 0
        num_ctx = 4096
    }
} | ConvertTo-Json
$r = Invoke-RestMethod     -Method Post
-Uri http://localhost:11434/api/generate     -ContentType "application/json"
-Body $body
[math]::Round($r.eval_count / ($r.eval_duration / 1e9), 2)

Record the runtime at the same time:

ollama ps
nvidia-smi
ollama show gemma3:1b

For a comparable result, keep the model tag, context, prompt and generation settings unchanged and record your Ollama and driver versions.

Limitations

This is a single-machine test, not a GPU-wide performance guarantee. It used one model, one quantization, a 4096 active context and a short fixed prompt. Windows desktop applications were running during the VRAM snapshots. Only three measured generation runs were used for the median.

The test demonstrates that this specific GTX 1060 3GB system could load the tested Gemma 3 1B configuration fully on GPU and generate at the recorded speed. It does not establish performance for larger context windows, other Gemma tags, other quantizations, different GTX 1060 cards or future Ollama versions.

Documentation used

• Ollama — Windows
• Ollama — Generate a response
• Ollama — CLI Reference
• Ollama — Context length