Managing Local Models with Ollama

Three useful model states

When troubleshooting Ollama, separate three questions:

1. Is the model downloaded? This is persistent storage.
2. Can the runtime load and execute it? This is an inference question.
3. Is it active right now, and where is it placed? This is a runtime/memory question.

Mixing these states creates common mistakes. A model can exist on disk without being loaded. A model can run with partial CPU/GPU placement even when its downloaded size looks smaller than the GPU's advertised VRAM.

Download and list local models

Pull a model:

ollama pull MODEL_NAME

List local models:

ollama ls

The local list answers a storage/inventory question. It does not tell you how the model will be placed once loaded.

Inspect model information

Use Ollama's show command for a selected local model:

ollama show MODEL_NAME

Record what the runtime actually reports instead of inferring architecture or configuration from a marketing label. For experiments, also record the exact model identifier and Ollama version.

Inspect active runtimes

After starting inference, run:

ollama ps

This is the operational view. In E—DOCEO Lab #001, Gemma 3 1B Q4_K_M was reported as 100% GPU on the GTX 1060 3GB test machine at active context 4096. In Lab #002, Qwen3 4B Q4_K_M was reported with mixed CPU/GPU placement on the same machine.

Those measurements demonstrate why model management is not just file management: the active configuration determines what the runtime must place in memory.

Stopping is not removing

Stop an active model:

ollama stop MODEL_NAME

Remove a downloaded model when you no longer want it stored locally:

ollama rm MODEL_NAME

Stopping manages the active runtime. Removing manages persistent local model storage. Use the command that matches the state you actually want to change.

Why downloaded size and runtime memory differ

A downloaded model file is only part of the active inference configuration. Runtime memory can also include context-related state and implementation overhead. This is why E—DOCEO keeps model-size estimation separate from the deployment-oriented VRAM calculator.

In Lab #002, the Qwen3 4B download was about 2.5 GB while ollama ps reported runtime sizes from 3.1 GB to 3.5 GB across the tested context configurations. Treat that as a measurement of that exact setup, not a universal multiplier.

A repeatable diagnostic workflow

Use this order when something looks wrong:

ollama --help
      ↓
ollama ls
      ↓
ollama show MODEL_NAME
      ↓
ollama run MODEL_NAME
      ↓
ollama ps

Each step answers a different question: CLI availability, local inventory, model metadata, inference success and active placement. Only after those checks should you change context, quantization or hardware expectations.

Continue with the step-by-step run tutorial or the local API tutorial.

Sources

• Ollama documentation
• Ollama CLI reference