Ollama Local LLM Guide for 2026: Installation, Model Management, and REST API Usage

Introduction: Run a local LLM with a few commands

Getting started with local LLMs can appear to require a working knowledge of model formats, quantization, inference engines, and GPU configuration. Ollama packages much of that setup into a runtime that can download, run, update, and remove models through one CLI.

This guide walks through installation on macOS, Windows, and Linux, terminal-based chat, model management, Modelfiles, the REST API, and integrations with Claude Code and Codex. The first milestone is running these two commands:

ollama pull qwen3.5:4b
ollama run qwen3.5:4b

Once the prompt appears, try:

>>> Briefly explain Ollama to a beginner.

The examples use qwen3.5:4b, a relatively small model listed in the Ollama library at publication time in July 2026. Model availability, sizes, and tags change quickly, so check the model page before following along. Model selection and quantization deserve their own guide; the focus here is operating Ollama.

1. What is Ollama?

1-1. A runtime, not a model

Ollama is not an LLM. Names such as qwen3.5 and gemma4 refer to models; Ollama is the runtime that retrieves, runs, updates, and removes them.

A useful analogy is Docker: Ollama resembles the runtime, while a model resembles a container image. Models are addressed with a name and tag, such as qwen3.5:4b. The exact meaning of a tag varies by model and may indicate parameter scale, quantization, or intended use.

Starting Ollama also starts a background server. Both CLI commands such as ollama run and requests from curl or an application go through that server.

flowchart TD
    accTitle: Relationship among the Ollama CLI, API, server, models, and hardware
    accDescr: Requests from a terminal, curl, or an application reach the Ollama server, which loads a stored model onto the CPU or GPU for inference.
    A[Terminal CLI] --> D[Ollama Server]
    B[curl] --> D
    C[Application] --> D
    D --> E[Stored models]
    E --> F[CPU or GPU]

Ollama architecture

1-2. Core capabilities

Ollama brings the following operations under one CLI:

• Downloading, updating, running, and removing models
• Inspecting models stored on disk and models loaded in memory
• Reusing system prompts and runtime parameters through Modelfiles
• Calling a REST API on localhost
• Using selected OpenAI-compatible API endpoints
• Configuring and launching Claude Code, Codex, VS Code, and other tools with ollama launch

The important mental model is that the Ollama server, a model stored on disk, and a model loaded into memory are three separate things. That distinction matters when stopping processes and diagnosing failures.

2. Checks before you begin

2-1. Memory and disk capacity

Successfully installing Ollama does not mean every model will run comfortably. Loading a model consumes system RAM or GPU VRAM. Increasing the context window or handling several requests in parallel raises memory requirements further.

For a first run, select a small model and close other memory-intensive applications. At publication time, the Ollama library lists qwen3.5:4b as an approximately 3.4GB model file. Runtime memory use is not necessarily equal to the downloaded file size.

Models remain on disk until removed. If you plan to experiment with several of them, reserve considerably more than a few gigabytes of free space.

2-2. What “local” covers

When you call a locally stored model through the local API, inference can run without sending the prompt to an external model API. That does not apply if you choose a cloud model, enable an external tool such as web search, or use an integration that makes its own network requests. Do not assume every workflow is offline merely because Ollama is involved; verify the selected model and connected tools.

3. Install Ollama

3-1. macOS

Download the DMG from the official macOS page, move Ollama.app into Applications, and launch it. If the CLI is not already on your PATH, the app prompts for permission to create a link in /usr/local/bin.

The current official requirements specify macOS Sonoma 14 or later. Apple M-series Macs support CPU and GPU execution; Intel Macs are documented as CPU-only. Verify the installation with:

ollama -v

3-2. Windows

Use the official installer, or run the following command in PowerShell:

irm https://ollama.com/install.ps1 | iex

The normal installation goes into the user account and generally does not require administrator privileges. Ollama then runs in the background, and its CLI is available from PowerShell or Command Prompt. The detailed current requirements specify Windows 10 22H2 or later.

ollama -v

3-3. Linux

Run the official installation script:

curl -fsSL https://ollama.com/install.sh | sh

Then verify the CLI:

ollama -v

If the installation created a systemd service, inspect its status:

sudo systemctl status ollama

For a manual installation or a distribution without systemd, run ollama serve in another terminal. GPU acceleration also requires supported hardware and the appropriate driver.

3-4. Verify the server

On any operating system, a successful call to the local API confirms that the server is reachable:

curl http://localhost:11434/api/version

You can also list locally stored models:

ollama ls

If either command reports connection refused, check that the Ollama application or service is running.

4. Download and run your first model

4-1. Download the model

Retrieve the example model with:

ollama pull qwen3.5:4b

The progress output includes manifest retrieval, blob downloads, and checksum verification. Ollama can share existing blobs among related model definitions, so downloading another model does not always duplicate every byte.

When the pull finishes, inspect the model list:

ollama ls

NAME identifies the model, ID is its identifier, SIZE is the stored size, and MODIFIED shows when it was last updated.

4-2. Start a chat

ollama run qwen3.5:4b

Enter a prompt when the interactive prompt appears:

>>> Explain three benefits of running an LLM locally.

If only the first answer is slow, the main cost is often loading the model from storage into RAM or VRAM. By default, Ollama keeps a model in memory for five minutes after use, so another request during that period may start faster. You can adjust this behavior with keep_alive or OLLAMA_KEEP_ALIVE.

5. Interactive and one-shot execution

5-1. Interactive mode

Run a model without a prompt argument to continue a conversation:

ollama run qwen3.5:4b

This mode works well for follow-up questions, prompt iteration, and quickly evaluating a model’s behavior.

For multiline input, wrap the text in """ inside the interactive prompt:

>>> """
... Follow these requirements:
... - Answer in English.
... - Use bullet points.
... - Write for a beginner.
...
... What is a Kubernetes Pod?
... """

5-2. One-shot execution

For a single answer, place the prompt after the model name:

ollama run qwen3.5:4b "Explain Java records."

Redirect the result into a file when appropriate:

ollama run qwen3.5:4b "Compare Docker containers and virtual machines." \
  > answer.txt

A heredoc is convenient for longer input:

ollama run qwen3.5:4b <<'EOF'
Review the following code.

public int add(int a, int b) {
    return a + b;
}
EOF

When scripting this workflow, also handle exit codes and timeouts, and validate the generated result. An LLM response is not guaranteed to be correct.

6. Essential commands and model states

The most common points of confusion are ls versus ps, and stop versus rm.

flowchart LR
    accTitle: Disk storage and memory states of an Ollama model
    accDescr: Pull stores a model on disk and run loads it into memory. Stop unloads only the in-memory copy, while rm deletes the stored model. ls reports disk state and ps reports memory state.
    A[Model library] -->|ollama pull| B[Model on disk]
    B -->|ollama run| C[Model in memory]
    C -->|ollama stop| B
    B -->|ollama rm| D[Not stored]
    B -. ollama ls .-> E[Stored list]
    C -. ollama ps .-> F[Running list]

Stored and running model states

ollama stop qwen3.5:4b releases RAM or VRAM but leaves the model files intact. ollama rm qwen3.5:4b removes the downloaded model. Neither command shuts down the Ollama server itself.

7. Switch models and manage storage

The model is selected for each command or API request. Ollama does not maintain one globally selected model.

ollama run qwen3.5:4b
ollama run gemma4

In qwen3.5:4b, qwen3.5 is the model name and 4b is the tag. Omitting the tag normally resolves to latest, but latest does not mean smallest, fastest, or best for your laptop. Check the model page for size and requirements.

Default model locations are:

Use OLLAMA_MODELS to choose another directory. For the macOS application, set it with launchctl setenv; on Windows, create a user environment variable; for a Linux systemd service, add it to a service override. Restart Ollama afterward. On Linux, the ollama service user must have read and write access to the new directory.

Prefer ollama rm for removal and OLLAMA_MODELS for relocation instead of manually deleting individual blobs.

8. Configure the context window

The context window determines how much input, conversation history, and generated content a model can consider at once. A longer context can help with large documents and codebases, but it also consumes more memory. Setting a value beyond the model’s supported limit does not create additional capability.

In interactive mode, change it with:

/set parameter num_ctx 8192

For a single API request, set options.num_ctx:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3.5:4b",
  "prompt": "Summarize this text.",
  "options": {
    "num_ctx": 8192
  },
  "stream": false
}'

For a manually started server, set the server-wide default through an environment variable:

OLLAMA_CONTEXT_LENGTH=8192 ollama serve

The official FAQ describes a default context window of 4,096 tokens, while the parameter table in the Modelfile reference currently states a different default. Behavior may also depend on the model, Ollama version, and launch path. Treat neither number as universal; verify the configuration and current documentation for your environment.

9. Reuse settings with a Modelfile

A Modelfile defines a system prompt and runtime parameters on top of an existing model. It is a configuration blueprint similar to a Dockerfile; creating one does not, by itself, retrain or fine-tune the model.

Create the following file:

FROM qwen3.5:4b

PARAMETER temperature 0.3
PARAMETER num_ctx 8192

SYSTEM """
You are an assistant that helps with software design.
Always answer in English.
Add a short explanation to each code example.
Do not present uncertain information as fact.
"""

Build a named model configuration:

ollama create backend-assistant -f ./Modelfile

Run it like any other model:

ollama run backend-assistant

You can inspect the generated definition with:

ollama show --modelfile backend-assistant

Names such as backend-assistant, sql-reviewer, and article-editor let you reuse task-specific behavior without pasting a large system prompt on every invocation. Ollama manages model data as blobs, so each named configuration does not necessarily create a complete copy of the large base-model file.

10. Call the local REST API

10-1. Generate versus chat

The default local API base URL is http://localhost:11434/api. Use /api/generate for generation from one prompt:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3.5:4b",
  "prompt": "Explain dependency injection in Spring Boot."
}'

Use /api/chat when sending conversation history:

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3.5:4b",
  "messages": [
    {
      "role": "system",
      "content": "You are a Java specialist."
    },
    {
      "role": "user",
      "content": "Explain records."
    }
  ]
}'

The messages array can contain roles including system, user, and assistant. A chat application should retain the required history and include it in subsequent requests.

10-2. Read streaming responses

Streaming is enabled by default for endpoints such as /api/generate. Chunks arrive as newline-delimited JSON, or NDJSON:

{"response":"Spring","done":false}
{"response":" Boot","done":false}
{"response":" is","done":false}
{"response":"...","done":true}

Append each response fragment in arrival order and stop when done becomes true. The /api/chat endpoint places generated text in message.content, so do not assume both endpoints return the same shape.

Disable streaming with stream: false:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3.5:4b",
  "prompt": "Explain Ollama.",
  "stream": false
}'

This returns one JSON document containing the complete response. It is easier for short operations and structured validation, while streaming provides better perceived latency for longer answers.

10-3. Inspect performance metrics

A completed response can include these metrics. Duration fields are measured in nanoseconds.

• total_duration: End-to-end request duration
• load_duration: Time spent loading the model
• prompt_eval_count: Number of input tokens
• prompt_eval_duration: Time spent processing the prompt
• eval_count: Number of generated tokens
• eval_duration: Time spent generating tokens

A high load_duration points toward model startup, while a high eval_duration focuses the investigation on generation performance.

Ollama also implements parts of the OpenAI API. Existing clients can target http://localhost:11434/v1/, but this is not complete API equivalence. Check the compatibility documentation for the exact endpoint and field support your application needs.

11. Connect Claude Code and Codex

Display the supported integrations interactively with:

ollama launch

Launch Claude Code, optionally selecting a model:

ollama launch claude
ollama launch claude --model qwen3.5:4b

Configure and launch Codex CLI with:

ollama launch codex
ollama launch codex --config

For a manual Codex configuration, the official integration guide also documents codex --oss for using the Ollama provider.

Coding agents generally need far more context than a short chat. Ollama’s Claude Code and Codex integration guides recommend at least a 64K-token context window. Giving a small model 64K context does not guarantee adequate coding quality, and it increases memory consumption. Treat the first run as a connectivity test, then evaluate tool-use support, code quality, and context requirements before relying on a local model for agentic work.

12. Stop and restart Ollama

To unload only one model from memory, run:

ollama stop qwen3.5:4b

On macOS and Windows, quit the entire server from the Ollama menu-bar or system-tray menu, then relaunch the application when needed.

Manage the Linux systemd service with:

sudo systemctl stop ollama
sudo systemctl start ollama
sudo systemctl restart ollama
sudo systemctl status ollama

If you started ollama serve directly in a terminal, press Ctrl+C in that terminal.

In short, ollama stop unloads a model, while Quit or systemctl stop shuts down the server. Choose the operation that matches the layer you are trying to stop.

13. Troubleshoot common errors

13-1. Cannot connect to the server

If you see connection refused, test the API first:

curl http://localhost:11434/api/version

Start the Ollama application on macOS or Windows, or the systemd service on Linux. If you run it manually, start ollama serve.

13-2. Model load fails or memory runs out

Inspect the running and selected model:

ollama ps
ollama show qwen3.5:4b

Try a smaller tag, unload unused models with ollama stop, lower num_ctx, reduce request concurrency, and close other memory-heavy applications. A large context window combined with several parallel requests can exhaust memory even when the base model is small.

13-3. Responses are slow

Check the PROCESSOR column in ollama ps. It can report 100% GPU, 100% CPU, or a CPU/GPU split. If only the first request is slow, investigate loading. If token generation remains slow, examine model size, CPU-only execution, and context length. API fields such as load_duration and eval_duration help separate these cases.

13-4. Port 11434 is already in use

On macOS or Linux, run:

lsof -i :11434

In Windows PowerShell, run:

Get-NetTCPConnection -LocalPort 11434

A common cause is starting a second ollama serve while the desktop application is already running, or exposing the same port from an Ollama container.

13-5. Read the logs

macOS:

cat ~/.ollama/logs/server.log

Linux:

journalctl -u ollama --no-pager --follow --pager-end

Windows:

%LOCALAPPDATA%\Ollama\server.log

When escalating a problem, collect the logs, Ollama version, operating system, GPU information, command, and reproducible steps. Remove secrets, source code, and sensitive prompts before sharing logs.

14. Be careful when exposing Ollama beyond localhost

By default, Ollama binds to 127.0.0.1:11434. A manually started server can listen on the local network with:

OLLAMA_HOST=0.0.0.0:11434 ollama serve

However, the local Ollama API does not require authentication for access through http://localhost:11434. Do not change the bind address and expose port 11434 directly to the internet.

If remote access is required, restrict source addresses with a firewall or VPN, and place a reverse proxy with TLS and authentication in front of Ollama. Avoid direct router port forwarding. For browser applications, minimize both network access and the allowed origins configured through OLLAMA_ORIGINS.

Conclusion

Ollama lets you retrieve a model with ollama pull and run a local LLM from the terminal with ollama run. ollama ls reports disk state, ollama ps reports memory state, and ollama stop and ollama rm remove different things.

A Modelfile makes system prompts and runtime parameters reusable, while the API on localhost:11434 provides an integration point for your own applications. ollama launch is the next step toward experimenting with Claude Code and Codex.

After mastering these fundamentals, consider adding a browser interface with Open WebUI, comparing coding-agent models, deploying an authenticated reverse proxy, or implementing robust streaming in your own application.