Running Claude Code with Local LLMs in 2026: LM Studio, Jan, and Ollama Setup and Caveats

Introduction: Can Claude Code Run with a Local LLM?

You may want to use Claude Code without watching API charges or subscription limits. You may also need to keep sensitive source code away from hosted AI services. If you already run models through Ollama or another local runtime, the obvious next question is whether that model can power Claude Code’s file editing and test execution as well as chat.

The short answer is yes. Claude Code normally uses Claude models, but you can redirect it to a local server that implements an Anthropic-compatible API. As of July 2026, LM Studio, Jan, and Ollama each publish official instructions for connecting their runtimes to Claude Code.

There is an important qualification: a model that connects successfully is not necessarily a useful coding agent. Reliable tool use, adequate context, and strong instruction following are essential. Anthropic also does not endorse third-party gateway products or support routing Claude Code to non-Claude models. Treat this setup as a compatibility path maintained by the local runtime, not as a first-party Anthropic configuration.Anthropic’s gateway guidance

1. Claude Code and Claude Models Are Separate Components

1-1. What Claude Code Does

Claude Code is an agentic coding tool that can inspect a codebase, edit files, and run commands.Official Claude Code overview

Unlike a basic chat interface, it does not necessarily stop after generating one answer. Claude Code sends your request to an LLM, executes the tool selected by the model, and returns the result to the model. The model then decides whether to inspect more files, make a change, run a test, or report completion.

flowchart TD
    accTitle: The Claude Code agent loop
    accDescr: A user request goes to an LLM, which selects a tool. Claude Code reads or edits files or runs tests, returns the result to the LLM, and repeats until the completion condition is met.
    A[User request] --> B[LLM decides the next action]
    B --> C[Claude Code executes a tool]
    C --> D[Read files, edit code, or run tests]
    D --> E[Return the result to the LLM]
    E --> F{Completion condition met?}
    F -- No --> B
    F -- Yes --> G[Report the result]

The Claude Code agent loop

1-2. What the Claude Model Does

It helps to treat the control layer and the reasoning model as separate components:

• Claude Code manages the conversation, permission checks, tool execution, and delivery of tool results.
• The Claude model decides what to do next and generates code or explanations.

Under the standard configuration, a Claude model such as Opus, Sonnet, or Haiku performs the reasoning. A local configuration replaces that reasoning backend with another model. Keeping Claude Code’s interface and tools does not reproduce the reasoning quality of a Claude model.

1-3. Claude Code Does Not Load GGUF Files

Claude Code has no built-in facility for loading a GGUF file or model weights. LM Studio, Jan, or Ollama loads the model into memory and exposes an HTTP server. Claude Code sends requests to that server through an API.

2. How the Local LLM Connection Works

Claude Code can use a local server when that server understands the Anthropic Messages format. The central endpoint is POST /v1/messages.

flowchart LR
    accTitle: Claude Code local LLM connection architecture
    accDescr: Claude Code receives a user request and sends an Anthropic Messages API request to LM Studio, Jan, or Ollama, which performs inference with a local LLM.
    U[User] --> C[Claude Code]
    C -->|Anthropic Messages API| S{Local model server}
    S --> L[LM Studio]
    S --> J[Jan]
    S --> O[Ollama]
    L --> M[Local LLM]
    J --> M
    O --> M

Local LLM connection architecture

2-1. What an Anthropic-Compatible API Means

A compatible API accepts the URL structure and request shape expected by another service’s client. If the local server can interpret the Messages API format expected by Claude Code, Claude Code does not need to manage where the model weights run.

These are the two main environment variables:

export ANTHROPIC_BASE_URL=http://localhost:<PORT>
export ANTHROPIC_AUTH_TOKEN=<TOKEN_OR_DUMMY_VALUE>

ANTHROPIC_BASE_URL selects the inference endpoint. ANTHROPIC_AUTH_TOKEN supplies a bearer token. A server with authentication disabled may use a dummy value to satisfy the client configuration. If authentication is enabled, use a real server token.

2-2. The Support Boundary

LM Studio, Jan, and Ollama document how their products integrate with Claude Code. Anthropic, however, does not support routing Claude Code to non-Claude models. New Claude Code releases may also introduce headers, API fields, or tool behavior that a compatibility server does not yet handle.

When a setup breaks, check both the Claude Code and local-server versions. Inspect the /v1/messages logs and the model’s tool-call response instead of assuming the model itself is the only possible cause.

3. Prerequisites

First, confirm that Claude Code is available:

claude --version

Do not begin with a critical repository. Prepare a small Python project such as:

local-claude-test/
├── calculator.py
└── test_calculator.py

Put it under Git so every edit is reversible and reviewable:

git init
git add .
git commit -m "Initial commit"

Do not place .env files, private keys, or cloud credentials in the test project. Unless remote access is intentional, bind the model server to localhost rather than exposing it on an external interface.

4. Method 1: Connect Through LM Studio

4-1. Prepare the Model and Server

Download a coding-oriented model in LM Studio and load it into memory. Check the model card for tool-use support and configure its context window. LM Studio recommends more than roughly 25K tokens for Claude Code.LM Studio’s Claude Code integration guide

Start the server from the Developer screen or with the CLI:

lms server start --port 1234

The default port is 1234. LM Studio exposes an Anthropic-compatible POST /v1/messages endpoint for Claude Code.

4-2. Test the API Independently

Before launching Claude Code, use the model ID displayed by LM Studio to test the endpoint:

curl http://localhost:1234/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer lmstudio" \
  -d '{
    "model": "<LM_STUDIO_MODEL_ID>",
    "max_tokens": 64,
    "messages": [{"role": "user", "content": "Reply with OK."}]
  }'

If this returns an HTTP error, fix the server state, model ID, or authentication before involving Claude Code. A successful text response does not prove that tool use works, so you still need the agent tests described later.

4-3. Launch Claude Code

Set the variables shown in LM Studio’s official instructions:

export ANTHROPIC_BASE_URL=http://localhost:1234
export ANTHROPIC_AUTH_TOKEN=lmstudio
export CLAUDE_CODE_ATTRIBUTION_HEADER=0

claude --model <LM_STUDIO_MODEL_ID>

CLAUDE_CODE_ATTRIBUTION_HEADER=0 omits the attribution block Claude Code otherwise adds to its system prompt. If Require Authentication is enabled in LM Studio, replace the dummy lmstudio value with an API token issued by LM Studio.

LM Studio makes model state, GPU offload, context configuration, and API logs easy to inspect from a GUI. It is a good starting point when learning how the compatibility layer behaves or comparing several models and quantizations.

5. Method 2: Connect Through Jan

5-1. Use the Integrations Screen

Open the following location in Jan Desktop:

Settings → Integrations → Claude Code

You can assign installed models to three tiers:

• Large Model corresponds to Opus and is intended for complex planning and reasoning.
• Medium Model corresponds to Sonnet and handles routine work.
• Small Model corresponds to Haiku and handles lightweight tasks.

Select the models and click Save & Enable. Jan will then serve as Claude Code’s local backend.Jan’s Claude Code integration guide

Do not base the assignments on parameter count alone. A smaller model that returns tool calls consistently may be suitable for light work, while a larger model with unreliable tool formatting may fail as an agent. Your machine may also be unable to keep three models resident at once. For an initial test, assigning the same dependable model to multiple tiers is reasonable.

5-2. Launch from the CLI

Jan CLI starts the model server and sets the environment variables for Claude Code. Claude Code itself must already be installed.

jan launch claude

To select a model directly:

jan launch claude --model <MODEL_ID>

Jan CLI uses port 6767 by default. Add -v when you need full server logs. Model inference settings are adjusted from Jan Desktop.Official Jan CLI documentation

Jan is particularly convenient when you want a dedicated Claude Code integration screen, tier-based model assignment, or a launch command that avoids manual environment management.

6. Method 3: Connect Through Ollama

6-1. Prepare a Model and Launch

Download a local model:

ollama pull <MODEL_NAME>

The shortest launch path is:

ollama launch claude

To pin a model, specify it explicitly:

ollama launch claude --model <MODEL_NAME>

For non-interactive use, arguments after -- are passed to Claude Code:

ollama launch claude \
  --model <MODEL_NAME> \
  --yes \
  -- -p "Explain the structure of this repository"

The --yes option skips selectors and pulls the model if necessary, so it requires --model.Ollama’s Claude Code integration guide

6-2. Connect Manually with Environment Variables

Ollama also documents a manual setup that makes the routing explicit:

export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""
export ANTHROPIC_BASE_URL=http://localhost:11434

claude --model <MODEL_NAME>

Clearing ANTHROPIC_API_KEY helps prevent a saved Anthropic API key from taking precedence over the token intended for the local endpoint.

Ollama supports cloud models as well as locally downloaded models. Running ollama launch claude is therefore not proof of a fully local session. Select a model that is present locally, then inspect the Ollama logs and network destinations. Ollama recommends a context window of at least 64K for local Claude Code models and the same or more for larger repositories.

7. Choosing a Local Model for Claude Code

7-1. Reliable Tool Use Is Close to Mandatory

Claude Code requires more than text generation. The model must choose tools such as file reading, editing, and shell execution, then supply valid arguments in the expected format.

A model without dependable tool use may:

• Describe an action without actually invoking a tool.
• Produce a broken tool name or malformed JSON arguments.
• Read files but never progress to editing.
• Repeat the same command indefinitely.
• Claim success while tests are still failing.

The ability to write code in chat is only one prerequisite. Test the model inside Claude Code across file reading, editing, command execution, and recovery from failure.

7-2. Provide Enough Context

Claude Code’s context includes its system prompt, tool definitions, conversation history, file contents, command output, error logs, and modified code. The model does not retain memory automatically between API requests, so Claude Code resends the context it needs.How Claude Code uses prompt caching

With insufficient context, a model may forget the original requirement, reread the same files, fail to process long logs, or change its implementation strategy midway through the task.

LM Studio’s recommendation of more than roughly 25K is best treated as a lower bound for basic experiments. Ollama’s recommendation of 64K or more is a more practical target for agent work. Larger context settings also consume more memory through the KV cache and related allocations. On a MacBook or Windows PC, balance model size, quantization, and context length, then measure them on a small repository first.

7-3. Map the Model Tiers

Claude Code model aliases can be redirected to provider-recognized model IDs with environment variables.Claude Code model configuration

export ANTHROPIC_DEFAULT_OPUS_MODEL=<LARGE_MODEL>
export ANTHROPIC_DEFAULT_SONNET_MODEL=<MEDIUM_MODEL>
export ANTHROPIC_DEFAULT_HAIKU_MODEL=<SMALL_MODEL>

Jan exposes the equivalent mapping through its GUI. With LM Studio or Ollama, use the exact IDs recognized by the server. Model aliases and configuration behavior can change as Claude Code evolves, so check the current model-configuration page before standardizing a deployment.

7-4. Evaluate Agent Behavior, Not Just Chat Quality

A code-generation benchmark is not enough to select an agent model. Practical Claude Code work requires the ability to:

• Preserve the tool-call format.
• Edit only the requested scope.
• Interpret command output and exit status.
• Recover after an error.
• Avoid inventing unspecified requirements.
• Retain the completion condition across several steps.

Even the same model and quantization can behave differently with another runtime, context setting, or prompt template. The most reliable approach is to maintain a small evaluation set based on your own representative tasks.

8. Practical Test: Edit a File and Run Tests

After connecting through any of the three runtimes, execute the same tasks in sequence. Start with a read-only request rather than granting immediate permission to modify the project.

8-1. Ask for an Explanation

Explain this project’s structure and the role of calculator.py.
Do not modify any files.

Check whether the model actually reads the files, invents nonexistent structure, or violates the no-edit instruction.

8-2. Request a Minimal Fix

Investigate the defect in calculator.py.
Explain the cause, then make the smallest necessary change.

Confirm that the model selects the right file, avoids inventing requirements, and produces a small diff. Review the result with git diff yourself.

8-3. Request Test Execution and Recovery

Run the tests. If any test fails, investigate the cause and fix it.
When all tests pass, summarize the changes.

Observe whether it chooses the correct test command, understands the error, reruns the test after editing, and identifies completion accurately.

Record results in a table like the one below. Enter measured values only.

If you calculate a success rate, repeat each model with the same initial state and prompt, and publish the number of trials. Do not generalize a single successful run into a claim about the model as a whole.

9. Risks of Giving a Small Model Execution Access

Local execution and safe execution are not the same thing. Even if no code is sent to a hosted model, Claude Code can manipulate local files and commands within the permissions you grant.

A small or tool-unstable model may edit files outside the requested scope, weaken tests to make them pass, hide an error, propose a dangerous command, rewrite too much code, or report success before the task is complete.

Keep these constraints in place during initial testing:

• Do not bypass Claude Code’s permission prompts.
• Do not use --dangerously-skip-permissions.
• Work in a disposable, Git-managed sample project.
• Keep .env files, SSH keys, and cloud credentials out of the workspace.
• Use Docker, a Dev Container, or a VM when stronger isolation is required.
• Review the diff and test result before committing.

Ollama’s integration documentation also limits permission-bypass advice to isolated environments. Treat file reads, edits, network access, and shell execution as separate capabilities, and grant only what the current task requires.

10. Verify That the Session Is Actually Local

“Local” can mean at least two different things:

1. Model inference runs on your own computer.
2. Claude Code and every related process make no external network connections.

The first does not guarantee the second. Update checks, metrics, error reports, WebFetch, web search, MCP servers, and plugins may still use the network.

10-1. Inspect Model-Server Logs

Open the LM Studio, Jan, or Ollama logs. Confirm that each Claude Code action reaches the expected localhost Messages endpoint. Check the selected model name as well, and make sure the runtime is not forwarding the request to a cloud model or another upstream service.

10-2. Reduce Nonessential Claude Code Traffic

Set the following variable to disable Claude Code’s nonessential traffic as a group:

export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1

According to Anthropic’s data-usage documentation, this disables usage metrics, applicable error reporting, and feedback surveys. It is not a universal offline switch. The WebFetch domain-safety check and official plugin marketplace auto-install have separate controls.Claude Code data usage and traffic controls

Claude Code also stores local session transcripts in plaintext under ~/.claude/projects/ for 30 days by default to support session resumption. That is separate from external transmission, but retention and filesystem permissions still matter on shared machines or when handling regulated data.

10-3. Observe Connections at the Operating-System Level

• macOS: nettop and lsof -i
• Linux: ss -tpn and lsof -i
• Windows: Resource Monitor and TCPView
• Stricter verification: a firewall with an outbound allowlist, a VM, or an isolated container

Confirm both that Claude Code connects to the expected endpoint—such as localhost:1234, localhost:6767, or localhost:11434—and that no connection carrying code goes to an external address. Networked tools such as web search, WebFetch, and MCP can create external traffic even while inference remains local.

11. Which Runtime Should You Choose?

LM Studio is a friendly first choice when you want to inspect every setting. Jan is attractive when you want GUI-based model tiers. Ollama is a natural fit for terminal-heavy and scripted workflows.

The runtime interface is rarely the largest determinant of output quality. The model, quantization, context window, prompt compatibility, memory bandwidth, and CPU or GPU performance matter more. Compare the runtimes with the same model and equivalent settings whenever possible.

12. The Quality Gap Versus Cloud Claude Models

Local models make it easier to keep inference data on your own hardware, iterate without per-token API charges, build an offline configuration, and compare models freely.

Cloud-hosted Claude models will often have an advantage on long-running agent tasks, large-repository comprehension, complex refactoring, dependable tool use, recovery from errors, and preservation of long instructions. The exact gap depends on your chosen local model and hardware, so it should not be inferred from a model name or parameter count alone.

A pragmatic split is to use a local model for repository explanations, routine transformations, small bug fixes, and initial inspection of sensitive code. Reserve difficult architecture work or large changes for a cloud Claude model when policy permits. Reassess whether the relevant source code may be transmitted before switching back to the cloud.

Conclusion

Claude Code and Claude models are separate components. When LM Studio, Jan, or Ollama exposes a model through an Anthropic-compatible API, Claude Code can route its reasoning requests to that local LLM.

A successful HTTP connection is not enough. Test reliable tool use, context settings ranging from more than 25K toward 64K or higher, instruction following, and your machine’s memory limits on real tasks. Do not grant broad execution permissions at the outset; work in an isolated Git repository and review every diff and test result.

If fully local operation is a requirement, verify more than the selected model and localhost logs. Audit Claude Code’s nonessential traffic, web-enabled tools, MCP servers, and operating-system connections. Once you have measured quality and safety, using local LLMs and cloud Claude models for different classes of work is usually the most practical approach.