Local models

Last updated October 15, 2026

Run the CLI's agent on a model on your own machine or network: Ollama, LM Studio, or any other server that speaks OpenAI's /v1/chat/completions, such as llama.cpp's server, vLLM or LocalAI. The CLI calls the server itself, so it can be on localhost, and the calls cost nothing.

Ollama and LM Studio

With Ollama or LM Studio running on its default port, there's nothing to set up. Tempr looks for both:

tempr models ollama/                       # the models Ollama has, priced free
tempr -m ollama/qwen3:8b "explain this repo"
tempr config set-model ollama/qwen3:8b     # the default for every run

A model's id is the provider's id, a slash, and the model's name on the server. tempr models lists the local models after your catalog's, and /model in the REPL offers them too. You still sign in to Tempr (tempr auth, or TEMPR_LICENSE_KEY in CI).

Other servers

Add any other server under provider in ~/.tempr/settings.json. It's OpenCode's shape, so a provider block from opencode.json works as it is:

{
  "provider": {
    "vllm": {
      "name": "vLLM on the GPU box",
      "options": {
        "baseURL": "http://192.168.1.20:8000/v1",
        "apiKey": "{env:VLLM_KEY}"
      },
      "models": {
        "qwen3-coder": { "name": "Qwen3 Coder", "limit": { "context": 131072, "output": 16384 } }
      }
    }
  },
  "disabled_providers": ["lmstudio"]
}

That server's models are then vllm/<model>: tempr -m vllm/qwen3-coder.

An entry called ollama or lmstudio changes the built-in one, to use another port, say. Without a baseURL it keeps the default address. --config <PATH> reads settings.json from that folder instead.

How a turn runs

Tempr's servers can't reach your machine, so with a local model the agent loop runs in the CLI: it calls the model, runs the tools it asks for, and calls it again, up to 100 times a turn unless --max-turns says otherwise. Approvals, permission rules, hooks, plan mode, /undo and /rewind, /compact and the sandbox work as they do with any other model.

• Free: tempr usage counts the tokens at $0, and they don't count toward --max-cost.
• Not on these turns: what only runs on Tempr's servers. That's delegating to a sub-agent, web_fetch, lookup_package and web search, and your organization's rules and skills. The workspace's own skills are still offered.
• Model size: Tempr's instructions and tools take about 2,000 tokens of every call, so agent work needs a model of a few billion parameters or more.
• Thinking: reasoning the server sends (reasoning_content, Ollama's reasoning, or <think> at the start of the text) shows as ✻ Thought for Ns, and --verbose prints it.

Tool calls

When a server says the model can't call tools, Tempr asks again without them. The model answers without reading or changing files, and a notice says so; pick a model that supports tool calling for agent work.

Small models sometimes write a tool call as text instead of making it. A reply that is nothing but calls to the tools the request offered runs as those calls: bare JSON, a ```json block, <tool_call> tags, [TOOL_CALLS], or an array of calls. It's held back while it streams, so the JSON doesn't show as an answer. Anything else is the answer, shown as written.

Reasoning

For a model that thinks, --reasoning and /reasoning work as for any other model, and the level is sent as reasoning_effort:

• Ollama says which of its models think. One that only switches thinking on and off, such as qwen3 or deepseek-r1, takes none and on. One with named levels, such as gpt-oss, takes those.
• Your own provider: "reasoning": true on a model offers none, low, medium and high; "reasoningLevels": [...] lists its own.
• default sends nothing, so the model reasons as it would.

tempr -m ollama/qwen3:8b --reasoning none "rename this variable"

Ollama's context window

Ollama runs a model with a small context window unless it's started with OLLAMA_CONTEXT_LENGTH, and silently drops the start of a request that doesn't fit, which is where the instructions and tools are. Tempr reads the window each loaded model runs with from Ollama's /api/ps, and uses it for the model list and for compacting before a turn. It also:

• says once when the window is under 8,192 tokens;
• says when a request didn't fit, and counts the request's whole size, so the next turn compacts first. /compact shortens the conversation at once.

For agent work, start Ollama with OLLAMA_CONTEXT_LENGTH=32768 or more.

In an organization

A local model's turns go from your computer to the model directly, so the Gateway's policies, your organization's rules and skills, and its usage reports don't cover them. An owner or admin can turn local models off for everyone in the Portal, under Agent playbook > Models. Then the CLI and Tempr Code stop listing them within 15 minutes, a turn on one is refused with the reason, and tempr doctor fails when your default model is a local one. The apps hold to the setting: it keeps Tempr from sending work to a local model, but it can't stop someone running a model outside Tempr.

Checking the setup

tempr doctor lists each local provider with how many models it has, or that it isn't running. One that isn't running is fine, unless your default model is on it: then the check fails.

Troubleshooting

Next steps

• CLI command reference: every command and flag.
• Settings files: what else goes in ~/.tempr/settings.json.
• Custom models: your own server, reached through Tempr.