Custom models

Run models on your own OpenAI-compatible server — vLLM, SGLang, a LiteLLM proxy, or a hosted OpenAI-compatible service — and call them through Tempr like any other model. Custom models are included with Chat Pro and Teams and on every Gateway plan.

Who gets it

On a Standard license, custom models don't appear in the model list, and calling one by id returns the Pro upgrade message. Models added while the license was Pro are kept and come back when it's Pro again.

Add an endpoint

An endpoint is one server: a base URL and the key Tempr sends with each request. In the Custom models section, fill in Add an endpoint and choose Test and add endpoint:

• Name — how the endpoint shows up in Tempr, for example Prod vLLM.
• Base URL — the server's OpenAI-compatible root, usually ending in /v1. Tempr calls <base URL>/chat/completions and <base URL>/models; a pasted …/chat/completions URL is trimmed back to the base.
• Auth scheme and key — how the key is sent, below. A key is required.
• Query parameters — optional, below.

Saving reads the server's /models list and sends one test chat completion capped at a single token; the endpoint is stored only if that call succeeds. If the server has no /models list, enter a model under Model to test with. When a list mixes chat models with embedding or image models, Tempr tries listed models that report a context length first, up to three.

How the key is sent

Custom endpoints must require a key; a server that accepts anyone can't be added. A custom header can't be one the HTTP stack owns: Host, Content-Length, Content-Type, Transfer-Encoding, Connection, or Cookie. The key is encrypted at rest and never shown again — leave it blank when editing an endpoint to keep the stored one.

Query parameters

Some servers need a fixed query string, such as ?api-version=2024-10-21. Add up to 5 name/value pairs, or paste a URL with a query string and Tempr splits it out; they're sent on every call, after the path. Query parameters are stored and shown in plain text, so a name that looks like a credential — one containing key, token, code, sig, secret, password, or auth — is refused. Send keys in a header instead.

Public HTTPS only

• The base URL must be https://, with no username or password in it.
• Tempr doesn't connect to localhost or private, loopback, or link-local addresses — including a public hostname that resolves to one. For a server on your own network, put it behind a public HTTPS address that requires the key.
• Redirects aren't followed. A server that answers with one gets a 502 that names the endpoint.

Add models

Choose Refresh models on an endpoint to see its /models list, with a suggested name for each model and its context length where the server reports one. Tick the ones you want and choose Add selected, or Add all. A model the server doesn't list can be added with Add a model by name, which sends one test call first.

Each model gets a name in Tempr, and you call it as custom/<name>. Names use lowercase letters, digits, ., _, and -, start with a letter or digit, and are up to 64 characters; they're unique within an account, organization, or license. The model's name on the server (for vLLM, its --served-model-name) can be anything, slashes included.

• Context length and Default output cap are optional. The cap is sent as max_tokens when a request doesn't set its own.
• An account, organization, or license can have up to 10 endpoints, each with up to 50 models.

Call a custom model

Gateway

curl https://api.temprhq.io/v1/chat/completions \
  -H "Authorization: Bearer $TEMPR_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "custom/prod-llama", "messages": [{"role": "user", "content": "Hello"}]}'

Custom models work on /v1/chat/completions — streaming or not, with tool calling — and /v1/messages, and GET /v1/models lists the ones a key can use.

• Allowlists — add custom/prod-llama to a virtual key's allowed models, or custom/* to allow every custom model. Per-member model limits in an organization match exact ids only, so list custom models there one by one.
• Fallback chains — a chain can mix custom and built-in models, for example fast: custom/prod-llama, deepinfra/meta-llama/Llama-3.3-70B-Instruct-Turbo. A failing endpoint only benches its own model.
• Request logs record which endpoint and which server-side model served each call.

Chat and the CLI

On a Pro or Teams license, custom models appear as custom/<name> in the model picker in Visual Studio, VS Code, and JetBrains IDEs, and in the CLI's model list. Pick one like any other model, or pass it to the CLI with --model custom/prod-llama. A Pro license uses the models added to it; a Teams seat uses its organization's. Agent mode needs a server with tool calling turned on (see Server notes).

Pricing and budgets

Tempr doesn't know what your server costs to run, so a custom model is unpriced unless you enter a price per million input and output tokens (both, or neither). Unpriced usage records its tokens at $0: budgets and cost totals don't count it, and the budget and key pages show those tokens separately so you can see what's missing. With a price set, usage is costed like any other model and counts toward budgets.

When a base URL belongs to a provider Tempr already supports — DeepInfra's API, for example — the Portal says so and marks the endpoint Built-in provider. Use the built-in provider for priced usage.

Edit, move, rename, remove

• Edit a model to change its server-side name, context length, output cap, price, or endpoint. Moving it to another endpoint, or changing the server-side name, sends a test call first. Its custom/ id stays the same, so callers don't change.
• Rename changes the custom/ id. The Portal first shows which virtual keys name the model, and renaming updates their allowlists and fallback chains. Apps and scripts that call the old id directly get model_not_found.
• Turn off a model or a whole endpoint to stop calls without deleting anything.
• Remove endpoint deletes it with all its models. The Portal lists those models, and the virtual keys that name them, before you confirm.

In an organization, each change is recorded in the audit log. Keys are never logged.

Server notes

• vLLM — for example vllm serve Qwen/Qwen2.5-7B-Instruct --api-key <key> --served-model-name qwen-7b. For tool calling, add --enable-auto-tool-choice --tool-call-parser <parser> with the parser for your model family (hermes for Qwen 2.5). Base URL: https://your-host/v1.
• SGLang — --api-key, --served-model-name, and --tool-call-parser for tool calling. Base URL: https://your-host/v1.
• LiteLLM proxy — the proxy's URL ending in /v1, with a LiteLLM key sent as Bearer. Its /models list shows the model names the proxy is configured with.
• Servers that scale to zero — saving waits 60 seconds for the test call. If a cold server takes longer, save again once it's up. Calls through the gateway, Chat, and the CLI wait up to 10 minutes.
• Servers differ: some don't report token usage while streaming, and tool calling depends on server flags, so agent mode can fail with the server's own error.

When a call fails