providers.md Raw History Folder

zot providers

zot ships with built-in providers and a model catalog. You can select models with /model, list them with zot --list-models, and add private models in $ZOT_HOME/models.json.

HTTP proxy

Set one global proxy for zot-managed HTTP and HTTPS traffic with the http_proxy key in $ZOT_HOME/config.json:

{
  "http_proxy": "http://127.0.0.1:7890"
}

zot applies this setting at startup. Existing HTTP_PROXY, HTTPS_PROXY, http_proxy, and https_proxy environment variables take precedence for their corresponding protocol. Standard NO_PROXY and no_proxy bypass lists remain effective. Restart zot after changing the setting.

config.json is not a credential store. If the proxy URL contains a username or password, prefer setting the proxy environment variables in a protected shell or service configuration rather than saving those credentials in config.json.

Use /login in interactive mode. Type in either provider picker to filter the list by provider ID or display name.

api key: stores an API key in $ZOT_HOME/auth.json when the provider uses a normal key.
subscription: stores OAuth credentials for subscription-backed providers.

Use /logout to remove stored credentials.

Some providers need more than a single pasted key. For those providers, /login shows setup instructions instead of opening a localhost browser form. This avoids broken browser flows in SSH, containers, and kubectl exec sessions.

Command-backed API keys

A provider can obtain its API key from a password manager or another local program. Configure this directly in $ZOT_HOME/auth.json:

{
  "openai": {
    "api_key_command": {
      "program": "op",
      "args": ["read", "op://Work/OpenAI/credential"],
      "timeout_ms": 120000
    }
  }
}

Custom provider credentials use the same object under additional_api_key_creds:

{
  "additional_api_key_creds": {
    "my-company": {
      "api_key_command": {
        "program": "secret-tool",
        "args": ["read", "my-company-api-key"]
      }
    }
  }
}

The program is started directly rather than through a shell. The timeout defaults to 120 seconds. It must write one non-empty line to stdout, with no more than 64 KiB of output. Trailing CR/LF characters are removed. Successful results stay only in process memory and are reused until zot exits.

Command-backed credentials count as logged in without being executed. zot materializes one only when selecting that provider; background model discovery skips it. /login with a normal key replaces the command, while /logout clears it. Because auth.json can cause program execution, keep it user-writable only and do not use files from untrusted sources.

Setup-instruction providers:

Amazon Bedrock
Google Vertex AI
Cloudflare Workers AI
Cloudflare AI Gateway
Azure OpenAI Responses

Subscription providers

These providers support subscription login:

Provider	Notes
Anthropic	Claude Pro/Max OAuth credentials.
OpenAI Codex	ChatGPT Plus/Pro Codex subscription route. Separate from the OpenAI API-key provider.
Kimi	Kimi subscription login.
xAI	SuperGrok or X Premium device-code login. The browser URL is prefilled with the device code.
GitHub Copilot	GitHub Copilot token flow.

OAuth tokens are stored in $ZOT_HOME/auth.json and refreshed when refresh is available.

GitHub Copilot model availability

The built-in Copilot catalog excludes retired models and utility-only models that are not selectable for chat. The default is claude-sonnet-5. Availability still depends on your plan and organization policies. At startup, zot queries Copilot's authenticated /models endpoint in the background and filters the runtime catalog using picker visibility, policy state, and tool-call support. Individual accounts with no eligible picker entries can fall back to explicitly enabled model policies; other account types cannot. zot never enables policies automatically.

Account availability stays in memory, not the shared model cache. Until discovery succeeds, the static catalog remains visible; failed refreshes retain the previous snapshot. Command-backed credentials are not executed for background discovery. Restart zot after changing accounts to refresh the snapshot. This filter is a selection aid, not an authorization boundary. The website displays the public catalog, not any individual account's availability.

Claude uses the Anthropic Messages API with Copilot Bearer authentication, without Claude Code identity injection. GPT, Grok, and MAI use Responses. Gemini and Kimi use Chat Completions without reasoning_effort; their reasoning is managed by the endpoint. All three routes send user/agent initiator metadata and image-request headers where appropriate.

As of September 8, 2026, the catalog includes 27 public chat models: Claude Fable 5/5.1, Haiku 4.5, Opus 4.7/4.8/4.8 Fast/5, Sonnet 5, Gemini 3.5/3.6/3.7/3.8 Flash, GPT-5 mini, GPT-5.3-Codex, GPT-5.4/5.4 mini/5.5, GPT-5.6 Luna/Sol/Terra, GPT-6 Astra, Grok 4.5/4.6, Kimi K2.7 Code/K3, and MAI-Code-1-Flash/1.1-Flash. MAI-Code-1-Flash uses the public picker ID mai-code-1-flash-picker and is scheduled to retire on September 10, 2026.

Internal search, execution, and compaction models are not chat choices. GPT-5.4 nano is excluded because GitHub offers it in the Codex VS Code extension, not Copilot Chat. New entries use Copilot-specific limits and token prices from models.dev; these are not a quote for your subscription bill. Existing zero-priced entries do not imply unlimited free use.

Claude Fable models have different data-retention terms: Anthropic retains prompts and outputs by default for safety classifiers. Eligible enterprises can request a time-limited zero-data-retention exception from GitHub; enabling a model does not itself grant that exception. Review GitHub's model availability and data-retention notes before using Fable with sensitive data.

Sonnet 4.6 is also excluded from the general catalog: GitHub retired it on September 1, 2026, except for individual annual Copilot Pro and Pro+ subscribers. Eligible subscribers can add it through $ZOT_HOME/models.json. See GitHub's model retirement history and plan exceptions.

GPT-6 Astra

Select gpt-6-astra with provider openai, openai-responses, openai-codex, or github-copilot, for example:

zot --provider github-copilot --model gpt-6-astra

Astra uses the Responses API and supports low, medium, high, xhigh, and max reasoning. OpenAI API-key and Codex subscription entries use the 272,000-token short-context tier by default; the Copilot entry uses a 1,050,000-token context window. All entries include the 128,000-token output limit and standard token prices, including the higher rates for prompts above 272,000 tokens. Subscription access depends on your plan, rollout, and organization policy; catalog availability does not grant access. See the OpenAI model specifications and GitHub Copilot availability.

API-key providers

These providers can use environment variables. Simple API-key providers can also be configured through /login. Providers that require extra cloud setup show instructions and should be configured with environment variables.

Provider	Environment variable	Stored key
Anthropic	`ANTHROPIC_API_KEY`	`anthropic`
OpenAI	`OPENAI_API_KEY`	`openai`
OpenAI Responses	`OPENAI_API_KEY`	`openai-responses`
Kimi	`KIMI_API_KEY` or `MOONSHOT_API_KEY`	`kimi`
Google Gemini	`GEMINI_API_KEY` or `GOOGLE_API_KEY`	`google`
DeepSeek	`DEEPSEEK_API_KEY`	`deepseek`
Moonshot AI	`MOONSHOT_API_KEY`	`moonshotai`
Moonshot AI China	`MOONSHOT_API_KEY`	`moonshotai-cn`
Groq	`GROQ_API_KEY`	`groq`
xAI	`XAI_API_KEY`	`xai`
Cerebras	`CEREBRAS_API_KEY`	`cerebras`
Together AI	`TOGETHER_API_KEY`	`together`
Hugging Face	`HF_TOKEN`	`huggingface`
OpenRouter	`OPENROUTER_API_KEY`	`openrouter`
Gondola	`GONDOLA_API_KEY`	`gondola`
Mistral	`MISTRAL_API_KEY`	`mistral`
ZAI	`ZAI_API_KEY`	`zai`
Xiaomi MiMo	`XIAOMI_API_KEY`	`xiaomi`
Xiaomi Token Plan Amsterdam	`XIAOMI_TOKEN_PLAN_AMS_API_KEY`	`xiaomi-token-plan-ams`
Xiaomi Token Plan China	`XIAOMI_TOKEN_PLAN_CN_API_KEY`	`xiaomi-token-plan-cn`
Xiaomi Token Plan Singapore	`XIAOMI_TOKEN_PLAN_SGP_API_KEY`	`xiaomi-token-plan-sgp`
MiniMax	`MINIMAX_API_KEY`	`minimax`
MiniMax China	`MINIMAX_CN_API_KEY` or `MINIMAX_API_KEY`	`minimax-cn`
Fireworks	`FIREWORKS_API_KEY`	`fireworks`
Vercel AI Gateway	`AI_GATEWAY_API_KEY`	`vercel-ai-gateway`
OpenCode Zen	`OPENCODE_API_KEY`	`opencode`
OpenCode Go	`OPENCODE_API_KEY`	`opencode-go`
GitHub Copilot token	`COPILOT_GITHUB_TOKEN` or `GITHUB_COPILOT_TOKEN`	`github-copilot`
Cloudflare Workers AI	`CLOUDFLARE_API_KEY`	`cloudflare-workers-ai`
Cloudflare AI Gateway	`CLOUDFLARE_API_KEY`	`cloudflare-ai-gateway`
Azure OpenAI Responses	`AZURE_OPENAI_API_KEY`	`azure-openai-responses`

When Gondola credentials are available, zot refreshes its public text-model catalog in the background and adds the discovered models to /model.

Example:

export OPENROUTER_API_KEY=...
zot --provider openrouter

Local LM Studio server

Register the lmstudio provider through /login, choose api key, and select LM Studio. Enter the server URL, for example http://localhost:1234, followed by an optional API key. Zot validates the URL and stores the connection in $ZOT_HOME/auth.json. Saving it does not require the server to be online.

A registered server URL is required for discovery. Zot does not probe localhost, and an API key alone does not enable discovery. Unlike llama.cpp's LLAMA_BASE_URL, LMSTUDIO_BASE_URL is not a registration mechanism. Your current provider and selected model do not change when registering a server.

Open /model to refresh and select a discovered model. Zot also refreshes after saving the connection, in the background on startup, and when selecting or resuming an LM Studio model. After registration, you can select a model directly:

zot --provider lmstudio --model my-local-model

Discovery prefers /api/v1/models and falls back to /v1/models when the native endpoint returns 404, 405, or 501. Authentication errors do not trigger fallback. The native list excludes entries explicitly marked as embedding models. The fallback only supplies IDs, so embedding filtering is best-effort.

Loaded models use their instance IDs and configured context lengths. Unloaded models use their keys and advertised maximum context lengths. Quantization-specific keys are preserved, but the separate native variant list is not expanded into additional choices. Listing models does not load or download them. Inference with an unloaded model depends on LM Studio's auto-loading settings. There is no /llama-style management interface for LM Studio.

Missing context metadata defaults to 32,768 tokens. Output defaults to at most 4,096 tokens, or a quarter of the context window for smaller models. These are estimates, not server guarantees. Discovered models have zero API pricing, and token usage comes from inference responses. Reasoning controls are not enabled automatically from capability metadata, which does not mean server-side reasoning is disabled. Override metadata under lmstudio in $ZOT_HOME/models.json, especially when an unloaded model will use a smaller context than advertised. User entries take precedence over discovery.

The discovered catalog stays in memory, separate from the six-hour cloud model cache and llama.cpp's entries. Failed refreshes retain the previous snapshot, and /model reports discovery errors while keeping other providers accessible. Use /logout lmstudio to remove the connection and stop discovery. Background refresh never executes API-key commands.

The existing coding loop, tools, permissions, and built-in provider pricing remain unchanged. For older builds without LM Studio in /login, the manual OpenAI-compatible setup in README.md remains available.

Local llama.cpp router

The llama.cpp provider connects to a multi-model router and is separate from the ollama provider. Ollama normally uses http://localhost:11434; entering that URL for llama.cpp management produces a 404 because Ollama does not implement the router endpoints.

Install a current llama.cpp build. On macOS:

brew install llama.cpp
# use `brew upgrade llama.cpp` for an existing installation

Start llama-server without --model, -m, or -hf. Those flags select one model and disable the router behavior zot expects.

mkdir -p ~/llama-models

llama-server \
  --models-dir ~/llama-models \
  --no-models-autoload \
  --jinja \
  --host 127.0.0.1 \
  --port 8080 \
  -ngl 999 \
  -c 32768

This configuration discovers GGUF files below ~/llama-models, leaves model loading under explicit user control, enables chat templates, requests GPU offload, and sets a 32K context. Tune the GPU layers and context size for the available memory.

Check the management API before opening zot:

curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/models

/models must return JSON with a data array. A 404 indicates the wrong URL, an older server build, or single-model mode.

To save the connection, run /login, choose api key, and select llama.cpp. Enter http://127.0.0.1:8080, without /v1, followed by an optional bearer key. zot validates the URL and stores both values in $ZOT_HOME/auth.json. Environment variables override the saved connection:

export LLAMA_BASE_URL=http://127.0.0.1:8080
export LLAMA_API_KEY=optional-secret

If a key is configured, start the router with the same --api-key value. A server bound only to 127.0.0.1 normally does not need authentication.

After login, /llama becomes visible. It shows model state, searches Hugging Face for GGUF repositories, offers available quantizations, reports download and load progress, and explicitly loads or unloads models. Press d on a router-downloaded cache model to ask the router to remove it after confirmation. Models discovered through --models-dir or a preset are not removable through the router API and must be deleted from their configured source. Some llama.cpp versions retain shared Hugging Face repository artifacts such as mmproj files after removing the selected GGUF; remove that repository's cache directory manually if those artifacts are no longer needed. HF_TOKEN raises Hugging Face API limits and may be required for repository metadata. For gated files, approve access first and provide the authorized token to the llama-server process because the router performs the download.

Opening /model refreshes the router and displays its loaded models under provider llama.cpp. Unloaded models cannot handle inference and therefore remain exclusive to /llama until loaded. Inference is sent to the router's derived /v1 endpoint.

Ollama downloads are stored in Ollama's internal layout and do not automatically become llama.cpp models. Obtain a GGUF copy through /llama or place GGUF files under ~/llama-models, then restart the router after adding files manually.

Cloud providers

Amazon Bedrock

Bedrock is configured with AWS credentials, not a generic zot API-key entry. Use one of these credential sources:

# AWS profile
export AWS_PROFILE=your-profile

# IAM access keys
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # only for temporary credentials

# Bedrock API key bearer token
export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...

# Region
export AWS_REGION=us-east-1

ECS task roles, IRSA, and other AWS SDK credential-chain sources are also supported.

Example:

AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-... AWS_REGION=us-east-1 \
  zot --provider amazon-bedrock --model anthropic.claude-sonnet-4-5-20250929-v1:0

Some Bedrock models require regional inference-profile IDs for on-demand throughput, such as us. or eu. prefixed model IDs. zot rewrites known families automatically where possible. Claude Opus 5 uses global.anthropic.claude-opus-5; zot maps the bare foundation-model ID to that global profile and supports its adaptive reasoning levels. Explicit profile IDs and ARNs are left unchanged.

Google Vertex AI

Vertex can use a Google API key when available:

export GOOGLE_CLOUD_API_KEY=...
zot --provider google-vertex

For service-account or application-default credentials, set the standard Google environment variables used by your deployment.

Cloudflare AI Gateway

Cloudflare AI Gateway needs a Cloudflare token plus account and gateway IDs:

export CLOUDFLARE_API_KEY=...
export CLOUDFLARE_ACCOUNT_ID=...
export CLOUDFLARE_GATEWAY_ID=...
zot --provider cloudflare-ai-gateway

Cloudflare Workers AI

Workers AI needs a Cloudflare token and account ID:

export CLOUDFLARE_API_KEY=...
export CLOUDFLARE_ACCOUNT_ID=...
zot --provider cloudflare-workers-ai

Azure OpenAI Responses

export AZURE_OPENAI_API_KEY=...
export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
export AZURE_OPENAI_API_VERSION=v1 # optional, v1 is the default
zot --provider azure-openai-responses

The provider uses Azure's Responses API. If deployment names differ from zot model IDs, map them without changing the catalog:

export AZURE_OPENAI_DEPLOYMENT_NAME_MAP='gpt-5.6-luna=luna-preview,gpt-5.6-sol=sol-preview'

Auth file

Credentials are stored in $ZOT_HOME/auth.json with user-only permissions when zot creates the file.

Example:

{
  "anthropic": { "api_key": "sk-ant-..." },
  "openai": { "api_key": "sk-..." },
  "google": { "api_key": "..." },
  "additional_api_key_creds": {
    "openrouter": { "api_key": "..." },
    "mistral": { "api_key": "..." }
  }
}

The top-level keys are used for providers with dedicated credential fields. Other API-key providers are stored under additional_api_key_creds. Prefer /login so zot writes the correct schema.

Custom providers and models

Use $ZOT_HOME/models.json for private models, deployment aliases, local servers, or OpenAI-compatible gateways that are not in the built-in catalog. User entries override built-in entries with the same provider and model ID, and adding a models.json no longer hides the built-in catalog: your entries are merged on top of the baked-in and live-discovered models.

A top-level provider key that is not a built-in id defines a custom provider. Give it a provider-level baseUrl and an api wire format (openai for OpenAI-compatible Chat Completions, the default, openai-responses for the OpenAI Responses API, or anthropic for the Anthropic Messages API). A model-level baseUrl overrides the provider-level one for that model, and an unknown api value falls back to openai with a warning.

{
  "providers": {
    "my-company": {
      "baseUrl": "https://llm.mycompany.com/v1",
      "api": "openai",
      "models": [
        { "id": "company-llm-v2", "name": "Company LLM v2" }
      ]
    }
  }
}

Custom providers are first-class: they appear in --list-models, /model, and /login. models.json never stores secrets. Supply the key through /login, --api-key, or a derived environment variable in upper snake case, so my-company reads MY_COMPANY_API_KEY. Because many self-hosted gateways do not expose a model-list endpoint, custom provider keys are accepted and stored without a verification probe; an invalid key surfaces on the first model call.

To retrieve this custom provider's key from a password manager, add a matching entry to $ZOT_HOME/auth.json:

{
  "additional_api_key_creds": {
    "my-company": {
      "api_key_command": {
        "program": "op",
        "args": ["read", "op://Work/OpenAI/credential"],
        "timeout_ms": 120000
      }
    }
  }
}

The provider IDs in models.json and auth.json must match. Then select the custom model directly:

zot --provider my-company --model company-llm-v2

Credential resolution

For each request, zot checks credentials in this order:

Explicit CLI key, such as --api-key.
Provider-specific environment variables (including derived custom-provider variables such as MY_COMPANY_API_KEY).
$ZOT_HOME/auth.json, including custom provider keys saved by /login.

models.json itself never stores credentials; it only describes models and their endpoints.

Bedrock then uses the AWS SDK credential chain for the actual request.

source

1# zot providers
2
3zot ships with built-in providers and a model catalog. You can select models
4with `/model`, list them with `zot --list-models`, and add private models in
5`$ZOT_HOME/models.json`.
6
7## HTTP proxy
8
9Set one global proxy for zot-managed HTTP and HTTPS traffic with the `http_proxy` key in `$ZOT_HOME/config.json`:
10
11```json
12{
13  "http_proxy": "http://127.0.0.1:7890"
14}
15```
16
17zot applies this setting at startup. Existing `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy`, and `https_proxy` environment variables take precedence for their corresponding protocol. Standard `NO_PROXY` and `no_proxy` bypass lists remain effective. Restart zot after changing the setting.
18
19`config.json` is not a credential store. If the proxy URL contains a username or password, prefer setting the proxy environment variables in a protected shell or service configuration rather than saving those credentials in `config.json`.
20
21## Login methods
22
23Use `/login` in interactive mode. Type in either provider picker to filter the list by provider ID or display name.
24
25- `api key`: stores an API key in `$ZOT_HOME/auth.json` when the provider uses a normal key.
26- `subscription`: stores OAuth credentials for subscription-backed providers.
27
28Use `/logout` to remove stored credentials.
29
30Some providers need more than a single pasted key. For those providers,
31`/login` shows setup instructions instead of opening a localhost browser form.
32This avoids broken browser flows in SSH, containers, and `kubectl exec`
33sessions.
34
35### Command-backed API keys
36
37A provider can obtain its API key from a password manager or another local program. Configure this directly in `$ZOT_HOME/auth.json`:
38
39```json
40{
41  "openai": {
42    "api_key_command": {
43      "program": "op",
44      "args": ["read", "op://Work/OpenAI/credential"],
45      "timeout_ms": 120000
46    }
47  }
48}
49```
50
51Custom provider credentials use the same object under `additional_api_key_creds`:
52
53```json
54{
55  "additional_api_key_creds": {
56    "my-company": {
57      "api_key_command": {
58        "program": "secret-tool",
59        "args": ["read", "my-company-api-key"]
60      }
61    }
62  }
63}
64```
65
66The program is started directly rather than through a shell. The timeout defaults to 120 seconds. It must write one non-empty line to stdout, with no more than 64 KiB of output. Trailing CR/LF characters are removed. Successful results stay only in process memory and are reused until zot exits.
67
68Command-backed credentials count as logged in without being executed. zot materializes one only when selecting that provider; background model discovery skips it. `/login` with a normal key replaces the command, while `/logout` clears it. Because `auth.json` can cause program execution, keep it user-writable only and do not use files from untrusted sources.
69
70Setup-instruction providers:
71
72- Amazon Bedrock
73- Google Vertex AI
74- Cloudflare Workers AI
75- Cloudflare AI Gateway
76- Azure OpenAI Responses
77
78## Subscription providers
79
80These providers support subscription login:
81
82| Provider | Notes |
83| --- | --- |
84| Anthropic | Claude Pro/Max OAuth credentials. |
85| OpenAI Codex | ChatGPT Plus/Pro Codex subscription route. Separate from the OpenAI API-key provider. |
86| Kimi | Kimi subscription login. |
87| xAI | SuperGrok or X Premium device-code login. The browser URL is prefilled with the device code. |
88| GitHub Copilot | GitHub Copilot token flow. |
89
90OAuth tokens are stored in `$ZOT_HOME/auth.json` and refreshed when refresh is
91available.
92
93## GitHub Copilot model availability
94
95The built-in Copilot catalog excludes retired models and utility-only models
96that are not selectable for chat. The default is `claude-sonnet-5`.
97Availability still depends on your plan and organization policies. At startup,
98zot queries Copilot's authenticated `/models` endpoint in the background and
99filters the runtime catalog using picker visibility, policy state, and tool-call
100support. Individual accounts with no eligible picker entries can fall back to
101explicitly enabled model policies; other account types cannot. zot never enables
102policies automatically.
103
104Account availability stays in memory, not the shared model cache. Until discovery
105succeeds, the static catalog remains visible; failed refreshes retain the previous
106snapshot. Command-backed credentials are not executed for background discovery.
107Restart zot after changing accounts to refresh the snapshot. This filter is a
108selection aid, not an authorization boundary. The website displays the public
109catalog, not any individual account's availability.
110
111Claude uses the Anthropic Messages API with Copilot Bearer authentication, without
112Claude Code identity injection. GPT, Grok, and MAI use Responses. Gemini and Kimi
113use Chat Completions without `reasoning_effort`; their reasoning is managed by the
114endpoint. All three routes send user/agent initiator metadata and image-request
115headers where appropriate.
116
117As of September 8, 2026, the catalog includes 27 public chat models: Claude
118Fable 5/5.1, Haiku 4.5, Opus 4.7/4.8/4.8 Fast/5, Sonnet 5, Gemini
1193.5/3.6/3.7/3.8 Flash, GPT-5 mini, GPT-5.3-Codex, GPT-5.4/5.4 mini/5.5,
120GPT-5.6 Luna/Sol/Terra, GPT-6 Astra, Grok 4.5/4.6, Kimi K2.7 Code/K3,
121and MAI-Code-1-Flash/1.1-Flash. MAI-Code-1-Flash uses the public picker ID
122`mai-code-1-flash-picker` and is scheduled to retire on September 10, 2026.
123
124Internal search, execution, and compaction models are not chat choices.
125GPT-5.4 nano is excluded because GitHub offers it in the Codex VS Code
126extension, not Copilot Chat. New entries use Copilot-specific limits and token
127prices from [models.dev](https://models.dev); these are not a quote for your
128subscription bill. Existing zero-priced entries do not imply unlimited free use.
129
130Claude Fable models have different data-retention terms: Anthropic retains
131prompts and outputs by default for safety classifiers. Eligible enterprises
132can request a time-limited zero-data-retention exception from GitHub; enabling
133a model does not itself grant that exception. Review
134[GitHub's model availability and data-retention notes](https://docs.github.com/en/copilot/reference/ai-models/supported-models)
135before using Fable with sensitive data.
136
137Sonnet 4.6 is also excluded from the general catalog: GitHub retired it on
138September 1, 2026, except for individual annual Copilot Pro and Pro+ subscribers.
139Eligible subscribers can add it through `$ZOT_HOME/models.json`.
140See [GitHub's model retirement history and plan exceptions](https://docs.github.com/en/copilot/reference/ai-models/supported-models#model-retirement-history).
141
142## GPT-6 Astra
143
144Select `gpt-6-astra` with provider `openai`, `openai-responses`,
145`openai-codex`, or `github-copilot`, for example:
146
147```bash
148zot --provider github-copilot --model gpt-6-astra
149```
150
151Astra uses the Responses API and supports `low`, `medium`, `high`, `xhigh`,
152and `max` reasoning. OpenAI API-key and Codex subscription entries use the
153272,000-token short-context tier by default; the Copilot entry uses a
1541,050,000-token context window. All entries include the 128,000-token output
155limit and standard token prices, including the higher rates for prompts above
156272,000 tokens. Subscription access depends on your
157plan, rollout, and organization policy; catalog availability does not grant access.
158See the [OpenAI model specifications](https://developers.openai.com/api/docs/models/gpt-6-astra)
159and [GitHub Copilot availability](https://github.blog/changelog/2026-09-04-gpt-6-astra-is-generally-available-in-github-copilot/).
160
161## API-key providers
162
163These providers can use environment variables. Simple API-key providers can
164also be configured through `/login`. Providers that require extra cloud setup
165show instructions and should be configured with environment variables.
166
167| Provider | Environment variable | Stored key |
168| --- | --- | --- |
169| Anthropic | `ANTHROPIC_API_KEY` | `anthropic` |
170| OpenAI | `OPENAI_API_KEY` | `openai` |
171| OpenAI Responses | `OPENAI_API_KEY` | `openai-responses` |
172| Kimi | `KIMI_API_KEY` or `MOONSHOT_API_KEY` | `kimi` |
173| Google Gemini | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | `google` |
174| DeepSeek | `DEEPSEEK_API_KEY` | `deepseek` |
175| Moonshot AI | `MOONSHOT_API_KEY` | `moonshotai` |
176| Moonshot AI China | `MOONSHOT_API_KEY` | `moonshotai-cn` |
177| Groq | `GROQ_API_KEY` | `groq` |
178| xAI | `XAI_API_KEY` | `xai` |
179| Cerebras | `CEREBRAS_API_KEY` | `cerebras` |
180| Together AI | `TOGETHER_API_KEY` | `together` |
181| Hugging Face | `HF_TOKEN` | `huggingface` |
182| OpenRouter | `OPENROUTER_API_KEY` | `openrouter` |
183| Gondola | `GONDOLA_API_KEY` | `gondola` |
184| Mistral | `MISTRAL_API_KEY` | `mistral` |
185| ZAI | `ZAI_API_KEY` | `zai` |
186| Xiaomi MiMo | `XIAOMI_API_KEY` | `xiaomi` |
187| Xiaomi Token Plan Amsterdam | `XIAOMI_TOKEN_PLAN_AMS_API_KEY` | `xiaomi-token-plan-ams` |
188| Xiaomi Token Plan China | `XIAOMI_TOKEN_PLAN_CN_API_KEY` | `xiaomi-token-plan-cn` |
189| Xiaomi Token Plan Singapore | `XIAOMI_TOKEN_PLAN_SGP_API_KEY` | `xiaomi-token-plan-sgp` |
190| MiniMax | `MINIMAX_API_KEY` | `minimax` |
191| MiniMax China | `MINIMAX_CN_API_KEY` or `MINIMAX_API_KEY` | `minimax-cn` |
192| Fireworks | `FIREWORKS_API_KEY` | `fireworks` |
193| Vercel AI Gateway | `AI_GATEWAY_API_KEY` | `vercel-ai-gateway` |
194| OpenCode Zen | `OPENCODE_API_KEY` | `opencode` |
195| OpenCode Go | `OPENCODE_API_KEY` | `opencode-go` |
196| GitHub Copilot token | `COPILOT_GITHUB_TOKEN` or `GITHUB_COPILOT_TOKEN` | `github-copilot` |
197| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` | `cloudflare-workers-ai` |
198| Cloudflare AI Gateway | `CLOUDFLARE_API_KEY` | `cloudflare-ai-gateway` |
199| Azure OpenAI Responses | `AZURE_OPENAI_API_KEY` | `azure-openai-responses` |
200
201When Gondola credentials are available, zot refreshes its public text-model
202catalog in the background and adds the discovered models to `/model`.
203
204Example:
205
206```bash
207export OPENROUTER_API_KEY=...
208zot --provider openrouter
209```
210
211## Local LM Studio server
212
213Register the `lmstudio` provider through `/login`, choose `api key`, and select
214`LM Studio`. Enter the server URL, for example `http://localhost:1234`, followed
215by an optional API key. Zot validates the URL and stores the connection in
216`$ZOT_HOME/auth.json`. Saving it does not require the server to be online.
217
218A registered server URL is required for discovery. Zot does not probe localhost,
219and an API key alone does not enable discovery. Unlike llama.cpp's
220`LLAMA_BASE_URL`, `LMSTUDIO_BASE_URL` is not a registration mechanism. Your
221current provider and selected model do not change when registering a server.
222
223Open `/model` to refresh and select a discovered model. Zot also refreshes after
224saving the connection, in the background on startup, and when selecting or
225resuming an LM Studio model. After registration, you can select a model directly:
226
227```sh
228zot --provider lmstudio --model my-local-model
229```
230
231Discovery prefers `/api/v1/models` and falls back to `/v1/models` when the native
232endpoint returns 404, 405, or 501. Authentication errors do not trigger fallback.
233The native list excludes entries explicitly marked as embedding models. The
234fallback only supplies IDs, so embedding filtering is best-effort.
235
236Loaded models use their instance IDs and configured context lengths. Unloaded
237models use their keys and advertised maximum context lengths. Quantization-specific
238keys are preserved, but the separate native variant list is not expanded into
239additional choices. Listing models does not load or download them. Inference
240with an unloaded model depends on LM Studio's auto-loading settings. There is
241no `/llama`-style management interface for LM Studio.
242
243Missing context metadata defaults to 32,768 tokens. Output defaults to at most
2444,096 tokens, or a quarter of the context window for smaller models. These are
245estimates, not server guarantees. Discovered models have zero API pricing, and
246token usage comes from inference responses. Reasoning controls are not enabled
247automatically from capability metadata, which does not mean server-side reasoning
248is disabled. Override metadata under `lmstudio` in `$ZOT_HOME/models.json`,
249especially when an unloaded model will use a smaller context than advertised.
250User entries take precedence over discovery.
251
252The discovered catalog stays in memory, separate from the six-hour cloud model
253cache and llama.cpp's entries. Failed refreshes retain the previous snapshot,
254and `/model` reports discovery errors while keeping other providers accessible.
255Use `/logout lmstudio` to remove the connection and stop discovery. Background
256refresh never executes API-key commands.
257
258The existing coding loop, tools, permissions, and built-in provider pricing
259remain unchanged. For older builds without `LM Studio` in `/login`, the manual
260OpenAI-compatible setup in [README.md](../README.md#local-models-with-ollama)
261remains available.
262
263## Local llama.cpp router
264
265The `llama.cpp` provider connects to a multi-model router and is separate from the `ollama` provider. Ollama normally uses `http://localhost:11434`; entering that URL for llama.cpp management produces a 404 because Ollama does not implement the router endpoints.
266
267Install a current llama.cpp build. On macOS:
268
269```bash
270brew install llama.cpp
271# use `brew upgrade llama.cpp` for an existing installation
272```
273
274Start `llama-server` without `--model`, `-m`, or `-hf`. Those flags select one model and disable the router behavior zot expects.
275
276```bash
277mkdir -p ~/llama-models
278
279llama-server \
280  --models-dir ~/llama-models \
281  --no-models-autoload \
282  --jinja \
283  --host 127.0.0.1 \
284  --port 8080 \
285  -ngl 999 \
286  -c 32768
287```
288
289This configuration discovers GGUF files below `~/llama-models`, leaves model loading under explicit user control, enables chat templates, requests GPU offload, and sets a 32K context. Tune the GPU layers and context size for the available memory.
290
291Check the management API before opening zot:
292
293```bash
294curl http://127.0.0.1:8080/health
295curl http://127.0.0.1:8080/models
296```
297
298`/models` must return JSON with a `data` array. A 404 indicates the wrong URL, an older server build, or single-model mode.
299
300To save the connection, run `/login`, choose `api key`, and select `llama.cpp`. Enter `http://127.0.0.1:8080`, without `/v1`, followed by an optional bearer key. zot validates the URL and stores both values in `$ZOT_HOME/auth.json`. Environment variables override the saved connection:
301
302```bash
303export LLAMA_BASE_URL=http://127.0.0.1:8080
304export LLAMA_API_KEY=optional-secret
305```
306
307If a key is configured, start the router with the same `--api-key` value. A server bound only to `127.0.0.1` normally does not need authentication.
308
309After login, `/llama` becomes visible. It shows model state, searches Hugging Face for GGUF repositories, offers available quantizations, reports download and load progress, and explicitly loads or unloads models. Press `d` on a router-downloaded cache model to ask the router to remove it after confirmation. Models discovered through `--models-dir` or a preset are not removable through the router API and must be deleted from their configured source. Some llama.cpp versions retain shared Hugging Face repository artifacts such as `mmproj` files after removing the selected GGUF; remove that repository's cache directory manually if those artifacts are no longer needed. `HF_TOKEN` raises Hugging Face API limits and may be required for repository metadata. For gated files, approve access first and provide the authorized token to the `llama-server` process because the router performs the download.
310
311Opening `/model` refreshes the router and displays its loaded models under provider `llama.cpp`. Unloaded models cannot handle inference and therefore remain exclusive to `/llama` until loaded. Inference is sent to the router's derived `/v1` endpoint.
312
313Ollama downloads are stored in Ollama's internal layout and do not automatically become llama.cpp models. Obtain a GGUF copy through `/llama` or place GGUF files under `~/llama-models`, then restart the router after adding files manually.
314
315## Cloud providers
316
317### Amazon Bedrock
318
319Bedrock is configured with AWS credentials, not a generic zot API-key entry.
320Use one of these credential sources:
321
322```bash
323# AWS profile
324export AWS_PROFILE=your-profile
325
326# IAM access keys
327export AWS_ACCESS_KEY_ID=AKIA...
328export AWS_SECRET_ACCESS_KEY=...
329export AWS_SESSION_TOKEN=... # only for temporary credentials
330
331# Bedrock API key bearer token
332export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...
333
334# Region
335export AWS_REGION=us-east-1
336```
337
338ECS task roles, IRSA, and other AWS SDK credential-chain sources are also
339supported.
340
341Example:
342
343```bash
344AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-... AWS_REGION=us-east-1 \
345  zot --provider amazon-bedrock --model anthropic.claude-sonnet-4-5-20250929-v1:0
346```
347
348Some Bedrock models require regional inference-profile IDs for on-demand
349throughput, such as `us.` or `eu.` prefixed model IDs. zot rewrites known
350families automatically where possible. Claude Opus 5 uses
351`global.anthropic.claude-opus-5`; zot maps the bare foundation-model ID to that
352global profile and supports its adaptive reasoning levels. Explicit profile IDs
353and ARNs are left unchanged.
354
355### Google Vertex AI
356
357Vertex can use a Google API key when available:
358
359```bash
360export GOOGLE_CLOUD_API_KEY=...
361zot --provider google-vertex
362```
363
364For service-account or application-default credentials, set the standard
365Google environment variables used by your deployment.
366
367### Cloudflare AI Gateway
368
369Cloudflare AI Gateway needs a Cloudflare token plus account and gateway IDs:
370
371```bash
372export CLOUDFLARE_API_KEY=...
373export CLOUDFLARE_ACCOUNT_ID=...
374export CLOUDFLARE_GATEWAY_ID=...
375zot --provider cloudflare-ai-gateway
376```
377
378### Cloudflare Workers AI
379
380Workers AI needs a Cloudflare token and account ID:
381
382```bash
383export CLOUDFLARE_API_KEY=...
384export CLOUDFLARE_ACCOUNT_ID=...
385zot --provider cloudflare-workers-ai
386```
387
388### Azure OpenAI Responses
389
390```bash
391export AZURE_OPENAI_API_KEY=...
392export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
393export AZURE_OPENAI_API_VERSION=v1 # optional, v1 is the default
394zot --provider azure-openai-responses
395```
396
397The provider uses Azure's Responses API. If deployment names differ from zot
398model IDs, map them without changing the catalog:
399
400```bash
401export AZURE_OPENAI_DEPLOYMENT_NAME_MAP='gpt-5.6-luna=luna-preview,gpt-5.6-sol=sol-preview'
402```
403
404## Auth file
405
406Credentials are stored in `$ZOT_HOME/auth.json` with user-only permissions
407when zot creates the file.
408
409Example:
410
411```json
412{
413  "anthropic": { "api_key": "sk-ant-..." },
414  "openai": { "api_key": "sk-..." },
415  "google": { "api_key": "..." },
416  "additional_api_key_creds": {
417    "openrouter": { "api_key": "..." },
418    "mistral": { "api_key": "..." }
419  }
420}
421```
422
423The top-level keys are used for providers with dedicated credential fields.
424Other API-key providers are stored under `additional_api_key_creds`. Prefer
425`/login` so zot writes the correct schema.
426
427## Custom providers and models
428
429Use `$ZOT_HOME/models.json` for private models, deployment aliases, local
430servers, or OpenAI-compatible gateways that are not in the built-in catalog.
431User entries override built-in entries with the same provider and model ID, and
432adding a `models.json` no longer hides the built-in catalog: your entries are
433merged on top of the baked-in and live-discovered models.
434
435A top-level provider key that is not a built-in id defines a custom provider.
436Give it a provider-level `baseUrl` and an `api` wire format (`openai` for
437OpenAI-compatible Chat Completions, the default, `openai-responses` for the
438OpenAI Responses API, or `anthropic` for the Anthropic Messages API). A
439model-level `baseUrl` overrides the provider-level one for that model, and an
440unknown `api` value falls back to `openai` with a warning.
441
442```json
443{
444  "providers": {
445    "my-company": {
446      "baseUrl": "https://llm.mycompany.com/v1",
447      "api": "openai",
448      "models": [
449        { "id": "company-llm-v2", "name": "Company LLM v2" }
450      ]
451    }
452  }
453}
454```
455
456Custom providers are first-class: they appear in `--list-models`, `/model`, and
457`/login`. `models.json` never stores secrets. Supply the key through `/login`,
458`--api-key`, or a derived environment variable in upper snake case, so
459`my-company` reads `MY_COMPANY_API_KEY`. Because many self-hosted gateways do
460not expose a model-list endpoint, custom provider keys are accepted and stored
461without a verification probe; an invalid key surfaces on the first model call.
462
463To retrieve this custom provider's key from a password manager, add a matching
464entry to `$ZOT_HOME/auth.json`:
465
466```json
467{
468  "additional_api_key_creds": {
469    "my-company": {
470      "api_key_command": {
471        "program": "op",
472        "args": ["read", "op://Work/OpenAI/credential"],
473        "timeout_ms": 120000
474      }
475    }
476  }
477}
478```
479
480The provider IDs in `models.json` and `auth.json` must match. Then select the
481custom model directly:
482
483```sh
484zot --provider my-company --model company-llm-v2
485```
486
487## Credential resolution
488
489For each request, zot checks credentials in this order:
490
4911. Explicit CLI key, such as `--api-key`.
4922. Provider-specific environment variables (including derived custom-provider
493   variables such as `MY_COMPANY_API_KEY`).
4943. `$ZOT_HOME/auth.json`, including custom provider keys saved by `/login`.
495
496`models.json` itself never stores credentials; it only describes models and
497their endpoints.
498
499Bedrock then uses the AWS SDK credential chain for the actual request.

providers.md RawHistoryFolder

zot providers

HTTP proxy

Login methods

Command-backed API keys

Subscription providers

GitHub Copilot model availability

GPT-6 Astra

API-key providers

Local LM Studio server

Local llama.cpp router

Cloud providers

Amazon Bedrock

Google Vertex AI

Cloudflare AI Gateway

Cloudflare Workers AI

Azure OpenAI Responses

Auth file

Custom providers and models

Credential resolution

providers.md Raw History Folder