providers.md Raw History Folder

zot providers

zot ships with built-in providers and a model catalog. You can select models with /model, list them with zot --list-models, and add private models in $ZOT_HOME/models.json.

Use /login in interactive mode. Type in either provider picker to filter the list by provider ID or display name.

api key: stores an API key in $ZOT_HOME/auth.json when the provider uses a normal key.
subscription: stores OAuth credentials for subscription-backed providers.

Use /logout to remove stored credentials.

Some providers need more than a single pasted key. For those providers, /login shows setup instructions instead of opening a localhost browser form. This avoids broken browser flows in SSH, containers, and kubectl exec sessions.

Setup-instruction providers:

Amazon Bedrock
Google Vertex AI
Cloudflare Workers AI
Cloudflare AI Gateway
Azure OpenAI Responses

Subscription providers

These providers support subscription login:

Provider	Notes
Anthropic	Claude Pro/Max OAuth credentials.
OpenAI Codex	ChatGPT Plus/Pro Codex subscription route. Separate from the OpenAI API-key provider.
Kimi	Kimi subscription login.
xAI	SuperGrok or X Premium device-code login. The browser URL is prefilled with the device code.
GitHub Copilot	GitHub Copilot token flow.

OAuth tokens are stored in $ZOT_HOME/auth.json and refreshed when refresh is available.

API-key providers

These providers can use environment variables. Simple API-key providers can also be configured through /login. Providers that require extra cloud setup show instructions and should be configured with environment variables.

Provider	Environment variable	Stored key
Anthropic	`ANTHROPIC_API_KEY`	`anthropic`
OpenAI	`OPENAI_API_KEY`	`openai`
OpenAI Responses	`OPENAI_API_KEY`	`openai-responses`
Kimi	`KIMI_API_KEY` or `MOONSHOT_API_KEY`	`kimi`
Google Gemini	`GEMINI_API_KEY` or `GOOGLE_API_KEY`	`google`
DeepSeek	`DEEPSEEK_API_KEY`	`deepseek`
Moonshot AI	`MOONSHOT_API_KEY`	`moonshotai`
Moonshot AI China	`MOONSHOT_API_KEY`	`moonshotai-cn`
Groq	`GROQ_API_KEY`	`groq`
xAI	`XAI_API_KEY`	`xai`
Cerebras	`CEREBRAS_API_KEY`	`cerebras`
Together AI	`TOGETHER_API_KEY`	`together`
Hugging Face	`HF_TOKEN`	`huggingface`
OpenRouter	`OPENROUTER_API_KEY`	`openrouter`
Mistral	`MISTRAL_API_KEY`	`mistral`
ZAI	`ZAI_API_KEY`	`zai`
Xiaomi MiMo	`XIAOMI_API_KEY`	`xiaomi`
Xiaomi Token Plan Amsterdam	`XIAOMI_TOKEN_PLAN_AMS_API_KEY`	`xiaomi-token-plan-ams`
Xiaomi Token Plan China	`XIAOMI_TOKEN_PLAN_CN_API_KEY`	`xiaomi-token-plan-cn`
Xiaomi Token Plan Singapore	`XIAOMI_TOKEN_PLAN_SGP_API_KEY`	`xiaomi-token-plan-sgp`
MiniMax	`MINIMAX_API_KEY`	`minimax`
MiniMax China	`MINIMAX_CN_API_KEY` or `MINIMAX_API_KEY`	`minimax-cn`
Fireworks	`FIREWORKS_API_KEY`	`fireworks`
Vercel AI Gateway	`AI_GATEWAY_API_KEY`	`vercel-ai-gateway`
OpenCode Zen	`OPENCODE_API_KEY`	`opencode`
OpenCode Go	`OPENCODE_API_KEY`	`opencode-go`
GitHub Copilot token	`COPILOT_GITHUB_TOKEN` or `GITHUB_COPILOT_TOKEN`	`github-copilot`
Cloudflare Workers AI	`CLOUDFLARE_API_KEY`	`cloudflare-workers-ai`
Cloudflare AI Gateway	`CLOUDFLARE_API_KEY`	`cloudflare-ai-gateway`
Azure OpenAI Responses	`AZURE_OPENAI_API_KEY`	`azure-openai-responses`

Example:

export OPENROUTER_API_KEY=...
zot --provider openrouter

Local llama.cpp router

The llama.cpp provider connects to a multi-model router and is separate from the ollama provider. Ollama normally uses http://localhost:11434; entering that URL for llama.cpp management produces a 404 because Ollama does not implement the router endpoints.

Install a current llama.cpp build. On macOS:

brew install llama.cpp
# use `brew upgrade llama.cpp` for an existing installation

Start llama-server without --model, -m, or -hf. Those flags select one model and disable the router behavior zot expects.

mkdir -p ~/llama-models

llama-server \
  --models-dir ~/llama-models \
  --no-models-autoload \
  --jinja \
  --host 127.0.0.1 \
  --port 8080 \
  -ngl 999 \
  -c 32768

This configuration discovers GGUF files below ~/llama-models, leaves model loading under explicit user control, enables chat templates, requests GPU offload, and sets a 32K context. Tune the GPU layers and context size for the available memory.

Check the management API before opening zot:

curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/models

/models must return JSON with a data array. A 404 indicates the wrong URL, an older server build, or single-model mode.

To save the connection, run /login, choose api key, and select llama.cpp. Enter http://127.0.0.1:8080, without /v1, followed by an optional bearer key. zot validates the URL and stores both values in $ZOT_HOME/auth.json. Environment variables override the saved connection:

export LLAMA_BASE_URL=http://127.0.0.1:8080
export LLAMA_API_KEY=optional-secret

If a key is configured, start the router with the same --api-key value. A server bound only to 127.0.0.1 normally does not need authentication.

After login, /llama becomes visible. It shows model state, searches Hugging Face for GGUF repositories, offers available quantizations, reports download and load progress, and explicitly loads or unloads models. Press d on a router-downloaded cache model to ask the router to remove it after confirmation. Models discovered through --models-dir or a preset are not removable through the router API and must be deleted from their configured source. Some llama.cpp versions retain shared Hugging Face repository artifacts such as mmproj files after removing the selected GGUF; remove that repository's cache directory manually if those artifacts are no longer needed. HF_TOKEN raises Hugging Face API limits and may be required for repository metadata. For gated files, approve access first and provide the authorized token to the llama-server process because the router performs the download.

Opening /model refreshes the router and displays its loaded models under provider llama.cpp. Unloaded models cannot handle inference and therefore remain exclusive to /llama until loaded. Inference is sent to the router's derived /v1 endpoint.

Ollama downloads are stored in Ollama's internal layout and do not automatically become llama.cpp models. Obtain a GGUF copy through /llama or place GGUF files under ~/llama-models, then restart the router after adding files manually.

Cloud providers

Amazon Bedrock

Bedrock is configured with AWS credentials, not a generic zot API-key entry. Use one of these credential sources:

# AWS profile
export AWS_PROFILE=your-profile

# IAM access keys
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # only for temporary credentials

# Bedrock API key bearer token
export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...

# Region
export AWS_REGION=us-east-1

ECS task roles, IRSA, and other AWS SDK credential-chain sources are also supported.

Example:

AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-... AWS_REGION=us-east-1 \
  zot --provider amazon-bedrock --model anthropic.claude-sonnet-4-5-20250929-v1:0

Some Bedrock models require regional inference-profile IDs for on-demand throughput, such as us. or eu. prefixed model IDs. zot rewrites known families automatically where possible. Explicit profile IDs and ARNs are left unchanged.

Google Vertex AI

Vertex can use a Google API key when available:

export GOOGLE_CLOUD_API_KEY=...
zot --provider google-vertex

For service-account or application-default credentials, set the standard Google environment variables used by your deployment.

Cloudflare AI Gateway

Cloudflare AI Gateway needs a Cloudflare token plus account and gateway IDs:

export CLOUDFLARE_API_KEY=...
export CLOUDFLARE_ACCOUNT_ID=...
export CLOUDFLARE_GATEWAY_ID=...
zot --provider cloudflare-ai-gateway

Cloudflare Workers AI

Workers AI needs a Cloudflare token and account ID:

export CLOUDFLARE_API_KEY=...
export CLOUDFLARE_ACCOUNT_ID=...
zot --provider cloudflare-workers-ai

Azure OpenAI Responses

export AZURE_OPENAI_API_KEY=...
export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
export AZURE_OPENAI_API_VERSION=v1 # optional, v1 is the default
zot --provider azure-openai-responses

The provider uses Azure's Responses API. If deployment names differ from zot model IDs, map them without changing the catalog:

export AZURE_OPENAI_DEPLOYMENT_NAME_MAP='gpt-5.6-luna=luna-preview,gpt-5.6-sol=sol-preview'

Auth file

Credentials are stored in $ZOT_HOME/auth.json with user-only permissions when zot creates the file.

Example:

{
  "anthropic": { "api_key": "sk-ant-..." },
  "openai": { "api_key": "sk-..." },
  "google": { "api_key": "..." },
  "additional_api_key_creds": {
    "openrouter": { "api_key": "..." },
    "mistral": { "api_key": "..." }
  }
}

The top-level keys are used for providers with dedicated credential fields. Other API-key providers are stored under additional_api_key_creds. Prefer /login so zot writes the correct schema.

Custom providers and models

Use $ZOT_HOME/models.json for private models, deployment aliases, local servers, or OpenAI-compatible gateways that are not in the built-in catalog. User entries override built-in entries with the same provider and model ID, and adding a models.json no longer hides the built-in catalog: your entries are merged on top of the baked-in and live-discovered models.

A top-level provider key that is not a built-in id defines a custom provider. Give it a provider-level baseUrl and an api wire format (openai for OpenAI-compatible chat completions, the default, or anthropic for the Anthropic messages API). A model-level baseUrl overrides the provider-level one for that model, and an unknown api value falls back to openai with a warning.

{
  "providers": {
    "my-company": {
      "baseUrl": "https://llm.mycompany.com/v1",
      "api": "openai",
      "models": [
        { "id": "company-llm-v2", "name": "Company LLM v2" }
      ]
    }
  }
}

Custom providers are first-class: they appear in --list-models, /model, and /login. models.json never stores secrets. Supply the key through /login, --api-key, or a derived environment variable in upper snake case, so my-company reads MY_COMPANY_API_KEY. Because many self-hosted gateways do not expose a model-list endpoint, custom provider keys are accepted and stored without a verification probe; an invalid key surfaces on the first model call.

Credential resolution

For each request, zot checks credentials in this order:

Explicit CLI key, such as --api-key.
Provider-specific environment variables (including derived custom-provider variables such as MY_COMPANY_API_KEY).
$ZOT_HOME/auth.json, including custom provider keys saved by /login.

models.json itself never stores credentials; it only describes models and their endpoints.

Bedrock then uses the AWS SDK credential chain for the actual request.

source

1# zot providers
2
3zot ships with built-in providers and a model catalog. You can select models
4with `/model`, list them with `zot --list-models`, and add private models in
5`$ZOT_HOME/models.json`.
6
7## Login methods
8
9Use `/login` in interactive mode. Type in either provider picker to filter the list by provider ID or display name.
10
11- `api key`: stores an API key in `$ZOT_HOME/auth.json` when the provider uses a normal key.
12- `subscription`: stores OAuth credentials for subscription-backed providers.
13
14Use `/logout` to remove stored credentials.
15
16Some providers need more than a single pasted key. For those providers,
17`/login` shows setup instructions instead of opening a localhost browser form.
18This avoids broken browser flows in SSH, containers, and `kubectl exec`
19sessions.
20
21Setup-instruction providers:
22
23- Amazon Bedrock
24- Google Vertex AI
25- Cloudflare Workers AI
26- Cloudflare AI Gateway
27- Azure OpenAI Responses
28
29## Subscription providers
30
31These providers support subscription login:
32
33| Provider | Notes |
34| --- | --- |
35| Anthropic | Claude Pro/Max OAuth credentials. |
36| OpenAI Codex | ChatGPT Plus/Pro Codex subscription route. Separate from the OpenAI API-key provider. |
37| Kimi | Kimi subscription login. |
38| xAI | SuperGrok or X Premium device-code login. The browser URL is prefilled with the device code. |
39| GitHub Copilot | GitHub Copilot token flow. |
40
41OAuth tokens are stored in `$ZOT_HOME/auth.json` and refreshed when refresh is
42available.
43
44## API-key providers
45
46These providers can use environment variables. Simple API-key providers can
47also be configured through `/login`. Providers that require extra cloud setup
48show instructions and should be configured with environment variables.
49
50| Provider | Environment variable | Stored key |
51| --- | --- | --- |
52| Anthropic | `ANTHROPIC_API_KEY` | `anthropic` |
53| OpenAI | `OPENAI_API_KEY` | `openai` |
54| OpenAI Responses | `OPENAI_API_KEY` | `openai-responses` |
55| Kimi | `KIMI_API_KEY` or `MOONSHOT_API_KEY` | `kimi` |
56| Google Gemini | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | `google` |
57| DeepSeek | `DEEPSEEK_API_KEY` | `deepseek` |
58| Moonshot AI | `MOONSHOT_API_KEY` | `moonshotai` |
59| Moonshot AI China | `MOONSHOT_API_KEY` | `moonshotai-cn` |
60| Groq | `GROQ_API_KEY` | `groq` |
61| xAI | `XAI_API_KEY` | `xai` |
62| Cerebras | `CEREBRAS_API_KEY` | `cerebras` |
63| Together AI | `TOGETHER_API_KEY` | `together` |
64| Hugging Face | `HF_TOKEN` | `huggingface` |
65| OpenRouter | `OPENROUTER_API_KEY` | `openrouter` |
66| Mistral | `MISTRAL_API_KEY` | `mistral` |
67| ZAI | `ZAI_API_KEY` | `zai` |
68| Xiaomi MiMo | `XIAOMI_API_KEY` | `xiaomi` |
69| Xiaomi Token Plan Amsterdam | `XIAOMI_TOKEN_PLAN_AMS_API_KEY` | `xiaomi-token-plan-ams` |
70| Xiaomi Token Plan China | `XIAOMI_TOKEN_PLAN_CN_API_KEY` | `xiaomi-token-plan-cn` |
71| Xiaomi Token Plan Singapore | `XIAOMI_TOKEN_PLAN_SGP_API_KEY` | `xiaomi-token-plan-sgp` |
72| MiniMax | `MINIMAX_API_KEY` | `minimax` |
73| MiniMax China | `MINIMAX_CN_API_KEY` or `MINIMAX_API_KEY` | `minimax-cn` |
74| Fireworks | `FIREWORKS_API_KEY` | `fireworks` |
75| Vercel AI Gateway | `AI_GATEWAY_API_KEY` | `vercel-ai-gateway` |
76| OpenCode Zen | `OPENCODE_API_KEY` | `opencode` |
77| OpenCode Go | `OPENCODE_API_KEY` | `opencode-go` |
78| GitHub Copilot token | `COPILOT_GITHUB_TOKEN` or `GITHUB_COPILOT_TOKEN` | `github-copilot` |
79| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` | `cloudflare-workers-ai` |
80| Cloudflare AI Gateway | `CLOUDFLARE_API_KEY` | `cloudflare-ai-gateway` |
81| Azure OpenAI Responses | `AZURE_OPENAI_API_KEY` | `azure-openai-responses` |
82
83Example:
84
85```bash
86export OPENROUTER_API_KEY=...
87zot --provider openrouter
88```
89
90## Local llama.cpp router
91
92The `llama.cpp` provider connects to a multi-model router and is separate from the `ollama` provider. Ollama normally uses `http://localhost:11434`; entering that URL for llama.cpp management produces a 404 because Ollama does not implement the router endpoints.
93
94Install a current llama.cpp build. On macOS:
95
96```bash
97brew install llama.cpp
98# use `brew upgrade llama.cpp` for an existing installation
99```
100
101Start `llama-server` without `--model`, `-m`, or `-hf`. Those flags select one model and disable the router behavior zot expects.
102
103```bash
104mkdir -p ~/llama-models
105
106llama-server \
107  --models-dir ~/llama-models \
108  --no-models-autoload \
109  --jinja \
110  --host 127.0.0.1 \
111  --port 8080 \
112  -ngl 999 \
113  -c 32768
114```
115
116This configuration discovers GGUF files below `~/llama-models`, leaves model loading under explicit user control, enables chat templates, requests GPU offload, and sets a 32K context. Tune the GPU layers and context size for the available memory.
117
118Check the management API before opening zot:
119
120```bash
121curl http://127.0.0.1:8080/health
122curl http://127.0.0.1:8080/models
123```
124
125`/models` must return JSON with a `data` array. A 404 indicates the wrong URL, an older server build, or single-model mode.
126
127To save the connection, run `/login`, choose `api key`, and select `llama.cpp`. Enter `http://127.0.0.1:8080`, without `/v1`, followed by an optional bearer key. zot validates the URL and stores both values in `$ZOT_HOME/auth.json`. Environment variables override the saved connection:
128
129```bash
130export LLAMA_BASE_URL=http://127.0.0.1:8080
131export LLAMA_API_KEY=optional-secret
132```
133
134If a key is configured, start the router with the same `--api-key` value. A server bound only to `127.0.0.1` normally does not need authentication.
135
136After login, `/llama` becomes visible. It shows model state, searches Hugging Face for GGUF repositories, offers available quantizations, reports download and load progress, and explicitly loads or unloads models. Press `d` on a router-downloaded cache model to ask the router to remove it after confirmation. Models discovered through `--models-dir` or a preset are not removable through the router API and must be deleted from their configured source. Some llama.cpp versions retain shared Hugging Face repository artifacts such as `mmproj` files after removing the selected GGUF; remove that repository's cache directory manually if those artifacts are no longer needed. `HF_TOKEN` raises Hugging Face API limits and may be required for repository metadata. For gated files, approve access first and provide the authorized token to the `llama-server` process because the router performs the download.
137
138Opening `/model` refreshes the router and displays its loaded models under provider `llama.cpp`. Unloaded models cannot handle inference and therefore remain exclusive to `/llama` until loaded. Inference is sent to the router's derived `/v1` endpoint.
139
140Ollama downloads are stored in Ollama's internal layout and do not automatically become llama.cpp models. Obtain a GGUF copy through `/llama` or place GGUF files under `~/llama-models`, then restart the router after adding files manually.
141
142## Cloud providers
143
144### Amazon Bedrock
145
146Bedrock is configured with AWS credentials, not a generic zot API-key entry.
147Use one of these credential sources:
148
149```bash
150# AWS profile
151export AWS_PROFILE=your-profile
152
153# IAM access keys
154export AWS_ACCESS_KEY_ID=AKIA...
155export AWS_SECRET_ACCESS_KEY=...
156export AWS_SESSION_TOKEN=... # only for temporary credentials
157
158# Bedrock API key bearer token
159export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...
160
161# Region
162export AWS_REGION=us-east-1
163```
164
165ECS task roles, IRSA, and other AWS SDK credential-chain sources are also
166supported.
167
168Example:
169
170```bash
171AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-... AWS_REGION=us-east-1 \
172  zot --provider amazon-bedrock --model anthropic.claude-sonnet-4-5-20250929-v1:0
173```
174
175Some Bedrock models require regional inference-profile IDs for on-demand
176throughput, such as `us.` or `eu.` prefixed model IDs. zot rewrites known
177families automatically where possible. Explicit profile IDs and ARNs are left
178unchanged.
179
180### Google Vertex AI
181
182Vertex can use a Google API key when available:
183
184```bash
185export GOOGLE_CLOUD_API_KEY=...
186zot --provider google-vertex
187```
188
189For service-account or application-default credentials, set the standard
190Google environment variables used by your deployment.
191
192### Cloudflare AI Gateway
193
194Cloudflare AI Gateway needs a Cloudflare token plus account and gateway IDs:
195
196```bash
197export CLOUDFLARE_API_KEY=...
198export CLOUDFLARE_ACCOUNT_ID=...
199export CLOUDFLARE_GATEWAY_ID=...
200zot --provider cloudflare-ai-gateway
201```
202
203### Cloudflare Workers AI
204
205Workers AI needs a Cloudflare token and account ID:
206
207```bash
208export CLOUDFLARE_API_KEY=...
209export CLOUDFLARE_ACCOUNT_ID=...
210zot --provider cloudflare-workers-ai
211```
212
213### Azure OpenAI Responses
214
215```bash
216export AZURE_OPENAI_API_KEY=...
217export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
218export AZURE_OPENAI_API_VERSION=v1 # optional, v1 is the default
219zot --provider azure-openai-responses
220```
221
222The provider uses Azure's Responses API. If deployment names differ from zot
223model IDs, map them without changing the catalog:
224
225```bash
226export AZURE_OPENAI_DEPLOYMENT_NAME_MAP='gpt-5.6-luna=luna-preview,gpt-5.6-sol=sol-preview'
227```
228
229## Auth file
230
231Credentials are stored in `$ZOT_HOME/auth.json` with user-only permissions
232when zot creates the file.
233
234Example:
235
236```json
237{
238  "anthropic": { "api_key": "sk-ant-..." },
239  "openai": { "api_key": "sk-..." },
240  "google": { "api_key": "..." },
241  "additional_api_key_creds": {
242    "openrouter": { "api_key": "..." },
243    "mistral": { "api_key": "..." }
244  }
245}
246```
247
248The top-level keys are used for providers with dedicated credential fields.
249Other API-key providers are stored under `additional_api_key_creds`. Prefer
250`/login` so zot writes the correct schema.
251
252## Custom providers and models
253
254Use `$ZOT_HOME/models.json` for private models, deployment aliases, local
255servers, or OpenAI-compatible gateways that are not in the built-in catalog.
256User entries override built-in entries with the same provider and model ID, and
257adding a `models.json` no longer hides the built-in catalog: your entries are
258merged on top of the baked-in and live-discovered models.
259
260A top-level provider key that is not a built-in id defines a custom provider.
261Give it a provider-level `baseUrl` and an `api` wire format (`openai` for
262OpenAI-compatible chat completions, the default, or `anthropic` for the
263Anthropic messages API). A model-level `baseUrl` overrides the provider-level
264one for that model, and an unknown `api` value falls back to `openai` with a
265warning.
266
267```json
268{
269  "providers": {
270    "my-company": {
271      "baseUrl": "https://llm.mycompany.com/v1",
272      "api": "openai",
273      "models": [
274        { "id": "company-llm-v2", "name": "Company LLM v2" }
275      ]
276    }
277  }
278}
279```
280
281Custom providers are first-class: they appear in `--list-models`, `/model`, and
282`/login`. `models.json` never stores secrets. Supply the key through `/login`,
283`--api-key`, or a derived environment variable in upper snake case, so
284`my-company` reads `MY_COMPANY_API_KEY`. Because many self-hosted gateways do
285not expose a model-list endpoint, custom provider keys are accepted and stored
286without a verification probe; an invalid key surfaces on the first model call.
287
288## Credential resolution
289
290For each request, zot checks credentials in this order:
291
2921. Explicit CLI key, such as `--api-key`.
2932. Provider-specific environment variables (including derived custom-provider
294   variables such as `MY_COMPANY_API_KEY`).
2953. `$ZOT_HOME/auth.json`, including custom provider keys saved by `/login`.
296
297`models.json` itself never stores credentials; it only describes models and
298their endpoints.
299
300Bedrock then uses the AWS SDK credential chain for the actual request.

providers.md RawHistoryFolder

zot providers

Login methods

Subscription providers

API-key providers

Local llama.cpp router

Cloud providers

Amazon Bedrock

Google Vertex AI

Cloudflare AI Gateway

Cloudflare Workers AI

Azure OpenAI Responses

Auth file

Custom providers and models

Credential resolution

providers.md Raw History Folder