zut providers
zut ships with built-in providers and a model catalog. You can select models
with /model, list them with zut --list-models, and add private models in
$ZUT_HOME/models.json.
HTTP proxy
Set one global proxy for zut-managed HTTP and HTTPS traffic with the http_proxy key in $ZUT_HOME/config.json:
{
"http_proxy": "http://127.0.0.1:7890"
}
zut applies this setting at startup. Existing HTTP_PROXY, HTTPS_PROXY, http_proxy, and https_proxy environment variables take precedence for their corresponding protocol. Standard NO_PROXY and no_proxy bypass lists remain effective. Restart zut after changing the setting.
config.json is not a credential store. If the proxy URL contains a username or password, prefer setting the proxy environment variables in a protected shell or service configuration rather than saving those credentials in config.json.
Login methods
Use /login in interactive mode. Type in either provider picker to filter the list by provider ID or display name.
api key: stores an API key in $ZUT_HOME/auth.json when the provider uses a normal key.subscription: stores OAuth credentials for subscription-backed providers.
Use /logout to remove stored credentials.
Some providers need more than a single pasted key. For those providers,
/login shows setup instructions instead of opening a localhost browser form.
This avoids broken browser flows in SSH, containers, and kubectl exec
sessions.
Command-backed API keys
A provider can obtain its API key from a password manager or another local program. Configure this directly in $ZUT_HOME/auth.json:
{
"openai": {
"api_key_command": {
"program": "op",
"args": ["read", "op://Work/OpenAI/credential"],
"timeout_ms": 120000
}
}
}
Custom provider credentials use the same object under additional_api_key_creds:
{
"additional_api_key_creds": {
"my-company": {
"api_key_command": {
"program": "secret-tool",
"args": ["read", "my-company-api-key"]
}
}
}
}
The program is started directly rather than through a shell. The timeout defaults to 120 seconds. It must write one non-empty line to stdout, with no more than 64 KiB of output. Trailing CR/LF characters are removed. Successful results stay only in process memory and are reused until zut exits.
Command-backed credentials count as logged in without being executed. zut materializes one only when selecting that provider; background model discovery skips it. /login with a normal key replaces the command, while /logout clears it. Because auth.json can cause program execution, keep it user-writable only and do not use files from untrusted sources.
Setup-instruction providers:
- Amazon Bedrock
- Google Vertex AI
- Cloudflare Workers AI
- Cloudflare AI Gateway
- Azure OpenAI Responses
Subscription providers
These providers support subscription login:
| Provider |
Notes |
| Anthropic |
Claude Pro/Max OAuth credentials. |
| OpenAI Codex |
ChatGPT Plus/Pro Codex subscription route. Separate from the OpenAI API-key provider. |
| Kimi |
Kimi subscription login. |
| xAI |
SuperGrok or X Premium device-code login. The browser URL is prefilled with the device code. |
| GitHub Copilot |
GitHub Copilot token flow. |
OAuth tokens are stored in $ZUT_HOME/auth.json and refreshed when refresh is
available.
Anthropic offers browser callback and copy-code login as separate choices.
With Anthropic OAuth, MCP resource helpers use Claude Code wire names while
zut retains mcp_list_resources and mcp_read_resource in tools and sessions.
API-key requests retain the zut names. OAuth acceptance and billing remain
controlled by Anthropic; a successful request does not guarantee subscription
coverage.
Browser login for Anthropic and OpenAI Codex also accepts a pasted callback URL
or code using the same authorization transaction; it does not start a second
login. A supplied state must match the pending transaction. Escape cancels the
pending flow. OpenAI Codex requires the browser callback transaction and has no
standalone manual-code login.
GitHub Copilot
Copilot uses the account's short-lived inference token. Claude models use the
Anthropic Messages API with Copilot Bearer authentication, without Claude Code
identity headers. Newer GPT, Grok, and MAI models use Responses; older GPT-4 and
Grok Code models remain on Chat Completions. Gemini and Kimi use Chat Completions
without reasoning_effort. Requests carry user/agent initiator metadata and an
image request header when appropriate.
Authenticated model discovery filters the runtime catalog by picker visibility,
policy state, and tool-call support. The individual-account endpoint may fall
back to explicitly enabled policies when no eligible picker entries exist;
other endpoints cannot. Zut never enables policies automatically. Availability
is a selection aid, not an authorization guarantee, and remains in memory rather
than the shared model cache. Failed discovery retains the previous snapshot.
Background discovery does not run command-backed credentials. Until discovery
succeeds, the static catalog is visible. Existing sessions and explicit model
selection still resolve catalog metadata for models hidden from the picker.
The default Copilot model is claude-sonnet-5. User model overrides retain
Copilot protocol defaults unless an explicit API or reasoningLevelMap overrides
them. For Copilot rows, the anthropic and openai API aliases select Messages
and Chat Completions respectively. Claude 4.5/4.6 rows use explicit thinking
budgets. The retired
mai-code-1-flash-picker is not in the static catalog; mai-code-1.1-flash is.
API-key providers
These providers can use environment variables. Simple API-key providers can
also be configured through /login. Providers that require extra cloud setup
show instructions and should be configured with environment variables.
| Provider |
Environment variable |
Stored key |
| Anthropic |
ANTHROPIC_API_KEY |
anthropic |
| OpenAI |
OPENAI_API_KEY |
openai |
| OpenAI Responses |
OPENAI_API_KEY |
openai-responses |
| Kimi |
KIMI_API_KEY or MOONSHOT_API_KEY |
kimi |
| Google Gemini |
GEMINI_API_KEY or GOOGLE_API_KEY |
google |
| DeepSeek |
DEEPSEEK_API_KEY |
deepseek |
| Ollama Cloud |
OLLAMA_API_KEY |
ollama |
| Moonshot AI |
MOONSHOT_API_KEY |
moonshotai |
| Moonshot AI China |
MOONSHOT_API_KEY |
moonshotai-cn |
| Groq |
GROQ_API_KEY |
groq |
| xAI |
XAI_API_KEY |
xai |
| Cerebras |
CEREBRAS_API_KEY |
cerebras |
| Together AI |
TOGETHER_API_KEY |
together |
| Hugging Face |
HF_TOKEN |
huggingface |
| OpenRouter |
OPENROUTER_API_KEY |
openrouter |
| Mistral |
MISTRAL_API_KEY |
mistral |
| ZAI |
ZAI_API_KEY |
zai |
| Xiaomi MiMo |
XIAOMI_API_KEY |
xiaomi |
| Xiaomi Token Plan Amsterdam |
XIAOMI_TOKEN_PLAN_AMS_API_KEY |
xiaomi-token-plan-ams |
| Xiaomi Token Plan China |
XIAOMI_TOKEN_PLAN_CN_API_KEY |
xiaomi-token-plan-cn |
| Xiaomi Token Plan Singapore |
XIAOMI_TOKEN_PLAN_SGP_API_KEY |
xiaomi-token-plan-sgp |
| MiniMax |
MINIMAX_API_KEY |
minimax |
| MiniMax China |
MINIMAX_CN_API_KEY or MINIMAX_API_KEY |
minimax-cn |
| Fireworks |
FIREWORKS_API_KEY |
fireworks |
| Vercel AI Gateway |
AI_GATEWAY_API_KEY |
vercel-ai-gateway |
| OpenCode Zen |
OPENCODE_API_KEY |
opencode |
| OpenCode Go |
OPENCODE_API_KEY |
opencode-go |
| GitHub Copilot token |
COPILOT_GITHUB_TOKEN or GITHUB_COPILOT_TOKEN |
github-copilot |
| Cloudflare Workers AI |
CLOUDFLARE_API_KEY |
cloudflare-workers-ai |
| Cloudflare AI Gateway |
CLOUDFLARE_API_KEY |
cloudflare-ai-gateway |
| Azure OpenAI Responses |
AZURE_OPENAI_API_KEY |
azure-openai-responses |
Example:
export OPENROUTER_API_KEY=...
zut --provider openrouter
Ollama local vs Cloud
The ollama provider serves both local and Cloud models over the OpenAI-compatible /v1 API. Secrets are sent only as Authorization: Bearer headers, never in URLs or logs.
- No key:
http://localhost:11434, no auth (local models).
- Real key (
--api-key, OLLAMA_API_KEY, or /login → Ollama) with no explicit endpoint: https://ollama.com/v1 (Cloud models).
--base-url always wins; a baseUrl in models.json (model- or provider-level) wins over the automatic default.
OpenCode Go model catalog
OpenCode Go is synchronized instead of using a hand-maintained model list. When
OPENCODE_API_KEY or a configured OpenCode Go credential is available, zut
reads the model IDs published by
https://opencode.ai/zen/go/v1/models
and enriches them with names, limits, prices, reasoning options, and adapter
metadata from the Models.dev JSON API, which
powers the OpenCode Go provider page.
The published list is authoritative for the catalog and picker, but it is not
an entitlement check: consent, regional eligibility, quota, and account policy
can still reject an inference request.
Models.dev adapter metadata selects the wire protocol for each model:
@ai-sdk/openai uses OpenAI Responses, @ai-sdk/openai-compatible uses Chat
Completions, and @ai-sdk/anthropic uses Anthropic Messages. If that metadata
is unavailable, zut keeps the published model ID and uses only its narrow
bootstrap fallback; protocol compatibility cannot be guaranteed until the
metadata is available. An explicit unsupported adapter fails with an
unsupported-protocol error instead of silently choosing Chat Completions.
Explicit and discovered base URLs remain the credential-scoped gateway
endpoint; public metadata does not redirect requests to another host.
The merged catalog is cached in $ZUT_HOME/models-cache.json for 24 hours;
OpenCode Go entries are scoped to both the credential fingerprint and endpoint.
--list-models waits for a refresh when the cache is stale, while SDK runtimes
load matching cached metadata before resolving. Synchronous refreshes resolve
configured api_key_command credentials; unsolicited startup refreshes leave
those commands untouched.
Fast mode
Use /fast or /settings to enable fast mode for subsequent model calls.
The setting is off by default and is stored as fast_mode in
$ZUT_HOME/config.json.
Fast mode currently requests OpenAI's Fast service tier for OpenAI Chat,
OpenAI Responses, and OpenAI Codex requests. Enabling it while using another
provider currently returns an error instead of silently changing that provider's
request. Subagent children inherit the setting from their parent unless their
profile sets fastMode explicitly.
GPT-6 Astra
gpt-6-astra is available through openai (Responses API with an API key)
and openai-codex (ChatGPT subscription). Both catalog entries deliberately
use a 500,000-token working context budget, rather than the larger advertised
maximum, and retain a 128,000-token maximum output. Native reasoning efforts
low, medium, high, xhigh, and max are supported. Selecting reasoning off
omits the effort parameter; it does not disable Astra's reasoning.
Standard API rates per million tokens are $10 input, $50 output, $1 cache reads,
and $12.50 cache writes. Above 272,000 total input tokens, including cache
reads and writes, rates for the entire request become $20 input, $75 output,
$2 cache reads, and $25 cache writes. The 500k working budget does not avoid
this pricing tier. Codex catalog dollar estimates use these API-equivalent
rates, not subscription billing or quota measurements.
Sources: Astra model specifications
and prompt caching.
GPT-6.1 Sol
gpt-6.1-sol is available through openai (Responses API) and openai-codex
with the same 500,000-token working budget, 128,000-token maximum output, and
native reasoning efforts low through max as Astra.
Standard API rates per million tokens are $2 input, $10 output, $0.10 cache
reads, and $2.50 cache writes. Above 272,000 total input tokens, the whole
request is billed at $4 input, $15 output, $0.20 cache reads, and $5 cache
writes.
Source: GPT-6.1 Sol model specifications.
Session identity and caching
zut supplies every provider request with a root cache identity, a
conversation-thread identity, and a per-prompt turn ID. They are opaque local
correlations; adapters translate them only when their exact provider, model,
and endpoint declaration supports it. Resident children share the root cache
identity but retain distinct thread identities. The OpenAI/Codex Responses
route uses the root identity as prompt_cache_key; declared Codex routes also
receive their documented session and thread routing headers. OpenCode Go
requests identify zut with x-opencode-client and User-Agent, use the stable
thread (or root cache) ID as x-opencode-session, and use the turn ID as
x-opencode-request. A generic OpenAI-compatible endpoint never receives
these extensions merely because it accepts an OpenAI-shaped request. JSON
event mode also reports sanitized cache diagnostics (eligible, mode,
transport, and continuation). eligible
is a conservative local prompt-length estimate; OpenAI performs the authoritative
tokenization and cache decision. These records never include prompts, durable
IDs, request bodies, or credentials.
When a provider reports cache detail, zut displays cache-read hit rate as
cache reads / cache-reporting prompt input. A shown 0% is a confirmed
cache miss; no percentage means the provider or historical session did not
report cache detail. Cache writes remain separate from that hit rate. Provider
retention and eviction, request routing, and OpenAI's 1,024-token eligibility
minimum can still produce a miss even when the stable prefix is unchanged.
On the official public OpenAI Responses endpoint, GPT-5.6 and GPT-6 Astra
requests include an explicit cache breakpoint after the stable instruction/tool
prefix while retaining implicit caching for conversation history. The Codex
subscription route and custom Responses endpoints do not receive these explicit
cache controls. Changing request-level reasoning effort can still disrupt prefix
reuse: zut does not yet send cache-preserving configuration_update items.
OpenAI Responses WebSocket mode
For the exact public endpoint https://api.openai.com/v1/responses, zut uses
the Responses API WebSocket mode for the OpenAI Responses route. It keeps one
connection per conversation thread, sends one response.create at a time,
and continues the most recent completed response with previous_response_id
only when the complete response-context configuration and prior input prefix
match exactly, including the completed assistant output that precedes a new
suffix. Compatible continuations send only new tool or user input.
The request uses store:false; its only provider-specific handshake header is
Authorization: Bearer ..., never the
ChatGPT/Codex account, routing, or session headers.
The capability is deliberately not inherited by --base-url, compatible
providers, Azure, or the ChatGPT Codex subscription endpoint. Those clients
continue to use their declared HTTP/SSE contracts. A request without a logical
session ID, a failed WebSocket setup,
an injected nonstandard RoundTripper, or a configured proxy falls back to
HTTP/SSE with the same logical session/cache identity. Cancellation closes
only the active session socket; a normally completed turn keeps it warm for
the next continuation.
Responses explicit continuation
When a Responses completion carries an explicit end_turn:false, zut treats the finished inference as a continuation (turn_end with stop:"continue") and issues another model call in the same turn, reusing its request context, guards, usage accounting, and queued-input boundary. Tool calls still take precedence over the signal, and errors or cancellation never become continuations. A missing, null, or true end_turn keeps the existing behavior and ends the turn.
Responses reasoning replay
You can switch models or providers within the same session. Assistant text and tool exchanges remain available; encrypted reasoning and thought signatures from a different provider or model are omitted from outgoing requests without editing session history. Switching back allows the matching model to replay its own metadata.
Responses requests run with store:false, so only reasoning blocks with an encrypted payload and a valid Responses item ID (when present) are replayed. Session loading recovers message provenance from the provider/model metadata recorded alongside turns. Older compacted messages without provenance retain best-effort replay; malformed reasoning IDs are omitted.
Local llama.cpp router
The llama.cpp provider connects to a multi-model router and is separate from the ollama provider. Ollama normally uses http://localhost:11434; entering that URL for llama.cpp management produces a 404 because Ollama does not implement the router endpoints.
Install a current llama.cpp build. On macOS:
brew install llama.cpp
# use `brew upgrade llama.cpp` for an existing installation
Start llama-server without --model, -m, or -hf. Those flags select one model and disable the router behavior zut expects.
mkdir -p ~/llama-models
llama-server \
--models-dir ~/llama-models \
--no-models-autoload \
--jinja \
--host 127.0.0.1 \
--port 8080 \
-ngl 999 \
-c 32768
This configuration discovers GGUF files below ~/llama-models, leaves model loading under explicit user control, enables chat templates, requests GPU offload, and sets a 32K context. Tune the GPU layers and context size for the available memory.
Check the management API before opening zut:
curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/models
/models must return JSON with a data array. A 404 indicates the wrong URL, an older server build, or single-model mode.
To save the connection, run /login, choose api key, and select llama.cpp. Enter http://127.0.0.1:8080, without /v1, followed by an optional bearer key. zut validates the URL and stores both values in $ZUT_HOME/auth.json. Environment variables override the saved connection:
export LLAMA_BASE_URL=http://127.0.0.1:8080
export LLAMA_API_KEY=optional-secret
If a key is configured, start the router with the same --api-key value. A server bound only to 127.0.0.1 normally does not need authentication.
After login, /llama becomes visible. It shows model state, searches Hugging Face for GGUF repositories, offers available quantizations, reports download and load progress, and explicitly loads or unloads models. Press d on a router-downloaded cache model to ask the router to remove it after confirmation. Models discovered through --models-dir or a preset are not removable through the router API and must be deleted from their configured source. Some llama.cpp versions retain shared Hugging Face repository artifacts such as mmproj files after removing the selected GGUF; remove that repository's cache directory manually if those artifacts are no longer needed. HF_TOKEN raises Hugging Face API limits and may be required for repository metadata. For gated files, approve access first and provide the authorized token to the llama-server process because the router performs the download.
Opening /model refreshes the router and displays its loaded models under provider llama.cpp. Unloaded models cannot handle inference and therefore remain exclusive to /llama until loaded. Inference is sent to the router's derived /v1 endpoint.
Ollama downloads are stored in Ollama's internal layout and do not automatically become llama.cpp models. Obtain a GGUF copy through /llama or place GGUF files under ~/llama-models, then restart the router after adding files manually.
Cloud providers
Amazon Bedrock
Bedrock is configured with AWS credentials, not a generic zut API-key entry.
Use one of these credential sources:
# AWS profile
export AWS_PROFILE=your-profile
# IAM access keys
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # only for temporary credentials
# Bedrock API key bearer token
export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...
# Region
export AWS_REGION=us-east-1
ECS task roles, IRSA, and other AWS SDK credential-chain sources are also
supported.
Example:
AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-... AWS_REGION=us-east-1 \
zut --provider amazon-bedrock --model anthropic.claude-sonnet-4-5-20250929-v1:0
Some Bedrock models require regional inference-profile IDs for on-demand
throughput, such as us. or eu. prefixed model IDs. zut rewrites known
families automatically where possible. Claude Opus 5 uses
global.anthropic.claude-opus-5; zut maps the bare foundation-model ID to that
global profile and supports its adaptive reasoning levels. Explicit profile IDs
and ARNs are left unchanged.
Google Vertex AI
Vertex can use a Google API key when available:
export GOOGLE_CLOUD_API_KEY=...
zut --provider google-vertex
For service-account or application-default credentials, set the standard
Google environment variables used by your deployment.
Cloudflare AI Gateway
Cloudflare AI Gateway needs a Cloudflare token plus account and gateway IDs:
export CLOUDFLARE_API_KEY=...
export CLOUDFLARE_ACCOUNT_ID=...
export CLOUDFLARE_GATEWAY_ID=...
zut --provider cloudflare-ai-gateway
Cloudflare Workers AI
Workers AI needs a Cloudflare token and account ID:
export CLOUDFLARE_API_KEY=...
export CLOUDFLARE_ACCOUNT_ID=...
zut --provider cloudflare-workers-ai
Azure OpenAI Responses
export AZURE_OPENAI_API_KEY=...
export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
export AZURE_OPENAI_API_VERSION=v1 # optional, v1 is the default
zut --provider azure-openai-responses
The provider uses Azure's Responses API. If deployment names differ from zut
model IDs, map them without changing the catalog:
export AZURE_OPENAI_DEPLOYMENT_NAME_MAP='gpt-5.6-luna=luna-preview,gpt-5.6-sol=sol-preview'
Auth file
Credentials are stored in $ZUT_HOME/auth.json with user-only permissions
when zut creates the file.
Example:
{
"anthropic": { "api_key": "sk-ant-..." },
"openai": { "api_key": "sk-..." },
"google": { "api_key": "..." },
"additional_api_key_creds": {
"openrouter": { "api_key": "..." },
"mistral": { "api_key": "..." }
}
}
The top-level keys are used for providers with dedicated credential fields.
Other API-key providers are stored under additional_api_key_creds. Prefer
/login so zut writes the correct schema.
Custom providers and models
Use $ZUT_HOME/models.json for private models, deployment aliases, local
servers, or OpenAI-compatible gateways that are not in the built-in catalog.
User entries override built-in entries with the same provider and model ID, and
adding a models.json no longer hides the built-in catalog: your entries are
merged on top of the baked-in and live-discovered models.
A top-level provider key that is not a built-in id defines a custom provider.
Give it a provider-level baseUrl and an api wire format (openai for
OpenAI-compatible Chat Completions, the default, openai-responses for the
OpenAI Responses API, or anthropic for the Anthropic Messages API). A
model-level baseUrl overrides the provider-level one for that model, and an
unknown api value falls back to openai with a warning.
{
"providers": {
"my-company": {
"baseUrl": "https://llm.mycompany.com/v1",
"api": "openai",
"models": [
{ "id": "company-llm-v2", "name": "Company LLM v2" }
]
}
}
}
Custom providers are first-class: they appear in --list-models, /model, and
/login. models.json never stores secrets. For authenticated endpoints, supply
the key through /login, --api-key, or a derived environment variable in upper
snake case, so my-company reads MY_COMPANY_API_KEY. Custom endpoints with a
configured base URL can also run without a key. Set "auth": "none" on a custom
provider to ignore inherited, CLI, environment, and stored credentials, including
command-backed keys, for inference and discovery. Use this only with trusted
keyless endpoints. Built-in providers retain their own authentication rules;
setting auth: "none" on a built-in provider is ignored with a warning.
Custom provider keys are accepted and stored without a verification probe,
because many self-hosted gateways do not expose a model-list endpoint; an invalid
key surfaces on the first model call.
To retrieve this custom provider's key from a password manager, add a matching
entry to $ZUT_HOME/auth.json:
{
"additional_api_key_creds": {
"my-company": {
"api_key_command": {
"program": "op",
"args": ["read", "op://Work/OpenAI/credential"],
"timeout_ms": 120000
}
}
}
}
The provider IDs in models.json and auth.json must match. Then select the
custom model directly:
zut --provider my-company --model company-llm-v2
Custom model discovery
Set "discover": true at the custom provider level to list models from its
OpenAI-style /models endpoint. Discovery is off by default and requires a
provider-level baseUrl; static model entries override discovered metadata.
For a local keyless server:
{
"providers": {
"local-gateway": {
"baseUrl": "http://127.0.0.1:48000/v1",
"api": "openai",
"auth": "none",
"discover": true,
"models": [
{ "id": "local-model", "contextWindow": 65536, "maxTokens": 8192 }
]
}
}
}
Zut requests <baseUrl>/models, or <baseUrl>/v1/models when the base has no
version suffix, with a three-second timeout and an eight-MiB response limit.
The response must contain {"data": [{"id": "model-id"}]}. Discovery trusts IDs
only: default metadata is 32,768 context tokens, 4,096 output tokens, zero pricing,
and no reasoning controls. IDs containing embed are skipped. Configure explicit
model metadata to raise limits or enable reasoning. Discovery always verifies
TLS certificates; --insecure and the saved insecure setting apply to inference,
not discovery. For self-signed servers, install a trusted certificate chain.
Discovery runs in the background at startup and when /model opens. Launching
with an unknown model from a discovery-enabled provider probes only that
selected provider, without touching other providers' endpoints or key commands.
/model refreshes configured llama.cpp and custom catalogs with independent three-second
timeouts and still opens the picker if a refresh fails, showing the error.
Lists stay in memory, separate from the cloud model cache. Failed refreshes
retain the previous provider snapshot. Background discovery never executes an
api_key_command and skips a provider whose only key is command-backed; an
explicit refresh may execute that command. The refresh budget also bounds the
key command; a slow or interactive password-manager prompt can time out.
Environment or stored keys are used when available unless auth is none; no
key is required for a keyless server. If an explicitly selected custom model is
unknown and discovery fails, inference can still attempt it with the existing
128,000-context / 16,384-output fallback. Configure static model limits to avoid
relying on either fallback.
Credential resolution
For each request, zut checks credentials in this order:
- Explicit CLI key, such as
--api-key.
- Provider-specific environment variables (including derived custom-provider
variables such as
MY_COMPANY_API_KEY).
$ZUT_HOME/auth.json, including custom provider keys saved by /login.
models.json itself never stores credentials; it only describes models and
their endpoints.
Bedrock then uses the AWS SDK credential chain for the actual request.
source
1# zut providers
2
3zut ships with built-in providers and a model catalog. You can select models
4with `/model`, list them with `zut --list-models`, and add private models in
5`$ZUT_HOME/models.json`.
6
7## HTTP proxy
8
9Set one global proxy for zut-managed HTTP and HTTPS traffic with the `http_proxy` key in `$ZUT_HOME/config.json`:
10
11```json
12{
13 "http_proxy": "http://127.0.0.1:7890"
14}
15```
16
17zut applies this setting at startup. Existing `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy`, and `https_proxy` environment variables take precedence for their corresponding protocol. Standard `NO_PROXY` and `no_proxy` bypass lists remain effective. Restart zut after changing the setting.
18
19`config.json` is not a credential store. If the proxy URL contains a username or password, prefer setting the proxy environment variables in a protected shell or service configuration rather than saving those credentials in `config.json`.
20
21## Login methods
22
23Use `/login` in interactive mode. Type in either provider picker to filter the list by provider ID or display name.
24
25- `api key`: stores an API key in `$ZUT_HOME/auth.json` when the provider uses a normal key.
26- `subscription`: stores OAuth credentials for subscription-backed providers.
27
28Use `/logout` to remove stored credentials.
29
30Some providers need more than a single pasted key. For those providers,
31`/login` shows setup instructions instead of opening a localhost browser form.
32This avoids broken browser flows in SSH, containers, and `kubectl exec`
33sessions.
34
35### Command-backed API keys
36
37A provider can obtain its API key from a password manager or another local program. Configure this directly in `$ZUT_HOME/auth.json`:
38
39```json
40{
41 "openai": {
42 "api_key_command": {
43 "program": "op",
44 "args": ["read", "op://Work/OpenAI/credential"],
45 "timeout_ms": 120000
46 }
47 }
48}
49```
50
51Custom provider credentials use the same object under `additional_api_key_creds`:
52
53```json
54{
55 "additional_api_key_creds": {
56 "my-company": {
57 "api_key_command": {
58 "program": "secret-tool",
59 "args": ["read", "my-company-api-key"]
60 }
61 }
62 }
63}
64```
65
66The program is started directly rather than through a shell. The timeout defaults to 120 seconds. It must write one non-empty line to stdout, with no more than 64 KiB of output. Trailing CR/LF characters are removed. Successful results stay only in process memory and are reused until zut exits.
67
68Command-backed credentials count as logged in without being executed. zut materializes one only when selecting that provider; background model discovery skips it. `/login` with a normal key replaces the command, while `/logout` clears it. Because `auth.json` can cause program execution, keep it user-writable only and do not use files from untrusted sources.
69
70Setup-instruction providers:
71
72- Amazon Bedrock
73- Google Vertex AI
74- Cloudflare Workers AI
75- Cloudflare AI Gateway
76- Azure OpenAI Responses
77
78## Subscription providers
79
80These providers support subscription login:
81
82| Provider | Notes |
83| --- | --- |
84| Anthropic | Claude Pro/Max OAuth credentials. |
85| OpenAI Codex | ChatGPT Plus/Pro Codex subscription route. Separate from the OpenAI API-key provider. |
86| Kimi | Kimi subscription login. |
87| xAI | SuperGrok or X Premium device-code login. The browser URL is prefilled with the device code. |
88| GitHub Copilot | GitHub Copilot token flow. |
89
90OAuth tokens are stored in `$ZUT_HOME/auth.json` and refreshed when refresh is
91available.
92
93Anthropic offers browser callback and copy-code login as separate choices.
94With Anthropic OAuth, MCP resource helpers use Claude Code wire names while
95zut retains `mcp_list_resources` and `mcp_read_resource` in tools and sessions.
96API-key requests retain the zut names. OAuth acceptance and billing remain
97controlled by Anthropic; a successful request does not guarantee subscription
98coverage.
99
100Browser login for Anthropic and OpenAI Codex also accepts a pasted callback URL
101or code using the same authorization transaction; it does not start a second
102login. A supplied state must match the pending transaction. Escape cancels the
103pending flow. OpenAI Codex requires the browser callback transaction and has no
104standalone manual-code login.
105
106### GitHub Copilot
107
108Copilot uses the account's short-lived inference token. Claude models use the
109Anthropic Messages API with Copilot Bearer authentication, without Claude Code
110identity headers. Newer GPT, Grok, and MAI models use Responses; older GPT-4 and
111Grok Code models remain on Chat Completions. Gemini and Kimi use Chat Completions
112without `reasoning_effort`. Requests carry user/agent initiator metadata and an
113image request header when appropriate.
114
115Authenticated model discovery filters the runtime catalog by picker visibility,
116policy state, and tool-call support. The individual-account endpoint may fall
117back to explicitly enabled policies when no eligible picker entries exist;
118other endpoints cannot. Zut never enables policies automatically. Availability
119is a selection aid, not an authorization guarantee, and remains in memory rather
120than the shared model cache. Failed discovery retains the previous snapshot.
121Background discovery does not run command-backed credentials. Until discovery
122succeeds, the static catalog is visible. Existing sessions and explicit model
123selection still resolve catalog metadata for models hidden from the picker.
124The default Copilot model is `claude-sonnet-5`. User model overrides retain
125Copilot protocol defaults unless an explicit API or `reasoningLevelMap` overrides
126them. For Copilot rows, the `anthropic` and `openai` API aliases select Messages
127and Chat Completions respectively. Claude 4.5/4.6 rows use explicit thinking
128budgets. The retired
129`mai-code-1-flash-picker` is not in the static catalog; `mai-code-1.1-flash` is.
130
131## API-key providers
132
133These providers can use environment variables. Simple API-key providers can
134also be configured through `/login`. Providers that require extra cloud setup
135show instructions and should be configured with environment variables.
136
137| Provider | Environment variable | Stored key |
138| --- | --- | --- |
139| Anthropic | `ANTHROPIC_API_KEY` | `anthropic` |
140| OpenAI | `OPENAI_API_KEY` | `openai` |
141| OpenAI Responses | `OPENAI_API_KEY` | `openai-responses` |
142| Kimi | `KIMI_API_KEY` or `MOONSHOT_API_KEY` | `kimi` |
143| Google Gemini | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | `google` |
144| DeepSeek | `DEEPSEEK_API_KEY` | `deepseek` |
145| Ollama Cloud | `OLLAMA_API_KEY` | `ollama` |
146| Moonshot AI | `MOONSHOT_API_KEY` | `moonshotai` |
147| Moonshot AI China | `MOONSHOT_API_KEY` | `moonshotai-cn` |
148| Groq | `GROQ_API_KEY` | `groq` |
149| xAI | `XAI_API_KEY` | `xai` |
150| Cerebras | `CEREBRAS_API_KEY` | `cerebras` |
151| Together AI | `TOGETHER_API_KEY` | `together` |
152| Hugging Face | `HF_TOKEN` | `huggingface` |
153| OpenRouter | `OPENROUTER_API_KEY` | `openrouter` |
154| Mistral | `MISTRAL_API_KEY` | `mistral` |
155| ZAI | `ZAI_API_KEY` | `zai` |
156| Xiaomi MiMo | `XIAOMI_API_KEY` | `xiaomi` |
157| Xiaomi Token Plan Amsterdam | `XIAOMI_TOKEN_PLAN_AMS_API_KEY` | `xiaomi-token-plan-ams` |
158| Xiaomi Token Plan China | `XIAOMI_TOKEN_PLAN_CN_API_KEY` | `xiaomi-token-plan-cn` |
159| Xiaomi Token Plan Singapore | `XIAOMI_TOKEN_PLAN_SGP_API_KEY` | `xiaomi-token-plan-sgp` |
160| MiniMax | `MINIMAX_API_KEY` | `minimax` |
161| MiniMax China | `MINIMAX_CN_API_KEY` or `MINIMAX_API_KEY` | `minimax-cn` |
162| Fireworks | `FIREWORKS_API_KEY` | `fireworks` |
163| Vercel AI Gateway | `AI_GATEWAY_API_KEY` | `vercel-ai-gateway` |
164| OpenCode Zen | `OPENCODE_API_KEY` | `opencode` |
165| OpenCode Go | `OPENCODE_API_KEY` | `opencode-go` |
166| GitHub Copilot token | `COPILOT_GITHUB_TOKEN` or `GITHUB_COPILOT_TOKEN` | `github-copilot` |
167| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` | `cloudflare-workers-ai` |
168| Cloudflare AI Gateway | `CLOUDFLARE_API_KEY` | `cloudflare-ai-gateway` |
169| Azure OpenAI Responses | `AZURE_OPENAI_API_KEY` | `azure-openai-responses` |
170
171Example:
172
173```bash
174export OPENROUTER_API_KEY=...
175zut --provider openrouter
176```
177
178### Ollama local vs Cloud
179
180The `ollama` provider serves both local and Cloud models over the OpenAI-compatible `/v1` API. Secrets are sent only as `Authorization: Bearer` headers, never in URLs or logs.
181
182- No key: `http://localhost:11434`, no auth (local models).
183- Real key (`--api-key`, `OLLAMA_API_KEY`, or `/login` → Ollama) with no explicit endpoint: `https://ollama.com/v1` (Cloud models).
184- `--base-url` always wins; a `baseUrl` in `models.json` (model- or provider-level) wins over the automatic default.
185
186### OpenCode Go model catalog
187
188OpenCode Go is synchronized instead of using a hand-maintained model list. When
189`OPENCODE_API_KEY` or a configured OpenCode Go credential is available, zut
190reads the model IDs published by
191[`https://opencode.ai/zen/go/v1/models`](https://opencode.ai/zen/go/v1/models)
192and enriches them with names, limits, prices, reasoning options, and adapter
193metadata from the [Models.dev JSON API](https://models.dev/api.json), which
194powers the [OpenCode Go provider page](https://models.dev/providers/opencode-go/).
195The published list is authoritative for the catalog and picker, but it is not
196an entitlement check: consent, regional eligibility, quota, and account policy
197can still reject an inference request.
198
199Models.dev adapter metadata selects the wire protocol for each model:
200`@ai-sdk/openai` uses OpenAI Responses, `@ai-sdk/openai-compatible` uses Chat
201Completions, and `@ai-sdk/anthropic` uses Anthropic Messages. If that metadata
202is unavailable, zut keeps the published model ID and uses only its narrow
203bootstrap fallback; protocol compatibility cannot be guaranteed until the
204metadata is available. An explicit unsupported adapter fails with an
205unsupported-protocol error instead of silently choosing Chat Completions.
206Explicit and discovered base URLs remain the credential-scoped gateway
207endpoint; public metadata does not redirect requests to another host.
208
209The merged catalog is cached in `$ZUT_HOME/models-cache.json` for 24 hours;
210OpenCode Go entries are scoped to both the credential fingerprint and endpoint.
211`--list-models` waits for a refresh when the cache is stale, while SDK runtimes
212load matching cached metadata before resolving. Synchronous refreshes resolve
213configured `api_key_command` credentials; unsolicited startup refreshes leave
214those commands untouched.
215
216## Fast mode
217
218Use `/fast` or `/settings` to enable **fast mode** for subsequent model calls.
219The setting is off by default and is stored as `fast_mode` in
220`$ZUT_HOME/config.json`.
221
222Fast mode currently requests OpenAI's Fast service tier for OpenAI Chat,
223OpenAI Responses, and OpenAI Codex requests. Enabling it while using another
224provider currently returns an error instead of silently changing that provider's
225request. Subagent children inherit the setting from their parent unless their
226profile sets `fastMode` explicitly.
227
228## GPT-6 Astra
229
230`gpt-6-astra` is available through `openai` (Responses API with an API key)
231and `openai-codex` (ChatGPT subscription). Both catalog entries deliberately
232use a **500,000-token working context budget**, rather than the larger advertised
233maximum, and retain a 128,000-token maximum output. Native reasoning efforts
234`low`, `medium`, `high`, `xhigh`, and `max` are supported. Selecting reasoning off
235omits the effort parameter; it does not disable Astra's reasoning.
236
237Standard API rates per million tokens are $10 input, $50 output, $1 cache reads,
238and $12.50 cache writes. Above **272,000 total input tokens**, including cache
239reads and writes, rates for the entire request become $20 input, $75 output,
240$2 cache reads, and $25 cache writes. The 500k working budget does not avoid
241this pricing tier. Codex catalog dollar estimates use these API-equivalent
242rates, not subscription billing or quota measurements.
243
244Sources: [Astra model specifications](https://developers.openai.com/api/docs/models/gpt-6-astra)
245and [prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching).
246
247## GPT-6.1 Sol
248
249`gpt-6.1-sol` is available through `openai` (Responses API) and `openai-codex`
250with the same 500,000-token working budget, 128,000-token maximum output, and
251native reasoning efforts `low` through `max` as Astra.
252
253Standard API rates per million tokens are $2 input, $10 output, $0.10 cache
254reads, and $2.50 cache writes. Above 272,000 total input tokens, the whole
255request is billed at $4 input, $15 output, $0.20 cache reads, and $5 cache
256writes.
257
258Source: [GPT-6.1 Sol model specifications](https://developers.openai.com/api/docs/models/gpt-6.1-sol).
259
260## Session identity and caching
261
262zut supplies every provider request with a root cache identity, a
263conversation-thread identity, and a per-prompt turn ID. They are opaque local
264correlations; adapters translate them only when their exact provider, model,
265and endpoint declaration supports it. Resident children share the root cache
266identity but retain distinct thread identities. The OpenAI/Codex Responses
267route uses the root identity as `prompt_cache_key`; declared Codex routes also
268receive their documented session and thread routing headers. OpenCode Go
269requests identify zut with `x-opencode-client` and `User-Agent`, use the stable
270thread (or root cache) ID as `x-opencode-session`, and use the turn ID as
271`x-opencode-request`. A generic OpenAI-compatible endpoint never receives
272these extensions merely because it accepts an OpenAI-shaped request. JSON
273event mode also reports sanitized cache diagnostics (`eligible`, `mode`,
274`transport`, and `continuation`). `eligible`
275is a conservative local prompt-length estimate; OpenAI performs the authoritative
276tokenization and cache decision. These records never include prompts, durable
277IDs, request bodies, or credentials.
278
279When a provider reports cache detail, zut displays cache-read hit rate as
280`cache reads / cache-reporting prompt input`. A shown `0%` is a confirmed
281cache miss; no percentage means the provider or historical session did not
282report cache detail. Cache writes remain separate from that hit rate. Provider
283retention and eviction, request routing, and OpenAI's 1,024-token eligibility
284minimum can still produce a miss even when the stable prefix is unchanged.
285
286On the official public OpenAI Responses endpoint, GPT-5.6 and GPT-6 Astra
287requests include an explicit cache breakpoint after the stable instruction/tool
288prefix while retaining implicit caching for conversation history. The Codex
289subscription route and custom Responses endpoints do not receive these explicit
290cache controls. Changing request-level reasoning effort can still disrupt prefix
291reuse: zut does not yet send cache-preserving `configuration_update` items.
292
293### OpenAI Responses WebSocket mode
294
295For the exact public endpoint `https://api.openai.com/v1/responses`, zut uses
296the Responses API WebSocket mode for the OpenAI Responses route. It keeps one
297connection per conversation thread, sends one `response.create` at a time,
298and continues the most recent completed response with `previous_response_id`
299only when the complete response-context configuration and prior input prefix
300match exactly, including the completed assistant output that precedes a new
301suffix. Compatible continuations send only new tool or user input.
302The request uses `store:false`; its only provider-specific handshake header is
303`Authorization: Bearer ...`, never the
304ChatGPT/Codex account, routing, or session headers.
305
306The capability is deliberately not inherited by `--base-url`, compatible
307providers, Azure, or the ChatGPT Codex subscription endpoint. Those clients
308continue to use their declared HTTP/SSE contracts. A request without a logical
309session ID, a failed WebSocket setup,
310an injected nonstandard RoundTripper, or a configured proxy falls back to
311HTTP/SSE with the same logical session/cache identity. Cancellation closes
312only the active session socket; a normally completed turn keeps it warm for
313the next continuation.
314
315### Responses explicit continuation
316
317When a Responses completion carries an explicit `end_turn:false`, zut treats the finished inference as a continuation (`turn_end` with `stop:"continue"`) and issues another model call in the same turn, reusing its request context, guards, usage accounting, and queued-input boundary. Tool calls still take precedence over the signal, and errors or cancellation never become continuations. A missing, null, or true `end_turn` keeps the existing behavior and ends the turn.
318
319### Responses reasoning replay
320
321You can switch models or providers within the same session. Assistant text and tool exchanges remain available; encrypted reasoning and thought signatures from a different provider or model are omitted from outgoing requests without editing session history. Switching back allows the matching model to replay its own metadata.
322
323Responses requests run with `store:false`, so only reasoning blocks with an encrypted payload and a valid Responses item ID (when present) are replayed. Session loading recovers message provenance from the provider/model metadata recorded alongside turns. Older compacted messages without provenance retain best-effort replay; malformed reasoning IDs are omitted.
324
325## Local llama.cpp router
326
327The `llama.cpp` provider connects to a multi-model router and is separate from the `ollama` provider. Ollama normally uses `http://localhost:11434`; entering that URL for llama.cpp management produces a 404 because Ollama does not implement the router endpoints.
328
329Install a current llama.cpp build. On macOS:
330
331```bash
332brew install llama.cpp
333# use `brew upgrade llama.cpp` for an existing installation
334```
335
336Start `llama-server` without `--model`, `-m`, or `-hf`. Those flags select one model and disable the router behavior zut expects.
337
338```bash
339mkdir -p ~/llama-models
340
341llama-server \
342 --models-dir ~/llama-models \
343 --no-models-autoload \
344 --jinja \
345 --host 127.0.0.1 \
346 --port 8080 \
347 -ngl 999 \
348 -c 32768
349```
350
351This configuration discovers GGUF files below `~/llama-models`, leaves model loading under explicit user control, enables chat templates, requests GPU offload, and sets a 32K context. Tune the GPU layers and context size for the available memory.
352
353Check the management API before opening zut:
354
355```bash
356curl http://127.0.0.1:8080/health
357curl http://127.0.0.1:8080/models
358```
359
360`/models` must return JSON with a `data` array. A 404 indicates the wrong URL, an older server build, or single-model mode.
361
362To save the connection, run `/login`, choose `api key`, and select `llama.cpp`. Enter `http://127.0.0.1:8080`, without `/v1`, followed by an optional bearer key. zut validates the URL and stores both values in `$ZUT_HOME/auth.json`. Environment variables override the saved connection:
363
364```bash
365export LLAMA_BASE_URL=http://127.0.0.1:8080
366export LLAMA_API_KEY=optional-secret
367```
368
369If a key is configured, start the router with the same `--api-key` value. A server bound only to `127.0.0.1` normally does not need authentication.
370
371After login, `/llama` becomes visible. It shows model state, searches Hugging Face for GGUF repositories, offers available quantizations, reports download and load progress, and explicitly loads or unloads models. Press `d` on a router-downloaded cache model to ask the router to remove it after confirmation. Models discovered through `--models-dir` or a preset are not removable through the router API and must be deleted from their configured source. Some llama.cpp versions retain shared Hugging Face repository artifacts such as `mmproj` files after removing the selected GGUF; remove that repository's cache directory manually if those artifacts are no longer needed. `HF_TOKEN` raises Hugging Face API limits and may be required for repository metadata. For gated files, approve access first and provide the authorized token to the `llama-server` process because the router performs the download.
372
373Opening `/model` refreshes the router and displays its loaded models under provider `llama.cpp`. Unloaded models cannot handle inference and therefore remain exclusive to `/llama` until loaded. Inference is sent to the router's derived `/v1` endpoint.
374
375Ollama downloads are stored in Ollama's internal layout and do not automatically become llama.cpp models. Obtain a GGUF copy through `/llama` or place GGUF files under `~/llama-models`, then restart the router after adding files manually.
376
377## Cloud providers
378
379### Amazon Bedrock
380
381Bedrock is configured with AWS credentials, not a generic zut API-key entry.
382Use one of these credential sources:
383
384```bash
385# AWS profile
386export AWS_PROFILE=your-profile
387
388# IAM access keys
389export AWS_ACCESS_KEY_ID=AKIA...
390export AWS_SECRET_ACCESS_KEY=...
391export AWS_SESSION_TOKEN=... # only for temporary credentials
392
393# Bedrock API key bearer token
394export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...
395
396# Region
397export AWS_REGION=us-east-1
398```
399
400ECS task roles, IRSA, and other AWS SDK credential-chain sources are also
401supported.
402
403Example:
404
405```bash
406AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-... AWS_REGION=us-east-1 \
407 zut --provider amazon-bedrock --model anthropic.claude-sonnet-4-5-20250929-v1:0
408```
409
410Some Bedrock models require regional inference-profile IDs for on-demand
411throughput, such as `us.` or `eu.` prefixed model IDs. zut rewrites known
412families automatically where possible. Claude Opus 5 uses
413`global.anthropic.claude-opus-5`; zut maps the bare foundation-model ID to that
414global profile and supports its adaptive reasoning levels. Explicit profile IDs
415and ARNs are left unchanged.
416
417### Google Vertex AI
418
419Vertex can use a Google API key when available:
420
421```bash
422export GOOGLE_CLOUD_API_KEY=...
423zut --provider google-vertex
424```
425
426For service-account or application-default credentials, set the standard
427Google environment variables used by your deployment.
428
429### Cloudflare AI Gateway
430
431Cloudflare AI Gateway needs a Cloudflare token plus account and gateway IDs:
432
433```bash
434export CLOUDFLARE_API_KEY=...
435export CLOUDFLARE_ACCOUNT_ID=...
436export CLOUDFLARE_GATEWAY_ID=...
437zut --provider cloudflare-ai-gateway
438```
439
440### Cloudflare Workers AI
441
442Workers AI needs a Cloudflare token and account ID:
443
444```bash
445export CLOUDFLARE_API_KEY=...
446export CLOUDFLARE_ACCOUNT_ID=...
447zut --provider cloudflare-workers-ai
448```
449
450### Azure OpenAI Responses
451
452```bash
453export AZURE_OPENAI_API_KEY=...
454export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
455export AZURE_OPENAI_API_VERSION=v1 # optional, v1 is the default
456zut --provider azure-openai-responses
457```
458
459The provider uses Azure's Responses API. If deployment names differ from zut
460model IDs, map them without changing the catalog:
461
462```bash
463export AZURE_OPENAI_DEPLOYMENT_NAME_MAP='gpt-5.6-luna=luna-preview,gpt-5.6-sol=sol-preview'
464```
465
466## Auth file
467
468Credentials are stored in `$ZUT_HOME/auth.json` with user-only permissions
469when zut creates the file.
470
471Example:
472
473```json
474{
475 "anthropic": { "api_key": "sk-ant-..." },
476 "openai": { "api_key": "sk-..." },
477 "google": { "api_key": "..." },
478 "additional_api_key_creds": {
479 "openrouter": { "api_key": "..." },
480 "mistral": { "api_key": "..." }
481 }
482}
483```
484
485The top-level keys are used for providers with dedicated credential fields.
486Other API-key providers are stored under `additional_api_key_creds`. Prefer
487`/login` so zut writes the correct schema.
488
489## Custom providers and models
490
491Use `$ZUT_HOME/models.json` for private models, deployment aliases, local
492servers, or OpenAI-compatible gateways that are not in the built-in catalog.
493User entries override built-in entries with the same provider and model ID, and
494adding a `models.json` no longer hides the built-in catalog: your entries are
495merged on top of the baked-in and live-discovered models.
496
497A top-level provider key that is not a built-in id defines a custom provider.
498Give it a provider-level `baseUrl` and an `api` wire format (`openai` for
499OpenAI-compatible Chat Completions, the default, `openai-responses` for the
500OpenAI Responses API, or `anthropic` for the Anthropic Messages API). A
501model-level `baseUrl` overrides the provider-level one for that model, and an
502unknown `api` value falls back to `openai` with a warning.
503
504```json
505{
506 "providers": {
507 "my-company": {
508 "baseUrl": "https://llm.mycompany.com/v1",
509 "api": "openai",
510 "models": [
511 { "id": "company-llm-v2", "name": "Company LLM v2" }
512 ]
513 }
514 }
515}
516```
517
518Custom providers are first-class: they appear in `--list-models`, `/model`, and
519`/login`. `models.json` never stores secrets. For authenticated endpoints, supply
520the key through `/login`, `--api-key`, or a derived environment variable in upper
521snake case, so `my-company` reads `MY_COMPANY_API_KEY`. Custom endpoints with a
522configured base URL can also run without a key. Set `"auth": "none"` on a custom
523provider to ignore inherited, CLI, environment, and stored credentials, including
524command-backed keys, for inference and discovery. Use this only with trusted
525keyless endpoints. Built-in providers retain their own authentication rules;
526setting `auth: "none"` on a built-in provider is ignored with a warning.
527
528Custom provider keys are accepted and stored without a verification probe,
529because many self-hosted gateways do not expose a model-list endpoint; an invalid
530key surfaces on the first model call.
531
532To retrieve this custom provider's key from a password manager, add a matching
533entry to `$ZUT_HOME/auth.json`:
534
535```json
536{
537 "additional_api_key_creds": {
538 "my-company": {
539 "api_key_command": {
540 "program": "op",
541 "args": ["read", "op://Work/OpenAI/credential"],
542 "timeout_ms": 120000
543 }
544 }
545 }
546}
547```
548
549The provider IDs in `models.json` and `auth.json` must match. Then select the
550custom model directly:
551
552```sh
553zut --provider my-company --model company-llm-v2
554```
555
556### Custom model discovery
557
558Set `"discover": true` at the custom provider level to list models from its
559OpenAI-style `/models` endpoint. Discovery is off by default and requires a
560provider-level `baseUrl`; static model entries override discovered metadata.
561For a local keyless server:
562
563```json
564{
565 "providers": {
566 "local-gateway": {
567 "baseUrl": "http://127.0.0.1:48000/v1",
568 "api": "openai",
569 "auth": "none",
570 "discover": true,
571 "models": [
572 { "id": "local-model", "contextWindow": 65536, "maxTokens": 8192 }
573 ]
574 }
575 }
576}
577```
578
579Zut requests `<baseUrl>/models`, or `<baseUrl>/v1/models` when the base has no
580version suffix, with a three-second timeout and an eight-MiB response limit.
581The response must contain `{"data": [{"id": "model-id"}]}`. Discovery trusts IDs
582only: default metadata is 32,768 context tokens, 4,096 output tokens, zero pricing,
583and no reasoning controls. IDs containing `embed` are skipped. Configure explicit
584model metadata to raise limits or enable reasoning. Discovery always verifies
585TLS certificates; `--insecure` and the saved insecure setting apply to inference,
586not discovery. For self-signed servers, install a trusted certificate chain.
587
588Discovery runs in the background at startup and when `/model` opens. Launching
589with an unknown model from a discovery-enabled provider probes only that
590selected provider, without touching other providers' endpoints or key commands.
591`/model` refreshes configured llama.cpp and custom catalogs with independent three-second
592timeouts and still opens the picker if a refresh fails, showing the error.
593Lists stay in memory, separate from the cloud model cache. Failed refreshes
594retain the previous provider snapshot. Background discovery never executes an
595`api_key_command` and skips a provider whose only key is command-backed; an
596explicit refresh may execute that command. The refresh budget also bounds the
597key command; a slow or interactive password-manager prompt can time out.
598Environment or stored keys are used when available unless `auth` is `none`; no
599key is required for a keyless server. If an explicitly selected custom model is
600unknown and discovery fails, inference can still attempt it with the existing
601128,000-context / 16,384-output fallback. Configure static model limits to avoid
602relying on either fallback.
603
604## Credential resolution
605
606For each request, zut checks credentials in this order:
607
6081. Explicit CLI key, such as `--api-key`.
6092. Provider-specific environment variables (including derived custom-provider
610 variables such as `MY_COMPANY_API_KEY`).
6113. `$ZUT_HOME/auth.json`, including custom provider keys saved by `/login`.
612
613`models.json` itself never stores credentials; it only describes models and
614their endpoints.
615
616Bedrock then uses the AWS SDK credential chain for the actual request.