1# zut providers
2
3zut ships with built-in providers and a model catalog. You can select models
4with `/model`, list them with `zut --list-models`, and add private models in
5`$ZUT_HOME/models.json`.
6
7## HTTP proxy
8
9Set one global proxy for zut-managed HTTP and HTTPS traffic with the `http_proxy` key in `$ZUT_HOME/config.json`:
10
11```json
12{
13  "http_proxy": "http://127.0.0.1:7890"
14}
15```
16
17zut applies this setting at startup. Existing `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy`, and `https_proxy` environment variables take precedence for their corresponding protocol. Standard `NO_PROXY` and `no_proxy` bypass lists remain effective. Restart zut after changing the setting.
18
19`config.json` is not a credential store. If the proxy URL contains a username or password, prefer setting the proxy environment variables in a protected shell or service configuration rather than saving those credentials in `config.json`.
20
21## Login methods
22
23Use `/login` in interactive mode. Type in either provider picker to filter the list by provider ID or display name.
24
25- `api key`: stores an API key in `$ZUT_HOME/auth.json` when the provider uses a normal key.
26- `subscription`: stores OAuth credentials for subscription-backed providers.
27
28Use `/logout` to remove stored credentials.
29
30Some providers need more than a single pasted key. For those providers,
31`/login` shows setup instructions instead of opening a localhost browser form.
32This avoids broken browser flows in SSH, containers, and `kubectl exec`
33sessions.
34
35### Command-backed API keys
36
37A provider can obtain its API key from a password manager or another local program. Configure this directly in `$ZUT_HOME/auth.json`:
38
39```json
40{
41  "openai": {
42    "api_key_command": {
43      "program": "op",
44      "args": ["read", "op://Work/OpenAI/credential"],
45      "timeout_ms": 120000
46    }
47  }
48}
49```
50
51Custom provider credentials use the same object under `additional_api_key_creds`:
52
53```json
54{
55  "additional_api_key_creds": {
56    "my-company": {
57      "api_key_command": {
58        "program": "secret-tool",
59        "args": ["read", "my-company-api-key"]
60      }
61    }
62  }
63}
64```
65
66The program is started directly rather than through a shell. The timeout defaults to 120 seconds. It must write one non-empty line to stdout, with no more than 64 KiB of output. Trailing CR/LF characters are removed. Successful results stay only in process memory and are reused until zut exits.
67
68Command-backed credentials count as logged in without being executed. zut materializes one only when selecting that provider; background model discovery skips it. `/login` with a normal key replaces the command, while `/logout` clears it. Because `auth.json` can cause program execution, keep it user-writable only and do not use files from untrusted sources.
69
70Setup-instruction providers:
71
72- Amazon Bedrock
73- Google Vertex AI
74- Cloudflare Workers AI
75- Cloudflare AI Gateway
76- Azure OpenAI Responses
77
78## Subscription providers
79
80These providers support subscription login:
81
82| Provider | Notes |
83| --- | --- |
84| Anthropic | Claude Pro/Max OAuth credentials. |
85| OpenAI Codex | ChatGPT Plus/Pro Codex subscription route. Separate from the OpenAI API-key provider. |
86| Kimi | Kimi subscription login. |
87| xAI | SuperGrok or X Premium device-code login. The browser URL is prefilled with the device code. |
88| GitHub Copilot | GitHub Copilot token flow. |
89
90OAuth tokens are stored in `$ZUT_HOME/auth.json` and refreshed when refresh is
91available.
92
93Anthropic offers browser callback and copy-code login as separate choices.
94Browser login for Anthropic and OpenAI Codex also accepts a pasted callback URL
95or code using the same authorization transaction; it does not start a second
96login. A supplied state must match the pending transaction. Escape cancels the
97pending flow. OpenAI Codex requires the browser callback transaction and has no
98standalone manual-code login.
99
100### GitHub Copilot
101
102Copilot uses the account's short-lived inference token. Claude models use the
103Anthropic Messages API with Copilot Bearer authentication, without Claude Code
104identity headers. Newer GPT, Grok, and MAI models use Responses; older GPT-4 and
105Grok Code models remain on Chat Completions. Gemini and Kimi use Chat Completions
106without `reasoning_effort`. Requests carry user/agent initiator metadata and an
107image request header when appropriate.
108
109Authenticated model discovery filters the runtime catalog by picker visibility,
110policy state, and tool-call support. The individual-account endpoint may fall
111back to explicitly enabled policies when no eligible picker entries exist;
112other endpoints cannot. Zut never enables policies automatically. Availability
113is a selection aid, not an authorization guarantee, and remains in memory rather
114than the shared model cache. Failed discovery retains the previous snapshot.
115Background discovery does not run command-backed credentials. Until discovery
116succeeds, the static catalog is visible. Existing sessions and explicit model
117selection still resolve catalog metadata for models hidden from the picker.
118The default Copilot model is `claude-sonnet-5`. User model overrides retain
119Copilot protocol defaults unless an explicit API or `reasoningLevelMap` overrides
120them. For Copilot rows, the `anthropic` and `openai` API aliases select Messages
121and Chat Completions respectively. Claude 4.5/4.6 rows use explicit thinking
122budgets. The retired
123`mai-code-1-flash-picker` is not in the static catalog; `mai-code-1.1-flash` is.
124
125## API-key providers
126
127These providers can use environment variables. Simple API-key providers can
128also be configured through `/login`. Providers that require extra cloud setup
129show instructions and should be configured with environment variables.
130
131| Provider | Environment variable | Stored key |
132| --- | --- | --- |
133| Anthropic | `ANTHROPIC_API_KEY` | `anthropic` |
134| OpenAI | `OPENAI_API_KEY` | `openai` |
135| OpenAI Responses | `OPENAI_API_KEY` | `openai-responses` |
136| Kimi | `KIMI_API_KEY` or `MOONSHOT_API_KEY` | `kimi` |
137| Google Gemini | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | `google` |
138| DeepSeek | `DEEPSEEK_API_KEY` | `deepseek` |
139| Ollama Cloud | `OLLAMA_API_KEY` | `ollama` |
140| Moonshot AI | `MOONSHOT_API_KEY` | `moonshotai` |
141| Moonshot AI China | `MOONSHOT_API_KEY` | `moonshotai-cn` |
142| Groq | `GROQ_API_KEY` | `groq` |
143| xAI | `XAI_API_KEY` | `xai` |
144| Cerebras | `CEREBRAS_API_KEY` | `cerebras` |
145| Together AI | `TOGETHER_API_KEY` | `together` |
146| Hugging Face | `HF_TOKEN` | `huggingface` |
147| OpenRouter | `OPENROUTER_API_KEY` | `openrouter` |
148| Mistral | `MISTRAL_API_KEY` | `mistral` |
149| ZAI | `ZAI_API_KEY` | `zai` |
150| Xiaomi MiMo | `XIAOMI_API_KEY` | `xiaomi` |
151| Xiaomi Token Plan Amsterdam | `XIAOMI_TOKEN_PLAN_AMS_API_KEY` | `xiaomi-token-plan-ams` |
152| Xiaomi Token Plan China | `XIAOMI_TOKEN_PLAN_CN_API_KEY` | `xiaomi-token-plan-cn` |
153| Xiaomi Token Plan Singapore | `XIAOMI_TOKEN_PLAN_SGP_API_KEY` | `xiaomi-token-plan-sgp` |
154| MiniMax | `MINIMAX_API_KEY` | `minimax` |
155| MiniMax China | `MINIMAX_CN_API_KEY` or `MINIMAX_API_KEY` | `minimax-cn` |
156| Fireworks | `FIREWORKS_API_KEY` | `fireworks` |
157| Vercel AI Gateway | `AI_GATEWAY_API_KEY` | `vercel-ai-gateway` |
158| OpenCode Zen | `OPENCODE_API_KEY` | `opencode` |
159| OpenCode Go | `OPENCODE_API_KEY` | `opencode-go` |
160| GitHub Copilot token | `COPILOT_GITHUB_TOKEN` or `GITHUB_COPILOT_TOKEN` | `github-copilot` |
161| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` | `cloudflare-workers-ai` |
162| Cloudflare AI Gateway | `CLOUDFLARE_API_KEY` | `cloudflare-ai-gateway` |
163| Azure OpenAI Responses | `AZURE_OPENAI_API_KEY` | `azure-openai-responses` |
164
165Example:
166
167```bash
168export OPENROUTER_API_KEY=...
169zut --provider openrouter
170```
171
172### Ollama local vs Cloud
173
174The `ollama` provider serves both local and Cloud models over the OpenAI-compatible `/v1` API. Secrets are sent only as `Authorization: Bearer` headers, never in URLs or logs.
175
176- No key: `http://localhost:11434`, no auth (local models).
177- Real key (`--api-key`, `OLLAMA_API_KEY`, or `/login` → Ollama) with no explicit endpoint: `https://ollama.com/v1` (Cloud models).
178- `--base-url` always wins; a `baseUrl` in `models.json` (model- or provider-level) wins over the automatic default.
179
180### OpenCode Go model catalog
181
182OpenCode Go is synchronized instead of using a hand-maintained model list. When
183`OPENCODE_API_KEY` or a configured OpenCode Go credential is available, zut
184reads the model IDs published by
185[`https://opencode.ai/zen/go/v1/models`](https://opencode.ai/zen/go/v1/models)
186and enriches them with names, limits, prices, reasoning options, and adapter
187metadata from the [Models.dev JSON API](https://models.dev/api.json), which
188powers the [OpenCode Go provider page](https://models.dev/providers/opencode-go/).
189The published list is authoritative for the catalog and picker, but it is not
190an entitlement check: consent, regional eligibility, quota, and account policy
191can still reject an inference request.
192
193Models.dev adapter metadata selects the wire protocol for each model:
194`@ai-sdk/openai` uses OpenAI Responses, `@ai-sdk/openai-compatible` uses Chat
195Completions, and `@ai-sdk/anthropic` uses Anthropic Messages. If that metadata
196is unavailable, zut keeps the published model ID and uses only its narrow
197bootstrap fallback; protocol compatibility cannot be guaranteed until the
198metadata is available. An explicit unsupported adapter fails with an
199unsupported-protocol error instead of silently choosing Chat Completions.
200Explicit and discovered base URLs remain the credential-scoped gateway
201endpoint; public metadata does not redirect requests to another host.
202
203The merged catalog is cached in `$ZUT_HOME/models-cache.json` for 24 hours;
204OpenCode Go entries are scoped to both the credential fingerprint and endpoint.
205`--list-models` waits for a refresh when the cache is stale, while SDK runtimes
206load matching cached metadata before resolving. Synchronous refreshes resolve
207configured `api_key_command` credentials; unsolicited startup refreshes leave
208those commands untouched.
209
210## Fast mode
211
212Use `/fast` or `/settings` to enable **fast mode** for subsequent model calls.
213The setting is off by default and is stored as `fast_mode` in
214`$ZUT_HOME/config.json`.
215
216Fast mode currently requests OpenAI's Fast service tier for OpenAI Chat,
217OpenAI Responses, and OpenAI Codex requests. Enabling it while using another
218provider currently returns an error instead of silently changing that provider's
219request. Subagent children inherit the setting from their parent unless their
220profile sets `fastMode` explicitly.
221
222## GPT-6 Astra
223
224`gpt-6-astra` is available through `openai` (Responses API with an API key)
225and `openai-codex` (ChatGPT subscription). Both catalog entries deliberately
226use a **500,000-token working context budget**, rather than the larger advertised
227maximum, and retain a 128,000-token maximum output. Native reasoning efforts
228`low`, `medium`, `high`, `xhigh`, and `max` are supported. Selecting reasoning off
229omits the effort parameter; it does not disable Astra's reasoning.
230
231Standard API rates per million tokens are $10 input, $50 output, $1 cache reads,
232and $12.50 cache writes. Above **272,000 total input tokens**, including cache
233reads and writes, rates for the entire request become $20 input, $75 output,
234$2 cache reads, and $25 cache writes. The 500k working budget does not avoid
235this pricing tier. Codex catalog dollar estimates use these API-equivalent
236rates, not subscription billing or quota measurements.
237
238Sources: [Astra model specifications](https://developers.openai.com/api/docs/models/gpt-6-astra)
239and [prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching).
240
241## GPT-6.1 Sol
242
243`gpt-6.1-sol` is available through `openai` (Responses API) and `openai-codex`
244with the same 500,000-token working budget, 128,000-token maximum output, and
245native reasoning efforts `low` through `max` as Astra.
246
247Standard API rates per million tokens are $2 input, $10 output, $0.10 cache
248reads, and $2.50 cache writes. Above 272,000 total input tokens, the whole
249request is billed at $4 input, $15 output, $0.20 cache reads, and $5 cache
250writes.
251
252Source: [GPT-6.1 Sol model specifications](https://developers.openai.com/api/docs/models/gpt-6.1-sol).
253
254## Session identity and caching
255
256zut supplies every provider request with a root cache identity, a
257conversation-thread identity, and a per-prompt turn ID. They are opaque local
258correlations; adapters translate them only when their exact provider, model,
259and endpoint declaration supports it. Resident children share the root cache
260identity but retain distinct thread identities. The OpenAI/Codex Responses
261route uses the root identity as `prompt_cache_key`; declared Codex routes also
262receive their documented session and thread routing headers. OpenCode Go
263requests identify zut with `x-opencode-client` and `User-Agent`, use the stable
264thread (or root cache) ID as `x-opencode-session`, and use the turn ID as
265`x-opencode-request`. A generic OpenAI-compatible endpoint never receives
266these extensions merely because it accepts an OpenAI-shaped request. JSON
267event mode also reports sanitized cache diagnostics (`eligible`, `mode`,
268`transport`, and `continuation`). `eligible`
269is a conservative local prompt-length estimate; OpenAI performs the authoritative
270tokenization and cache decision. These records never include prompts, durable
271IDs, request bodies, or credentials.
272
273When a provider reports cache detail, zut displays cache-read hit rate as
274`cache reads / cache-reporting prompt input`. A shown `0%` is a confirmed
275cache miss; no percentage means the provider or historical session did not
276report cache detail. Cache writes remain separate from that hit rate. Provider
277retention and eviction, request routing, and OpenAI's 1,024-token eligibility
278minimum can still produce a miss even when the stable prefix is unchanged.
279
280On the official public OpenAI Responses endpoint, GPT-5.6 and GPT-6 Astra
281requests include an explicit cache breakpoint after the stable instruction/tool
282prefix while retaining implicit caching for conversation history. The Codex
283subscription route and custom Responses endpoints do not receive these explicit
284cache controls. Changing request-level reasoning effort can still disrupt prefix
285reuse: zut does not yet send cache-preserving `configuration_update` items.
286
287### OpenAI Responses WebSocket mode
288
289For the exact public endpoint `https://api.openai.com/v1/responses`, zut uses
290the Responses API WebSocket mode for the OpenAI Responses route. It keeps one
291connection per conversation thread, sends one `response.create` at a time,
292and continues the most recent completed response with `previous_response_id`
293only when the complete response-context configuration and prior input prefix
294match exactly, including the completed assistant output that precedes a new
295suffix. Compatible continuations send only new tool or user input.
296The request uses `store:false`; its only provider-specific handshake header is
297`Authorization: Bearer ...`, never the
298ChatGPT/Codex account, routing, or session headers.
299
300The capability is deliberately not inherited by `--base-url`, compatible
301providers, Azure, or the ChatGPT Codex subscription endpoint. Those clients
302continue to use their declared HTTP/SSE contracts. A request without a logical
303session ID, a failed WebSocket setup,
304an injected nonstandard RoundTripper, or a configured proxy falls back to
305HTTP/SSE with the same logical session/cache identity. Cancellation closes
306only the active session socket; a normally completed turn keeps it warm for
307the next continuation.
308
309### Responses explicit continuation
310
311When a Responses completion carries an explicit `end_turn:false`, zut treats the finished inference as a continuation (`turn_end` with `stop:"continue"`) and issues another model call in the same turn, reusing its request context, guards, usage accounting, and queued-input boundary. Tool calls still take precedence over the signal, and errors or cancellation never become continuations. A missing, null, or true `end_turn` keeps the existing behavior and ends the turn.
312
313### Responses reasoning replay
314
315You can switch models or providers within the same session. Assistant text and tool exchanges remain available; encrypted reasoning and thought signatures from a different provider or model are omitted from outgoing requests without editing session history. Switching back allows the matching model to replay its own metadata.
316
317Responses requests run with `store:false`, so only reasoning blocks with an encrypted payload and a valid Responses item ID (when present) are replayed. Session loading recovers message provenance from the provider/model metadata recorded alongside turns. Older compacted messages without provenance retain best-effort replay; malformed reasoning IDs are omitted.
318
319## Local llama.cpp router
320
321The `llama.cpp` provider connects to a multi-model router and is separate from the `ollama` provider. Ollama normally uses `http://localhost:11434`; entering that URL for llama.cpp management produces a 404 because Ollama does not implement the router endpoints.
322
323Install a current llama.cpp build. On macOS:
324
325```bash
326brew install llama.cpp
327# use `brew upgrade llama.cpp` for an existing installation
328```
329
330Start `llama-server` without `--model`, `-m`, or `-hf`. Those flags select one model and disable the router behavior zut expects.
331
332```bash
333mkdir -p ~/llama-models
334
335llama-server \
336  --models-dir ~/llama-models \
337  --no-models-autoload \
338  --jinja \
339  --host 127.0.0.1 \
340  --port 8080 \
341  -ngl 999 \
342  -c 32768
343```
344
345This configuration discovers GGUF files below `~/llama-models`, leaves model loading under explicit user control, enables chat templates, requests GPU offload, and sets a 32K context. Tune the GPU layers and context size for the available memory.
346
347Check the management API before opening zut:
348
349```bash
350curl http://127.0.0.1:8080/health
351curl http://127.0.0.1:8080/models
352```
353
354`/models` must return JSON with a `data` array. A 404 indicates the wrong URL, an older server build, or single-model mode.
355
356To save the connection, run `/login`, choose `api key`, and select `llama.cpp`. Enter `http://127.0.0.1:8080`, without `/v1`, followed by an optional bearer key. zut validates the URL and stores both values in `$ZUT_HOME/auth.json`. Environment variables override the saved connection:
357
358```bash
359export LLAMA_BASE_URL=http://127.0.0.1:8080
360export LLAMA_API_KEY=optional-secret
361```
362
363If a key is configured, start the router with the same `--api-key` value. A server bound only to `127.0.0.1` normally does not need authentication.
364
365After login, `/llama` becomes visible. It shows model state, searches Hugging Face for GGUF repositories, offers available quantizations, reports download and load progress, and explicitly loads or unloads models. Press `d` on a router-downloaded cache model to ask the router to remove it after confirmation. Models discovered through `--models-dir` or a preset are not removable through the router API and must be deleted from their configured source. Some llama.cpp versions retain shared Hugging Face repository artifacts such as `mmproj` files after removing the selected GGUF; remove that repository's cache directory manually if those artifacts are no longer needed. `HF_TOKEN` raises Hugging Face API limits and may be required for repository metadata. For gated files, approve access first and provide the authorized token to the `llama-server` process because the router performs the download.
366
367Opening `/model` refreshes the router and displays its loaded models under provider `llama.cpp`. Unloaded models cannot handle inference and therefore remain exclusive to `/llama` until loaded. Inference is sent to the router's derived `/v1` endpoint.
368
369Ollama downloads are stored in Ollama's internal layout and do not automatically become llama.cpp models. Obtain a GGUF copy through `/llama` or place GGUF files under `~/llama-models`, then restart the router after adding files manually.
370
371## Cloud providers
372
373### Amazon Bedrock
374
375Bedrock is configured with AWS credentials, not a generic zut API-key entry.
376Use one of these credential sources:
377
378```bash
379# AWS profile
380export AWS_PROFILE=your-profile
381
382# IAM access keys
383export AWS_ACCESS_KEY_ID=AKIA...
384export AWS_SECRET_ACCESS_KEY=...
385export AWS_SESSION_TOKEN=... # only for temporary credentials
386
387# Bedrock API key bearer token
388export AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-...
389
390# Region
391export AWS_REGION=us-east-1
392```
393
394ECS task roles, IRSA, and other AWS SDK credential-chain sources are also
395supported.
396
397Example:
398
399```bash
400AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-... AWS_REGION=us-east-1 \
401  zut --provider amazon-bedrock --model anthropic.claude-sonnet-4-5-20250929-v1:0
402```
403
404Some Bedrock models require regional inference-profile IDs for on-demand
405throughput, such as `us.` or `eu.` prefixed model IDs. zut rewrites known
406families automatically where possible. Claude Opus 5 uses
407`global.anthropic.claude-opus-5`; zut maps the bare foundation-model ID to that
408global profile and supports its adaptive reasoning levels. Explicit profile IDs
409and ARNs are left unchanged.
410
411### Google Vertex AI
412
413Vertex can use a Google API key when available:
414
415```bash
416export GOOGLE_CLOUD_API_KEY=...
417zut --provider google-vertex
418```
419
420For service-account or application-default credentials, set the standard
421Google environment variables used by your deployment.
422
423### Cloudflare AI Gateway
424
425Cloudflare AI Gateway needs a Cloudflare token plus account and gateway IDs:
426
427```bash
428export CLOUDFLARE_API_KEY=...
429export CLOUDFLARE_ACCOUNT_ID=...
430export CLOUDFLARE_GATEWAY_ID=...
431zut --provider cloudflare-ai-gateway
432```
433
434### Cloudflare Workers AI
435
436Workers AI needs a Cloudflare token and account ID:
437
438```bash
439export CLOUDFLARE_API_KEY=...
440export CLOUDFLARE_ACCOUNT_ID=...
441zut --provider cloudflare-workers-ai
442```
443
444### Azure OpenAI Responses
445
446```bash
447export AZURE_OPENAI_API_KEY=...
448export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
449export AZURE_OPENAI_API_VERSION=v1 # optional, v1 is the default
450zut --provider azure-openai-responses
451```
452
453The provider uses Azure's Responses API. If deployment names differ from zut
454model IDs, map them without changing the catalog:
455
456```bash
457export AZURE_OPENAI_DEPLOYMENT_NAME_MAP='gpt-5.6-luna=luna-preview,gpt-5.6-sol=sol-preview'
458```
459
460## Auth file
461
462Credentials are stored in `$ZUT_HOME/auth.json` with user-only permissions
463when zut creates the file.
464
465Example:
466
467```json
468{
469  "anthropic": { "api_key": "sk-ant-..." },
470  "openai": { "api_key": "sk-..." },
471  "google": { "api_key": "..." },
472  "additional_api_key_creds": {
473    "openrouter": { "api_key": "..." },
474    "mistral": { "api_key": "..." }
475  }
476}
477```
478
479The top-level keys are used for providers with dedicated credential fields.
480Other API-key providers are stored under `additional_api_key_creds`. Prefer
481`/login` so zut writes the correct schema.
482
483## Custom providers and models
484
485Use `$ZUT_HOME/models.json` for private models, deployment aliases, local
486servers, or OpenAI-compatible gateways that are not in the built-in catalog.
487User entries override built-in entries with the same provider and model ID, and
488adding a `models.json` no longer hides the built-in catalog: your entries are
489merged on top of the baked-in and live-discovered models.
490
491A top-level provider key that is not a built-in id defines a custom provider.
492Give it a provider-level `baseUrl` and an `api` wire format (`openai` for
493OpenAI-compatible Chat Completions, the default, `openai-responses` for the
494OpenAI Responses API, or `anthropic` for the Anthropic Messages API). A
495model-level `baseUrl` overrides the provider-level one for that model, and an
496unknown `api` value falls back to `openai` with a warning.
497
498```json
499{
500  "providers": {
501    "my-company": {
502      "baseUrl": "https://llm.mycompany.com/v1",
503      "api": "openai",
504      "models": [
505        { "id": "company-llm-v2", "name": "Company LLM v2" }
506      ]
507    }
508  }
509}
510```
511
512Custom providers are first-class: they appear in `--list-models`, `/model`, and
513`/login`. `models.json` never stores secrets. For authenticated endpoints, supply
514the key through `/login`, `--api-key`, or a derived environment variable in upper
515snake case, so `my-company` reads `MY_COMPANY_API_KEY`. Custom endpoints with a
516configured base URL can also run without a key. Set `"auth": "none"` on a custom
517provider to ignore inherited, CLI, environment, and stored credentials, including
518command-backed keys, for inference and discovery. Use this only with trusted
519keyless endpoints. Built-in providers retain their own authentication rules;
520setting `auth: "none"` on a built-in provider is ignored with a warning.
521
522Custom provider keys are accepted and stored without a verification probe,
523because many self-hosted gateways do not expose a model-list endpoint; an invalid
524key surfaces on the first model call.
525
526To retrieve this custom provider's key from a password manager, add a matching
527entry to `$ZUT_HOME/auth.json`:
528
529```json
530{
531  "additional_api_key_creds": {
532    "my-company": {
533      "api_key_command": {
534        "program": "op",
535        "args": ["read", "op://Work/OpenAI/credential"],
536        "timeout_ms": 120000
537      }
538    }
539  }
540}
541```
542
543The provider IDs in `models.json` and `auth.json` must match. Then select the
544custom model directly:
545
546```sh
547zut --provider my-company --model company-llm-v2
548```
549
550### Custom model discovery
551
552Set `"discover": true` at the custom provider level to list models from its
553OpenAI-style `/models` endpoint. Discovery is off by default and requires a
554provider-level `baseUrl`; static model entries override discovered metadata.
555For a local keyless server:
556
557```json
558{
559  "providers": {
560    "local-gateway": {
561      "baseUrl": "http://127.0.0.1:48000/v1",
562      "api": "openai",
563      "auth": "none",
564      "discover": true,
565      "models": [
566        { "id": "local-model", "contextWindow": 65536, "maxTokens": 8192 }
567      ]
568    }
569  }
570}
571```
572
573Zut requests `<baseUrl>/models`, or `<baseUrl>/v1/models` when the base has no
574version suffix, with a three-second timeout and an eight-MiB response limit.
575The response must contain `{"data": [{"id": "model-id"}]}`. Discovery trusts IDs
576only: default metadata is 32,768 context tokens, 4,096 output tokens, zero pricing,
577and no reasoning controls. IDs containing `embed` are skipped. Configure explicit
578model metadata to raise limits or enable reasoning. Discovery always verifies
579TLS certificates; `--insecure` and the saved insecure setting apply to inference,
580not discovery. For self-signed servers, install a trusted certificate chain.
581
582Discovery runs in the background at startup and when `/model` opens. Launching
583with an unknown model from a discovery-enabled provider probes only that
584selected provider, without touching other providers' endpoints or key commands.
585`/model` refreshes configured llama.cpp and custom catalogs with independent three-second
586timeouts and still opens the picker if a refresh fails, showing the error.
587Lists stay in memory, separate from the cloud model cache. Failed refreshes
588retain the previous provider snapshot. Background discovery never executes an
589`api_key_command` and skips a provider whose only key is command-backed; an
590explicit refresh may execute that command. The refresh budget also bounds the
591key command; a slow or interactive password-manager prompt can time out.
592Environment or stored keys are used when available unless `auth` is `none`; no
593key is required for a keyless server. If an explicitly selected custom model is
594unknown and discovery fails, inference can still attempt it with the existing
595128,000-context / 16,384-output fallback. Configure static model limits to avoid
596relying on either fallback.
597
598## Credential resolution
599
600For each request, zut checks credentials in this order:
601
6021. Explicit CLI key, such as `--api-key`.
6032. Provider-specific environment variables (including derived custom-provider
604   variables such as `MY_COMPANY_API_KEY`).
6053. `$ZUT_HOME/auth.json`, including custom provider keys saved by `/login`.
606
607`models.json` itself never stores credentials; it only describes models and
608their endpoints.
609
610Bedrock then uses the AWS SDK credential chain for the actual request.