Give discovered Bedrock models real output limits
Live discovery returns model IDs only. IDs without an exact catalog entry, such as apac. or us-gov. inference profiles, were merged with MaxOutput 0, so the Bedrock client fell back to maxTokens 4096 and long responses stopped with max_tokens. Discovered profile IDs now inherit limits, reasoning flags, and pricing from their base model's catalog entry. Claude 4+ models absent from the catalog get a 64k output / 200k context default (Haiku 4.5's limits, the smallest among current Claude models). Legacy Claude and other unknown models keep the existing fallback. Also recognize the jp. geo prefix, which the catalog already uses.
- commit
- 8cb588bf0844a8febc1c198da22fe71e6e0841a9
- author
- Raymond Gasper · · 2026-09-29 13:33 UTC
- parent
- 2ff82fa
- signature
- unsigned
4 files · +230 -8
M README.md +1 -1
| @@ -798,7 +798,7 @@ If both are present, the bearer token wins for inference, while live model disco | |||
| 798 | 798 | Unlike most providers, Bedrock model **pricing, context windows, and capability flags are hand-maintained** in zot's baked-in catalog. AWS does not expose usable pricing programmatically for current models — its Price List API returns only stale, input-only rates for a few legacy models — so the catalog is authoritative for cost and metadata. This means: | |
| 799 | 799 | ||
| 800 | 800 | - **Bearer-token-only users** still get the full curated Bedrock model list with correct pricing from the catalog. No AWS API listing is attempted. | |
| 801 | - | - **Users with SigV4 credentials** additionally get **live model discovery**, even if inference uses a bearer token: on startup, zot lists the foundation models and cross-region inference profiles your account can actually use (via `ListFoundationModels` and `ListInferenceProfiles`) and merges them into the catalog. New Bedrock models therefore appear in `/model` without waiting for a zot release. Discovered IDs already known to the catalog keep their curated pricing; genuinely new IDs show up with placeholder pricing until the catalog is updated. | |
| 801 | + | - **Users with SigV4 credentials** additionally get **live model discovery**, even if inference uses a bearer token: on startup, zot lists the foundation models and cross-region inference profiles your account can actually use (via `ListFoundationModels` and `ListInferenceProfiles`) and merges them into the catalog. New Bedrock models therefore appear in `/model` without waiting for a zot release. Discovered IDs already known to the catalog keep their curated metadata. An inference profile the catalog doesn't list under that exact ID (for example `apac.` or `us-gov.` variants) inherits the limits, reasoning flags, and pricing of its base model. Claude models that aren't in the catalog at all get a 64k output / 200k context default; other unknown IDs show up with placeholder pricing and default limits until the catalog is updated. | |
| 802 | 802 | ||
| 803 | 803 | Live discovery requires SigV4 because the listing APIs live on the Bedrock control plane, which rejects the runtime bearer token. It runs in the background at startup and is cached for 6h like other providers, so `/model` reflects the latest list without any manual refresh. If discovery fails (no SigV4 creds, expired SSO login, missing IAM permissions), zot silently falls back to the catalog. Discovery needs `bedrock:ListFoundationModels` and `bedrock:ListInferenceProfiles`. | |
| 804 | 804 | ||
| @@ -798,7 +798,7 @@ If both are present, the bearer token wins for inference, while live model disco | |||||
| 798 | Unlike most providers, Bedrock model **pricing, context windows, and capability flags are hand-maintained** in zot's baked-in catalog. AWS does not expose usable pricing programmatically for current models — its Price List API returns only stale, input-only rates for a few legacy models — so the catalog is authoritative for cost and metadata. This means: | 798 | Unlike most providers, Bedrock model **pricing, context windows, and capability flags are hand-maintained** in zot's baked-in catalog. AWS does not expose usable pricing programmatically for current models — its Price List API returns only stale, input-only rates for a few legacy models — so the catalog is authoritative for cost and metadata. This means: | ||
| 799 | 799 | ||||
| 800 | - **Bearer-token-only users** still get the full curated Bedrock model list with correct pricing from the catalog. No AWS API listing is attempted. | 800 | - **Bearer-token-only users** still get the full curated Bedrock model list with correct pricing from the catalog. No AWS API listing is attempted. | ||
| 801 | - | - **Users with SigV4 credentials** additionally get **live model discovery**, even if inference uses a bearer token: on startup, zot lists the foundation models and cross-region inference profiles your account can actually use (via `ListFoundationModels` and `ListInferenceProfiles`) and merges them into the catalog. New Bedrock models therefore appear in `/model` without waiting for a zot release. Discovered IDs already known to the catalog keep their curated pricing; genuinely new IDs show up with placeholder pricing until the catalog is updated. | 801 | + | - **Users with SigV4 credentials** additionally get **live model discovery**, even if inference uses a bearer token: on startup, zot lists the foundation models and cross-region inference profiles your account can actually use (via `ListFoundationModels` and `ListInferenceProfiles`) and merges them into the catalog. New Bedrock models therefore appear in `/model` without waiting for a zot release. Discovered IDs already known to the catalog keep their curated metadata. An inference profile the catalog doesn't list under that exact ID (for example `apac.` or `us-gov.` variants) inherits the limits, reasoning flags, and pricing of its base model. Claude models that aren't in the catalog at all get a 64k output / 200k context default; other unknown IDs show up with placeholder pricing and default limits until the catalog is updated. |
| 802 | 802 | ||||
| 803 | Live discovery requires SigV4 because the listing APIs live on the Bedrock control plane, which rejects the runtime bearer token. It runs in the background at startup and is cached for 6h like other providers, so `/model` reflects the latest list without any manual refresh. If discovery fails (no SigV4 creds, expired SSO login, missing IAM permissions), zot silently falls back to the catalog. Discovery needs `bedrock:ListFoundationModels` and `bedrock:ListInferenceProfiles`. | 803 | Live discovery requires SigV4 because the listing APIs live on the Bedrock control plane, which rejects the runtime bearer token. It runs in the background at startup and is cached for 6h like other providers, so `/model` reflects the latest list without any manual refresh. If discovery fails (no SigV4 creds, expired SSO login, missing IAM permissions), zot silently falls back to the catalog. Discovery needs `bedrock:ListFoundationModels` and `bedrock:ListInferenceProfiles`. | ||
| 804 | 804 | ||||
M packages/provider/amazon_bedrock.go +1 -1
| @@ -600,7 +600,7 @@ func resolveBedrockInferenceProfileID(modelID, region string) string { | |||
| 600 | 600 | // bedrockGeoPrefixes are the cross-region inference-profile geo | |
| 601 | 601 | // prefixes Bedrock uses. A model ID that starts with one of these | |
| 602 | 602 | // (followed by a dot) is already a profile reference. | |
| 603 | - | var bedrockGeoPrefixes = []string{"us-gov", "us", "eu", "apac", "ap", "global", "au"} | |
| 603 | + | var bedrockGeoPrefixes = []string{"us-gov", "us", "eu", "apac", "ap", "global", "au", "jp"} | |
| 604 | 604 | ||
| 605 | 605 | func bedrockHasGeoPrefix(modelID string) bool { | |
| 606 | 606 | for _, p := range bedrockGeoPrefixes { | |
| @@ -600,7 +600,7 @@ func resolveBedrockInferenceProfileID(modelID, region string) string { | |||||
| 600 | // bedrockGeoPrefixes are the cross-region inference-profile geo | 600 | // bedrockGeoPrefixes are the cross-region inference-profile geo | ||
| 601 | // prefixes Bedrock uses. A model ID that starts with one of these | 601 | // prefixes Bedrock uses. A model ID that starts with one of these | ||
| 602 | // (followed by a dot) is already a profile reference. | 602 | // (followed by a dot) is already a profile reference. | ||
| 603 | - | var bedrockGeoPrefixes = []string{"us-gov", "us", "eu", "apac", "ap", "global", "au"} | 603 | + | var bedrockGeoPrefixes = []string{"us-gov", "us", "eu", "apac", "ap", "global", "au", "jp"} |
| 604 | 604 | ||||
| 605 | func bedrockHasGeoPrefix(modelID string) bool { | 605 | func bedrockHasGeoPrefix(modelID string) bool { | ||
| 606 | for _, p := range bedrockGeoPrefixes { | 606 | for _, p := range bedrockGeoPrefixes { | ||
M packages/provider/discover_bedrock.go +125 -6
| @@ -34,6 +34,7 @@ import ( | |||
| 34 | 34 | "encoding/json" | |
| 35 | 35 | "fmt" | |
| 36 | 36 | "io" | |
| 37 | + | "maps" | |
| 37 | 38 | "net/http" | |
| 38 | 39 | "net/url" | |
| 39 | 40 | "strings" | |
| @@ -80,16 +81,134 @@ func DiscoverBedrock(ctx context.Context, region string) ([]Model, error) { | |||
| 80 | 81 | ||
| 81 | 82 | out := make([]Model, 0, len(ids)) | |
| 82 | 83 | for _, id := range ids { | |
| 83 | - | out = append(out, Model{ | |
| 84 | - | Provider: "amazon-bedrock", | |
| 85 | - | ID: id, | |
| 86 | - | Source: "live", | |
| 87 | - | BaseURL: "https://bedrock-runtime." + region + ".amazonaws.com", | |
| 88 | - | }) | |
| 84 | + | out = append(out, bedrockDiscoveredModel(id, region)) | |
| 89 | 85 | } | |
| 90 | 86 | return out, nil | |
| 91 | 87 | } | |
| 92 | 88 | ||
| 89 | + | // bedrockAnthropicDefaultMaxOutput is the output budget assumed for a | |
| 90 | + | // discovered Claude model with no catalog entry. It matches Claude Haiku | |
| 91 | + | // 4.5, the smallest output limit among current Claude models, so it is | |
| 92 | + | // safe for any Claude 4+ model and far above the client's 4096 fallback. | |
| 93 | + | const bedrockAnthropicDefaultMaxOutput = 64000 | |
| 94 | + | ||
| 95 | + | // bedrockAnthropicDefaultContextWindow is the context window assumed | |
| 96 | + | // for an uncatalogued Claude model, the smallest among current models. | |
| 97 | + | const bedrockAnthropicDefaultContextWindow = 200000 | |
| 98 | + | ||
| 99 | + | // bedrockDiscoveredModel builds the Model for a discovered ID. The | |
| 100 | + | // control plane returns IDs only, so limits and capabilities come from | |
| 101 | + | // the static catalog. MergeCatalog already uses an exact catalog match; | |
| 102 | + | // this handles IDs without one, most often a geo-prefixed inference | |
| 103 | + | // profile (apac., us-gov., ...) whose base model is catalogued. Without | |
| 104 | + | // this the model would carry MaxOutput 0 and the Bedrock client would | |
| 105 | + | // cap every response at its 4096-token fallback. | |
| 106 | + | func bedrockDiscoveredModel(id, region string) Model { | |
| 107 | + | m := Model{ | |
| 108 | + | Provider: "amazon-bedrock", | |
| 109 | + | ID: id, | |
| 110 | + | Source: "live", | |
| 111 | + | BaseURL: "https://bedrock-runtime." + region + ".amazonaws.com", | |
| 112 | + | } | |
| 113 | + | if bedrockInCatalog(id) { | |
| 114 | + | // MergeCatalog keeps the exact catalog entry's metadata. | |
| 115 | + | return m | |
| 116 | + | } | |
| 117 | + | prefix, base := bedrockSplitGeoPrefix(id) | |
| 118 | + | if c, ok := bedrockCatalogBase(base); ok { | |
| 119 | + | m.ContextWindow = c.ContextWindow | |
| 120 | + | m.MaxOutput = c.MaxOutput | |
| 121 | + | m.Reasoning = c.Reasoning | |
| 122 | + | m.ReasoningLevelMap = maps.Clone(c.ReasoningLevelMap) | |
| 123 | + | m.AdaptiveThinking = c.AdaptiveThinking | |
| 124 | + | m.PriceInput = c.PriceInput | |
| 125 | + | m.PriceOutput = c.PriceOutput | |
| 126 | + | m.PriceCacheRead = c.PriceCacheRead | |
| 127 | + | m.PriceCacheWrite = c.PriceCacheWrite | |
| 128 | + | m.PriceTierInputTokens = c.PriceTierInputTokens | |
| 129 | + | m.PriceInputAbove = c.PriceInputAbove | |
| 130 | + | m.PriceOutputAbove = c.PriceOutputAbove | |
| 131 | + | m.PriceCacheReadAbove = c.PriceCacheReadAbove | |
| 132 | + | m.PriceCacheWriteAbove = c.PriceCacheWriteAbove | |
| 133 | + | if c.DisplayName != "" { | |
| 134 | + | m.DisplayName = c.DisplayName | |
| 135 | + | if cp, _ := bedrockSplitGeoPrefix(c.ID); cp != "" { | |
| 136 | + | m.DisplayName = strings.TrimSuffix(m.DisplayName, " ("+strings.ToUpper(cp)+")") | |
| 137 | + | } | |
| 138 | + | if prefix != "" { | |
| 139 | + | m.DisplayName += " (" + strings.ToUpper(prefix) + ")" | |
| 140 | + | } | |
| 141 | + | } | |
| 142 | + | return m | |
| 143 | + | } | |
| 144 | + | if bedrockIsModernClaude(base) { | |
| 145 | + | m.ContextWindow = bedrockAnthropicDefaultContextWindow | |
| 146 | + | m.MaxOutput = bedrockAnthropicDefaultMaxOutput | |
| 147 | + | } | |
| 148 | + | return m | |
| 149 | + | } | |
| 150 | + | ||
| 151 | + | // bedrockSplitGeoPrefix separates a cross-region inference-profile geo | |
| 152 | + | // prefix from the foundation-model ID. prefix is "" when there is none. | |
| 153 | + | func bedrockSplitGeoPrefix(id string) (prefix, base string) { | |
| 154 | + | for _, p := range bedrockGeoPrefixes { | |
| 155 | + | if strings.HasPrefix(id, p+".") { | |
| 156 | + | return p, id[len(p)+1:] | |
| 157 | + | } | |
| 158 | + | } | |
| 159 | + | return "", id | |
| 160 | + | } | |
| 161 | + | ||
| 162 | + | func bedrockInCatalog(id string) bool { | |
| 163 | + | for _, c := range Catalog { | |
| 164 | + | if c.Provider == "amazon-bedrock" && c.ID == id { | |
| 165 | + | return true | |
| 166 | + | } | |
| 167 | + | } | |
| 168 | + | return false | |
| 169 | + | } | |
| 170 | + | ||
| 171 | + | // bedrockCatalogBase finds catalog metadata for a foundation-model ID: | |
| 172 | + | // the bare entry if present, otherwise any geo-prefixed variant of it. | |
| 173 | + | func bedrockCatalogBase(base string) (Model, bool) { | |
| 174 | + | var variant Model | |
| 175 | + | found := false | |
| 176 | + | for _, c := range Catalog { | |
| 177 | + | if c.Provider != "amazon-bedrock" { | |
| 178 | + | continue | |
| 179 | + | } | |
| 180 | + | if c.ID == base { | |
| 181 | + | return c, true | |
| 182 | + | } | |
| 183 | + | if !found { | |
| 184 | + | if _, b := bedrockSplitGeoPrefix(c.ID); b == base { | |
| 185 | + | variant, found = c, true | |
| 186 | + | } | |
| 187 | + | } | |
| 188 | + | } | |
| 189 | + | return variant, found | |
| 190 | + | } | |
| 191 | + | ||
| 192 | + | // bedrockIsModernClaude reports whether a foundation-model ID is a Claude | |
| 193 | + | // model whose output limit is at least bedrockAnthropicDefaultMaxOutput. | |
| 194 | + | // Legacy Claude 2 / Instant / 3 / 3.5 models cap output at 4096-8192 and | |
| 195 | + | // would reject a 64000-token request, so they keep the client fallback. | |
| 196 | + | func bedrockIsModernClaude(base string) bool { | |
| 197 | + | if !strings.HasPrefix(base, "anthropic.claude-") { | |
| 198 | + | return false | |
| 199 | + | } | |
| 200 | + | switch { | |
| 201 | + | case strings.HasPrefix(base, "anthropic.claude-v"), | |
| 202 | + | strings.HasPrefix(base, "anthropic.claude-instant"), | |
| 203 | + | strings.HasPrefix(base, "anthropic.claude-3-5-"), | |
| 204 | + | strings.HasPrefix(base, "anthropic.claude-3-haiku"), | |
| 205 | + | strings.HasPrefix(base, "anthropic.claude-3-sonnet"), | |
| 206 | + | strings.HasPrefix(base, "anthropic.claude-3-opus"): | |
| 207 | + | return false | |
| 208 | + | } | |
| 209 | + | return true | |
| 210 | + | } | |
| 211 | + | ||
| 93 | 212 | // bedrockListModelIDs enumerates foundation-model IDs and inference- | |
| 94 | 213 | // profile IDs available in the region. Profile IDs are the geo-prefixed | |
| 95 | 214 | // forms zot invokes; both are returned so the picker shows every | |
| @@ -34,6 +34,7 @@ import ( | |||||
| 34 | "encoding/json" | 34 | "encoding/json" | ||
| 35 | "fmt" | 35 | "fmt" | ||
| 36 | "io" | 36 | "io" | ||
| 37 | + | "maps" | |||
| 37 | "net/http" | 38 | "net/http" | ||
| 38 | "net/url" | 39 | "net/url" | ||
| 39 | "strings" | 40 | "strings" | ||
| @@ -80,16 +81,134 @@ func DiscoverBedrock(ctx context.Context, region string) ([]Model, error) { | |||||
| 80 | 81 | ||||
| 81 | out := make([]Model, 0, len(ids)) | 82 | out := make([]Model, 0, len(ids)) | ||
| 82 | for _, id := range ids { | 83 | for _, id := range ids { | ||
| 83 | - | out = append(out, Model{ | 84 | + | out = append(out, bedrockDiscoveredModel(id, region)) |
| 84 | - | Provider: "amazon-bedrock", | |||
| 85 | - | ID: id, | |||
| 86 | - | Source: "live", | |||
| 87 | - | BaseURL: "https://bedrock-runtime." + region + ".amazonaws.com", | |||
| 88 | - | }) | |||
| 89 | } | 85 | } | ||
| 90 | return out, nil | 86 | return out, nil | ||
| 91 | } | 87 | } | ||
| 92 | 88 | ||||
| 89 | + | // bedrockAnthropicDefaultMaxOutput is the output budget assumed for a | |||
| 90 | + | // discovered Claude model with no catalog entry. It matches Claude Haiku | |||
| 91 | + | // 4.5, the smallest output limit among current Claude models, so it is | |||
| 92 | + | // safe for any Claude 4+ model and far above the client's 4096 fallback. | |||
| 93 | + | const bedrockAnthropicDefaultMaxOutput = 64000 | |||
| 94 | + | ||||
| 95 | + | // bedrockAnthropicDefaultContextWindow is the context window assumed | |||
| 96 | + | // for an uncatalogued Claude model, the smallest among current models. | |||
| 97 | + | const bedrockAnthropicDefaultContextWindow = 200000 | |||
| 98 | + | ||||
| 99 | + | // bedrockDiscoveredModel builds the Model for a discovered ID. The | |||
| 100 | + | // control plane returns IDs only, so limits and capabilities come from | |||
| 101 | + | // the static catalog. MergeCatalog already uses an exact catalog match; | |||
| 102 | + | // this handles IDs without one, most often a geo-prefixed inference | |||
| 103 | + | // profile (apac., us-gov., ...) whose base model is catalogued. Without | |||
| 104 | + | // this the model would carry MaxOutput 0 and the Bedrock client would | |||
| 105 | + | // cap every response at its 4096-token fallback. | |||
| 106 | + | func bedrockDiscoveredModel(id, region string) Model { | |||
| 107 | + | m := Model{ | |||
| 108 | + | Provider: "amazon-bedrock", | |||
| 109 | + | ID: id, | |||
| 110 | + | Source: "live", | |||
| 111 | + | BaseURL: "https://bedrock-runtime." + region + ".amazonaws.com", | |||
| 112 | + | } | |||
| 113 | + | if bedrockInCatalog(id) { | |||
| 114 | + | // MergeCatalog keeps the exact catalog entry's metadata. | |||
| 115 | + | return m | |||
| 116 | + | } | |||
| 117 | + | prefix, base := bedrockSplitGeoPrefix(id) | |||
| 118 | + | if c, ok := bedrockCatalogBase(base); ok { | |||
| 119 | + | m.ContextWindow = c.ContextWindow | |||
| 120 | + | m.MaxOutput = c.MaxOutput | |||
| 121 | + | m.Reasoning = c.Reasoning | |||
| 122 | + | m.ReasoningLevelMap = maps.Clone(c.ReasoningLevelMap) | |||
| 123 | + | m.AdaptiveThinking = c.AdaptiveThinking | |||
| 124 | + | m.PriceInput = c.PriceInput | |||
| 125 | + | m.PriceOutput = c.PriceOutput | |||
| 126 | + | m.PriceCacheRead = c.PriceCacheRead | |||
| 127 | + | m.PriceCacheWrite = c.PriceCacheWrite | |||
| 128 | + | m.PriceTierInputTokens = c.PriceTierInputTokens | |||
| 129 | + | m.PriceInputAbove = c.PriceInputAbove | |||
| 130 | + | m.PriceOutputAbove = c.PriceOutputAbove | |||
| 131 | + | m.PriceCacheReadAbove = c.PriceCacheReadAbove | |||
| 132 | + | m.PriceCacheWriteAbove = c.PriceCacheWriteAbove | |||
| 133 | + | if c.DisplayName != "" { | |||
| 134 | + | m.DisplayName = c.DisplayName | |||
| 135 | + | if cp, _ := bedrockSplitGeoPrefix(c.ID); cp != "" { | |||
| 136 | + | m.DisplayName = strings.TrimSuffix(m.DisplayName, " ("+strings.ToUpper(cp)+")") | |||
| 137 | + | } | |||
| 138 | + | if prefix != "" { | |||
| 139 | + | m.DisplayName += " (" + strings.ToUpper(prefix) + ")" | |||
| 140 | + | } | |||
| 141 | + | } | |||
| 142 | + | return m | |||
| 143 | + | } | |||
| 144 | + | if bedrockIsModernClaude(base) { | |||
| 145 | + | m.ContextWindow = bedrockAnthropicDefaultContextWindow | |||
| 146 | + | m.MaxOutput = bedrockAnthropicDefaultMaxOutput | |||
| 147 | + | } | |||
| 148 | + | return m | |||
| 149 | + | } | |||
| 150 | + | ||||
| 151 | + | // bedrockSplitGeoPrefix separates a cross-region inference-profile geo | |||
| 152 | + | // prefix from the foundation-model ID. prefix is "" when there is none. | |||
| 153 | + | func bedrockSplitGeoPrefix(id string) (prefix, base string) { | |||
| 154 | + | for _, p := range bedrockGeoPrefixes { | |||
| 155 | + | if strings.HasPrefix(id, p+".") { | |||
| 156 | + | return p, id[len(p)+1:] | |||
| 157 | + | } | |||
| 158 | + | } | |||
| 159 | + | return "", id | |||
| 160 | + | } | |||
| 161 | + | ||||
| 162 | + | func bedrockInCatalog(id string) bool { | |||
| 163 | + | for _, c := range Catalog { | |||
| 164 | + | if c.Provider == "amazon-bedrock" && c.ID == id { | |||
| 165 | + | return true | |||
| 166 | + | } | |||
| 167 | + | } | |||
| 168 | + | return false | |||
| 169 | + | } | |||
| 170 | + | ||||
| 171 | + | // bedrockCatalogBase finds catalog metadata for a foundation-model ID: | |||
| 172 | + | // the bare entry if present, otherwise any geo-prefixed variant of it. | |||
| 173 | + | func bedrockCatalogBase(base string) (Model, bool) { | |||
| 174 | + | var variant Model | |||
| 175 | + | found := false | |||
| 176 | + | for _, c := range Catalog { | |||
| 177 | + | if c.Provider != "amazon-bedrock" { | |||
| 178 | + | continue | |||
| 179 | + | } | |||
| 180 | + | if c.ID == base { | |||
| 181 | + | return c, true | |||
| 182 | + | } | |||
| 183 | + | if !found { | |||
| 184 | + | if _, b := bedrockSplitGeoPrefix(c.ID); b == base { | |||
| 185 | + | variant, found = c, true | |||
| 186 | + | } | |||
| 187 | + | } | |||
| 188 | + | } | |||
| 189 | + | return variant, found | |||
| 190 | + | } | |||
| 191 | + | ||||
| 192 | + | // bedrockIsModernClaude reports whether a foundation-model ID is a Claude | |||
| 193 | + | // model whose output limit is at least bedrockAnthropicDefaultMaxOutput. | |||
| 194 | + | // Legacy Claude 2 / Instant / 3 / 3.5 models cap output at 4096-8192 and | |||
| 195 | + | // would reject a 64000-token request, so they keep the client fallback. | |||
| 196 | + | func bedrockIsModernClaude(base string) bool { | |||
| 197 | + | if !strings.HasPrefix(base, "anthropic.claude-") { | |||
| 198 | + | return false | |||
| 199 | + | } | |||
| 200 | + | switch { | |||
| 201 | + | case strings.HasPrefix(base, "anthropic.claude-v"), | |||
| 202 | + | strings.HasPrefix(base, "anthropic.claude-instant"), | |||
| 203 | + | strings.HasPrefix(base, "anthropic.claude-3-5-"), | |||
| 204 | + | strings.HasPrefix(base, "anthropic.claude-3-haiku"), | |||
| 205 | + | strings.HasPrefix(base, "anthropic.claude-3-sonnet"), | |||
| 206 | + | strings.HasPrefix(base, "anthropic.claude-3-opus"): | |||
| 207 | + | return false | |||
| 208 | + | } | |||
| 209 | + | return true | |||
| 210 | + | } | |||
| 211 | + | ||||
| 93 | // bedrockListModelIDs enumerates foundation-model IDs and inference- | 212 | // bedrockListModelIDs enumerates foundation-model IDs and inference- | ||
| 94 | // profile IDs available in the region. Profile IDs are the geo-prefixed | 213 | // profile IDs available in the region. Profile IDs are the geo-prefixed | ||
| 95 | // forms zot invokes; both are returned so the picker shows every | 214 | // forms zot invokes; both are returned so the picker shows every | ||
A packages/provider/discover_bedrock_metadata_test.go +103 -0
| @@ -0,0 +1,103 @@ | |||
| 1 | + | package provider | |
| 2 | + | ||
| 3 | + | import "testing" | |
| 4 | + | ||
| 5 | + | // discoveredWireMaxTokens runs a discovered ID through the same path as | |
| 6 | + | // production (discovery metadata -> MergeCatalog -> FindModel -> the | |
| 7 | + | // agent's MaxTokens -> Bedrock buildRequest) and returns the merged model | |
| 8 | + | // plus the maxTokens that would be sent on the wire. | |
| 9 | + | func discoveredWireMaxTokens(t *testing.T, id string) (Model, int) { | |
| 10 | + | t.Helper() | |
| 11 | + | withLiveModels(t, []Model{bedrockDiscoveredModel(id, "us-east-1")}) | |
| 12 | + | m, err := FindModel("amazon-bedrock", id) | |
| 13 | + | if err != nil { | |
| 14 | + | t.Fatalf("FindModel(%q): %v", id, err) | |
| 15 | + | } | |
| 16 | + | c := &bedrockClient{region: "us-east-1"} | |
| 17 | + | req, err := c.buildRequest(Request{Model: id, MaxTokens: m.MaxOutput}) | |
| 18 | + | if err != nil { | |
| 19 | + | t.Fatalf("buildRequest(%q): %v", id, err) | |
| 20 | + | } | |
| 21 | + | return m, req.InferenceConfig.MaxTokens | |
| 22 | + | } | |
| 23 | + | ||
| 24 | + | // Discovered inference-profile IDs with no exact catalog entry used to | |
| 25 | + | // carry MaxOutput 0, so every request fell back to maxTokens 4096 and | |
| 26 | + | // long responses stopped with max_tokens. | |
| 27 | + | func TestBedrockDiscoveredProfileInheritsBaseMetadata(t *testing.T) { | |
| 28 | + | base, err := FindModel("amazon-bedrock", "anthropic.claude-sonnet-4-6") | |
| 29 | + | if err != nil { | |
| 30 | + | t.Fatal(err) | |
| 31 | + | } | |
| 32 | + | for _, id := range []string{"apac.anthropic.claude-sonnet-4-6", "us-gov.anthropic.claude-sonnet-4-6"} { | |
| 33 | + | m, wire := discoveredWireMaxTokens(t, id) | |
| 34 | + | if m.MaxOutput != base.MaxOutput || wire != base.MaxOutput { | |
| 35 | + | t.Errorf("%s: MaxOutput=%d wire=%d, want %d", id, m.MaxOutput, wire, base.MaxOutput) | |
| 36 | + | } | |
| 37 | + | if m.ContextWindow != base.ContextWindow || m.Reasoning != base.Reasoning { | |
| 38 | + | t.Errorf("%s: metadata not inherited: %+v", id, m) | |
| 39 | + | } | |
| 40 | + | if m.PriceOutput != base.PriceOutput { | |
| 41 | + | t.Errorf("%s: PriceOutput=%v, want %v", id, m.PriceOutput, base.PriceOutput) | |
| 42 | + | } | |
| 43 | + | } | |
| 44 | + | ||
| 45 | + | m, _ := discoveredWireMaxTokens(t, "apac.anthropic.claude-sonnet-4-6") | |
| 46 | + | if m.DisplayName != "Claude Sonnet 4.6 (APAC)" { | |
| 47 | + | t.Errorf("DisplayName = %q", m.DisplayName) | |
| 48 | + | } | |
| 49 | + | } | |
| 50 | + | ||
| 51 | + | // A base model catalogued only under geo-prefixed IDs still donates its | |
| 52 | + | // metadata, and the donor's region label is not carried over. | |
| 53 | + | func TestBedrockDiscoveredProfileUsesPrefixedCatalogVariant(t *testing.T) { | |
| 54 | + | const id = "apac.anthropic.claude-opus-5-5" | |
| 55 | + | m, wire := discoveredWireMaxTokens(t, id) | |
| 56 | + | if wire != 128000 || !m.AdaptiveThinking { | |
| 57 | + | t.Errorf("%s: wire=%d adaptive=%v, want 128000 adaptive", id, wire, m.AdaptiveThinking) | |
| 58 | + | } | |
| 59 | + | if m.DisplayName != "Claude Opus 5.5 (APAC)" { | |
| 60 | + | t.Errorf("DisplayName = %q", m.DisplayName) | |
| 61 | + | } | |
| 62 | + | } | |
| 63 | + | ||
| 64 | + | // Claude models absent from the catalog get Haiku 4.5's 64k output, the | |
| 65 | + | // smallest among current Claude models, instead of the 4096 fallback. | |
| 66 | + | func TestBedrockDiscoveredUnknownClaudeUses64kFloor(t *testing.T) { | |
| 67 | + | for _, id := range []string{"anthropic.claude-future-9", "us.anthropic.claude-future-9-v1:0"} { | |
| 68 | + | m, wire := discoveredWireMaxTokens(t, id) | |
| 69 | + | if wire != bedrockAnthropicDefaultMaxOutput || m.ContextWindow != bedrockAnthropicDefaultContextWindow { | |
| 70 | + | t.Errorf("%s: wire=%d ctx=%d", id, wire, m.ContextWindow) | |
| 71 | + | } | |
| 72 | + | } | |
| 73 | + | } | |
| 74 | + | ||
| 75 | + | // Legacy Claude and non-Anthropic models cap output well below 64k and | |
| 76 | + | // would reject a larger request, so they keep the client fallback. | |
| 77 | + | func TestBedrockDiscoveredLegacyAndOtherModelsKeepFallback(t *testing.T) { | |
| 78 | + | for _, id := range []string{ | |
| 79 | + | "anthropic.claude-v2:1", | |
| 80 | + | "anthropic.claude-instant-v1", | |
| 81 | + | "us.anthropic.claude-3-5-sonnet-20241022-v2:0", | |
| 82 | + | "anthropic.claude-3-haiku-20240307-v1:0", | |
| 83 | + | "mistral.unknown-model-v1:0", | |
| 84 | + | } { | |
| 85 | + | m, wire := discoveredWireMaxTokens(t, id) | |
| 86 | + | if m.MaxOutput != 0 || wire != 4096 { | |
| 87 | + | t.Errorf("%s: MaxOutput=%d wire=%d, want 0/4096", id, m.MaxOutput, wire) | |
| 88 | + | } | |
| 89 | + | } | |
| 90 | + | } | |
| 91 | + | ||
| 92 | + | // Exact catalog IDs keep their own catalog metadata through MergeCatalog. | |
| 93 | + | func TestBedrockDiscoveredExactCatalogIDUnchanged(t *testing.T) { | |
| 94 | + | const id = "au.anthropic.claude-sonnet-4-6" | |
| 95 | + | want, err := FindModel("amazon-bedrock", id) | |
| 96 | + | if err != nil { | |
| 97 | + | t.Fatal(err) | |
| 98 | + | } | |
| 99 | + | m, wire := discoveredWireMaxTokens(t, id) | |
| 100 | + | if m.MaxOutput != want.MaxOutput || wire != want.MaxOutput || m.DisplayName != want.DisplayName { | |
| 101 | + | t.Errorf("got MaxOutput=%d wire=%d name=%q, want %d %q", m.MaxOutput, wire, m.DisplayName, want.MaxOutput, want.DisplayName) | |
| 102 | + | } | |
| 103 | + | } | |
| @@ -0,0 +1,103 @@ | |||||
| 1 | + | package provider | |||
| 2 | + | ||||
| 3 | + | import "testing" | |||
| 4 | + | ||||
| 5 | + | // discoveredWireMaxTokens runs a discovered ID through the same path as | |||
| 6 | + | // production (discovery metadata -> MergeCatalog -> FindModel -> the | |||
| 7 | + | // agent's MaxTokens -> Bedrock buildRequest) and returns the merged model | |||
| 8 | + | // plus the maxTokens that would be sent on the wire. | |||
| 9 | + | func discoveredWireMaxTokens(t *testing.T, id string) (Model, int) { | |||
| 10 | + | t.Helper() | |||
| 11 | + | withLiveModels(t, []Model{bedrockDiscoveredModel(id, "us-east-1")}) | |||
| 12 | + | m, err := FindModel("amazon-bedrock", id) | |||
| 13 | + | if err != nil { | |||
| 14 | + | t.Fatalf("FindModel(%q): %v", id, err) | |||
| 15 | + | } | |||
| 16 | + | c := &bedrockClient{region: "us-east-1"} | |||
| 17 | + | req, err := c.buildRequest(Request{Model: id, MaxTokens: m.MaxOutput}) | |||
| 18 | + | if err != nil { | |||
| 19 | + | t.Fatalf("buildRequest(%q): %v", id, err) | |||
| 20 | + | } | |||
| 21 | + | return m, req.InferenceConfig.MaxTokens | |||
| 22 | + | } | |||
| 23 | + | ||||
| 24 | + | // Discovered inference-profile IDs with no exact catalog entry used to | |||
| 25 | + | // carry MaxOutput 0, so every request fell back to maxTokens 4096 and | |||
| 26 | + | // long responses stopped with max_tokens. | |||
| 27 | + | func TestBedrockDiscoveredProfileInheritsBaseMetadata(t *testing.T) { | |||
| 28 | + | base, err := FindModel("amazon-bedrock", "anthropic.claude-sonnet-4-6") | |||
| 29 | + | if err != nil { | |||
| 30 | + | t.Fatal(err) | |||
| 31 | + | } | |||
| 32 | + | for _, id := range []string{"apac.anthropic.claude-sonnet-4-6", "us-gov.anthropic.claude-sonnet-4-6"} { | |||
| 33 | + | m, wire := discoveredWireMaxTokens(t, id) | |||
| 34 | + | if m.MaxOutput != base.MaxOutput || wire != base.MaxOutput { | |||
| 35 | + | t.Errorf("%s: MaxOutput=%d wire=%d, want %d", id, m.MaxOutput, wire, base.MaxOutput) | |||
| 36 | + | } | |||
| 37 | + | if m.ContextWindow != base.ContextWindow || m.Reasoning != base.Reasoning { | |||
| 38 | + | t.Errorf("%s: metadata not inherited: %+v", id, m) | |||
| 39 | + | } | |||
| 40 | + | if m.PriceOutput != base.PriceOutput { | |||
| 41 | + | t.Errorf("%s: PriceOutput=%v, want %v", id, m.PriceOutput, base.PriceOutput) | |||
| 42 | + | } | |||
| 43 | + | } | |||
| 44 | + | ||||
| 45 | + | m, _ := discoveredWireMaxTokens(t, "apac.anthropic.claude-sonnet-4-6") | |||
| 46 | + | if m.DisplayName != "Claude Sonnet 4.6 (APAC)" { | |||
| 47 | + | t.Errorf("DisplayName = %q", m.DisplayName) | |||
| 48 | + | } | |||
| 49 | + | } | |||
| 50 | + | ||||
| 51 | + | // A base model catalogued only under geo-prefixed IDs still donates its | |||
| 52 | + | // metadata, and the donor's region label is not carried over. | |||
| 53 | + | func TestBedrockDiscoveredProfileUsesPrefixedCatalogVariant(t *testing.T) { | |||
| 54 | + | const id = "apac.anthropic.claude-opus-5-5" | |||
| 55 | + | m, wire := discoveredWireMaxTokens(t, id) | |||
| 56 | + | if wire != 128000 || !m.AdaptiveThinking { | |||
| 57 | + | t.Errorf("%s: wire=%d adaptive=%v, want 128000 adaptive", id, wire, m.AdaptiveThinking) | |||
| 58 | + | } | |||
| 59 | + | if m.DisplayName != "Claude Opus 5.5 (APAC)" { | |||
| 60 | + | t.Errorf("DisplayName = %q", m.DisplayName) | |||
| 61 | + | } | |||
| 62 | + | } | |||
| 63 | + | ||||
| 64 | + | // Claude models absent from the catalog get Haiku 4.5's 64k output, the | |||
| 65 | + | // smallest among current Claude models, instead of the 4096 fallback. | |||
| 66 | + | func TestBedrockDiscoveredUnknownClaudeUses64kFloor(t *testing.T) { | |||
| 67 | + | for _, id := range []string{"anthropic.claude-future-9", "us.anthropic.claude-future-9-v1:0"} { | |||
| 68 | + | m, wire := discoveredWireMaxTokens(t, id) | |||
| 69 | + | if wire != bedrockAnthropicDefaultMaxOutput || m.ContextWindow != bedrockAnthropicDefaultContextWindow { | |||
| 70 | + | t.Errorf("%s: wire=%d ctx=%d", id, wire, m.ContextWindow) | |||
| 71 | + | } | |||
| 72 | + | } | |||
| 73 | + | } | |||
| 74 | + | ||||
| 75 | + | // Legacy Claude and non-Anthropic models cap output well below 64k and | |||
| 76 | + | // would reject a larger request, so they keep the client fallback. | |||
| 77 | + | func TestBedrockDiscoveredLegacyAndOtherModelsKeepFallback(t *testing.T) { | |||
| 78 | + | for _, id := range []string{ | |||
| 79 | + | "anthropic.claude-v2:1", | |||
| 80 | + | "anthropic.claude-instant-v1", | |||
| 81 | + | "us.anthropic.claude-3-5-sonnet-20241022-v2:0", | |||
| 82 | + | "anthropic.claude-3-haiku-20240307-v1:0", | |||
| 83 | + | "mistral.unknown-model-v1:0", | |||
| 84 | + | } { | |||
| 85 | + | m, wire := discoveredWireMaxTokens(t, id) | |||
| 86 | + | if m.MaxOutput != 0 || wire != 4096 { | |||
| 87 | + | t.Errorf("%s: MaxOutput=%d wire=%d, want 0/4096", id, m.MaxOutput, wire) | |||
| 88 | + | } | |||
| 89 | + | } | |||
| 90 | + | } | |||
| 91 | + | ||||
| 92 | + | // Exact catalog IDs keep their own catalog metadata through MergeCatalog. | |||
| 93 | + | func TestBedrockDiscoveredExactCatalogIDUnchanged(t *testing.T) { | |||
| 94 | + | const id = "au.anthropic.claude-sonnet-4-6" | |||
| 95 | + | want, err := FindModel("amazon-bedrock", id) | |||
| 96 | + | if err != nil { | |||
| 97 | + | t.Fatal(err) | |||
| 98 | + | } | |||
| 99 | + | m, wire := discoveredWireMaxTokens(t, id) | |||
| 100 | + | if m.MaxOutput != want.MaxOutput || wire != want.MaxOutput || m.DisplayName != want.DisplayName { | |||
| 101 | + | t.Errorf("got MaxOutput=%d wire=%d name=%q, want %d %q", m.MaxOutput, wire, m.DisplayName, want.MaxOutput, want.DisplayName) | |||
| 102 | + | } | |||
| 103 | + | } | |||