Fix Bedrock discovery credential selection and pagination
- commit
- 82d7882329684267576ad5966b6b68e994c02b54
- author
- patriceckhart · · 2026-09-26 10:27 UTC
- parent
- a3a6316
- signature
- unsigned
7 files · +124 -26
M README.md +3 -3
| @@ -790,16 +790,16 @@ If none of these are available, zot errors with `vertex: no auth — set GOOGLE_ | |||
| 790 | 790 | zot talks to Bedrock's Converse streaming API and supports two credential styles: | |
| 791 | 791 | ||
| 792 | 792 | - **Bedrock API key** (bearer token) via `AWS_BEARER_TOKEN_BEDROCK`. This is the simplest setup: a single opaque string, region-scoped to wherever it was minted. It requires only the `bedrock:CallWithBearerToken` permission. | |
| 793 | - | - **SigV4** via `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` (plus optional `AWS_SESSION_TOKEN`), an `AWS_PROFILE`, or an SSO / assume-role / `credential_process` profile resolved through the AWS CLI. SigV4 (AWS Signature Version 4) is AWS's request-signing scheme: your secret key signs each request locally and only the signature travels over the wire. zot implements it directly so it does not depend on the AWS SDK. | |
| 793 | + | - **SigV4** via `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` (plus optional `AWS_SESSION_TOKEN`), an `AWS_PROFILE`, or a default / SSO / assume-role / `credential_process` profile resolved through the AWS CLI. SigV4 (AWS Signature Version 4) is AWS's request-signing scheme: your secret key signs each request locally and only the signature travels over the wire. zot implements it directly so it does not depend on the AWS SDK. | |
| 794 | 794 | ||
| 795 | - | If both are present, the bearer token wins for inference. The region defaults to `AWS_REGION` / `AWS_DEFAULT_REGION`, falling back to `us-east-1`. A `403` on a freshly-copied API key almost always means a region mismatch — set `AWS_REGION` to the region the key was created in, or pass `--base-url https://bedrock-runtime.<region>.amazonaws.com`. | |
| 795 | + | If both are present, the bearer token wins for inference, while live model discovery uses SigV4. The region defaults to `AWS_REGION` / `AWS_DEFAULT_REGION`, falling back to `us-east-1`. A `403` on a freshly-copied API key almost always means a region mismatch — set `AWS_REGION` to the region the key was created in, or pass `--base-url https://bedrock-runtime.<region>.amazonaws.com`. | |
| 796 | 796 | ||
| 797 | 797 | #### Catalog vs. live discovery | |
| 798 | 798 | ||
| 799 | 799 | Unlike most providers, Bedrock model **pricing, context windows, and capability flags are hand-maintained** in zot's baked-in catalog. AWS does not expose usable pricing programmatically for current models — its Price List API returns only stale, input-only rates for a few legacy models — so the catalog is authoritative for cost and metadata. This means: | |
| 800 | 800 | ||
| 801 | 801 | - **Bearer-token-only users** still get the full curated Bedrock model list with correct pricing from the catalog. No AWS API listing is attempted. | |
| 802 | - | - **SigV4 users** additionally get **live model discovery**: on startup, zot lists the foundation models and cross-region inference profiles your account can actually use (via `ListFoundationModels` and `ListInferenceProfiles`) and merges them into the catalog. New Bedrock models therefore appear in `/model` without waiting for a zot release. Discovered IDs already known to the catalog keep their curated pricing; genuinely new IDs show up with placeholder pricing until the catalog is updated. | |
| 802 | + | - **Users with SigV4 credentials** additionally get **live model discovery**, even if inference uses a bearer token: on startup, zot lists the foundation models and cross-region inference profiles your account can actually use (via `ListFoundationModels` and `ListInferenceProfiles`) and merges them into the catalog. New Bedrock models therefore appear in `/model` without waiting for a zot release. Discovered IDs already known to the catalog keep their curated pricing; genuinely new IDs show up with placeholder pricing until the catalog is updated. | |
| 803 | 803 | ||
| 804 | 804 | Live discovery requires SigV4 because the listing APIs live on the Bedrock control plane, which rejects the runtime bearer token. It runs in the background at startup and is cached for 6h like other providers, so `/model` reflects the latest list without any manual refresh. If discovery fails (no SigV4 creds, expired SSO login, missing IAM permissions), zot silently falls back to the catalog. Discovery needs `bedrock:ListFoundationModels` and `bedrock:ListInferenceProfiles`. | |
| 805 | 805 | ||
| @@ -790,16 +790,16 @@ If none of these are available, zot errors with `vertex: no auth — set GOOGLE_ | |||||
| 790 | zot talks to Bedrock's Converse streaming API and supports two credential styles: | 790 | zot talks to Bedrock's Converse streaming API and supports two credential styles: | ||
| 791 | 791 | ||||
| 792 | - **Bedrock API key** (bearer token) via `AWS_BEARER_TOKEN_BEDROCK`. This is the simplest setup: a single opaque string, region-scoped to wherever it was minted. It requires only the `bedrock:CallWithBearerToken` permission. | 792 | - **Bedrock API key** (bearer token) via `AWS_BEARER_TOKEN_BEDROCK`. This is the simplest setup: a single opaque string, region-scoped to wherever it was minted. It requires only the `bedrock:CallWithBearerToken` permission. | ||
| 793 | - | - **SigV4** via `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` (plus optional `AWS_SESSION_TOKEN`), an `AWS_PROFILE`, or an SSO / assume-role / `credential_process` profile resolved through the AWS CLI. SigV4 (AWS Signature Version 4) is AWS's request-signing scheme: your secret key signs each request locally and only the signature travels over the wire. zot implements it directly so it does not depend on the AWS SDK. | 793 | + | - **SigV4** via `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` (plus optional `AWS_SESSION_TOKEN`), an `AWS_PROFILE`, or a default / SSO / assume-role / `credential_process` profile resolved through the AWS CLI. SigV4 (AWS Signature Version 4) is AWS's request-signing scheme: your secret key signs each request locally and only the signature travels over the wire. zot implements it directly so it does not depend on the AWS SDK. |
| 794 | 794 | ||||
| 795 | - | If both are present, the bearer token wins for inference. The region defaults to `AWS_REGION` / `AWS_DEFAULT_REGION`, falling back to `us-east-1`. A `403` on a freshly-copied API key almost always means a region mismatch — set `AWS_REGION` to the region the key was created in, or pass `--base-url https://bedrock-runtime.<region>.amazonaws.com`. | 795 | + | If both are present, the bearer token wins for inference, while live model discovery uses SigV4. The region defaults to `AWS_REGION` / `AWS_DEFAULT_REGION`, falling back to `us-east-1`. A `403` on a freshly-copied API key almost always means a region mismatch — set `AWS_REGION` to the region the key was created in, or pass `--base-url https://bedrock-runtime.<region>.amazonaws.com`. |
| 796 | 796 | ||||
| 797 | #### Catalog vs. live discovery | 797 | #### Catalog vs. live discovery | ||
| 798 | 798 | ||||
| 799 | Unlike most providers, Bedrock model **pricing, context windows, and capability flags are hand-maintained** in zot's baked-in catalog. AWS does not expose usable pricing programmatically for current models — its Price List API returns only stale, input-only rates for a few legacy models — so the catalog is authoritative for cost and metadata. This means: | 799 | Unlike most providers, Bedrock model **pricing, context windows, and capability flags are hand-maintained** in zot's baked-in catalog. AWS does not expose usable pricing programmatically for current models — its Price List API returns only stale, input-only rates for a few legacy models — so the catalog is authoritative for cost and metadata. This means: | ||
| 800 | 800 | ||||
| 801 | - **Bearer-token-only users** still get the full curated Bedrock model list with correct pricing from the catalog. No AWS API listing is attempted. | 801 | - **Bearer-token-only users** still get the full curated Bedrock model list with correct pricing from the catalog. No AWS API listing is attempted. | ||
| 802 | - | - **SigV4 users** additionally get **live model discovery**: on startup, zot lists the foundation models and cross-region inference profiles your account can actually use (via `ListFoundationModels` and `ListInferenceProfiles`) and merges them into the catalog. New Bedrock models therefore appear in `/model` without waiting for a zot release. Discovered IDs already known to the catalog keep their curated pricing; genuinely new IDs show up with placeholder pricing until the catalog is updated. | 802 | + | - **Users with SigV4 credentials** additionally get **live model discovery**, even if inference uses a bearer token: on startup, zot lists the foundation models and cross-region inference profiles your account can actually use (via `ListFoundationModels` and `ListInferenceProfiles`) and merges them into the catalog. New Bedrock models therefore appear in `/model` without waiting for a zot release. Discovered IDs already known to the catalog keep their curated pricing; genuinely new IDs show up with placeholder pricing until the catalog is updated. |
| 803 | 803 | ||||
| 804 | Live discovery requires SigV4 because the listing APIs live on the Bedrock control plane, which rejects the runtime bearer token. It runs in the background at startup and is cached for 6h like other providers, so `/model` reflects the latest list without any manual refresh. If discovery fails (no SigV4 creds, expired SSO login, missing IAM permissions), zot silently falls back to the catalog. Discovery needs `bedrock:ListFoundationModels` and `bedrock:ListInferenceProfiles`. | 804 | Live discovery requires SigV4 because the listing APIs live on the Bedrock control plane, which rejects the runtime bearer token. It runs in the background at startup and is cached for 6h like other providers, so `/model` reflects the latest list without any manual refresh. If discovery fails (no SigV4 creds, expired SSO login, missing IAM permissions), zot silently falls back to the catalog. Discovery needs `bedrock:ListFoundationModels` and `bedrock:ListInferenceProfiles`. | ||
| 805 | 805 | ||||
M packages/agent/build.go +6 -0
| @@ -596,6 +596,12 @@ func Resolve(args Args, requireCred bool) (Resolved, error) { | |||
| 596 | 596 | // models like ollama don't need real API keys). | |
| 597 | 597 | if resolvedModel.BaseURL != "" && credErr != nil { | |
| 598 | 598 | cred = "ollama" | |
| 599 | + | if provName == "amazon-bedrock" { | |
| 600 | + | // The default AWS CLI profile is resolved by the Bedrock client, | |
| 601 | + | // not the general credential resolver. Never pass the local-model | |
| 602 | + | // placeholder as a bearer token. | |
| 603 | + | cred = "<aws>" | |
| 604 | + | } | |
| 599 | 605 | credErr = nil | |
| 600 | 606 | requireCred = false | |
| 601 | 607 | } | |
| @@ -596,6 +596,12 @@ func Resolve(args Args, requireCred bool) (Resolved, error) { | |||||
| 596 | // models like ollama don't need real API keys). | 596 | // models like ollama don't need real API keys). | ||
| 597 | if resolvedModel.BaseURL != "" && credErr != nil { | 597 | if resolvedModel.BaseURL != "" && credErr != nil { | ||
| 598 | cred = "ollama" | 598 | cred = "ollama" | ||
| 599 | + | if provName == "amazon-bedrock" { | |||
| 600 | + | // The default AWS CLI profile is resolved by the Bedrock client, | |||
| 601 | + | // not the general credential resolver. Never pass the local-model | |||
| 602 | + | // placeholder as a bearer token. | |||
| 603 | + | cred = "<aws>" | |||
| 604 | + | } | |||
| 599 | credErr = nil | 605 | credErr = nil | ||
| 600 | requireCred = false | 606 | requireCred = false | ||
| 601 | } | 607 | } | ||
M packages/agent/modelsync.go +5 -7
| @@ -255,13 +255,11 @@ func refreshModels() { | |||
| 255 | 255 | all = append(all, live...) | |
| 256 | 256 | } | |
| 257 | 257 | } | |
| 258 | - | if cred, _, err := resolveCredentialForBackground(ctx, "amazon-bedrock"); err == nil { | |
| 259 | - | // Bedrock discovery + pricing needs SigV4 credentials | |
| 260 | - | // (env keys or AWS_PROFILE); a bearer-only setup returns | |
| 261 | - | // (nil, nil) and is skipped inside DiscoverBedrock. | |
| 262 | - | if live, err := provider.DiscoverBedrock(ctx, cred, ""); err == nil { | |
| 263 | - | all = append(all, live...) | |
| 264 | - | } | |
| 258 | + | // Bedrock resolves SigV4 credentials independently of inference auth. | |
| 259 | + | // A default AWS CLI profile can be usable even when zot has no | |
| 260 | + | // Bedrock credential registered with its general resolver. | |
| 261 | + | if live, err := provider.DiscoverBedrock(ctx, ""); err == nil { | |
| 262 | + | all = append(all, live...) | |
| 265 | 263 | } | |
| 266 | 264 | if haveOpenRouter { | |
| 267 | 265 | // /models is public; gate on a credential so the picker only | |
| @@ -255,13 +255,11 @@ func refreshModels() { | |||||
| 255 | all = append(all, live...) | 255 | all = append(all, live...) | ||
| 256 | } | 256 | } | ||
| 257 | } | 257 | } | ||
| 258 | - | if cred, _, err := resolveCredentialForBackground(ctx, "amazon-bedrock"); err == nil { | 258 | + | // Bedrock resolves SigV4 credentials independently of inference auth. |
| 259 | - | // Bedrock discovery + pricing needs SigV4 credentials | 259 | + | // A default AWS CLI profile can be usable even when zot has no |
| 260 | - | // (env keys or AWS_PROFILE); a bearer-only setup returns | 260 | + | // Bedrock credential registered with its general resolver. |
| 261 | - | // (nil, nil) and is skipped inside DiscoverBedrock. | 261 | + | if live, err := provider.DiscoverBedrock(ctx, ""); err == nil { |
| 262 | - | if live, err := provider.DiscoverBedrock(ctx, cred, ""); err == nil { | 262 | + | all = append(all, live...) |
| 263 | - | all = append(all, live...) | |||
| 264 | - | } | |||
| 265 | } | 263 | } | ||
| 266 | if haveOpenRouter { | 264 | if haveOpenRouter { | ||
| 267 | // /models is public; gate on a credential so the picker only | 265 | // /models is public; gate on a credential so the picker only | ||
M packages/agent/modelsync_test.go +52 -0
| @@ -4,16 +4,68 @@ import ( | |||
| 4 | 4 | "context" | |
| 5 | 5 | "encoding/json" | |
| 6 | 6 | "fmt" | |
| 7 | + | "io" | |
| 7 | 8 | "net/http" | |
| 8 | 9 | "net/http/httptest" | |
| 9 | 10 | "os" | |
| 10 | 11 | "path/filepath" | |
| 12 | + | "runtime" | |
| 13 | + | "strings" | |
| 11 | 14 | "testing" | |
| 12 | 15 | "time" | |
| 13 | 16 | ||
| 14 | 17 | "github.com/patriceckhart/zot/packages/provider" | |
| 15 | 18 | ) | |
| 16 | 19 | ||
| 20 | + | func TestRefreshModelsBedrockWithDefaultCLIProfile(t *testing.T) { | |
| 21 | + | if runtime.GOOS == "windows" { | |
| 22 | + | t.Skip("test uses a shell script to simulate the AWS CLI") | |
| 23 | + | } | |
| 24 | + | t.Setenv("ZOT_HOME", t.TempDir()) | |
| 25 | + | t.Setenv("AWS_PROFILE", "") | |
| 26 | + | t.Setenv("AWS_ACCESS_KEY_ID", "") | |
| 27 | + | t.Setenv("AWS_SECRET_ACCESS_KEY", "") | |
| 28 | + | t.Setenv("AWS_BEARER_TOKEN_BEDROCK", "") | |
| 29 | + | for _, name := range []string{"ANTHROPIC_API_KEY", "OPENAI_API_KEY", "KIMI_API_KEY", "GOOGLE_API_KEY", "OPENROUTER_API_KEY", "GONDOLA_API_KEY", "YOLO_AUTO_API_KEY"} { | |
| 30 | + | t.Setenv(name, "") | |
| 31 | + | } | |
| 32 | + | bin := t.TempDir() | |
| 33 | + | script := "#!/bin/sh\nprintf '%s\\n' '{\"AccessKeyId\":\"AKID\",\"SecretAccessKey\":\"SECRET\"}'\n" | |
| 34 | + | if err := os.WriteFile(filepath.Join(bin, "aws"), []byte(script), 0o755); err != nil { | |
| 35 | + | t.Fatal(err) | |
| 36 | + | } | |
| 37 | + | t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) | |
| 38 | + | orig := http.DefaultTransport | |
| 39 | + | t.Cleanup(func() { | |
| 40 | + | http.DefaultTransport = orig | |
| 41 | + | provider.SetLiveModels(nil) | |
| 42 | + | }) | |
| 43 | + | http.DefaultTransport = yoloAutoTestTransport(func(r *http.Request) (*http.Response, error) { | |
| 44 | + | var body string | |
| 45 | + | switch r.URL.Path { | |
| 46 | + | case "/foundation-models": | |
| 47 | + | body = `{"modelSummaries":[{"modelId":"example.cli-only-model","outputModalities":["TEXT"]}]}` | |
| 48 | + | case "/inference-profiles": | |
| 49 | + | body = `{"inferenceProfileSummaries":[]}` | |
| 50 | + | default: | |
| 51 | + | return nil, fmt.Errorf("unexpected discovery request: %s", r.URL) | |
| 52 | + | } | |
| 53 | + | return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(body)), Header: make(http.Header)}, nil | |
| 54 | + | }) | |
| 55 | + | ||
| 56 | + | refreshModels() | |
| 57 | + | if _, err := provider.FindModel("amazon-bedrock", "example.cli-only-model"); err != nil { | |
| 58 | + | t.Fatal("default AWS CLI profile was not discovered:", err) | |
| 59 | + | } | |
| 60 | + | resolved, err := Resolve(Args{Provider: "amazon-bedrock", Model: "example.cli-only-model"}, true) | |
| 61 | + | if err != nil { | |
| 62 | + | t.Fatal("default AWS CLI profile could not resolve for inference:", err) | |
| 63 | + | } | |
| 64 | + | if resolved.Credential != "<aws>" { | |
| 65 | + | t.Fatalf("inference credential = %q, want AWS sentinel", resolved.Credential) | |
| 66 | + | } | |
| 67 | + | } | |
| 68 | + | ||
| 17 | 69 | func TestRefreshLlamaCPPModelsAddsOnlyLoadedModels(t *testing.T) { | |
| 18 | 70 | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | |
| 19 | 71 | if r.URL.Path != "/models" { | |
| @@ -4,16 +4,68 @@ import ( | |||||
| 4 | "context" | 4 | "context" | ||
| 5 | "encoding/json" | 5 | "encoding/json" | ||
| 6 | "fmt" | 6 | "fmt" | ||
| 7 | + | "io" | |||
| 7 | "net/http" | 8 | "net/http" | ||
| 8 | "net/http/httptest" | 9 | "net/http/httptest" | ||
| 9 | "os" | 10 | "os" | ||
| 10 | "path/filepath" | 11 | "path/filepath" | ||
| 12 | + | "runtime" | |||
| 13 | + | "strings" | |||
| 11 | "testing" | 14 | "testing" | ||
| 12 | "time" | 15 | "time" | ||
| 13 | 16 | ||||
| 14 | "github.com/patriceckhart/zot/packages/provider" | 17 | "github.com/patriceckhart/zot/packages/provider" | ||
| 15 | ) | 18 | ) | ||
| 16 | 19 | ||||
| 20 | + | func TestRefreshModelsBedrockWithDefaultCLIProfile(t *testing.T) { | |||
| 21 | + | if runtime.GOOS == "windows" { | |||
| 22 | + | t.Skip("test uses a shell script to simulate the AWS CLI") | |||
| 23 | + | } | |||
| 24 | + | t.Setenv("ZOT_HOME", t.TempDir()) | |||
| 25 | + | t.Setenv("AWS_PROFILE", "") | |||
| 26 | + | t.Setenv("AWS_ACCESS_KEY_ID", "") | |||
| 27 | + | t.Setenv("AWS_SECRET_ACCESS_KEY", "") | |||
| 28 | + | t.Setenv("AWS_BEARER_TOKEN_BEDROCK", "") | |||
| 29 | + | for _, name := range []string{"ANTHROPIC_API_KEY", "OPENAI_API_KEY", "KIMI_API_KEY", "GOOGLE_API_KEY", "OPENROUTER_API_KEY", "GONDOLA_API_KEY", "YOLO_AUTO_API_KEY"} { | |||
| 30 | + | t.Setenv(name, "") | |||
| 31 | + | } | |||
| 32 | + | bin := t.TempDir() | |||
| 33 | + | script := "#!/bin/sh\nprintf '%s\\n' '{\"AccessKeyId\":\"AKID\",\"SecretAccessKey\":\"SECRET\"}'\n" | |||
| 34 | + | if err := os.WriteFile(filepath.Join(bin, "aws"), []byte(script), 0o755); err != nil { | |||
| 35 | + | t.Fatal(err) | |||
| 36 | + | } | |||
| 37 | + | t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) | |||
| 38 | + | orig := http.DefaultTransport | |||
| 39 | + | t.Cleanup(func() { | |||
| 40 | + | http.DefaultTransport = orig | |||
| 41 | + | provider.SetLiveModels(nil) | |||
| 42 | + | }) | |||
| 43 | + | http.DefaultTransport = yoloAutoTestTransport(func(r *http.Request) (*http.Response, error) { | |||
| 44 | + | var body string | |||
| 45 | + | switch r.URL.Path { | |||
| 46 | + | case "/foundation-models": | |||
| 47 | + | body = `{"modelSummaries":[{"modelId":"example.cli-only-model","outputModalities":["TEXT"]}]}` | |||
| 48 | + | case "/inference-profiles": | |||
| 49 | + | body = `{"inferenceProfileSummaries":[]}` | |||
| 50 | + | default: | |||
| 51 | + | return nil, fmt.Errorf("unexpected discovery request: %s", r.URL) | |||
| 52 | + | } | |||
| 53 | + | return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(body)), Header: make(http.Header)}, nil | |||
| 54 | + | }) | |||
| 55 | + | ||||
| 56 | + | refreshModels() | |||
| 57 | + | if _, err := provider.FindModel("amazon-bedrock", "example.cli-only-model"); err != nil { | |||
| 58 | + | t.Fatal("default AWS CLI profile was not discovered:", err) | |||
| 59 | + | } | |||
| 60 | + | resolved, err := Resolve(Args{Provider: "amazon-bedrock", Model: "example.cli-only-model"}, true) | |||
| 61 | + | if err != nil { | |||
| 62 | + | t.Fatal("default AWS CLI profile could not resolve for inference:", err) | |||
| 63 | + | } | |||
| 64 | + | if resolved.Credential != "<aws>" { | |||
| 65 | + | t.Fatalf("inference credential = %q, want AWS sentinel", resolved.Credential) | |||
| 66 | + | } | |||
| 67 | + | } | |||
| 68 | + | ||||
| 17 | func TestRefreshLlamaCPPModelsAddsOnlyLoadedModels(t *testing.T) { | 69 | func TestRefreshLlamaCPPModelsAddsOnlyLoadedModels(t *testing.T) { | ||
| 18 | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | 70 | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | ||
| 19 | if r.URL.Path != "/models" { | 71 | if r.URL.Path != "/models" { | ||
M packages/provider/amazon_bedrock.go +10 -4
| @@ -145,16 +145,22 @@ func resolveBedrockAuth(apiKey string) (bearer string, sigv4 *bedrockSigV4Creds) | |||
| 145 | 145 | return token, nil | |
| 146 | 146 | } | |
| 147 | 147 | ||
| 148 | + | return "", resolveBedrockDiscoveryCreds() | |
| 149 | + | } | |
| 150 | + | ||
| 151 | + | // resolveBedrockDiscoveryCreds ignores inference bearer tokens: Bedrock's | |
| 152 | + | // control-plane listing endpoints require SigV4 even when inference uses a bearer. | |
| 153 | + | func resolveBedrockDiscoveryCreds() *bedrockSigV4Creds { | |
| 148 | 154 | ak := os.Getenv("AWS_ACCESS_KEY_ID") | |
| 149 | 155 | sk := os.Getenv("AWS_SECRET_ACCESS_KEY") | |
| 150 | 156 | st := os.Getenv("AWS_SESSION_TOKEN") | |
| 151 | 157 | if ak != "" && sk != "" { | |
| 152 | - | return "", &bedrockSigV4Creds{accessKeyID: ak, secretAccessKey: sk, sessionToken: st} | |
| 158 | + | return &bedrockSigV4Creds{accessKeyID: ak, secretAccessKey: sk, sessionToken: st} | |
| 153 | 159 | } | |
| 154 | 160 | ||
| 155 | 161 | if profile := os.Getenv("AWS_PROFILE"); profile != "" { | |
| 156 | 162 | if creds, err := readAWSCredentialsFile(profile); err == nil { | |
| 157 | - | return "", creds | |
| 163 | + | return creds | |
| 158 | 164 | } | |
| 159 | 165 | } | |
| 160 | 166 | ||
| @@ -164,10 +170,10 @@ func resolveBedrockAuth(apiKey string) (bearer string, sigv4 *bedrockSigV4Creds) | |||
| 164 | 170 | // in ~/.aws/credentials. Best-effort: absent CLI or an expired login | |
| 165 | 171 | // simply yields no credentials. | |
| 166 | 172 | if creds := resolveBedrockCredsViaCLI(); creds != nil { | |
| 167 | - | return "", creds | |
| 173 | + | return creds | |
| 168 | 174 | } | |
| 169 | 175 | ||
| 170 | - | return "", nil | |
| 176 | + | return nil | |
| 171 | 177 | } | |
| 172 | 178 | ||
| 173 | 179 | // resolveBedrockCredsViaCLI shells out to `aws configure | |
| @@ -145,16 +145,22 @@ func resolveBedrockAuth(apiKey string) (bearer string, sigv4 *bedrockSigV4Creds) | |||||
| 145 | return token, nil | 145 | return token, nil | ||
| 146 | } | 146 | } | ||
| 147 | 147 | ||||
| 148 | + | return "", resolveBedrockDiscoveryCreds() | |||
| 149 | + | } | |||
| 150 | + | ||||
| 151 | + | // resolveBedrockDiscoveryCreds ignores inference bearer tokens: Bedrock's | |||
| 152 | + | // control-plane listing endpoints require SigV4 even when inference uses a bearer. | |||
| 153 | + | func resolveBedrockDiscoveryCreds() *bedrockSigV4Creds { | |||
| 148 | ak := os.Getenv("AWS_ACCESS_KEY_ID") | 154 | ak := os.Getenv("AWS_ACCESS_KEY_ID") | ||
| 149 | sk := os.Getenv("AWS_SECRET_ACCESS_KEY") | 155 | sk := os.Getenv("AWS_SECRET_ACCESS_KEY") | ||
| 150 | st := os.Getenv("AWS_SESSION_TOKEN") | 156 | st := os.Getenv("AWS_SESSION_TOKEN") | ||
| 151 | if ak != "" && sk != "" { | 157 | if ak != "" && sk != "" { | ||
| 152 | - | return "", &bedrockSigV4Creds{accessKeyID: ak, secretAccessKey: sk, sessionToken: st} | 158 | + | return &bedrockSigV4Creds{accessKeyID: ak, secretAccessKey: sk, sessionToken: st} |
| 153 | } | 159 | } | ||
| 154 | 160 | ||||
| 155 | if profile := os.Getenv("AWS_PROFILE"); profile != "" { | 161 | if profile := os.Getenv("AWS_PROFILE"); profile != "" { | ||
| 156 | if creds, err := readAWSCredentialsFile(profile); err == nil { | 162 | if creds, err := readAWSCredentialsFile(profile); err == nil { | ||
| 157 | - | return "", creds | 163 | + | return creds |
| 158 | } | 164 | } | ||
| 159 | } | 165 | } | ||
| 160 | 166 | ||||
| @@ -164,10 +170,10 @@ func resolveBedrockAuth(apiKey string) (bearer string, sigv4 *bedrockSigV4Creds) | |||||
| 164 | // in ~/.aws/credentials. Best-effort: absent CLI or an expired login | 170 | // in ~/.aws/credentials. Best-effort: absent CLI or an expired login | ||
| 165 | // simply yields no credentials. | 171 | // simply yields no credentials. | ||
| 166 | if creds := resolveBedrockCredsViaCLI(); creds != nil { | 172 | if creds := resolveBedrockCredsViaCLI(); creds != nil { | ||
| 167 | - | return "", creds | 173 | + | return creds |
| 168 | } | 174 | } | ||
| 169 | 175 | ||||
| 170 | - | return "", nil | 176 | + | return nil |
| 171 | } | 177 | } | ||
| 172 | 178 | ||||
| 173 | // resolveBedrockCredsViaCLI shells out to `aws configure | 179 | // resolveBedrockCredsViaCLI shells out to `aws configure | ||
M packages/provider/discover_bedrock.go +8 -8
| @@ -35,6 +35,7 @@ import ( | |||
| 35 | 35 | "fmt" | |
| 36 | 36 | "io" | |
| 37 | 37 | "net/http" | |
| 38 | + | "net/url" | |
| 38 | 39 | "strings" | |
| 39 | 40 | "time" | |
| 40 | 41 | ) | |
| @@ -51,19 +52,17 @@ func bytesReader(payload []byte) io.Reader { | |||
| 51 | 52 | // DiscoverBedrock lists the foundation models and inference profiles | |
| 52 | 53 | // available to the caller's AWS credentials in the given region. | |
| 53 | 54 | // | |
| 54 | - | // apiKey is the credential zot resolved for the amazon-bedrock provider; | |
| 55 | - | // it is used only to detect a bearer token (which cannot reach the | |
| 56 | - | // control-plane APIs). SigV4 credentials are resolved from the standard | |
| 57 | - | // AWS environment / profile / CLI sources, matching the runtime client. | |
| 55 | + | // SigV4 credentials are resolved from the standard AWS environment / | |
| 56 | + | // profile / CLI sources, even when a bearer token is used for inference. | |
| 58 | 57 | // | |
| 59 | 58 | // region defaults to AWS_REGION / AWS_DEFAULT_REGION / us-east-1 when | |
| 60 | 59 | // empty. A nil error with a nil slice means "no SigV4 credentials" or | |
| 61 | 60 | // "no models"; callers should treat that as a skip. | |
| 62 | - | func DiscoverBedrock(ctx context.Context, apiKey, region string) ([]Model, error) { | |
| 61 | + | func DiscoverBedrock(ctx context.Context, region string) ([]Model, error) { | |
| 63 | 62 | if region == "" { | |
| 64 | 63 | region = bedrockResolveRegion() | |
| 65 | 64 | } | |
| 66 | - | _, sigv4 := resolveBedrockAuth(apiKey) | |
| 65 | + | sigv4 := resolveBedrockDiscoveryCreds() | |
| 67 | 66 | if sigv4 == nil { | |
| 68 | 67 | // Bearer-only or unauthenticated: control-plane APIs are | |
| 69 | 68 | // unreachable. Skip quietly so the catalog stands. | |
| @@ -134,10 +133,11 @@ func bedrockListModelIDs(ctx context.Context, client *http.Client, creds *bedroc | |||
| 134 | 133 | // ListInferenceProfiles: GET, paginated via ?nextToken=. | |
| 135 | 134 | next := "" | |
| 136 | 135 | for { | |
| 137 | - | ipURL := controlPlaneBase + "/inference-profiles?maxResults=1000" | |
| 136 | + | query := url.Values{"maxResults": {"1000"}} | |
| 138 | 137 | if next != "" { | |
| 139 | - | ipURL += "&nextToken=" + next | |
| 138 | + | query.Set("nextToken", next) | |
| 140 | 139 | } | |
| 140 | + | ipURL := controlPlaneBase + "/inference-profiles?" + query.Encode() | |
| 141 | 141 | ipBody, err := bedrockSignedControlPlane(ctx, client, creds, region, http.MethodGet, ipURL, nil) | |
| 142 | 142 | if err != nil { | |
| 143 | 143 | // Non-fatal: foundation models already gathered. | |
| @@ -35,6 +35,7 @@ import ( | |||||
| 35 | "fmt" | 35 | "fmt" | ||
| 36 | "io" | 36 | "io" | ||
| 37 | "net/http" | 37 | "net/http" | ||
| 38 | + | "net/url" | |||
| 38 | "strings" | 39 | "strings" | ||
| 39 | "time" | 40 | "time" | ||
| 40 | ) | 41 | ) | ||
| @@ -51,19 +52,17 @@ func bytesReader(payload []byte) io.Reader { | |||||
| 51 | // DiscoverBedrock lists the foundation models and inference profiles | 52 | // DiscoverBedrock lists the foundation models and inference profiles | ||
| 52 | // available to the caller's AWS credentials in the given region. | 53 | // available to the caller's AWS credentials in the given region. | ||
| 53 | // | 54 | // | ||
| 54 | - | // apiKey is the credential zot resolved for the amazon-bedrock provider; | 55 | + | // SigV4 credentials are resolved from the standard AWS environment / |
| 55 | - | // it is used only to detect a bearer token (which cannot reach the | 56 | + | // profile / CLI sources, even when a bearer token is used for inference. |
| 56 | - | // control-plane APIs). SigV4 credentials are resolved from the standard | |||
| 57 | - | // AWS environment / profile / CLI sources, matching the runtime client. | |||
| 58 | // | 57 | // | ||
| 59 | // region defaults to AWS_REGION / AWS_DEFAULT_REGION / us-east-1 when | 58 | // region defaults to AWS_REGION / AWS_DEFAULT_REGION / us-east-1 when | ||
| 60 | // empty. A nil error with a nil slice means "no SigV4 credentials" or | 59 | // empty. A nil error with a nil slice means "no SigV4 credentials" or | ||
| 61 | // "no models"; callers should treat that as a skip. | 60 | // "no models"; callers should treat that as a skip. | ||
| 62 | - | func DiscoverBedrock(ctx context.Context, apiKey, region string) ([]Model, error) { | 61 | + | func DiscoverBedrock(ctx context.Context, region string) ([]Model, error) { |
| 63 | if region == "" { | 62 | if region == "" { | ||
| 64 | region = bedrockResolveRegion() | 63 | region = bedrockResolveRegion() | ||
| 65 | } | 64 | } | ||
| 66 | - | _, sigv4 := resolveBedrockAuth(apiKey) | 65 | + | sigv4 := resolveBedrockDiscoveryCreds() |
| 67 | if sigv4 == nil { | 66 | if sigv4 == nil { | ||
| 68 | // Bearer-only or unauthenticated: control-plane APIs are | 67 | // Bearer-only or unauthenticated: control-plane APIs are | ||
| 69 | // unreachable. Skip quietly so the catalog stands. | 68 | // unreachable. Skip quietly so the catalog stands. | ||
| @@ -134,10 +133,11 @@ func bedrockListModelIDs(ctx context.Context, client *http.Client, creds *bedroc | |||||
| 134 | // ListInferenceProfiles: GET, paginated via ?nextToken=. | 133 | // ListInferenceProfiles: GET, paginated via ?nextToken=. | ||
| 135 | next := "" | 134 | next := "" | ||
| 136 | for { | 135 | for { | ||
| 137 | - | ipURL := controlPlaneBase + "/inference-profiles?maxResults=1000" | 136 | + | query := url.Values{"maxResults": {"1000"}} |
| 138 | if next != "" { | 137 | if next != "" { | ||
| 139 | - | ipURL += "&nextToken=" + next | 138 | + | query.Set("nextToken", next) |
| 140 | } | 139 | } | ||
| 140 | + | ipURL := controlPlaneBase + "/inference-profiles?" + query.Encode() | |||
| 141 | ipBody, err := bedrockSignedControlPlane(ctx, client, creds, region, http.MethodGet, ipURL, nil) | 141 | ipBody, err := bedrockSignedControlPlane(ctx, client, creds, region, http.MethodGet, ipURL, nil) | ||
| 142 | if err != nil { | 142 | if err != nil { | ||
| 143 | // Non-fatal: foundation models already gathered. | 143 | // Non-fatal: foundation models already gathered. | ||
M packages/provider/discover_bedrock_test.go +40 -4
| @@ -2,12 +2,44 @@ package provider | |||
| 2 | 2 | ||
| 3 | 3 | import ( | |
| 4 | 4 | "context" | |
| 5 | + | "io" | |
| 5 | 6 | "net/http" | |
| 6 | 7 | "net/http/httptest" | |
| 7 | 8 | "strings" | |
| 8 | 9 | "testing" | |
| 9 | 10 | ) | |
| 10 | 11 | ||
| 12 | + | type bedrockDiscoveryTestTransport func(*http.Request) (*http.Response, error) | |
| 13 | + | ||
| 14 | + | func (f bedrockDiscoveryTestTransport) RoundTrip(r *http.Request) (*http.Response, error) { | |
| 15 | + | return f(r) | |
| 16 | + | } | |
| 17 | + | ||
| 18 | + | func TestDiscoverBedrockUsesSigV4WithBearer(t *testing.T) { | |
| 19 | + | t.Setenv("AWS_BEARER_TOKEN_BEDROCK", "bearer") | |
| 20 | + | t.Setenv("AWS_ACCESS_KEY_ID", "AKID") | |
| 21 | + | t.Setenv("AWS_SECRET_ACCESS_KEY", "SECRET") | |
| 22 | + | original := http.DefaultTransport | |
| 23 | + | t.Cleanup(func() { http.DefaultTransport = original }) | |
| 24 | + | http.DefaultTransport = bedrockDiscoveryTestTransport(func(r *http.Request) (*http.Response, error) { | |
| 25 | + | if !strings.Contains(r.Header.Get("Authorization"), "Credential=AKID/") { | |
| 26 | + | t.Errorf("discovery request was not signed with SigV4") | |
| 27 | + | } | |
| 28 | + | body := `{"modelSummaries":[{"modelId":"test.new-model","outputModalities":["TEXT"]}]}` | |
| 29 | + | if r.URL.Path == "/inference-profiles" { | |
| 30 | + | body = `{"inferenceProfileSummaries":[]}` | |
| 31 | + | } | |
| 32 | + | return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(body)), Header: make(http.Header)}, nil | |
| 33 | + | }) | |
| 34 | + | models, err := DiscoverBedrock(context.Background(), "us-east-1") | |
| 35 | + | if err != nil || len(models) != 1 || models[0].ID != "test.new-model" { | |
| 36 | + | t.Fatalf("discovered models = %v, err = %v", models, err) | |
| 37 | + | } | |
| 38 | + | if bearer, sigv4 := resolveBedrockAuth("bearer"); bearer != "bearer" || sigv4 != nil { | |
| 39 | + | t.Fatal("inference must continue to prefer the bearer token") | |
| 40 | + | } | |
| 41 | + | } | |
| 42 | + | ||
| 11 | 43 | func TestBedrockEmitsText(t *testing.T) { | |
| 12 | 44 | if !bedrockEmitsText(nil) { | |
| 13 | 45 | t.Error("nil modalities should default to text") | |
| @@ -55,7 +87,7 @@ func TestResolveBedrockAuthBearerVsSigV4(t *testing.T) { | |||
| 55 | 87 | // nextToken. Embedding/image-only models are filtered out; profile IDs | |
| 56 | 88 | // are included; duplicates are de-duplicated. | |
| 57 | 89 | func TestBedrockListModelIDs(t *testing.T) { | |
| 58 | - | page1 := `{"inferenceProfileSummaries":[{"inferenceProfileId":"us.anthropic.claude-opus-5-5"}],"nextToken":"NEXT"}` | |
| 90 | + | page1 := `{"inferenceProfileSummaries":[{"inferenceProfileId":"us.anthropic.claude-opus-5-5"}],"nextToken":"N+E/T= &"}` | |
| 59 | 91 | page2 := `{"inferenceProfileSummaries":[{"inferenceProfileId":"eu.anthropic.claude-sonnet-5"}]}` | |
| 60 | 92 | foundation := `{"modelSummaries":[ | |
| 61 | 93 | {"modelId":"anthropic.claude-opus-5-5","outputModalities":["TEXT"]}, | |
| @@ -72,10 +104,14 @@ func TestBedrockListModelIDs(t *testing.T) { | |||
| 72 | 104 | w.Write([]byte(foundation)) | |
| 73 | 105 | case strings.Contains(r.URL.Path, "/inference-profiles"): | |
| 74 | 106 | ipHits++ | |
| 75 | - | if r.URL.Query().Get("nextToken") == "NEXT" { | |
| 76 | - | w.Write([]byte(page2)) | |
| 77 | - | } else { | |
| 107 | + | switch token := r.URL.Query().Get("nextToken"); token { | |
| 108 | + | case "": | |
| 78 | 109 | w.Write([]byte(page1)) | |
| 110 | + | case "N+E/T= &": | |
| 111 | + | w.Write([]byte(page2)) | |
| 112 | + | default: | |
| 113 | + | t.Errorf("unexpected pagination token %q", token) | |
| 114 | + | w.WriteHeader(http.StatusBadRequest) | |
| 79 | 115 | } | |
| 80 | 116 | default: | |
| 81 | 117 | w.WriteHeader(http.StatusNotFound) | |
| @@ -2,12 +2,44 @@ package provider | |||||
| 2 | 2 | ||||
| 3 | import ( | 3 | import ( | ||
| 4 | "context" | 4 | "context" | ||
| 5 | + | "io" | |||
| 5 | "net/http" | 6 | "net/http" | ||
| 6 | "net/http/httptest" | 7 | "net/http/httptest" | ||
| 7 | "strings" | 8 | "strings" | ||
| 8 | "testing" | 9 | "testing" | ||
| 9 | ) | 10 | ) | ||
| 10 | 11 | ||||
| 12 | + | type bedrockDiscoveryTestTransport func(*http.Request) (*http.Response, error) | |||
| 13 | + | ||||
| 14 | + | func (f bedrockDiscoveryTestTransport) RoundTrip(r *http.Request) (*http.Response, error) { | |||
| 15 | + | return f(r) | |||
| 16 | + | } | |||
| 17 | + | ||||
| 18 | + | func TestDiscoverBedrockUsesSigV4WithBearer(t *testing.T) { | |||
| 19 | + | t.Setenv("AWS_BEARER_TOKEN_BEDROCK", "bearer") | |||
| 20 | + | t.Setenv("AWS_ACCESS_KEY_ID", "AKID") | |||
| 21 | + | t.Setenv("AWS_SECRET_ACCESS_KEY", "SECRET") | |||
| 22 | + | original := http.DefaultTransport | |||
| 23 | + | t.Cleanup(func() { http.DefaultTransport = original }) | |||
| 24 | + | http.DefaultTransport = bedrockDiscoveryTestTransport(func(r *http.Request) (*http.Response, error) { | |||
| 25 | + | if !strings.Contains(r.Header.Get("Authorization"), "Credential=AKID/") { | |||
| 26 | + | t.Errorf("discovery request was not signed with SigV4") | |||
| 27 | + | } | |||
| 28 | + | body := `{"modelSummaries":[{"modelId":"test.new-model","outputModalities":["TEXT"]}]}` | |||
| 29 | + | if r.URL.Path == "/inference-profiles" { | |||
| 30 | + | body = `{"inferenceProfileSummaries":[]}` | |||
| 31 | + | } | |||
| 32 | + | return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(body)), Header: make(http.Header)}, nil | |||
| 33 | + | }) | |||
| 34 | + | models, err := DiscoverBedrock(context.Background(), "us-east-1") | |||
| 35 | + | if err != nil || len(models) != 1 || models[0].ID != "test.new-model" { | |||
| 36 | + | t.Fatalf("discovered models = %v, err = %v", models, err) | |||
| 37 | + | } | |||
| 38 | + | if bearer, sigv4 := resolveBedrockAuth("bearer"); bearer != "bearer" || sigv4 != nil { | |||
| 39 | + | t.Fatal("inference must continue to prefer the bearer token") | |||
| 40 | + | } | |||
| 41 | + | } | |||
| 42 | + | ||||
| 11 | func TestBedrockEmitsText(t *testing.T) { | 43 | func TestBedrockEmitsText(t *testing.T) { | ||
| 12 | if !bedrockEmitsText(nil) { | 44 | if !bedrockEmitsText(nil) { | ||
| 13 | t.Error("nil modalities should default to text") | 45 | t.Error("nil modalities should default to text") | ||
| @@ -55,7 +87,7 @@ func TestResolveBedrockAuthBearerVsSigV4(t *testing.T) { | |||||
| 55 | // nextToken. Embedding/image-only models are filtered out; profile IDs | 87 | // nextToken. Embedding/image-only models are filtered out; profile IDs | ||
| 56 | // are included; duplicates are de-duplicated. | 88 | // are included; duplicates are de-duplicated. | ||
| 57 | func TestBedrockListModelIDs(t *testing.T) { | 89 | func TestBedrockListModelIDs(t *testing.T) { | ||
| 58 | - | page1 := `{"inferenceProfileSummaries":[{"inferenceProfileId":"us.anthropic.claude-opus-5-5"}],"nextToken":"NEXT"}` | 90 | + | page1 := `{"inferenceProfileSummaries":[{"inferenceProfileId":"us.anthropic.claude-opus-5-5"}],"nextToken":"N+E/T= &"}` |
| 59 | page2 := `{"inferenceProfileSummaries":[{"inferenceProfileId":"eu.anthropic.claude-sonnet-5"}]}` | 91 | page2 := `{"inferenceProfileSummaries":[{"inferenceProfileId":"eu.anthropic.claude-sonnet-5"}]}` | ||
| 60 | foundation := `{"modelSummaries":[ | 92 | foundation := `{"modelSummaries":[ | ||
| 61 | {"modelId":"anthropic.claude-opus-5-5","outputModalities":["TEXT"]}, | 93 | {"modelId":"anthropic.claude-opus-5-5","outputModalities":["TEXT"]}, | ||
| @@ -72,10 +104,14 @@ func TestBedrockListModelIDs(t *testing.T) { | |||||
| 72 | w.Write([]byte(foundation)) | 104 | w.Write([]byte(foundation)) | ||
| 73 | case strings.Contains(r.URL.Path, "/inference-profiles"): | 105 | case strings.Contains(r.URL.Path, "/inference-profiles"): | ||
| 74 | ipHits++ | 106 | ipHits++ | ||
| 75 | - | if r.URL.Query().Get("nextToken") == "NEXT" { | 107 | + | switch token := r.URL.Query().Get("nextToken"); token { |
| 76 | - | w.Write([]byte(page2)) | 108 | + | case "": |
| 77 | - | } else { | |||
| 78 | w.Write([]byte(page1)) | 109 | w.Write([]byte(page1)) | ||
| 110 | + | case "N+E/T= &": | |||
| 111 | + | w.Write([]byte(page2)) | |||
| 112 | + | default: | |||
| 113 | + | t.Errorf("unexpected pagination token %q", token) | |||
| 114 | + | w.WriteHeader(http.StatusBadRequest) | |||
| 79 | } | 115 | } | ||
| 80 | default: | 116 | default: | ||
| 81 | w.WriteHeader(http.StatusNotFound) | 117 | w.WriteHeader(http.StatusNotFound) | ||