Recover once from silent normal-turn endings (#195)
* feat(core): recover once from silent normal-turn endings Preserve Responses text phases (commentary/final_answer) through streaming, replay, and sessions. When a successful normal stop carries no final answer, append one bounded continuation in the same turn; a second incomplete ending returns ErrIncompleteTurn instead of silent success. Surface recovery in all modes and refine task execution guidance. * fix(core): treat tool execution refusals as denials for turn recovery Permission, allowlist, jail, and admission refusals returned from tool Execute bypassed the denial latch, so a later incomplete normal stop gained a recovery prompt urging more actions. Mark typed ToolDeniedError refusals at the execution boundary; ordinary tool failures keep recovery eligibility. Harden the goal single-owner test assertion. * fix(tests): drop unused mutex in turn recovery script client The script client uses atomics for call counting and never mutates shared state, so the mutex tripped the ubuntu golangci-lint gate.
- commit
- a939a2a3dfafe4d6d3c99461c1ab177ccd8c7808
- author
- bnema · · 2026-09-10 03:59 UTC
- committer
- GitHub · · 2026-09-10 03:59 UTC
- parent
- 5f1c8e9
- signature
- GPG Unverified
B5690EEEBB952194The key is unknown - refs
- tag: v0.34.0
25 files · +1752 -51
M README.md +2 -0
| @@ -294,6 +294,8 @@ The resident manager enforces global fair concurrency (six turns by default), op | |||
| 294 | 294 | ||
| 295 | 295 | When an initial print, stream, or JSON request exceeds the provider context window, zut compacts the existing transcript and continues the already-appended prompt once. Print still writes only the recovered final text, stream keeps assistant text on stdout and tool diagnostics on stderr, and JSON suppresses the recoverable first terminal error so stdout remains JSONL for the successful turn. A compaction failure or a second context-window error remains terminal; this is not a general autonomous-follow-up policy. The same one-shot recovery applies to resident child turns. It does not apply to RPC, SDK, bot, or Telegram requests. | |
| 296 | 296 | ||
| 297 | + | When a normal turn ends without a final answer (empty output, or commentary with no result), zut appends one bounded continuation and keeps working in the same turn. Interactive shows `Continuing: no final answer received.`; print writes only the recovered answer to stdout; stream keeps assistant text on stdout and the notice on stderr; JSON and RPC emit a `turn_recovery` event with `reason` (`missing_answer` or `commentary_only`) and `attempt` (always 1). If the continuation still produces no final answer, the turn ends with a visible incomplete-turn error instead of silent success. This does not guarantee task completion and does not override permissions, confirmations, or denials. | |
| 298 | + | ||
| 297 | 299 | ## zutfile agents | |
| 298 | 300 | ||
| 299 | 301 | A zutfile packages an agent's instructions, skills, requirements, and enforced tool permissions as a shareable agent. Run one from a local directory, a packed `.zut` artifact, a short name, or directly from a public GitHub repository: | |
| @@ -294,6 +294,8 @@ The resident manager enforces global fair concurrency (six turns by default), op | |||||
| 294 | 294 | ||||
| 295 | When an initial print, stream, or JSON request exceeds the provider context window, zut compacts the existing transcript and continues the already-appended prompt once. Print still writes only the recovered final text, stream keeps assistant text on stdout and tool diagnostics on stderr, and JSON suppresses the recoverable first terminal error so stdout remains JSONL for the successful turn. A compaction failure or a second context-window error remains terminal; this is not a general autonomous-follow-up policy. The same one-shot recovery applies to resident child turns. It does not apply to RPC, SDK, bot, or Telegram requests. | 295 | When an initial print, stream, or JSON request exceeds the provider context window, zut compacts the existing transcript and continues the already-appended prompt once. Print still writes only the recovered final text, stream keeps assistant text on stdout and tool diagnostics on stderr, and JSON suppresses the recoverable first terminal error so stdout remains JSONL for the successful turn. A compaction failure or a second context-window error remains terminal; this is not a general autonomous-follow-up policy. The same one-shot recovery applies to resident child turns. It does not apply to RPC, SDK, bot, or Telegram requests. | ||
| 296 | 296 | ||||
| 297 | + | When a normal turn ends without a final answer (empty output, or commentary with no result), zut appends one bounded continuation and keeps working in the same turn. Interactive shows `Continuing: no final answer received.`; print writes only the recovered answer to stdout; stream keeps assistant text on stdout and the notice on stderr; JSON and RPC emit a `turn_recovery` event with `reason` (`missing_answer` or `commentary_only`) and `attempt` (always 1). If the continuation still produces no final answer, the turn ends with a visible incomplete-turn error instead of silent success. This does not guarantee task completion and does not override permissions, confirmations, or denials. | |||
| 298 | + | ||||
| 297 | ## zutfile agents | 299 | ## zutfile agents | ||
| 298 | 300 | ||||
| 299 | A zutfile packages an agent's instructions, skills, requirements, and enforced tool permissions as a shareable agent. Run one from a local directory, a packed `.zut` artifact, a short name, or directly from a public GitHub repository: | 301 | A zutfile packages an agent's instructions, skills, requirements, and enforced tool permissions as a shareable agent. Run one from a local directory, a packed `.zut` artifact, a short name, or directly from a public GitHub repository: | ||
M docs/rpc.md +1 -0
| @@ -209,6 +209,7 @@ Stream notifications during a `prompt` or `compact`. None carry an `id`. | |||
| 209 | 209 | | `assistant_message` | `content`, `time` | Final assistant message after the model turn ends | | |
| 210 | 210 | | `usage` | `input`, `output`, `reasoning`, `cache_read`, `cache_write`, `cache_measured_prompt`, `cache_measured_read`, `cost_usd`, `cumulative` | Per-turn + cumulative tokens / cost. Cache ratios use only the measured cache fields; `reasoning` is null when unavailable. | | |
| 211 | 211 | | `turn_end` | `stop`, optional `error` | One model call finished. `stop` is `end`, `continue`, `tool_use`, `length`, `error`, or `aborted`. `continue` ends one inference, not the logical task: the agent issues another model call in the same turn | | |
| 212 | + | | `turn_recovery` | `reason`, `attempt` | The turn ended without a final answer, so the agent appended one bounded continuation. `reason` is `missing_answer` or `commentary_only`; `attempt` is always 1 | | |
| 212 | 213 | | `done` | (none) | The whole prompt/compact completed (success or error) | | |
| 213 | 214 | | `error` | `message` | Non-fatal error message | | |
| 214 | 215 | | `compact_done` | `summary` | Compaction finished, summary text included | | |
| @@ -209,6 +209,7 @@ Stream notifications during a `prompt` or `compact`. None carry an `id`. | |||||
| 209 | | `assistant_message` | `content`, `time` | Final assistant message after the model turn ends | | 209 | | `assistant_message` | `content`, `time` | Final assistant message after the model turn ends | | ||
| 210 | | `usage` | `input`, `output`, `reasoning`, `cache_read`, `cache_write`, `cache_measured_prompt`, `cache_measured_read`, `cost_usd`, `cumulative` | Per-turn + cumulative tokens / cost. Cache ratios use only the measured cache fields; `reasoning` is null when unavailable. | | 210 | | `usage` | `input`, `output`, `reasoning`, `cache_read`, `cache_write`, `cache_measured_prompt`, `cache_measured_read`, `cost_usd`, `cumulative` | Per-turn + cumulative tokens / cost. Cache ratios use only the measured cache fields; `reasoning` is null when unavailable. | | ||
| 211 | | `turn_end` | `stop`, optional `error` | One model call finished. `stop` is `end`, `continue`, `tool_use`, `length`, `error`, or `aborted`. `continue` ends one inference, not the logical task: the agent issues another model call in the same turn | | 211 | | `turn_end` | `stop`, optional `error` | One model call finished. `stop` is `end`, `continue`, `tool_use`, `length`, `error`, or `aborted`. `continue` ends one inference, not the logical task: the agent issues another model call in the same turn | | ||
| 212 | + | | `turn_recovery` | `reason`, `attempt` | The turn ended without a final answer, so the agent appended one bounded continuation. `reason` is `missing_answer` or `commentary_only`; `attempt` is always 1 | | |||
| 212 | | `done` | (none) | The whole prompt/compact completed (success or error) | | 213 | | `done` | (none) | The whole prompt/compact completed (success or error) | | ||
| 213 | | `error` | `message` | Non-fatal error message | | 214 | | `error` | `message` | Non-fatal error message | | ||
| 214 | | `compact_done` | `summary` | Compaction finished, summary text included | | 215 | | `compact_done` | `summary` | Compaction finished, summary text included | | ||
M packages/agent/modes/compact_queue_test.go +11 -2
| @@ -100,7 +100,10 @@ func (c *goalLengthClient) Stream(_ context.Context, req provider.Request) (<-ch | |||
| 100 | 100 | } | |
| 101 | 101 | case 2: | |
| 102 | 102 | out <- provider.EventTextDelta{Delta: "durable summary"} | |
| 103 | - | out <- provider.EventDone{Stop: provider.StopEnd} | |
| 103 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ | |
| 104 | + | Role: provider.RoleAssistant, | |
| 105 | + | Content: []provider.Content{provider.TextBlock{Text: "durable summary"}}, | |
| 106 | + | }} | |
| 104 | 107 | default: | |
| 105 | 108 | if c.onThird != nil { | |
| 106 | 109 | c.onThird() | |
| @@ -532,7 +535,10 @@ func (c *thresholdAutoCompactClient) Stream(ctx context.Context, req provider.Re | |||
| 532 | 535 | out <- provider.EventDone{Stop: provider.StopAborted, Err: ctx.Err()} | |
| 533 | 536 | case <-c.releaseCompaction: | |
| 534 | 537 | out <- provider.EventTextDelta{Delta: "summary"} | |
| 535 | - | out <- provider.EventDone{Stop: provider.StopEnd} | |
| 538 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ | |
| 539 | + | Role: provider.RoleAssistant, | |
| 540 | + | Content: []provider.Content{provider.TextBlock{Text: "summary"}}, | |
| 541 | + | }} | |
| 536 | 542 | } | |
| 537 | 543 | default: | |
| 538 | 544 | c.mu.Lock() | |
| @@ -903,6 +909,9 @@ func TestThresholdAutoCompactionContinuesMostRecentIntent(t *testing.T) { | |||
| 903 | 909 | compactionStarted: make(chan struct{}), | |
| 904 | 910 | releaseCompaction: make(chan struct{}), | |
| 905 | 911 | followUpRequest: make(chan provider.Request, 1), | |
| 912 | + | firstContent: []provider.Content{ | |
| 913 | + | provider.TextBlock{Text: "Next I will inspect the remaining call sites."}, | |
| 914 | + | }, | |
| 906 | 915 | } | |
| 907 | 916 | agent := core.NewAgent(client, "test-model", "", nil) | |
| 908 | 917 | threshold := 70 | |
| @@ -100,7 +100,10 @@ func (c *goalLengthClient) Stream(_ context.Context, req provider.Request) (<-ch | |||||
| 100 | } | 100 | } | ||
| 101 | case 2: | 101 | case 2: | ||
| 102 | out <- provider.EventTextDelta{Delta: "durable summary"} | 102 | out <- provider.EventTextDelta{Delta: "durable summary"} | ||
| 103 | - | out <- provider.EventDone{Stop: provider.StopEnd} | 103 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ |
| 104 | + | Role: provider.RoleAssistant, | |||
| 105 | + | Content: []provider.Content{provider.TextBlock{Text: "durable summary"}}, | |||
| 106 | + | }} | |||
| 104 | default: | 107 | default: | ||
| 105 | if c.onThird != nil { | 108 | if c.onThird != nil { | ||
| 106 | c.onThird() | 109 | c.onThird() | ||
| @@ -532,7 +535,10 @@ func (c *thresholdAutoCompactClient) Stream(ctx context.Context, req provider.Re | |||||
| 532 | out <- provider.EventDone{Stop: provider.StopAborted, Err: ctx.Err()} | 535 | out <- provider.EventDone{Stop: provider.StopAborted, Err: ctx.Err()} | ||
| 533 | case <-c.releaseCompaction: | 536 | case <-c.releaseCompaction: | ||
| 534 | out <- provider.EventTextDelta{Delta: "summary"} | 537 | out <- provider.EventTextDelta{Delta: "summary"} | ||
| 535 | - | out <- provider.EventDone{Stop: provider.StopEnd} | 538 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ |
| 539 | + | Role: provider.RoleAssistant, | |||
| 540 | + | Content: []provider.Content{provider.TextBlock{Text: "summary"}}, | |||
| 541 | + | }} | |||
| 536 | } | 542 | } | ||
| 537 | default: | 543 | default: | ||
| 538 | c.mu.Lock() | 544 | c.mu.Lock() | ||
| @@ -903,6 +909,9 @@ func TestThresholdAutoCompactionContinuesMostRecentIntent(t *testing.T) { | |||||
| 903 | compactionStarted: make(chan struct{}), | 909 | compactionStarted: make(chan struct{}), | ||
| 904 | releaseCompaction: make(chan struct{}), | 910 | releaseCompaction: make(chan struct{}), | ||
| 905 | followUpRequest: make(chan provider.Request, 1), | 911 | followUpRequest: make(chan provider.Request, 1), | ||
| 912 | + | firstContent: []provider.Content{ | |||
| 913 | + | provider.TextBlock{Text: "Next I will inspect the remaining call sites."}, | |||
| 914 | + | }, | |||
| 906 | } | 915 | } | ||
| 907 | agent := core.NewAgent(client, "test-model", "", nil) | 916 | agent := core.NewAgent(client, "test-model", "", nil) | ||
| 908 | threshold := 70 | 917 | threshold := 70 | ||
M packages/agent/modes/interactive_events.go +6 -0
| @@ -194,6 +194,12 @@ func (i *Interactive) handleEvent(ev core.AgentEvent) { | |||
| 194 | 194 | if contextUsed := e.Usage.InputTokens + e.Usage.CacheReadTokens + e.Usage.CacheWriteTokens; contextUsed > 0 { | |
| 195 | 195 | i.lastCtxInput = contextUsed | |
| 196 | 196 | } | |
| 197 | + | case core.EvTurnRecovery: | |
| 198 | + | // One bounded core continuation fired because the turn ended | |
| 199 | + | // without a final answer. Surface it once so the extended work | |
| 200 | + | // does not read as a stall. The inner continuation is owned by | |
| 201 | + | // core, so no goal or scheduler follow-up is started here. | |
| 202 | + | i.extNotes = append(i.extNotes, " "+i.cfg.Theme.FGColor(i.cfg.Theme.Tool, "Continuing: no final answer received.")) | |
| 197 | 203 | case core.EvTurnEnd: | |
| 198 | 204 | if e.Stop == provider.StopAborted { | |
| 199 | 205 | i.resetStreamingStateLocked() | |
| @@ -194,6 +194,12 @@ func (i *Interactive) handleEvent(ev core.AgentEvent) { | |||||
| 194 | if contextUsed := e.Usage.InputTokens + e.Usage.CacheReadTokens + e.Usage.CacheWriteTokens; contextUsed > 0 { | 194 | if contextUsed := e.Usage.InputTokens + e.Usage.CacheReadTokens + e.Usage.CacheWriteTokens; contextUsed > 0 { | ||
| 195 | i.lastCtxInput = contextUsed | 195 | i.lastCtxInput = contextUsed | ||
| 196 | } | 196 | } | ||
| 197 | + | case core.EvTurnRecovery: | |||
| 198 | + | // One bounded core continuation fired because the turn ended | |||
| 199 | + | // without a final answer. Surface it once so the extended work | |||
| 200 | + | // does not read as a stall. The inner continuation is owned by | |||
| 201 | + | // core, so no goal or scheduler follow-up is started here. | |||
| 202 | + | i.extNotes = append(i.extNotes, " "+i.cfg.Theme.FGColor(i.cfg.Theme.Tool, "Continuing: no final answer received.")) | |||
| 197 | case core.EvTurnEnd: | 203 | case core.EvTurnEnd: | ||
| 198 | if e.Stop == provider.StopAborted { | 204 | if e.Stop == provider.StopAborted { | ||
| 199 | i.resetStreamingStateLocked() | 205 | i.resetStreamingStateLocked() | ||
M packages/agent/modes/json.go +3 -0
| @@ -139,6 +139,9 @@ func EventToJSON(ev core.AgentEvent) map[string]any { | |||
| 139 | 139 | "cache_measured_read": e.Cumulative.CacheMeasuredReadTokens, | |
| 140 | 140 | "cost_usd": e.Cumulative.CostUSD, | |
| 141 | 141 | } | |
| 142 | + | case core.EvTurnRecovery: | |
| 143 | + | m["reason"] = e.Reason | |
| 144 | + | m["attempt"] = e.Attempt | |
| 142 | 145 | case core.EvTurnEnd: | |
| 143 | 146 | m["stop"] = string(e.Stop) | |
| 144 | 147 | if e.Err != nil { | |
| @@ -139,6 +139,9 @@ func EventToJSON(ev core.AgentEvent) map[string]any { | |||||
| 139 | "cache_measured_read": e.Cumulative.CacheMeasuredReadTokens, | 139 | "cache_measured_read": e.Cumulative.CacheMeasuredReadTokens, | ||
| 140 | "cost_usd": e.Cumulative.CostUSD, | 140 | "cost_usd": e.Cumulative.CostUSD, | ||
| 141 | } | 141 | } | ||
| 142 | + | case core.EvTurnRecovery: | |||
| 143 | + | m["reason"] = e.Reason | |||
| 144 | + | m["attempt"] = e.Attempt | |||
| 142 | case core.EvTurnEnd: | 145 | case core.EvTurnEnd: | ||
| 143 | m["stop"] = string(e.Stop) | 146 | m["stop"] = string(e.Stop) | ||
| 144 | if e.Err != nil { | 147 | if e.Err != nil { | ||
M packages/agent/modes/print.go +7 -0
| @@ -5,6 +5,7 @@ import ( | |||
| 5 | 5 | "context" | |
| 6 | 6 | "fmt" | |
| 7 | 7 | "io" | |
| 8 | + | "os" | |
| 8 | 9 | "strings" | |
| 9 | 10 | ||
| 10 | 11 | "github.com/bnema/zut/packages/core" | |
| @@ -52,6 +53,12 @@ func RunPrintWithContextRecovery(ctx context.Context, ag *core.Agent, prompt str | |||
| 52 | 53 | usage = e.Usage | |
| 53 | 54 | haveUsage = true | |
| 54 | 55 | } | |
| 56 | + | case core.EvTurnRecovery: | |
| 57 | + | // The core owns the single bounded continuation; surface it | |
| 58 | + | // on stderr so stdout keeps only the final answer. An | |
| 59 | + | // exhausted allowance returns ErrIncompleteTurn from Prompt | |
| 60 | + | // and never reaches here as success. | |
| 61 | + | fmt.Fprintln(os.Stderr, "Continuing: no final answer received.") | |
| 55 | 62 | case core.EvTurnEnd: | |
| 56 | 63 | if e.Err != nil { | |
| 57 | 64 | runErr = e.Err | |
| @@ -5,6 +5,7 @@ import ( | |||||
| 5 | "context" | 5 | "context" | ||
| 6 | "fmt" | 6 | "fmt" | ||
| 7 | "io" | 7 | "io" | ||
| 8 | + | "os" | |||
| 8 | "strings" | 9 | "strings" | ||
| 9 | 10 | ||||
| 10 | "github.com/bnema/zut/packages/core" | 11 | "github.com/bnema/zut/packages/core" | ||
| @@ -52,6 +53,12 @@ func RunPrintWithContextRecovery(ctx context.Context, ag *core.Agent, prompt str | |||||
| 52 | usage = e.Usage | 53 | usage = e.Usage | ||
| 53 | haveUsage = true | 54 | haveUsage = true | ||
| 54 | } | 55 | } | ||
| 56 | + | case core.EvTurnRecovery: | |||
| 57 | + | // The core owns the single bounded continuation; surface it | |||
| 58 | + | // on stderr so stdout keeps only the final answer. An | |||
| 59 | + | // exhausted allowance returns ErrIncompleteTurn from Prompt | |||
| 60 | + | // and never reaches here as success. | |||
| 61 | + | fmt.Fprintln(os.Stderr, "Continuing: no final answer received.") | |||
| 55 | case core.EvTurnEnd: | 62 | case core.EvTurnEnd: | ||
| 56 | if e.Err != nil { | 63 | if e.Err != nil { | ||
| 57 | runErr = e.Err | 64 | runErr = e.Err | ||
M packages/agent/modes/stream.go +4 -0
| @@ -99,6 +99,10 @@ func RunStreamWithContextRecovery(ctx context.Context, ag *core.Agent, prompt st | |||
| 99 | 99 | if e.Result.IsError { | |
| 100 | 100 | fmt.Fprintln(diag, "[tool error]") | |
| 101 | 101 | } | |
| 102 | + | case core.EvTurnRecovery: | |
| 103 | + | // Assistant text stays on stdout; the recovery notice is a | |
| 104 | + | // host diagnostic and belongs on the diag channel. | |
| 105 | + | fmt.Fprintln(diag, "Continuing: no final answer received.") | |
| 102 | 106 | case core.EvTurnEnd: | |
| 103 | 107 | if e.Err != nil { | |
| 104 | 108 | runErr = e.Err | |
| @@ -99,6 +99,10 @@ func RunStreamWithContextRecovery(ctx context.Context, ag *core.Agent, prompt st | |||||
| 99 | if e.Result.IsError { | 99 | if e.Result.IsError { | ||
| 100 | fmt.Fprintln(diag, "[tool error]") | 100 | fmt.Fprintln(diag, "[tool error]") | ||
| 101 | } | 101 | } | ||
| 102 | + | case core.EvTurnRecovery: | |||
| 103 | + | // Assistant text stays on stdout; the recovery notice is a | |||
| 104 | + | // host diagnostic and belongs on the diag channel. | |||
| 105 | + | fmt.Fprintln(diag, "Continuing: no final answer received.") | |||
| 102 | case core.EvTurnEnd: | 106 | case core.EvTurnEnd: | ||
| 103 | if e.Err != nil { | 107 | if e.Err != nil { | ||
| 104 | runErr = e.Err | 108 | runErr = e.Err | ||
A packages/agent/modes/turn_recovery_test.go +279 -0
| @@ -0,0 +1,279 @@ | |||
| 1 | + | package modes | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "bytes" | |
| 5 | + | "context" | |
| 6 | + | "encoding/json" | |
| 7 | + | "errors" | |
| 8 | + | "strings" | |
| 9 | + | "sync" | |
| 10 | + | "testing" | |
| 11 | + | "time" | |
| 12 | + | ||
| 13 | + | "github.com/bnema/zut/packages/core" | |
| 14 | + | "github.com/bnema/zut/packages/provider" | |
| 15 | + | ) | |
| 16 | + | ||
| 17 | + | // turnRecoveryModeClient replays one assistant message per Stream call so | |
| 18 | + | // mode tests can drive the core recovery allowance without timing sleeps. | |
| 19 | + | type turnRecoveryModeClient struct { | |
| 20 | + | mu sync.Mutex | |
| 21 | + | calls int | |
| 22 | + | turns [][]provider.Content | |
| 23 | + | } | |
| 24 | + | ||
| 25 | + | func (c *turnRecoveryModeClient) Name() string { return "turn-recovery-mode-test" } | |
| 26 | + | ||
| 27 | + | func (c *turnRecoveryModeClient) Stream(context.Context, provider.Request) (<-chan provider.Event, error) { | |
| 28 | + | c.mu.Lock() | |
| 29 | + | call := c.calls | |
| 30 | + | c.calls++ | |
| 31 | + | c.mu.Unlock() | |
| 32 | + | var content []provider.Content | |
| 33 | + | if call < len(c.turns) { | |
| 34 | + | content = c.turns[call] | |
| 35 | + | } | |
| 36 | + | out := make(chan provider.Event, 4) | |
| 37 | + | out <- provider.EventStart{Provider: c.Name(), Model: "test-model"} | |
| 38 | + | out <- provider.EventUsage{Usage: provider.Usage{InputTokens: 3, OutputTokens: 5}} | |
| 39 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ | |
| 40 | + | Role: provider.RoleAssistant, | |
| 41 | + | Content: content, | |
| 42 | + | Time: time.Now(), | |
| 43 | + | }} | |
| 44 | + | close(out) | |
| 45 | + | return out, nil | |
| 46 | + | } | |
| 47 | + | ||
| 48 | + | func (c *turnRecoveryModeClient) Calls() int { | |
| 49 | + | c.mu.Lock() | |
| 50 | + | defer c.mu.Unlock() | |
| 51 | + | return c.calls | |
| 52 | + | } | |
| 53 | + | ||
| 54 | + | func commentaryContent(text string) []provider.Content { | |
| 55 | + | return []provider.Content{provider.TextBlock{Text: text, Phase: provider.TextPhaseCommentary}} | |
| 56 | + | } | |
| 57 | + | ||
| 58 | + | func answerContent(text string) []provider.Content { | |
| 59 | + | return []provider.Content{provider.TextBlock{Text: text}} | |
| 60 | + | } | |
| 61 | + | ||
| 62 | + | func TestTurnRecoveryModeRendersOneRecoveryStatus(t *testing.T) { | |
| 63 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |
| 64 | + | commentaryContent("still working on it"), | |
| 65 | + | answerContent("finished"), | |
| 66 | + | }} | |
| 67 | + | ag := core.NewAgent(client, "test-model", "", nil) | |
| 68 | + | i := newNotesTestInteractive() | |
| 69 | + | i.cfg.Agent = ag | |
| 70 | + | i.agent = ag | |
| 71 | + | ||
| 72 | + | i.handleEvent(core.EvTurnRecovery{Reason: core.TurnRecoveryReasonCommentaryOnly, Attempt: 1}) | |
| 73 | + | ||
| 74 | + | if len(i.extNotes) != 1 { | |
| 75 | + | t.Fatalf("extNotes = %d, want exactly one recovery status", len(i.extNotes)) | |
| 76 | + | } | |
| 77 | + | if got := stripANSIBytes(i.extNotes[0]); !strings.Contains(got, "Continuing: no final answer received.") { | |
| 78 | + | t.Fatalf("recovery status = %q, want exact recovery line", got) | |
| 79 | + | } | |
| 80 | + | } | |
| 81 | + | ||
| 82 | + | func TestTurnRecoveryPrintSurfacesRecoveryAndCompletes(t *testing.T) { | |
| 83 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |
| 84 | + | commentaryContent("still working on it"), | |
| 85 | + | answerContent("finished"), | |
| 86 | + | }} | |
| 87 | + | ag := core.NewAgent(client, "test-model", "", nil) | |
| 88 | + | var out bytes.Buffer | |
| 89 | + | usage, err := RunPrint(context.Background(), ag, "work", nil, &out) | |
| 90 | + | if err != nil { | |
| 91 | + | t.Fatalf("RunPrint error = %v", err) | |
| 92 | + | } | |
| 93 | + | _ = usage | |
| 94 | + | if out.String() != "finished\n" { | |
| 95 | + | t.Fatalf("print output = %q, want recovered final answer", out.String()) | |
| 96 | + | } | |
| 97 | + | if client.Calls() != 2 { | |
| 98 | + | t.Fatalf("provider calls = %d, want 2 (one bounded recovery)", client.Calls()) | |
| 99 | + | } | |
| 100 | + | } | |
| 101 | + | ||
| 102 | + | func TestTurnRecoveryStreamKeepsTranscriptOnStdout(t *testing.T) { | |
| 103 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |
| 104 | + | nil, | |
| 105 | + | answerContent("finished"), | |
| 106 | + | }} | |
| 107 | + | ag := core.NewAgent(client, "test-model", "", nil) | |
| 108 | + | var out, diag bytes.Buffer | |
| 109 | + | if err := RunStreamWithDiag(context.Background(), ag, "work", nil, &out, &diag); err != nil { | |
| 110 | + | t.Fatalf("RunStreamWithDiag error = %v", err) | |
| 111 | + | } | |
| 112 | + | if !strings.Contains(out.String(), "finished") { | |
| 113 | + | t.Fatalf("stdout = %q, want recovered answer text", out.String()) | |
| 114 | + | } | |
| 115 | + | if !strings.Contains(diag.String(), "Continuing: no final answer received.") { | |
| 116 | + | t.Fatalf("diag = %q, want one recovery status", diag.String()) | |
| 117 | + | } | |
| 118 | + | if strings.Contains(out.String(), "Continuing: no final answer received.") { | |
| 119 | + | t.Fatalf("stdout = %q, recovery status must not pollute the transcript", out.String()) | |
| 120 | + | } | |
| 121 | + | } | |
| 122 | + | ||
| 123 | + | func TestTurnRecoveryExhaustionSurfacesIncompleteError(t *testing.T) { | |
| 124 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |
| 125 | + | nil, | |
| 126 | + | nil, | |
| 127 | + | }} | |
| 128 | + | ag := core.NewAgent(client, "test-model", "", nil) | |
| 129 | + | var out bytes.Buffer | |
| 130 | + | _, err := RunPrint(context.Background(), ag, "work", nil, &out) | |
| 131 | + | if !errors.Is(err, core.ErrIncompleteTurn) { | |
| 132 | + | t.Fatalf("RunPrint error = %v, want ErrIncompleteTurn via errors.Is", err) | |
| 133 | + | } | |
| 134 | + | if out.String() != "" { | |
| 135 | + | t.Fatalf("print output = %q, exhaustion must not report success text", out.String()) | |
| 136 | + | } | |
| 137 | + | if client.Calls() != 2 { | |
| 138 | + | t.Fatalf("provider calls = %d, want exactly 2 (no unbounded retry)", client.Calls()) | |
| 139 | + | } | |
| 140 | + | } | |
| 141 | + | ||
| 142 | + | func TestTurnRecoveryGoalDoesNotScheduleDuplicateWork(t *testing.T) { | |
| 143 | + | var mu sync.Mutex | |
| 144 | + | goal := &core.SessionGoal{ID: "goal-1", Status: core.GoalActive, Objective: "finish the work"} | |
| 145 | + | current := func() *core.SessionGoal { | |
| 146 | + | mu.Lock() | |
| 147 | + | defer mu.Unlock() | |
| 148 | + | cp := *goal | |
| 149 | + | return &cp | |
| 150 | + | } | |
| 151 | + | persist := func(next *core.SessionGoal) error { | |
| 152 | + | mu.Lock() | |
| 153 | + | goal = next | |
| 154 | + | mu.Unlock() | |
| 155 | + | return nil | |
| 156 | + | } | |
| 157 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |
| 158 | + | commentaryContent("still working on it"), | |
| 159 | + | answerContent("finished"), | |
| 160 | + | }} | |
| 161 | + | ag := core.NewAgent(client, "test-model", "", goalToolRegistry()) | |
| 162 | + | i := NewInteractive(InteractiveConfig{ | |
| 163 | + | Agent: ag, | |
| 164 | + | CurrentGoal: current, | |
| 165 | + | PersistGoal: persist, | |
| 166 | + | PersistGoalRuntime: persist, | |
| 167 | + | }) | |
| 168 | + | i.runCtx = context.Background() | |
| 169 | + | run, err := i.startGoalRun(current()) | |
| 170 | + | if err != nil { | |
| 171 | + | t.Fatalf("startGoalRun error = %v", err) | |
| 172 | + | } | |
| 173 | + | if run == nil { | |
| 174 | + | t.Fatal("startGoalRun did not start a run") | |
| 175 | + | } | |
| 176 | + | var events []core.AgentEvent | |
| 177 | + | if err := ag.Prompt(context.Background(), "work", nil, func(ev core.AgentEvent) { | |
| 178 | + | events = append(events, ev) | |
| 179 | + | i.observeGoalRun(ev) | |
| 180 | + | i.handleEvent(ev) | |
| 181 | + | }); err != nil { | |
| 182 | + | t.Fatalf("Prompt error = %v", err) | |
| 183 | + | } | |
| 184 | + | recoveries := 0 | |
| 185 | + | for _, ev := range events { | |
| 186 | + | if _, ok := ev.(core.EvTurnRecovery); ok { | |
| 187 | + | recoveries++ | |
| 188 | + | } | |
| 189 | + | } | |
| 190 | + | if recoveries != 1 { | |
| 191 | + | t.Fatalf("recovery events = %d, want exactly one inner continuation", recoveries) | |
| 192 | + | } | |
| 193 | + | // The inner continuation must not start a second goal run: the run | |
| 194 | + | // started above is still the single owner, and finishing it once must | |
| 195 | + | // record accounting without scheduling another continuation here. | |
| 196 | + | i.mu.Lock() | |
| 197 | + | owner := i.goalRun | |
| 198 | + | i.mu.Unlock() | |
| 199 | + | if owner != run { | |
| 200 | + | t.Fatal("inner continuation replaced the goal run: want the single owner started above") | |
| 201 | + | } | |
| 202 | + | if owner.goalID != "goal-1" { | |
| 203 | + | t.Fatalf("goal run goalID = %q, want %q", owner.goalID, "goal-1") | |
| 204 | + | } | |
| 205 | + | if !i.finishGoalRun(false) { | |
| 206 | + | t.Fatalf("finishGoalRun = false, want accounting for the single owner run") | |
| 207 | + | } | |
| 208 | + | mu.Lock() | |
| 209 | + | used := goal.TokensUsed | |
| 210 | + | mu.Unlock() | |
| 211 | + | if used == 0 { | |
| 212 | + | t.Fatalf("goal tokens used = 0, want usage recorded once for the single run") | |
| 213 | + | } | |
| 214 | + | } | |
| 215 | + | ||
| 216 | + | func TestTurnRecoveryExhaustionKeepsGoalBlocked(t *testing.T) { | |
| 217 | + | var mu sync.Mutex | |
| 218 | + | goal := &core.SessionGoal{ID: "goal-1", Status: core.GoalActive, Objective: "finish the work"} | |
| 219 | + | current := func() *core.SessionGoal { | |
| 220 | + | mu.Lock() | |
| 221 | + | defer mu.Unlock() | |
| 222 | + | cp := *goal | |
| 223 | + | return &cp | |
| 224 | + | } | |
| 225 | + | var persisted []*core.SessionGoal | |
| 226 | + | persist := func(next *core.SessionGoal) error { | |
| 227 | + | mu.Lock() | |
| 228 | + | cp := *next | |
| 229 | + | persisted = append(persisted, &cp) | |
| 230 | + | goal = &cp | |
| 231 | + | mu.Unlock() | |
| 232 | + | return nil | |
| 233 | + | } | |
| 234 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{nil, nil}} | |
| 235 | + | ag := core.NewAgent(client, "test-model", "", goalToolRegistry()) | |
| 236 | + | i := NewInteractive(InteractiveConfig{ | |
| 237 | + | Agent: ag, | |
| 238 | + | CurrentGoal: current, | |
| 239 | + | PersistGoal: persist, | |
| 240 | + | PersistGoalRuntime: persist, | |
| 241 | + | }) | |
| 242 | + | err := ag.Prompt(context.Background(), "work", nil, func(core.AgentEvent) {}) | |
| 243 | + | if !errors.Is(err, core.ErrIncompleteTurn) { | |
| 244 | + | t.Fatalf("Prompt error = %v, want ErrIncompleteTurn", err) | |
| 245 | + | } | |
| 246 | + | // Exhaustion follows established blocking semantics: the goal is marked | |
| 247 | + | // blocked with the incomplete-turn cause, never auto-cleared or rerun. | |
| 248 | + | i.updateActiveGoal(core.GoalBlocked, "turn ended with an error: "+core.ErrIncompleteTurn.Error()) | |
| 249 | + | mu.Lock() | |
| 250 | + | status := goal.Status | |
| 251 | + | mu.Unlock() | |
| 252 | + | if status != core.GoalBlocked { | |
| 253 | + | t.Fatalf("goal status = %v, want blocked after exhaustion", status) | |
| 254 | + | } | |
| 255 | + | if len(persisted) == 0 { | |
| 256 | + | t.Fatal("goal exhaustion was not persisted") | |
| 257 | + | } | |
| 258 | + | } | |
| 259 | + | ||
| 260 | + | func TestTurnRecoveryEventJSONEncoding(t *testing.T) { | |
| 261 | + | encoded := EventToJSON(core.EvTurnRecovery{Reason: core.TurnRecoveryReasonMissingAnswer, Attempt: 1}) | |
| 262 | + | raw, err := json.Marshal(encoded) | |
| 263 | + | if err != nil { | |
| 264 | + | t.Fatal(err) | |
| 265 | + | } | |
| 266 | + | var decoded map[string]any | |
| 267 | + | if err := json.Unmarshal(raw, &decoded); err != nil { | |
| 268 | + | t.Fatalf("turn_recovery event is not JSON: %v", err) | |
| 269 | + | } | |
| 270 | + | if decoded["type"] != "turn_recovery" { | |
| 271 | + | t.Fatalf("type = %#v, want turn_recovery", decoded["type"]) | |
| 272 | + | } | |
| 273 | + | if decoded["reason"] != core.TurnRecoveryReasonMissingAnswer { | |
| 274 | + | t.Fatalf("reason = %#v, want missing_answer", decoded["reason"]) | |
| 275 | + | } | |
| 276 | + | if decoded["attempt"] != float64(1) { | |
| 277 | + | t.Fatalf("attempt = %#v, want 1", decoded["attempt"]) | |
| 278 | + | } | |
| 279 | + | } | |
| @@ -0,0 +1,279 @@ | |||||
| 1 | + | package modes | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "bytes" | |||
| 5 | + | "context" | |||
| 6 | + | "encoding/json" | |||
| 7 | + | "errors" | |||
| 8 | + | "strings" | |||
| 9 | + | "sync" | |||
| 10 | + | "testing" | |||
| 11 | + | "time" | |||
| 12 | + | ||||
| 13 | + | "github.com/bnema/zut/packages/core" | |||
| 14 | + | "github.com/bnema/zut/packages/provider" | |||
| 15 | + | ) | |||
| 16 | + | ||||
| 17 | + | // turnRecoveryModeClient replays one assistant message per Stream call so | |||
| 18 | + | // mode tests can drive the core recovery allowance without timing sleeps. | |||
| 19 | + | type turnRecoveryModeClient struct { | |||
| 20 | + | mu sync.Mutex | |||
| 21 | + | calls int | |||
| 22 | + | turns [][]provider.Content | |||
| 23 | + | } | |||
| 24 | + | ||||
| 25 | + | func (c *turnRecoveryModeClient) Name() string { return "turn-recovery-mode-test" } | |||
| 26 | + | ||||
| 27 | + | func (c *turnRecoveryModeClient) Stream(context.Context, provider.Request) (<-chan provider.Event, error) { | |||
| 28 | + | c.mu.Lock() | |||
| 29 | + | call := c.calls | |||
| 30 | + | c.calls++ | |||
| 31 | + | c.mu.Unlock() | |||
| 32 | + | var content []provider.Content | |||
| 33 | + | if call < len(c.turns) { | |||
| 34 | + | content = c.turns[call] | |||
| 35 | + | } | |||
| 36 | + | out := make(chan provider.Event, 4) | |||
| 37 | + | out <- provider.EventStart{Provider: c.Name(), Model: "test-model"} | |||
| 38 | + | out <- provider.EventUsage{Usage: provider.Usage{InputTokens: 3, OutputTokens: 5}} | |||
| 39 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ | |||
| 40 | + | Role: provider.RoleAssistant, | |||
| 41 | + | Content: content, | |||
| 42 | + | Time: time.Now(), | |||
| 43 | + | }} | |||
| 44 | + | close(out) | |||
| 45 | + | return out, nil | |||
| 46 | + | } | |||
| 47 | + | ||||
| 48 | + | func (c *turnRecoveryModeClient) Calls() int { | |||
| 49 | + | c.mu.Lock() | |||
| 50 | + | defer c.mu.Unlock() | |||
| 51 | + | return c.calls | |||
| 52 | + | } | |||
| 53 | + | ||||
| 54 | + | func commentaryContent(text string) []provider.Content { | |||
| 55 | + | return []provider.Content{provider.TextBlock{Text: text, Phase: provider.TextPhaseCommentary}} | |||
| 56 | + | } | |||
| 57 | + | ||||
| 58 | + | func answerContent(text string) []provider.Content { | |||
| 59 | + | return []provider.Content{provider.TextBlock{Text: text}} | |||
| 60 | + | } | |||
| 61 | + | ||||
| 62 | + | func TestTurnRecoveryModeRendersOneRecoveryStatus(t *testing.T) { | |||
| 63 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |||
| 64 | + | commentaryContent("still working on it"), | |||
| 65 | + | answerContent("finished"), | |||
| 66 | + | }} | |||
| 67 | + | ag := core.NewAgent(client, "test-model", "", nil) | |||
| 68 | + | i := newNotesTestInteractive() | |||
| 69 | + | i.cfg.Agent = ag | |||
| 70 | + | i.agent = ag | |||
| 71 | + | ||||
| 72 | + | i.handleEvent(core.EvTurnRecovery{Reason: core.TurnRecoveryReasonCommentaryOnly, Attempt: 1}) | |||
| 73 | + | ||||
| 74 | + | if len(i.extNotes) != 1 { | |||
| 75 | + | t.Fatalf("extNotes = %d, want exactly one recovery status", len(i.extNotes)) | |||
| 76 | + | } | |||
| 77 | + | if got := stripANSIBytes(i.extNotes[0]); !strings.Contains(got, "Continuing: no final answer received.") { | |||
| 78 | + | t.Fatalf("recovery status = %q, want exact recovery line", got) | |||
| 79 | + | } | |||
| 80 | + | } | |||
| 81 | + | ||||
| 82 | + | func TestTurnRecoveryPrintSurfacesRecoveryAndCompletes(t *testing.T) { | |||
| 83 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |||
| 84 | + | commentaryContent("still working on it"), | |||
| 85 | + | answerContent("finished"), | |||
| 86 | + | }} | |||
| 87 | + | ag := core.NewAgent(client, "test-model", "", nil) | |||
| 88 | + | var out bytes.Buffer | |||
| 89 | + | usage, err := RunPrint(context.Background(), ag, "work", nil, &out) | |||
| 90 | + | if err != nil { | |||
| 91 | + | t.Fatalf("RunPrint error = %v", err) | |||
| 92 | + | } | |||
| 93 | + | _ = usage | |||
| 94 | + | if out.String() != "finished\n" { | |||
| 95 | + | t.Fatalf("print output = %q, want recovered final answer", out.String()) | |||
| 96 | + | } | |||
| 97 | + | if client.Calls() != 2 { | |||
| 98 | + | t.Fatalf("provider calls = %d, want 2 (one bounded recovery)", client.Calls()) | |||
| 99 | + | } | |||
| 100 | + | } | |||
| 101 | + | ||||
| 102 | + | func TestTurnRecoveryStreamKeepsTranscriptOnStdout(t *testing.T) { | |||
| 103 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |||
| 104 | + | nil, | |||
| 105 | + | answerContent("finished"), | |||
| 106 | + | }} | |||
| 107 | + | ag := core.NewAgent(client, "test-model", "", nil) | |||
| 108 | + | var out, diag bytes.Buffer | |||
| 109 | + | if err := RunStreamWithDiag(context.Background(), ag, "work", nil, &out, &diag); err != nil { | |||
| 110 | + | t.Fatalf("RunStreamWithDiag error = %v", err) | |||
| 111 | + | } | |||
| 112 | + | if !strings.Contains(out.String(), "finished") { | |||
| 113 | + | t.Fatalf("stdout = %q, want recovered answer text", out.String()) | |||
| 114 | + | } | |||
| 115 | + | if !strings.Contains(diag.String(), "Continuing: no final answer received.") { | |||
| 116 | + | t.Fatalf("diag = %q, want one recovery status", diag.String()) | |||
| 117 | + | } | |||
| 118 | + | if strings.Contains(out.String(), "Continuing: no final answer received.") { | |||
| 119 | + | t.Fatalf("stdout = %q, recovery status must not pollute the transcript", out.String()) | |||
| 120 | + | } | |||
| 121 | + | } | |||
| 122 | + | ||||
| 123 | + | func TestTurnRecoveryExhaustionSurfacesIncompleteError(t *testing.T) { | |||
| 124 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |||
| 125 | + | nil, | |||
| 126 | + | nil, | |||
| 127 | + | }} | |||
| 128 | + | ag := core.NewAgent(client, "test-model", "", nil) | |||
| 129 | + | var out bytes.Buffer | |||
| 130 | + | _, err := RunPrint(context.Background(), ag, "work", nil, &out) | |||
| 131 | + | if !errors.Is(err, core.ErrIncompleteTurn) { | |||
| 132 | + | t.Fatalf("RunPrint error = %v, want ErrIncompleteTurn via errors.Is", err) | |||
| 133 | + | } | |||
| 134 | + | if out.String() != "" { | |||
| 135 | + | t.Fatalf("print output = %q, exhaustion must not report success text", out.String()) | |||
| 136 | + | } | |||
| 137 | + | if client.Calls() != 2 { | |||
| 138 | + | t.Fatalf("provider calls = %d, want exactly 2 (no unbounded retry)", client.Calls()) | |||
| 139 | + | } | |||
| 140 | + | } | |||
| 141 | + | ||||
| 142 | + | func TestTurnRecoveryGoalDoesNotScheduleDuplicateWork(t *testing.T) { | |||
| 143 | + | var mu sync.Mutex | |||
| 144 | + | goal := &core.SessionGoal{ID: "goal-1", Status: core.GoalActive, Objective: "finish the work"} | |||
| 145 | + | current := func() *core.SessionGoal { | |||
| 146 | + | mu.Lock() | |||
| 147 | + | defer mu.Unlock() | |||
| 148 | + | cp := *goal | |||
| 149 | + | return &cp | |||
| 150 | + | } | |||
| 151 | + | persist := func(next *core.SessionGoal) error { | |||
| 152 | + | mu.Lock() | |||
| 153 | + | goal = next | |||
| 154 | + | mu.Unlock() | |||
| 155 | + | return nil | |||
| 156 | + | } | |||
| 157 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{ | |||
| 158 | + | commentaryContent("still working on it"), | |||
| 159 | + | answerContent("finished"), | |||
| 160 | + | }} | |||
| 161 | + | ag := core.NewAgent(client, "test-model", "", goalToolRegistry()) | |||
| 162 | + | i := NewInteractive(InteractiveConfig{ | |||
| 163 | + | Agent: ag, | |||
| 164 | + | CurrentGoal: current, | |||
| 165 | + | PersistGoal: persist, | |||
| 166 | + | PersistGoalRuntime: persist, | |||
| 167 | + | }) | |||
| 168 | + | i.runCtx = context.Background() | |||
| 169 | + | run, err := i.startGoalRun(current()) | |||
| 170 | + | if err != nil { | |||
| 171 | + | t.Fatalf("startGoalRun error = %v", err) | |||
| 172 | + | } | |||
| 173 | + | if run == nil { | |||
| 174 | + | t.Fatal("startGoalRun did not start a run") | |||
| 175 | + | } | |||
| 176 | + | var events []core.AgentEvent | |||
| 177 | + | if err := ag.Prompt(context.Background(), "work", nil, func(ev core.AgentEvent) { | |||
| 178 | + | events = append(events, ev) | |||
| 179 | + | i.observeGoalRun(ev) | |||
| 180 | + | i.handleEvent(ev) | |||
| 181 | + | }); err != nil { | |||
| 182 | + | t.Fatalf("Prompt error = %v", err) | |||
| 183 | + | } | |||
| 184 | + | recoveries := 0 | |||
| 185 | + | for _, ev := range events { | |||
| 186 | + | if _, ok := ev.(core.EvTurnRecovery); ok { | |||
| 187 | + | recoveries++ | |||
| 188 | + | } | |||
| 189 | + | } | |||
| 190 | + | if recoveries != 1 { | |||
| 191 | + | t.Fatalf("recovery events = %d, want exactly one inner continuation", recoveries) | |||
| 192 | + | } | |||
| 193 | + | // The inner continuation must not start a second goal run: the run | |||
| 194 | + | // started above is still the single owner, and finishing it once must | |||
| 195 | + | // record accounting without scheduling another continuation here. | |||
| 196 | + | i.mu.Lock() | |||
| 197 | + | owner := i.goalRun | |||
| 198 | + | i.mu.Unlock() | |||
| 199 | + | if owner != run { | |||
| 200 | + | t.Fatal("inner continuation replaced the goal run: want the single owner started above") | |||
| 201 | + | } | |||
| 202 | + | if owner.goalID != "goal-1" { | |||
| 203 | + | t.Fatalf("goal run goalID = %q, want %q", owner.goalID, "goal-1") | |||
| 204 | + | } | |||
| 205 | + | if !i.finishGoalRun(false) { | |||
| 206 | + | t.Fatalf("finishGoalRun = false, want accounting for the single owner run") | |||
| 207 | + | } | |||
| 208 | + | mu.Lock() | |||
| 209 | + | used := goal.TokensUsed | |||
| 210 | + | mu.Unlock() | |||
| 211 | + | if used == 0 { | |||
| 212 | + | t.Fatalf("goal tokens used = 0, want usage recorded once for the single run") | |||
| 213 | + | } | |||
| 214 | + | } | |||
| 215 | + | ||||
| 216 | + | func TestTurnRecoveryExhaustionKeepsGoalBlocked(t *testing.T) { | |||
| 217 | + | var mu sync.Mutex | |||
| 218 | + | goal := &core.SessionGoal{ID: "goal-1", Status: core.GoalActive, Objective: "finish the work"} | |||
| 219 | + | current := func() *core.SessionGoal { | |||
| 220 | + | mu.Lock() | |||
| 221 | + | defer mu.Unlock() | |||
| 222 | + | cp := *goal | |||
| 223 | + | return &cp | |||
| 224 | + | } | |||
| 225 | + | var persisted []*core.SessionGoal | |||
| 226 | + | persist := func(next *core.SessionGoal) error { | |||
| 227 | + | mu.Lock() | |||
| 228 | + | cp := *next | |||
| 229 | + | persisted = append(persisted, &cp) | |||
| 230 | + | goal = &cp | |||
| 231 | + | mu.Unlock() | |||
| 232 | + | return nil | |||
| 233 | + | } | |||
| 234 | + | client := &turnRecoveryModeClient{turns: [][]provider.Content{nil, nil}} | |||
| 235 | + | ag := core.NewAgent(client, "test-model", "", goalToolRegistry()) | |||
| 236 | + | i := NewInteractive(InteractiveConfig{ | |||
| 237 | + | Agent: ag, | |||
| 238 | + | CurrentGoal: current, | |||
| 239 | + | PersistGoal: persist, | |||
| 240 | + | PersistGoalRuntime: persist, | |||
| 241 | + | }) | |||
| 242 | + | err := ag.Prompt(context.Background(), "work", nil, func(core.AgentEvent) {}) | |||
| 243 | + | if !errors.Is(err, core.ErrIncompleteTurn) { | |||
| 244 | + | t.Fatalf("Prompt error = %v, want ErrIncompleteTurn", err) | |||
| 245 | + | } | |||
| 246 | + | // Exhaustion follows established blocking semantics: the goal is marked | |||
| 247 | + | // blocked with the incomplete-turn cause, never auto-cleared or rerun. | |||
| 248 | + | i.updateActiveGoal(core.GoalBlocked, "turn ended with an error: "+core.ErrIncompleteTurn.Error()) | |||
| 249 | + | mu.Lock() | |||
| 250 | + | status := goal.Status | |||
| 251 | + | mu.Unlock() | |||
| 252 | + | if status != core.GoalBlocked { | |||
| 253 | + | t.Fatalf("goal status = %v, want blocked after exhaustion", status) | |||
| 254 | + | } | |||
| 255 | + | if len(persisted) == 0 { | |||
| 256 | + | t.Fatal("goal exhaustion was not persisted") | |||
| 257 | + | } | |||
| 258 | + | } | |||
| 259 | + | ||||
| 260 | + | func TestTurnRecoveryEventJSONEncoding(t *testing.T) { | |||
| 261 | + | encoded := EventToJSON(core.EvTurnRecovery{Reason: core.TurnRecoveryReasonMissingAnswer, Attempt: 1}) | |||
| 262 | + | raw, err := json.Marshal(encoded) | |||
| 263 | + | if err != nil { | |||
| 264 | + | t.Fatal(err) | |||
| 265 | + | } | |||
| 266 | + | var decoded map[string]any | |||
| 267 | + | if err := json.Unmarshal(raw, &decoded); err != nil { | |||
| 268 | + | t.Fatalf("turn_recovery event is not JSON: %v", err) | |||
| 269 | + | } | |||
| 270 | + | if decoded["type"] != "turn_recovery" { | |||
| 271 | + | t.Fatalf("type = %#v, want turn_recovery", decoded["type"]) | |||
| 272 | + | } | |||
| 273 | + | if decoded["reason"] != core.TurnRecoveryReasonMissingAnswer { | |||
| 274 | + | t.Fatalf("reason = %#v, want missing_answer", decoded["reason"]) | |||
| 275 | + | } | |||
| 276 | + | if decoded["attempt"] != float64(1) { | |||
| 277 | + | t.Fatalf("attempt = %#v, want 1", decoded["attempt"]) | |||
| 278 | + | } | |||
| 279 | + | } | |||
M packages/agent/systemprompt.go +1 -1
| @@ -103,7 +103,7 @@ For focused changes to an existing file, inspect its current contents and use ed | |||
| 103 | 103 | ||
| 104 | 104 | const compactedSummaryHandoffInstruction = `When you see a "## Context Summary (compacted)" message, treat it as a handoff from earlier work. Keep its active constraints and preferences in force, continue the most recent unresolved user request without waiting for the user to type "continue", and follow a newer user request when it supersedes the summary.` | |
| 105 | 105 | ||
| 106 | - | const taskExecutionGuidance = `Treat requests such as "can you" as instructions to do the requested work within the user's intended scope; do not stop at a plan or an offer to continue. Proceed with authorized, non-destructive work and reasonable assumptions until the task is complete or genuinely blocked. Respect requests for explanation, planning, or review without making unsolicited changes. Ask only when a missing decision blocks safe, correct progress; complete independent authorized work first. Preserve tool permissions, required confirmations, and explicit approval requirements; do not infer permission for destructive or irreversible actions.` | |
| 106 | + | const taskExecutionGuidance = `Treat requests such as "can you" as instructions to do the requested work within the user's intended scope; do not stop at a plan or an offer to continue. Proceed with authorized, non-destructive work and reasonable assumptions until the task is complete or genuinely blocked. Follow each progress statement with the action it describes: pair every intermediate update with tool calls or a concrete next step in the same turn. Final answers contain the result of the work or a concrete blocker, never a bare promise to continue. Respect requests for explanation, planning, or review without making unsolicited changes. Ask only when a necessary decision blocks safe, correct progress; complete independent authorized work first. Preserve tool permissions, required confirmations, and explicit approval requirements; do not infer permission for destructive or irreversible actions.` | |
| 107 | 107 | ||
| 108 | 108 | const skillPriorityGuidance = `Explicit user instructions take precedence over skill guidelines. Skills do not grant permissions or override system or developer constraints. If a skill would block requested work or change its scope, identify its source and the relevant instruction, explain the conflict, and follow the applicable higher-priority instruction.` | |
| 109 | 109 | ||
| @@ -103,7 +103,7 @@ For focused changes to an existing file, inspect its current contents and use ed | |||||
| 103 | 103 | ||||
| 104 | const compactedSummaryHandoffInstruction = `When you see a "## Context Summary (compacted)" message, treat it as a handoff from earlier work. Keep its active constraints and preferences in force, continue the most recent unresolved user request without waiting for the user to type "continue", and follow a newer user request when it supersedes the summary.` | 104 | const compactedSummaryHandoffInstruction = `When you see a "## Context Summary (compacted)" message, treat it as a handoff from earlier work. Keep its active constraints and preferences in force, continue the most recent unresolved user request without waiting for the user to type "continue", and follow a newer user request when it supersedes the summary.` | ||
| 105 | 105 | ||||
| 106 | - | const taskExecutionGuidance = `Treat requests such as "can you" as instructions to do the requested work within the user's intended scope; do not stop at a plan or an offer to continue. Proceed with authorized, non-destructive work and reasonable assumptions until the task is complete or genuinely blocked. Respect requests for explanation, planning, or review without making unsolicited changes. Ask only when a missing decision blocks safe, correct progress; complete independent authorized work first. Preserve tool permissions, required confirmations, and explicit approval requirements; do not infer permission for destructive or irreversible actions.` | 106 | + | const taskExecutionGuidance = `Treat requests such as "can you" as instructions to do the requested work within the user's intended scope; do not stop at a plan or an offer to continue. Proceed with authorized, non-destructive work and reasonable assumptions until the task is complete or genuinely blocked. Follow each progress statement with the action it describes: pair every intermediate update with tool calls or a concrete next step in the same turn. Final answers contain the result of the work or a concrete blocker, never a bare promise to continue. Respect requests for explanation, planning, or review without making unsolicited changes. Ask only when a necessary decision blocks safe, correct progress; complete independent authorized work first. Preserve tool permissions, required confirmations, and explicit approval requirements; do not infer permission for destructive or irreversible actions.` |
| 107 | 107 | ||||
| 108 | const skillPriorityGuidance = `Explicit user instructions take precedence over skill guidelines. Skills do not grant permissions or override system or developer constraints. If a skill would block requested work or change its scope, identify its source and the relevant instruction, explain the conflict, and follow the applicable higher-priority instruction.` | 108 | const skillPriorityGuidance = `Explicit user instructions take precedence over skill guidelines. Skills do not grant permissions or override system or developer constraints. If a skill would block requested work or change its scope, identify its source and the relevant instruction, explain the conflict, and follow the applicable higher-priority instruction.` | ||
| 109 | 109 | ||||
M packages/agent/systemprompt_test.go +3 -1
| @@ -64,8 +64,10 @@ func TestBuildSystemPromptAlwaysIncludesTaskAndSkillGuidance(t *testing.T) { | |||
| 64 | 64 | for _, want := range []string{ | |
| 65 | 65 | "Treat requests such as \"can you\" as instructions to do the requested work", | |
| 66 | 66 | "do not stop at a plan or an offer to continue", | |
| 67 | + | "Follow each progress statement with the action it describes", | |
| 68 | + | "Final answers contain the result of the work or a concrete blocker", | |
| 67 | 69 | "Respect requests for explanation, planning, or review without making unsolicited changes", | |
| 68 | - | "Ask only when a missing decision blocks safe, correct progress", | |
| 70 | + | "Ask only when a necessary decision blocks safe, correct progress", | |
| 69 | 71 | "Preserve tool permissions, required confirmations, and explicit approval requirements", | |
| 70 | 72 | "Explicit user instructions take precedence over skill guidelines", | |
| 71 | 73 | "Skills do not grant permissions or override system or developer constraints", | |
| @@ -64,8 +64,10 @@ func TestBuildSystemPromptAlwaysIncludesTaskAndSkillGuidance(t *testing.T) { | |||||
| 64 | for _, want := range []string{ | 64 | for _, want := range []string{ | ||
| 65 | "Treat requests such as \"can you\" as instructions to do the requested work", | 65 | "Treat requests such as \"can you\" as instructions to do the requested work", | ||
| 66 | "do not stop at a plan or an offer to continue", | 66 | "do not stop at a plan or an offer to continue", | ||
| 67 | + | "Follow each progress statement with the action it describes", | |||
| 68 | + | "Final answers contain the result of the work or a concrete blocker", | |||
| 67 | "Respect requests for explanation, planning, or review without making unsolicited changes", | 69 | "Respect requests for explanation, planning, or review without making unsolicited changes", | ||
| 68 | - | "Ask only when a missing decision blocks safe, correct progress", | 70 | + | "Ask only when a necessary decision blocks safe, correct progress", |
| 69 | "Preserve tool permissions, required confirmations, and explicit approval requirements", | 71 | "Preserve tool permissions, required confirmations, and explicit approval requirements", | ||
| 70 | "Explicit user instructions take precedence over skill guidelines", | 72 | "Explicit user instructions take precedence over skill guidelines", | ||
| 71 | "Skills do not grant permissions or override system or developer constraints", | 73 | "Skills do not grant permissions or override system or developer constraints", | ||
M packages/agent/tools/permissions.go +17 -8
| @@ -4,8 +4,17 @@ import ( | |||
| 4 | 4 | "fmt" | |
| 5 | 5 | "path/filepath" | |
| 6 | 6 | "strings" | |
| 7 | + | ||
| 8 | + | "github.com/bnema/zut/packages/core" | |
| 7 | 9 | ) | |
| 8 | 10 | ||
| 11 | + | // denied marks a policy refusal (permission scope, allowlist, jail | |
| 12 | + | // confinement, or admission gate) so core can distinguish it from an | |
| 13 | + | // ordinary tool failure. The message text shown to the model is unchanged. | |
| 14 | + | func denied(format string, args ...any) error { | |
| 15 | + | return &core.ToolDeniedError{Reason: fmt.Sprintf(format, args...)} | |
| 16 | + | } | |
| 17 | + | ||
| 9 | 18 | // PermissionSet is the local-runtime permission contract for a packaged .zut agent. | |
| 10 | 19 | type PermissionSet struct { | |
| 11 | 20 | FS struct { | |
| @@ -63,7 +72,7 @@ func (s *Sandbox) CheckReadPath(path string) error { | |||
| 63 | 72 | return nil | |
| 64 | 73 | } | |
| 65 | 74 | if len(s.Permissions.FS.Read) == 0 { | |
| 66 | - | return fmt.Errorf("permission denied: this agent has no filesystem read permission") | |
| 75 | + | return denied("permission denied: this agent has no filesystem read permission") | |
| 67 | 76 | } | |
| 68 | 77 | return checkScopedPath("read", path, s.Permissions.FS.Read) | |
| 69 | 78 | } | |
| @@ -76,7 +85,7 @@ func (s *Sandbox) CheckWritePath(path string) error { | |||
| 76 | 85 | return nil | |
| 77 | 86 | } | |
| 78 | 87 | if len(s.Permissions.FS.Write) == 0 { | |
| 79 | - | return fmt.Errorf("permission denied: this agent has no filesystem write permission") | |
| 88 | + | return denied("permission denied: this agent has no filesystem write permission") | |
| 80 | 89 | } | |
| 81 | 90 | return checkScopedPath("write", path, s.Permissions.FS.Write) | |
| 82 | 91 | } | |
| @@ -95,7 +104,7 @@ func checkScopedPath(op, path string, scopes []string) error { | |||
| 95 | 104 | return nil | |
| 96 | 105 | } | |
| 97 | 106 | } | |
| 98 | - | return fmt.Errorf("permission denied: %s %q is outside declared scopes", op, path) | |
| 107 | + | return denied("permission denied: %s %q is outside declared scopes", op, path) | |
| 99 | 108 | } | |
| 100 | 109 | ||
| 101 | 110 | func (s *Sandbox) CheckBashPermission(cmd string) error { | |
| @@ -108,18 +117,18 @@ func (s *Sandbox) CheckBashPermission(cmd string) error { | |||
| 108 | 117 | } | |
| 109 | 118 | switch mode { | |
| 110 | 119 | case "none": | |
| 111 | - | return fmt.Errorf("permission denied: this agent has no bash permission") | |
| 120 | + | return denied("permission denied: this agent has no bash permission") | |
| 112 | 121 | case "ask": | |
| 113 | 122 | // The local runtime obtains explicit consent for this capability at | |
| 114 | 123 | // every launch. Unlike durable consent, ask mode is never cached. | |
| 115 | 124 | return nil | |
| 116 | 125 | case "allowlist": | |
| 117 | 126 | if strings.ContainsAny(cmd, "`\n\r<>") || strings.Contains(cmd, "$(") { | |
| 118 | - | return fmt.Errorf("permission denied: bash allowlist does not permit substitution or redirection") | |
| 127 | + | return denied("permission denied: bash allowlist does not permit substitution or redirection") | |
| 119 | 128 | } | |
| 120 | 129 | names := commandNames(cmd) | |
| 121 | 130 | if len(names) == 0 { | |
| 122 | - | return fmt.Errorf("permission denied: empty command") | |
| 131 | + | return denied("permission denied: empty command") | |
| 123 | 132 | } | |
| 124 | 133 | allowed := map[string]bool{} | |
| 125 | 134 | for _, name := range s.Permissions.Bash.Allow { | |
| @@ -127,12 +136,12 @@ func (s *Sandbox) CheckBashPermission(cmd string) error { | |||
| 127 | 136 | } | |
| 128 | 137 | for _, name := range names { | |
| 129 | 138 | if !allowed[name] { | |
| 130 | - | return fmt.Errorf("permission denied: bash command %q is not in allowlist", name) | |
| 139 | + | return denied("permission denied: bash command %q is not in allowlist", name) | |
| 131 | 140 | } | |
| 132 | 141 | } | |
| 133 | 142 | return nil | |
| 134 | 143 | default: | |
| 135 | - | return fmt.Errorf("permission denied: unsupported bash mode %q", mode) | |
| 144 | + | return denied("permission denied: unsupported bash mode %q", mode) | |
| 136 | 145 | } | |
| 137 | 146 | } | |
| 138 | 147 | ||
| @@ -4,8 +4,17 @@ import ( | |||||
| 4 | "fmt" | 4 | "fmt" | ||
| 5 | "path/filepath" | 5 | "path/filepath" | ||
| 6 | "strings" | 6 | "strings" | ||
| 7 | + | ||||
| 8 | + | "github.com/bnema/zut/packages/core" | |||
| 7 | ) | 9 | ) | ||
| 8 | 10 | ||||
| 11 | + | // denied marks a policy refusal (permission scope, allowlist, jail | |||
| 12 | + | // confinement, or admission gate) so core can distinguish it from an | |||
| 13 | + | // ordinary tool failure. The message text shown to the model is unchanged. | |||
| 14 | + | func denied(format string, args ...any) error { | |||
| 15 | + | return &core.ToolDeniedError{Reason: fmt.Sprintf(format, args...)} | |||
| 16 | + | } | |||
| 17 | + | ||||
| 9 | // PermissionSet is the local-runtime permission contract for a packaged .zut agent. | 18 | // PermissionSet is the local-runtime permission contract for a packaged .zut agent. | ||
| 10 | type PermissionSet struct { | 19 | type PermissionSet struct { | ||
| 11 | FS struct { | 20 | FS struct { | ||
| @@ -63,7 +72,7 @@ func (s *Sandbox) CheckReadPath(path string) error { | |||||
| 63 | return nil | 72 | return nil | ||
| 64 | } | 73 | } | ||
| 65 | if len(s.Permissions.FS.Read) == 0 { | 74 | if len(s.Permissions.FS.Read) == 0 { | ||
| 66 | - | return fmt.Errorf("permission denied: this agent has no filesystem read permission") | 75 | + | return denied("permission denied: this agent has no filesystem read permission") |
| 67 | } | 76 | } | ||
| 68 | return checkScopedPath("read", path, s.Permissions.FS.Read) | 77 | return checkScopedPath("read", path, s.Permissions.FS.Read) | ||
| 69 | } | 78 | } | ||
| @@ -76,7 +85,7 @@ func (s *Sandbox) CheckWritePath(path string) error { | |||||
| 76 | return nil | 85 | return nil | ||
| 77 | } | 86 | } | ||
| 78 | if len(s.Permissions.FS.Write) == 0 { | 87 | if len(s.Permissions.FS.Write) == 0 { | ||
| 79 | - | return fmt.Errorf("permission denied: this agent has no filesystem write permission") | 88 | + | return denied("permission denied: this agent has no filesystem write permission") |
| 80 | } | 89 | } | ||
| 81 | return checkScopedPath("write", path, s.Permissions.FS.Write) | 90 | return checkScopedPath("write", path, s.Permissions.FS.Write) | ||
| 82 | } | 91 | } | ||
| @@ -95,7 +104,7 @@ func checkScopedPath(op, path string, scopes []string) error { | |||||
| 95 | return nil | 104 | return nil | ||
| 96 | } | 105 | } | ||
| 97 | } | 106 | } | ||
| 98 | - | return fmt.Errorf("permission denied: %s %q is outside declared scopes", op, path) | 107 | + | return denied("permission denied: %s %q is outside declared scopes", op, path) |
| 99 | } | 108 | } | ||
| 100 | 109 | ||||
| 101 | func (s *Sandbox) CheckBashPermission(cmd string) error { | 110 | func (s *Sandbox) CheckBashPermission(cmd string) error { | ||
| @@ -108,18 +117,18 @@ func (s *Sandbox) CheckBashPermission(cmd string) error { | |||||
| 108 | } | 117 | } | ||
| 109 | switch mode { | 118 | switch mode { | ||
| 110 | case "none": | 119 | case "none": | ||
| 111 | - | return fmt.Errorf("permission denied: this agent has no bash permission") | 120 | + | return denied("permission denied: this agent has no bash permission") |
| 112 | case "ask": | 121 | case "ask": | ||
| 113 | // The local runtime obtains explicit consent for this capability at | 122 | // The local runtime obtains explicit consent for this capability at | ||
| 114 | // every launch. Unlike durable consent, ask mode is never cached. | 123 | // every launch. Unlike durable consent, ask mode is never cached. | ||
| 115 | return nil | 124 | return nil | ||
| 116 | case "allowlist": | 125 | case "allowlist": | ||
| 117 | if strings.ContainsAny(cmd, "`\n\r<>") || strings.Contains(cmd, "$(") { | 126 | if strings.ContainsAny(cmd, "`\n\r<>") || strings.Contains(cmd, "$(") { | ||
| 118 | - | return fmt.Errorf("permission denied: bash allowlist does not permit substitution or redirection") | 127 | + | return denied("permission denied: bash allowlist does not permit substitution or redirection") |
| 119 | } | 128 | } | ||
| 120 | names := commandNames(cmd) | 129 | names := commandNames(cmd) | ||
| 121 | if len(names) == 0 { | 130 | if len(names) == 0 { | ||
| 122 | - | return fmt.Errorf("permission denied: empty command") | 131 | + | return denied("permission denied: empty command") |
| 123 | } | 132 | } | ||
| 124 | allowed := map[string]bool{} | 133 | allowed := map[string]bool{} | ||
| 125 | for _, name := range s.Permissions.Bash.Allow { | 134 | for _, name := range s.Permissions.Bash.Allow { | ||
| @@ -127,12 +136,12 @@ func (s *Sandbox) CheckBashPermission(cmd string) error { | |||||
| 127 | } | 136 | } | ||
| 128 | for _, name := range names { | 137 | for _, name := range names { | ||
| 129 | if !allowed[name] { | 138 | if !allowed[name] { | ||
| 130 | - | return fmt.Errorf("permission denied: bash command %q is not in allowlist", name) | 139 | + | return denied("permission denied: bash command %q is not in allowlist", name) |
| 131 | } | 140 | } | ||
| 132 | } | 141 | } | ||
| 133 | return nil | 142 | return nil | ||
| 134 | default: | 143 | default: | ||
| 135 | - | return fmt.Errorf("permission denied: unsupported bash mode %q", mode) | 144 | + | return denied("permission denied: unsupported bash mode %q", mode) |
| 136 | } | 145 | } | ||
| 137 | } | 146 | } | ||
| 138 | 147 | ||||
M packages/agent/tools/python.go +5 -3
| @@ -126,12 +126,14 @@ func (t *PythonTool) Execute(ctx context.Context, raw json.RawMessage, progress | |||
| 126 | 126 | return protocolToolError(err.Error()) | |
| 127 | 127 | } | |
| 128 | 128 | // Admission checks run before any interpreter discovery, so a denied | |
| 129 | - | // call never probes the filesystem or spawns a process. | |
| 129 | + | // call never probes the filesystem or spawns a process. The denial | |
| 130 | + | // is typed (not a plain tool-error result) so core records the | |
| 131 | + | // refusal and does not append a recovery prompt urging more actions. | |
| 130 | 132 | if t.Sandbox != nil && t.Sandbox.Permissions != nil { | |
| 131 | - | return protocolToolError("permission denied: this agent cannot run python") | |
| 133 | + | return core.ToolResult{}, denied("permission denied: this agent cannot run python") | |
| 132 | 134 | } | |
| 133 | 135 | if t.Sandbox.Locked() { | |
| 134 | - | return protocolToolError("jailed: arbitrary Python cannot be confined by the current jail (use /unjail to disable)") | |
| 136 | + | return core.ToolResult{}, denied("jailed: arbitrary Python cannot be confined by the current jail (use /unjail to disable)") | |
| 135 | 137 | } | |
| 136 | 138 | cwd := t.CWD | |
| 137 | 139 | if cwd == "" { | |
| @@ -126,12 +126,14 @@ func (t *PythonTool) Execute(ctx context.Context, raw json.RawMessage, progress | |||||
| 126 | return protocolToolError(err.Error()) | 126 | return protocolToolError(err.Error()) | ||
| 127 | } | 127 | } | ||
| 128 | // Admission checks run before any interpreter discovery, so a denied | 128 | // Admission checks run before any interpreter discovery, so a denied | ||
| 129 | - | // call never probes the filesystem or spawns a process. | 129 | + | // call never probes the filesystem or spawns a process. The denial |
| 130 | + | // is typed (not a plain tool-error result) so core records the | |||
| 131 | + | // refusal and does not append a recovery prompt urging more actions. | |||
| 130 | if t.Sandbox != nil && t.Sandbox.Permissions != nil { | 132 | if t.Sandbox != nil && t.Sandbox.Permissions != nil { | ||
| 131 | - | return protocolToolError("permission denied: this agent cannot run python") | 133 | + | return core.ToolResult{}, denied("permission denied: this agent cannot run python") |
| 132 | } | 134 | } | ||
| 133 | if t.Sandbox.Locked() { | 135 | if t.Sandbox.Locked() { | ||
| 134 | - | return protocolToolError("jailed: arbitrary Python cannot be confined by the current jail (use /unjail to disable)") | 136 | + | return core.ToolResult{}, denied("jailed: arbitrary Python cannot be confined by the current jail (use /unjail to disable)") |
| 135 | } | 137 | } | ||
| 136 | cwd := t.CWD | 138 | cwd := t.CWD | ||
| 137 | if cwd == "" { | 139 | if cwd == "" { | ||
M packages/agent/tools/python_test.go +13 -16
| @@ -10,6 +10,7 @@ import ( | |||
| 10 | 10 | "testing" | |
| 11 | 11 | "time" | |
| 12 | 12 | ||
| 13 | + | "github.com/bnema/zut/packages/core" | |
| 13 | 14 | "github.com/bnema/zut/packages/provider" | |
| 14 | 15 | ) | |
| 15 | 16 | ||
| @@ -116,27 +117,23 @@ func TestPythonDeniedNeverResolves(t *testing.T) { | |||
| 116 | 117 | } | |
| 117 | 118 | jailed := NewSandbox("/session") | |
| 118 | 119 | jailed.Lock() | |
| 119 | - | res, err := mk(jailed).Execute(context.Background(), pythonArgsJSON(t, "print(1)", 5), nil) | |
| 120 | - | if err != nil { | |
| 121 | - | t.Fatalf("jail err = %v, want model-visible IsError result", err) | |
| 122 | - | } | |
| 123 | - | if !res.IsError { | |
| 124 | - | t.Fatal("want IsError for jailed execution") | |
| 120 | + | _, err := mk(jailed).Execute(context.Background(), pythonArgsJSON(t, "print(1)", 5), nil) | |
| 121 | + | if err == nil || !strings.Contains(err.Error(), "cannot be confined") { | |
| 122 | + | t.Fatalf("jail err = %v, want typed confinement refusal", err) | |
| 125 | 123 | } | |
| 126 | - | if got := res.Content[0].(provider.TextBlock).Text; !strings.Contains(got, "cannot be confined") { | |
| 127 | - | t.Fatalf("jail result = %q", got) | |
| 124 | + | var jailDenied *core.ToolDeniedError | |
| 125 | + | if !errors.As(err, &jailDenied) { | |
| 126 | + | t.Fatalf("jail err type = %T, want *core.ToolDeniedError", err) | |
| 128 | 127 | } | |
| 129 | 128 | restricted := NewSandbox("/session") | |
| 130 | 129 | restricted.SetPermissions(&PermissionSet{}) | |
| 131 | - | res, err = mk(restricted).Execute(context.Background(), pythonArgsJSON(t, "print(1)", 5), nil) | |
| 132 | - | if err != nil { | |
| 133 | - | t.Fatalf("permission err = %v, want model-visible IsError result", err) | |
| 134 | - | } | |
| 135 | - | if !res.IsError { | |
| 136 | - | t.Fatal("want IsError for permission-denied execution") | |
| 130 | + | _, err = mk(restricted).Execute(context.Background(), pythonArgsJSON(t, "print(1)", 5), nil) | |
| 131 | + | if err == nil || !strings.Contains(err.Error(), "permission denied") { | |
| 132 | + | t.Fatalf("permission err = %v, want typed permission refusal", err) | |
| 137 | 133 | } | |
| 138 | - | if got := res.Content[0].(provider.TextBlock).Text; !strings.Contains(got, "permission denied") { | |
| 139 | - | t.Fatalf("permission result = %q", got) | |
| 134 | + | var permDenied *core.ToolDeniedError | |
| 135 | + | if !errors.As(err, &permDenied) { | |
| 136 | + | t.Fatalf("permission err type = %T, want *core.ToolDeniedError", err) | |
| 140 | 137 | } | |
| 141 | 138 | if resolveCalls != 0 || execCalls != 0 { | |
| 142 | 139 | t.Fatalf("denied calls must never resolve or execute (resolve=%d exec=%d)", resolveCalls, execCalls) | |
| @@ -10,6 +10,7 @@ import ( | |||||
| 10 | "testing" | 10 | "testing" | ||
| 11 | "time" | 11 | "time" | ||
| 12 | 12 | ||||
| 13 | + | "github.com/bnema/zut/packages/core" | |||
| 13 | "github.com/bnema/zut/packages/provider" | 14 | "github.com/bnema/zut/packages/provider" | ||
| 14 | ) | 15 | ) | ||
| 15 | 16 | ||||
| @@ -116,27 +117,23 @@ func TestPythonDeniedNeverResolves(t *testing.T) { | |||||
| 116 | } | 117 | } | ||
| 117 | jailed := NewSandbox("/session") | 118 | jailed := NewSandbox("/session") | ||
| 118 | jailed.Lock() | 119 | jailed.Lock() | ||
| 119 | - | res, err := mk(jailed).Execute(context.Background(), pythonArgsJSON(t, "print(1)", 5), nil) | 120 | + | _, err := mk(jailed).Execute(context.Background(), pythonArgsJSON(t, "print(1)", 5), nil) |
| 120 | - | if err != nil { | 121 | + | if err == nil || !strings.Contains(err.Error(), "cannot be confined") { |
| 121 | - | t.Fatalf("jail err = %v, want model-visible IsError result", err) | 122 | + | t.Fatalf("jail err = %v, want typed confinement refusal", err) |
| 122 | - | } | |||
| 123 | - | if !res.IsError { | |||
| 124 | - | t.Fatal("want IsError for jailed execution") | |||
| 125 | } | 123 | } | ||
| 126 | - | if got := res.Content[0].(provider.TextBlock).Text; !strings.Contains(got, "cannot be confined") { | 124 | + | var jailDenied *core.ToolDeniedError |
| 127 | - | t.Fatalf("jail result = %q", got) | 125 | + | if !errors.As(err, &jailDenied) { |
| 126 | + | t.Fatalf("jail err type = %T, want *core.ToolDeniedError", err) | |||
| 128 | } | 127 | } | ||
| 129 | restricted := NewSandbox("/session") | 128 | restricted := NewSandbox("/session") | ||
| 130 | restricted.SetPermissions(&PermissionSet{}) | 129 | restricted.SetPermissions(&PermissionSet{}) | ||
| 131 | - | res, err = mk(restricted).Execute(context.Background(), pythonArgsJSON(t, "print(1)", 5), nil) | 130 | + | _, err = mk(restricted).Execute(context.Background(), pythonArgsJSON(t, "print(1)", 5), nil) |
| 132 | - | if err != nil { | 131 | + | if err == nil || !strings.Contains(err.Error(), "permission denied") { |
| 133 | - | t.Fatalf("permission err = %v, want model-visible IsError result", err) | 132 | + | t.Fatalf("permission err = %v, want typed permission refusal", err) |
| 134 | - | } | |||
| 135 | - | if !res.IsError { | |||
| 136 | - | t.Fatal("want IsError for permission-denied execution") | |||
| 137 | } | 133 | } | ||
| 138 | - | if got := res.Content[0].(provider.TextBlock).Text; !strings.Contains(got, "permission denied") { | 134 | + | var permDenied *core.ToolDeniedError |
| 139 | - | t.Fatalf("permission result = %q", got) | 135 | + | if !errors.As(err, &permDenied) { |
| 136 | + | t.Fatalf("permission err type = %T, want *core.ToolDeniedError", err) | |||
| 140 | } | 137 | } | ||
| 141 | if resolveCalls != 0 || execCalls != 0 { | 138 | if resolveCalls != 0 || execCalls != 0 { | ||
| 142 | t.Fatalf("denied calls must never resolve or execute (resolve=%d exec=%d)", resolveCalls, execCalls) | 139 | t.Fatalf("denied calls must never resolve or execute (resolve=%d exec=%d)", resolveCalls, execCalls) | ||
M packages/agent/tools/sandbox.go +6 -6
| @@ -53,7 +53,7 @@ func (s *Sandbox) CheckPath(path string) error { | |||
| 53 | 53 | return fmt.Errorf("sandbox path: %w", err) | |
| 54 | 54 | } | |
| 55 | 55 | if !isUnder(rootAbs, target) { | |
| 56 | - | return fmt.Errorf("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) | |
| 56 | + | return denied("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) | |
| 57 | 57 | } | |
| 58 | 58 | return nil | |
| 59 | 59 | } | |
| @@ -116,7 +116,7 @@ func (s *Sandbox) CheckCommand(cmd string) error { | |||
| 116 | 116 | lower := strings.ToLower(cmd) | |
| 117 | 117 | for _, b := range banned { | |
| 118 | 118 | if strings.Contains(lower, strings.ToLower(b)) { | |
| 119 | - | return fmt.Errorf("jailed: command contains banned pattern %q (use /unjail to disable)", b) | |
| 119 | + | return denied("jailed: command contains banned pattern %q (use /unjail to disable)", b) | |
| 120 | 120 | } | |
| 121 | 121 | } | |
| 122 | 122 | // Heuristic: reject a leading `cd` that tries to move the shell out | |
| @@ -261,7 +261,7 @@ func (s *Sandbox) checkCommandPath(path string) error { | |||
| 261 | 261 | } | |
| 262 | 262 | expanded := expandHome(path) | |
| 263 | 263 | if strings.HasPrefix(expanded, "/") && !filepath.IsAbs(expanded) { | |
| 264 | - | return fmt.Errorf("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) | |
| 264 | + | return denied("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) | |
| 265 | 265 | } | |
| 266 | 266 | if !filepath.IsAbs(expanded) { | |
| 267 | 267 | expanded = filepath.Join(s.Root, filepath.FromSlash(expanded)) | |
| @@ -271,7 +271,7 @@ func (s *Sandbox) checkCommandPath(path string) error { | |||
| 271 | 271 | return fmt.Errorf("sandbox path: %w", err) | |
| 272 | 272 | } | |
| 273 | 273 | if !isUnder(rootAbs, target) { | |
| 274 | - | return fmt.Errorf("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) | |
| 274 | + | return denied("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) | |
| 275 | 275 | } | |
| 276 | 276 | return nil | |
| 277 | 277 | } | |
| @@ -291,7 +291,7 @@ func (s *Sandbox) checkCDTarget(dir string) error { | |||
| 291 | 291 | // back inside root, so a `cd /etc` escape would slip through. Treat | |
| 292 | 292 | // it as an unconditional escape attempt: outside any project root. | |
| 293 | 293 | if strings.HasPrefix(expanded, "/") && !filepath.IsAbs(expanded) { | |
| 294 | - | return fmt.Errorf("jailed: cd outside sandbox root is not allowed (use /unjail to disable)") | |
| 294 | + | return denied("jailed: cd outside sandbox root is not allowed (use /unjail to disable)") | |
| 295 | 295 | } | |
| 296 | 296 | if !filepath.IsAbs(expanded) { | |
| 297 | 297 | // Relative targets (including `..`) resolve against the sandbox | |
| @@ -303,7 +303,7 @@ func (s *Sandbox) checkCDTarget(dir string) error { | |||
| 303 | 303 | return fmt.Errorf("sandbox path: %w", err) | |
| 304 | 304 | } | |
| 305 | 305 | if !isUnder(rootAbs, target) { | |
| 306 | - | return fmt.Errorf("jailed: cd outside sandbox root is not allowed (use /unjail to disable)") | |
| 306 | + | return denied("jailed: cd outside sandbox root is not allowed (use /unjail to disable)") | |
| 307 | 307 | } | |
| 308 | 308 | return nil | |
| 309 | 309 | } | |
| @@ -53,7 +53,7 @@ func (s *Sandbox) CheckPath(path string) error { | |||||
| 53 | return fmt.Errorf("sandbox path: %w", err) | 53 | return fmt.Errorf("sandbox path: %w", err) | ||
| 54 | } | 54 | } | ||
| 55 | if !isUnder(rootAbs, target) { | 55 | if !isUnder(rootAbs, target) { | ||
| 56 | - | return fmt.Errorf("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) | 56 | + | return denied("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) |
| 57 | } | 57 | } | ||
| 58 | return nil | 58 | return nil | ||
| 59 | } | 59 | } | ||
| @@ -116,7 +116,7 @@ func (s *Sandbox) CheckCommand(cmd string) error { | |||||
| 116 | lower := strings.ToLower(cmd) | 116 | lower := strings.ToLower(cmd) | ||
| 117 | for _, b := range banned { | 117 | for _, b := range banned { | ||
| 118 | if strings.Contains(lower, strings.ToLower(b)) { | 118 | if strings.Contains(lower, strings.ToLower(b)) { | ||
| 119 | - | return fmt.Errorf("jailed: command contains banned pattern %q (use /unjail to disable)", b) | 119 | + | return denied("jailed: command contains banned pattern %q (use /unjail to disable)", b) |
| 120 | } | 120 | } | ||
| 121 | } | 121 | } | ||
| 122 | // Heuristic: reject a leading `cd` that tries to move the shell out | 122 | // Heuristic: reject a leading `cd` that tries to move the shell out | ||
| @@ -261,7 +261,7 @@ func (s *Sandbox) checkCommandPath(path string) error { | |||||
| 261 | } | 261 | } | ||
| 262 | expanded := expandHome(path) | 262 | expanded := expandHome(path) | ||
| 263 | if strings.HasPrefix(expanded, "/") && !filepath.IsAbs(expanded) { | 263 | if strings.HasPrefix(expanded, "/") && !filepath.IsAbs(expanded) { | ||
| 264 | - | return fmt.Errorf("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) | 264 | + | return denied("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) |
| 265 | } | 265 | } | ||
| 266 | if !filepath.IsAbs(expanded) { | 266 | if !filepath.IsAbs(expanded) { | ||
| 267 | expanded = filepath.Join(s.Root, filepath.FromSlash(expanded)) | 267 | expanded = filepath.Join(s.Root, filepath.FromSlash(expanded)) | ||
| @@ -271,7 +271,7 @@ func (s *Sandbox) checkCommandPath(path string) error { | |||||
| 271 | return fmt.Errorf("sandbox path: %w", err) | 271 | return fmt.Errorf("sandbox path: %w", err) | ||
| 272 | } | 272 | } | ||
| 273 | if !isUnder(rootAbs, target) { | 273 | if !isUnder(rootAbs, target) { | ||
| 274 | - | return fmt.Errorf("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) | 274 | + | return denied("jailed: path %q is outside sandbox root %q (use /unjail to disable)", path, s.Root) |
| 275 | } | 275 | } | ||
| 276 | return nil | 276 | return nil | ||
| 277 | } | 277 | } | ||
| @@ -291,7 +291,7 @@ func (s *Sandbox) checkCDTarget(dir string) error { | |||||
| 291 | // back inside root, so a `cd /etc` escape would slip through. Treat | 291 | // back inside root, so a `cd /etc` escape would slip through. Treat | ||
| 292 | // it as an unconditional escape attempt: outside any project root. | 292 | // it as an unconditional escape attempt: outside any project root. | ||
| 293 | if strings.HasPrefix(expanded, "/") && !filepath.IsAbs(expanded) { | 293 | if strings.HasPrefix(expanded, "/") && !filepath.IsAbs(expanded) { | ||
| 294 | - | return fmt.Errorf("jailed: cd outside sandbox root is not allowed (use /unjail to disable)") | 294 | + | return denied("jailed: cd outside sandbox root is not allowed (use /unjail to disable)") |
| 295 | } | 295 | } | ||
| 296 | if !filepath.IsAbs(expanded) { | 296 | if !filepath.IsAbs(expanded) { | ||
| 297 | // Relative targets (including `..`) resolve against the sandbox | 297 | // Relative targets (including `..`) resolve against the sandbox | ||
| @@ -303,7 +303,7 @@ func (s *Sandbox) checkCDTarget(dir string) error { | |||||
| 303 | return fmt.Errorf("sandbox path: %w", err) | 303 | return fmt.Errorf("sandbox path: %w", err) | ||
| 304 | } | 304 | } | ||
| 305 | if !isUnder(rootAbs, target) { | 305 | if !isUnder(rootAbs, target) { | ||
| 306 | - | return fmt.Errorf("jailed: cd outside sandbox root is not allowed (use /unjail to disable)") | 306 | + | return denied("jailed: cd outside sandbox root is not allowed (use /unjail to disable)") |
| 307 | } | 307 | } | ||
| 308 | return nil | 308 | return nil | ||
| 309 | } | 309 | } | ||
M packages/core/agent.go +41 -1
| @@ -174,7 +174,12 @@ type Agent struct { | |||
| 174 | 174 | // boundaries: before the next model call after a tool batch, or | |
| 175 | 175 | // after a text-only assistant turn finishes. It never interrupts | |
| 176 | 176 | // a running tool or cancels an in-flight provider request. | |
| 177 | - | queued []queuedMessage | |
| 177 | + | queued []queuedMessage | |
| 178 | + | // deniedToolCall tracks whether any tool invocation was refused at the | |
| 179 | + | // execution boundary (guard or confirmation denial) during the current | |
| 180 | + | // top-level runLoop invocation. A denied invocation must not gain a | |
| 181 | + | // recovery prompt urging more actions. | |
| 182 | + | deniedToolCall bool | |
| 178 | 183 | timeContext agentTimeContext | |
| 179 | 184 | hasSessionTimeContext bool | |
| 180 | 185 | // sessionID identifies this conversation thread. cacheSessionID identifies | |
| @@ -624,6 +629,8 @@ func (a *Agent) wrapSink(sink func(AgentEvent)) func(AgentEvent) { | |||
| 624 | 629 | } | |
| 625 | 630 | ||
| 626 | 631 | func (a *Agent) runLoop(ctx context.Context, sink func(AgentEvent), requestContext provider.RequestContext) error { | |
| 632 | + | a.resetDeniedToolCalls() | |
| 633 | + | recoveryUsed := false | |
| 627 | 634 | for step := 1; a.MaxSteps <= 0 || step <= a.MaxSteps; step++ { | |
| 628 | 635 | // Messages queued while the agent was busy are delivered | |
| 629 | 636 | // before the next model call. This is the safe boundary: | |
| @@ -781,6 +788,28 @@ func (a *Agent) runLoop(ctx context.Context, sink func(AgentEvent), requestConte | |||
| 781 | 788 | continue | |
| 782 | 789 | } | |
| 783 | 790 | ||
| 791 | + | // Bounded normal-turn recovery: a successful normal stop whose | |
| 792 | + | // current terminal assistant message carries no final answer gets | |
| 793 | + | // one continuation per top-level invocation. Never on errors | |
| 794 | + | // (err != nil returns above), cancellation, output/context | |
| 795 | + | // limits, or a permission/confirmation refusal. | |
| 796 | + | if stop == provider.StopEnd && !a.sawDeniedToolCall() { | |
| 797 | + | if reason, incomplete := classifyTerminalAssistant(assistantMsg); incomplete { | |
| 798 | + | if !recoveryUsed { | |
| 799 | + | if err := ctx.Err(); err != nil { | |
| 800 | + | sink(EvDone{}) | |
| 801 | + | return err | |
| 802 | + | } | |
| 803 | + | recoveryUsed = true | |
| 804 | + | a.AppendUserContext(turnRecoveryPrompt, map[string]string{turnRecoveryMetaKey: "true"}) | |
| 805 | + | sink(EvTurnRecovery{Reason: reason, Attempt: 1}) | |
| 806 | + | continue | |
| 807 | + | } | |
| 808 | + | sink(EvDone{}) | |
| 809 | + | return ErrIncompleteTurn | |
| 810 | + | } | |
| 811 | + | } | |
| 812 | + | ||
| 784 | 813 | // Terminal stop (end, length, error, aborted). | |
| 785 | 814 | sink(EvDone{}) | |
| 786 | 815 | return nil | |
| @@ -1369,6 +1398,10 @@ func (a *Agent) runOneTool(ctx context.Context, tc provider.ToolCallBlock, tools | |||
| 1369 | 1398 | if a.BeforeToolExecute != nil { | |
| 1370 | 1399 | allowed, reason, modified := a.BeforeToolExecute(tc) | |
| 1371 | 1400 | if !allowed { | |
| 1401 | + | // A guard or confirmation denial is tracked separately from an | |
| 1402 | + | // ordinary tool error: a denied invocation must not gain a | |
| 1403 | + | // recovery prompt urging more actions. | |
| 1404 | + | a.markDeniedToolCall() | |
| 1372 | 1405 | if reason == "" { | |
| 1373 | 1406 | reason = "tool call refused by extension guard" | |
| 1374 | 1407 | } | |
| @@ -1399,6 +1432,13 @@ func (a *Agent) runOneTool(ctx context.Context, tc provider.ToolCallBlock, tools | |||
| 1399 | 1432 | sink(EvToolProgress{ID: tc.ID, Text: text}) | |
| 1400 | 1433 | }) | |
| 1401 | 1434 | if err != nil { | |
| 1435 | + | // A refused execution (permission scope, allowlist, jail, or | |
| 1436 | + | // admission gate) is tracked separately from an ordinary tool | |
| 1437 | + | // error: like a guard or confirmation denial, it must not gain | |
| 1438 | + | // a recovery prompt urging more actions. | |
| 1439 | + | if asToolDeniedError(err) { | |
| 1440 | + | a.markDeniedToolCall() | |
| 1441 | + | } | |
| 1402 | 1442 | if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { | |
| 1403 | 1443 | return ToolResult{ | |
| 1404 | 1444 | Content: []provider.Content{provider.TextBlock{Text: "aborted: " + err.Error()}}, | |
| @@ -174,7 +174,12 @@ type Agent struct { | |||||
| 174 | // boundaries: before the next model call after a tool batch, or | 174 | // boundaries: before the next model call after a tool batch, or | ||
| 175 | // after a text-only assistant turn finishes. It never interrupts | 175 | // after a text-only assistant turn finishes. It never interrupts | ||
| 176 | // a running tool or cancels an in-flight provider request. | 176 | // a running tool or cancels an in-flight provider request. | ||
| 177 | - | queued []queuedMessage | 177 | + | queued []queuedMessage |
| 178 | + | // deniedToolCall tracks whether any tool invocation was refused at the | |||
| 179 | + | // execution boundary (guard or confirmation denial) during the current | |||
| 180 | + | // top-level runLoop invocation. A denied invocation must not gain a | |||
| 181 | + | // recovery prompt urging more actions. | |||
| 182 | + | deniedToolCall bool | |||
| 178 | timeContext agentTimeContext | 183 | timeContext agentTimeContext | ||
| 179 | hasSessionTimeContext bool | 184 | hasSessionTimeContext bool | ||
| 180 | // sessionID identifies this conversation thread. cacheSessionID identifies | 185 | // sessionID identifies this conversation thread. cacheSessionID identifies | ||
| @@ -624,6 +629,8 @@ func (a *Agent) wrapSink(sink func(AgentEvent)) func(AgentEvent) { | |||||
| 624 | } | 629 | } | ||
| 625 | 630 | ||||
| 626 | func (a *Agent) runLoop(ctx context.Context, sink func(AgentEvent), requestContext provider.RequestContext) error { | 631 | func (a *Agent) runLoop(ctx context.Context, sink func(AgentEvent), requestContext provider.RequestContext) error { | ||
| 632 | + | a.resetDeniedToolCalls() | |||
| 633 | + | recoveryUsed := false | |||
| 627 | for step := 1; a.MaxSteps <= 0 || step <= a.MaxSteps; step++ { | 634 | for step := 1; a.MaxSteps <= 0 || step <= a.MaxSteps; step++ { | ||
| 628 | // Messages queued while the agent was busy are delivered | 635 | // Messages queued while the agent was busy are delivered | ||
| 629 | // before the next model call. This is the safe boundary: | 636 | // before the next model call. This is the safe boundary: | ||
| @@ -781,6 +788,28 @@ func (a *Agent) runLoop(ctx context.Context, sink func(AgentEvent), requestConte | |||||
| 781 | continue | 788 | continue | ||
| 782 | } | 789 | } | ||
| 783 | 790 | ||||
| 791 | + | // Bounded normal-turn recovery: a successful normal stop whose | |||
| 792 | + | // current terminal assistant message carries no final answer gets | |||
| 793 | + | // one continuation per top-level invocation. Never on errors | |||
| 794 | + | // (err != nil returns above), cancellation, output/context | |||
| 795 | + | // limits, or a permission/confirmation refusal. | |||
| 796 | + | if stop == provider.StopEnd && !a.sawDeniedToolCall() { | |||
| 797 | + | if reason, incomplete := classifyTerminalAssistant(assistantMsg); incomplete { | |||
| 798 | + | if !recoveryUsed { | |||
| 799 | + | if err := ctx.Err(); err != nil { | |||
| 800 | + | sink(EvDone{}) | |||
| 801 | + | return err | |||
| 802 | + | } | |||
| 803 | + | recoveryUsed = true | |||
| 804 | + | a.AppendUserContext(turnRecoveryPrompt, map[string]string{turnRecoveryMetaKey: "true"}) | |||
| 805 | + | sink(EvTurnRecovery{Reason: reason, Attempt: 1}) | |||
| 806 | + | continue | |||
| 807 | + | } | |||
| 808 | + | sink(EvDone{}) | |||
| 809 | + | return ErrIncompleteTurn | |||
| 810 | + | } | |||
| 811 | + | } | |||
| 812 | + | ||||
| 784 | // Terminal stop (end, length, error, aborted). | 813 | // Terminal stop (end, length, error, aborted). | ||
| 785 | sink(EvDone{}) | 814 | sink(EvDone{}) | ||
| 786 | return nil | 815 | return nil | ||
| @@ -1369,6 +1398,10 @@ func (a *Agent) runOneTool(ctx context.Context, tc provider.ToolCallBlock, tools | |||||
| 1369 | if a.BeforeToolExecute != nil { | 1398 | if a.BeforeToolExecute != nil { | ||
| 1370 | allowed, reason, modified := a.BeforeToolExecute(tc) | 1399 | allowed, reason, modified := a.BeforeToolExecute(tc) | ||
| 1371 | if !allowed { | 1400 | if !allowed { | ||
| 1401 | + | // A guard or confirmation denial is tracked separately from an | |||
| 1402 | + | // ordinary tool error: a denied invocation must not gain a | |||
| 1403 | + | // recovery prompt urging more actions. | |||
| 1404 | + | a.markDeniedToolCall() | |||
| 1372 | if reason == "" { | 1405 | if reason == "" { | ||
| 1373 | reason = "tool call refused by extension guard" | 1406 | reason = "tool call refused by extension guard" | ||
| 1374 | } | 1407 | } | ||
| @@ -1399,6 +1432,13 @@ func (a *Agent) runOneTool(ctx context.Context, tc provider.ToolCallBlock, tools | |||||
| 1399 | sink(EvToolProgress{ID: tc.ID, Text: text}) | 1432 | sink(EvToolProgress{ID: tc.ID, Text: text}) | ||
| 1400 | }) | 1433 | }) | ||
| 1401 | if err != nil { | 1434 | if err != nil { | ||
| 1435 | + | // A refused execution (permission scope, allowlist, jail, or | |||
| 1436 | + | // admission gate) is tracked separately from an ordinary tool | |||
| 1437 | + | // error: like a guard or confirmation denial, it must not gain | |||
| 1438 | + | // a recovery prompt urging more actions. | |||
| 1439 | + | if asToolDeniedError(err) { | |||
| 1440 | + | a.markDeniedToolCall() | |||
| 1441 | + | } | |||
| 1402 | if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { | 1442 | if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { | ||
| 1403 | return ToolResult{ | 1443 | return ToolResult{ | ||
| 1404 | Content: []provider.Content{provider.TextBlock{Text: "aborted: " + err.Error()}}, | 1444 | Content: []provider.Content{provider.TextBlock{Text: "aborted: " + err.Error()}}, | ||
M packages/core/agent_retry_test.go +4 -1
| @@ -69,7 +69,10 @@ func (activeStreamClient) Name() string { return "active-stream" } | |||
| 69 | 69 | func (activeStreamClient) Stream(context.Context, provider.Request) (<-chan provider.Event, error) { | |
| 70 | 70 | out := make(chan provider.Event, 2) | |
| 71 | 71 | out <- provider.EventStart{} | |
| 72 | - | out <- provider.EventDone{Stop: provider.StopEnd} | |
| 72 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ | |
| 73 | + | Role: provider.RoleAssistant, | |
| 74 | + | Content: []provider.Content{provider.TextBlock{Text: "streaming"}}, | |
| 75 | + | }} | |
| 73 | 76 | close(out) | |
| 74 | 77 | return out, nil | |
| 75 | 78 | } | |
| @@ -69,7 +69,10 @@ func (activeStreamClient) Name() string { return "active-stream" } | |||||
| 69 | func (activeStreamClient) Stream(context.Context, provider.Request) (<-chan provider.Event, error) { | 69 | func (activeStreamClient) Stream(context.Context, provider.Request) (<-chan provider.Event, error) { | ||
| 70 | out := make(chan provider.Event, 2) | 70 | out := make(chan provider.Event, 2) | ||
| 71 | out <- provider.EventStart{} | 71 | out <- provider.EventStart{} | ||
| 72 | - | out <- provider.EventDone{Stop: provider.StopEnd} | 72 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ |
| 73 | + | Role: provider.RoleAssistant, | |||
| 74 | + | Content: []provider.Content{provider.TextBlock{Text: "streaming"}}, | |||
| 75 | + | }} | |||
| 73 | close(out) | 76 | close(out) | ||
| 74 | return out, nil | 77 | return out, nil | ||
| 75 | } | 78 | } | ||
M packages/core/events.go +9 -0
| @@ -226,6 +226,15 @@ type EvTurnEnd struct { | |||
| 226 | 226 | ||
| 227 | 227 | func (EvTurnEnd) Type() string { return "turn_end" } | |
| 228 | 228 | ||
| 229 | + | type EvTurnRecovery struct { | |
| 230 | + | // Reason is missing_answer or commentary_only. | |
| 231 | + | Reason string | |
| 232 | + | // Attempt is always 1: one recovery allowance per top-level invocation. | |
| 233 | + | Attempt int | |
| 234 | + | } | |
| 235 | + | ||
| 236 | + | func (EvTurnRecovery) Type() string { return "turn_recovery" } | |
| 237 | + | ||
| 229 | 238 | type EvDone struct{} | |
| 230 | 239 | ||
| 231 | 240 | func (EvDone) Type() string { return "done" } | |
| @@ -226,6 +226,15 @@ type EvTurnEnd struct { | |||||
| 226 | 226 | ||||
| 227 | func (EvTurnEnd) Type() string { return "turn_end" } | 227 | func (EvTurnEnd) Type() string { return "turn_end" } | ||
| 228 | 228 | ||||
| 229 | + | type EvTurnRecovery struct { | |||
| 230 | + | // Reason is missing_answer or commentary_only. | |||
| 231 | + | Reason string | |||
| 232 | + | // Attempt is always 1: one recovery allowance per top-level invocation. | |||
| 233 | + | Attempt int | |||
| 234 | + | } | |||
| 235 | + | ||||
| 236 | + | func (EvTurnRecovery) Type() string { return "turn_recovery" } | |||
| 237 | + | ||||
| 229 | type EvDone struct{} | 238 | type EvDone struct{} | ||
| 230 | 239 | ||||
| 231 | func (EvDone) Type() string { return "done" } | 240 | func (EvDone) Type() string { return "done" } | ||
M packages/core/fast_mode_test.go +4 -1
| @@ -21,7 +21,10 @@ func (c *fastModeTestClient) Stream(_ context.Context, req provider.Request) (<- | |||
| 21 | 21 | c.lastReq = req | |
| 22 | 22 | out := make(chan provider.Event, 2) | |
| 23 | 23 | out <- provider.EventTextDelta{Delta: "ok"} | |
| 24 | - | out <- provider.EventDone{Stop: provider.StopEnd} | |
| 24 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ | |
| 25 | + | Role: provider.RoleAssistant, | |
| 26 | + | Content: []provider.Content{provider.TextBlock{Text: "ok"}}, | |
| 27 | + | }} | |
| 25 | 28 | close(out) | |
| 26 | 29 | return out, nil | |
| 27 | 30 | } | |
| @@ -21,7 +21,10 @@ func (c *fastModeTestClient) Stream(_ context.Context, req provider.Request) (<- | |||||
| 21 | c.lastReq = req | 21 | c.lastReq = req | ||
| 22 | out := make(chan provider.Event, 2) | 22 | out := make(chan provider.Event, 2) | ||
| 23 | out <- provider.EventTextDelta{Delta: "ok"} | 23 | out <- provider.EventTextDelta{Delta: "ok"} | ||
| 24 | - | out <- provider.EventDone{Stop: provider.StopEnd} | 24 | + | out <- provider.EventDone{Stop: provider.StopEnd, Message: provider.Message{ |
| 25 | + | Role: provider.RoleAssistant, | |||
| 26 | + | Content: []provider.Content{provider.TextBlock{Text: "ok"}}, | |||
| 27 | + | }} | |||
| 25 | close(out) | 28 | close(out) | ||
| 26 | return out, nil | 29 | return out, nil | ||
| 27 | } | 30 | } | ||
M packages/core/session.go +2 -0
| @@ -1735,6 +1735,7 @@ func hydrateMessageObject(rawMessage []byte) (provider.Message, error) { | |||
| 1735 | 1735 | Summary string `json:"summary"` | |
| 1736 | 1736 | Encrypted string `json:"encrypted_content"` | |
| 1737 | 1737 | ThoughtSignature string `json:"thought_signature"` | |
| 1738 | + | Phase string `json:"phase"` | |
| 1738 | 1739 | // ToolCallBlock also has Arguments, ToolResultBlock has Content + IsError | |
| 1739 | 1740 | } | |
| 1740 | 1741 | if err := json.Unmarshal(raw, &head); err != nil { | |
| @@ -1827,6 +1828,7 @@ func hydrateMessageObject(rawMessage []byte) (provider.Message, error) { | |||
| 1827 | 1828 | msg.Content = append(msg.Content, provider.TextBlock{ | |
| 1828 | 1829 | Text: head.Text, | |
| 1829 | 1830 | ThoughtSignature: head.ThoughtSignature, | |
| 1831 | + | Phase: head.Phase, | |
| 1830 | 1832 | }) | |
| 1831 | 1833 | } | |
| 1832 | 1834 | } | |
| @@ -1735,6 +1735,7 @@ func hydrateMessageObject(rawMessage []byte) (provider.Message, error) { | |||||
| 1735 | Summary string `json:"summary"` | 1735 | Summary string `json:"summary"` | ||
| 1736 | Encrypted string `json:"encrypted_content"` | 1736 | Encrypted string `json:"encrypted_content"` | ||
| 1737 | ThoughtSignature string `json:"thought_signature"` | 1737 | ThoughtSignature string `json:"thought_signature"` | ||
| 1738 | + | Phase string `json:"phase"` | |||
| 1738 | // ToolCallBlock also has Arguments, ToolResultBlock has Content + IsError | 1739 | // ToolCallBlock also has Arguments, ToolResultBlock has Content + IsError | ||
| 1739 | } | 1740 | } | ||
| 1740 | if err := json.Unmarshal(raw, &head); err != nil { | 1741 | if err := json.Unmarshal(raw, &head); err != nil { | ||
| @@ -1827,6 +1828,7 @@ func hydrateMessageObject(rawMessage []byte) (provider.Message, error) { | |||||
| 1827 | msg.Content = append(msg.Content, provider.TextBlock{ | 1828 | msg.Content = append(msg.Content, provider.TextBlock{ | ||
| 1828 | Text: head.Text, | 1829 | Text: head.Text, | ||
| 1829 | ThoughtSignature: head.ThoughtSignature, | 1830 | ThoughtSignature: head.ThoughtSignature, | ||
| 1831 | + | Phase: head.Phase, | |||
| 1830 | }) | 1832 | }) | ||
| 1831 | } | 1833 | } | ||
| 1832 | } | 1834 | } | ||
A packages/core/session_phase_test.go +250 -0
| @@ -0,0 +1,250 @@ | |||
| 1 | + | package core | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "encoding/json" | |
| 5 | + | "os" | |
| 6 | + | "path/filepath" | |
| 7 | + | "testing" | |
| 8 | + | ||
| 9 | + | "github.com/bnema/zut/packages/provider" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | func phaseOf(block provider.Content) string { | |
| 13 | + | tb, ok := block.(provider.TextBlock) | |
| 14 | + | if !ok { | |
| 15 | + | return "<non-text>" | |
| 16 | + | } | |
| 17 | + | return tb.Phase | |
| 18 | + | } | |
| 19 | + | ||
| 20 | + | func appendPhasedAssistant(t *testing.T, s *Session) { | |
| 21 | + | t.Helper() | |
| 22 | + | if err := s.AppendMessage(provider.Message{ | |
| 23 | + | Role: provider.RoleAssistant, | |
| 24 | + | Content: []provider.Content{ | |
| 25 | + | provider.TextBlock{Text: "working on it", Phase: provider.TextPhaseCommentary}, | |
| 26 | + | provider.TextBlock{Text: "done", Phase: provider.TextPhaseFinal}, | |
| 27 | + | provider.TextBlock{Text: "plain"}, | |
| 28 | + | }, | |
| 29 | + | }); err != nil { | |
| 30 | + | t.Fatal(err) | |
| 31 | + | } | |
| 32 | + | } | |
| 33 | + | ||
| 34 | + | func TestSessionPhaseOldUnphasedFixture(t *testing.T) { | |
| 35 | + | raw := `{"role":"assistant","content":[{"text":"hello"}],"time":"2026-01-01T00:00:00Z"}` | |
| 36 | + | msg, err := DecodeMessageJSON([]byte(raw)) | |
| 37 | + | if err != nil { | |
| 38 | + | t.Fatalf("DecodeMessageJSON: %v", err) | |
| 39 | + | } | |
| 40 | + | if len(msg.Content) != 1 { | |
| 41 | + | t.Fatalf("content = %#v, want 1 block", msg.Content) | |
| 42 | + | } | |
| 43 | + | if got := phaseOf(msg.Content[0]); got != "" { | |
| 44 | + | t.Fatalf("phase = %q, want empty", got) | |
| 45 | + | } | |
| 46 | + | } | |
| 47 | + | ||
| 48 | + | func TestSessionPhaseMixedRoundTrip(t *testing.T) { | |
| 49 | + | root := t.TempDir() | |
| 50 | + | cwd := t.TempDir() | |
| 51 | + | s, err := NewSession(root, cwd, "provider", "model", "test") | |
| 52 | + | if err != nil { | |
| 53 | + | t.Fatal(err) | |
| 54 | + | } | |
| 55 | + | appendPhasedAssistant(t, s) | |
| 56 | + | path := s.Path | |
| 57 | + | if err := s.Close(); err != nil { | |
| 58 | + | t.Fatal(err) | |
| 59 | + | } | |
| 60 | + | reopened, msgs, err := OpenSession(path) | |
| 61 | + | if err != nil { | |
| 62 | + | t.Fatalf("OpenSession: %v", err) | |
| 63 | + | } | |
| 64 | + | defer reopened.Close() | |
| 65 | + | if len(msgs) != 1 || len(msgs[0].Content) != 3 { | |
| 66 | + | t.Fatalf("msgs = %#v, want 1 message with 3 blocks", msgs) | |
| 67 | + | } | |
| 68 | + | want := []string{provider.TextPhaseCommentary, provider.TextPhaseFinal, ""} | |
| 69 | + | for i, w := range want { | |
| 70 | + | if got := phaseOf(msgs[0].Content[i]); got != w { | |
| 71 | + | t.Fatalf("block %d phase = %q, want %q", i, got, w) | |
| 72 | + | } | |
| 73 | + | } | |
| 74 | + | } | |
| 75 | + | ||
| 76 | + | func TestSessionPhaseResumeFork(t *testing.T) { | |
| 77 | + | root := t.TempDir() | |
| 78 | + | cwd := t.TempDir() | |
| 79 | + | s, err := NewSession(root, cwd, "provider", "model", "test") | |
| 80 | + | if err != nil { | |
| 81 | + | t.Fatal(err) | |
| 82 | + | } | |
| 83 | + | if err := s.AppendMessage(provider.Message{ | |
| 84 | + | Role: provider.RoleUser, | |
| 85 | + | Content: []provider.Content{provider.TextBlock{Text: "hi"}}, | |
| 86 | + | }); err != nil { | |
| 87 | + | t.Fatal(err) | |
| 88 | + | } | |
| 89 | + | appendPhasedAssistant(t, s) | |
| 90 | + | parentPath := s.Path | |
| 91 | + | if err := s.Close(); err != nil { | |
| 92 | + | t.Fatal(err) | |
| 93 | + | } | |
| 94 | + | ||
| 95 | + | // Resume: reopening the same file preserves phases. | |
| 96 | + | reopened, msgs, err := OpenSession(parentPath) | |
| 97 | + | if err != nil { | |
| 98 | + | t.Fatalf("OpenSession: %v", err) | |
| 99 | + | } | |
| 100 | + | reopened.Close() | |
| 101 | + | if len(msgs) != 2 || len(msgs[1].Content) != 3 { | |
| 102 | + | t.Fatalf("resume msgs = %#v, want 2 messages", msgs) | |
| 103 | + | } | |
| 104 | + | if got := phaseOf(msgs[1].Content[0]); got != provider.TextPhaseCommentary { | |
| 105 | + | t.Fatalf("resume phase = %q, want commentary", got) | |
| 106 | + | } | |
| 107 | + | ||
| 108 | + | // Fork: branching preserves phases. | |
| 109 | + | branchPath, err := BranchSession(parentPath, root, cwd, "test", 2) | |
| 110 | + | if err != nil { | |
| 111 | + | t.Fatalf("BranchSession: %v", err) | |
| 112 | + | } | |
| 113 | + | branch, branchMsgs, err := OpenSession(branchPath) | |
| 114 | + | if err != nil { | |
| 115 | + | t.Fatalf("OpenSession(branch): %v", err) | |
| 116 | + | } | |
| 117 | + | defer branch.Close() | |
| 118 | + | if len(branchMsgs) != 2 || len(branchMsgs[1].Content) != 3 { | |
| 119 | + | t.Fatalf("fork msgs = %#v, want 2 messages", branchMsgs) | |
| 120 | + | } | |
| 121 | + | want := []string{provider.TextPhaseCommentary, provider.TextPhaseFinal, ""} | |
| 122 | + | for i, w := range want { | |
| 123 | + | if got := phaseOf(branchMsgs[1].Content[i]); got != w { | |
| 124 | + | t.Fatalf("fork block %d phase = %q, want %q", i, got, w) | |
| 125 | + | } | |
| 126 | + | } | |
| 127 | + | } | |
| 128 | + | ||
| 129 | + | func TestSessionPhasePortableImportExport(t *testing.T) { | |
| 130 | + | root := t.TempDir() | |
| 131 | + | cwd := t.TempDir() | |
| 132 | + | s, err := NewSession(root, cwd, "provider", "model", "test") | |
| 133 | + | if err != nil { | |
| 134 | + | t.Fatal(err) | |
| 135 | + | } | |
| 136 | + | appendPhasedAssistant(t, s) | |
| 137 | + | srcPath := s.Path | |
| 138 | + | if err := s.Close(); err != nil { | |
| 139 | + | t.Fatal(err) | |
| 140 | + | } | |
| 141 | + | exportPath := filepath.Join(t.TempDir(), "export.json") | |
| 142 | + | out, err := ExportSession(srcPath, exportPath) | |
| 143 | + | if err != nil { | |
| 144 | + | t.Fatalf("ExportSession: %v", err) | |
| 145 | + | } | |
| 146 | + | importedPath, err := ImportSession(out, root, cwd, "test") | |
| 147 | + | if err != nil { | |
| 148 | + | t.Fatalf("ImportSession: %v", err) | |
| 149 | + | } | |
| 150 | + | imported, msgs, err := OpenSession(importedPath) | |
| 151 | + | if err != nil { | |
| 152 | + | t.Fatalf("OpenSession(imported): %v", err) | |
| 153 | + | } | |
| 154 | + | defer imported.Close() | |
| 155 | + | if len(msgs) != 1 || len(msgs[0].Content) != 3 { | |
| 156 | + | t.Fatalf("imported msgs = %#v, want 1 message with 3 blocks", msgs) | |
| 157 | + | } | |
| 158 | + | want := []string{provider.TextPhaseCommentary, provider.TextPhaseFinal, ""} | |
| 159 | + | for i, w := range want { | |
| 160 | + | if got := phaseOf(msgs[0].Content[i]); got != w { | |
| 161 | + | t.Fatalf("imported block %d phase = %q, want %q", i, got, w) | |
| 162 | + | } | |
| 163 | + | } | |
| 164 | + | } | |
| 165 | + | ||
| 166 | + | func TestSessionPhaseRetainedProjection(t *testing.T) { | |
| 167 | + | msgs := []provider.Message{ | |
| 168 | + | {Role: provider.RoleAssistant, Content: []provider.Content{ | |
| 169 | + | provider.TextBlock{Text: "progress", Phase: provider.TextPhaseCommentary}, | |
| 170 | + | provider.TextBlock{Text: "answer", Phase: provider.TextPhaseFinal}, | |
| 171 | + | }}, | |
| 172 | + | } | |
| 173 | + | projected := projectProviderMessages(msgs) | |
| 174 | + | if len(projected) != 1 || len(projected[0].Content) != 2 { | |
| 175 | + | t.Fatalf("projected = %#v, want 2 retained blocks", projected) | |
| 176 | + | } | |
| 177 | + | if got := phaseOf(projected[0].Content[0]); got != provider.TextPhaseCommentary { | |
| 178 | + | t.Fatalf("projected[0] phase = %q, want commentary", got) | |
| 179 | + | } | |
| 180 | + | if got := phaseOf(projected[0].Content[1]); got != provider.TextPhaseFinal { | |
| 181 | + | t.Fatalf("projected[1] phase = %q, want final_answer", got) | |
| 182 | + | } | |
| 183 | + | // Original must not be mutated by projection. | |
| 184 | + | if got := phaseOf(msgs[0].Content[0]); got != provider.TextPhaseCommentary { | |
| 185 | + | t.Fatalf("original mutated: phase = %q", got) | |
| 186 | + | } | |
| 187 | + | } | |
| 188 | + | ||
| 189 | + | func TestSessionPhaseUnknownPreserved(t *testing.T) { | |
| 190 | + | root := t.TempDir() | |
| 191 | + | cwd := t.TempDir() | |
| 192 | + | s, err := NewSession(root, cwd, "provider", "model", "test") | |
| 193 | + | if err != nil { | |
| 194 | + | t.Fatal(err) | |
| 195 | + | } | |
| 196 | + | if err := s.AppendMessage(provider.Message{ | |
| 197 | + | Role: provider.RoleAssistant, | |
| 198 | + | Content: []provider.Content{provider.TextBlock{Text: "odd", Phase: "mystery"}}, | |
| 199 | + | }); err != nil { | |
| 200 | + | t.Fatal(err) | |
| 201 | + | } | |
| 202 | + | path := s.Path | |
| 203 | + | if err := s.Close(); err != nil { | |
| 204 | + | t.Fatal(err) | |
| 205 | + | } | |
| 206 | + | // Raw file must carry the unrecognized phase verbatim. | |
| 207 | + | data, err := os.ReadFile(path) | |
| 208 | + | if err != nil { | |
| 209 | + | t.Fatal(err) | |
| 210 | + | } | |
| 211 | + | var found bool | |
| 212 | + | f, err := os.Open(path) | |
| 213 | + | if err != nil { | |
| 214 | + | t.Fatal(err) | |
| 215 | + | } | |
| 216 | + | defer f.Close() | |
| 217 | + | if err := forEachJSONLLine(f, func(line []byte) error { | |
| 218 | + | var row struct { | |
| 219 | + | Message *json.RawMessage `json:"message"` | |
| 220 | + | } | |
| 221 | + | if err := json.Unmarshal(line, &row); err != nil || row.Message == nil { | |
| 222 | + | return nil | |
| 223 | + | } | |
| 224 | + | var msg struct { | |
| 225 | + | Content []map[string]any `json:"content"` | |
| 226 | + | } | |
| 227 | + | if err := json.Unmarshal(*row.Message, &msg); err != nil { | |
| 228 | + | return nil | |
| 229 | + | } | |
| 230 | + | for _, c := range msg.Content { | |
| 231 | + | if c["phase"] == "mystery" { | |
| 232 | + | found = true | |
| 233 | + | } | |
| 234 | + | } | |
| 235 | + | return nil | |
| 236 | + | }); err != nil { | |
| 237 | + | t.Fatal(err) | |
| 238 | + | } | |
| 239 | + | if !found { | |
| 240 | + | t.Fatalf("unrecognized phase not persisted in %s", data) | |
| 241 | + | } | |
| 242 | + | reopened, msgs, err := OpenSession(path) | |
| 243 | + | if err != nil { | |
| 244 | + | t.Fatalf("OpenSession: %v", err) | |
| 245 | + | } | |
| 246 | + | defer reopened.Close() | |
| 247 | + | if len(msgs) != 1 || phaseOf(msgs[0].Content[0]) != "mystery" { | |
| 248 | + | t.Fatalf("msgs = %#v, want preserved mystery phase", msgs) | |
| 249 | + | } | |
| 250 | + | } | |
| @@ -0,0 +1,250 @@ | |||||
| 1 | + | package core | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "encoding/json" | |||
| 5 | + | "os" | |||
| 6 | + | "path/filepath" | |||
| 7 | + | "testing" | |||
| 8 | + | ||||
| 9 | + | "github.com/bnema/zut/packages/provider" | |||
| 10 | + | ) | |||
| 11 | + | ||||
| 12 | + | func phaseOf(block provider.Content) string { | |||
| 13 | + | tb, ok := block.(provider.TextBlock) | |||
| 14 | + | if !ok { | |||
| 15 | + | return "<non-text>" | |||
| 16 | + | } | |||
| 17 | + | return tb.Phase | |||
| 18 | + | } | |||
| 19 | + | ||||
| 20 | + | func appendPhasedAssistant(t *testing.T, s *Session) { | |||
| 21 | + | t.Helper() | |||
| 22 | + | if err := s.AppendMessage(provider.Message{ | |||
| 23 | + | Role: provider.RoleAssistant, | |||
| 24 | + | Content: []provider.Content{ | |||
| 25 | + | provider.TextBlock{Text: "working on it", Phase: provider.TextPhaseCommentary}, | |||
| 26 | + | provider.TextBlock{Text: "done", Phase: provider.TextPhaseFinal}, | |||
| 27 | + | provider.TextBlock{Text: "plain"}, | |||
| 28 | + | }, | |||
| 29 | + | }); err != nil { | |||
| 30 | + | t.Fatal(err) | |||
| 31 | + | } | |||
| 32 | + | } | |||
| 33 | + | ||||
| 34 | + | func TestSessionPhaseOldUnphasedFixture(t *testing.T) { | |||
| 35 | + | raw := `{"role":"assistant","content":[{"text":"hello"}],"time":"2026-01-01T00:00:00Z"}` | |||
| 36 | + | msg, err := DecodeMessageJSON([]byte(raw)) | |||
| 37 | + | if err != nil { | |||
| 38 | + | t.Fatalf("DecodeMessageJSON: %v", err) | |||
| 39 | + | } | |||
| 40 | + | if len(msg.Content) != 1 { | |||
| 41 | + | t.Fatalf("content = %#v, want 1 block", msg.Content) | |||
| 42 | + | } | |||
| 43 | + | if got := phaseOf(msg.Content[0]); got != "" { | |||
| 44 | + | t.Fatalf("phase = %q, want empty", got) | |||
| 45 | + | } | |||
| 46 | + | } | |||
| 47 | + | ||||
| 48 | + | func TestSessionPhaseMixedRoundTrip(t *testing.T) { | |||
| 49 | + | root := t.TempDir() | |||
| 50 | + | cwd := t.TempDir() | |||
| 51 | + | s, err := NewSession(root, cwd, "provider", "model", "test") | |||
| 52 | + | if err != nil { | |||
| 53 | + | t.Fatal(err) | |||
| 54 | + | } | |||
| 55 | + | appendPhasedAssistant(t, s) | |||
| 56 | + | path := s.Path | |||
| 57 | + | if err := s.Close(); err != nil { | |||
| 58 | + | t.Fatal(err) | |||
| 59 | + | } | |||
| 60 | + | reopened, msgs, err := OpenSession(path) | |||
| 61 | + | if err != nil { | |||
| 62 | + | t.Fatalf("OpenSession: %v", err) | |||
| 63 | + | } | |||
| 64 | + | defer reopened.Close() | |||
| 65 | + | if len(msgs) != 1 || len(msgs[0].Content) != 3 { | |||
| 66 | + | t.Fatalf("msgs = %#v, want 1 message with 3 blocks", msgs) | |||
| 67 | + | } | |||
| 68 | + | want := []string{provider.TextPhaseCommentary, provider.TextPhaseFinal, ""} | |||
| 69 | + | for i, w := range want { | |||
| 70 | + | if got := phaseOf(msgs[0].Content[i]); got != w { | |||
| 71 | + | t.Fatalf("block %d phase = %q, want %q", i, got, w) | |||
| 72 | + | } | |||
| 73 | + | } | |||
| 74 | + | } | |||
| 75 | + | ||||
| 76 | + | func TestSessionPhaseResumeFork(t *testing.T) { | |||
| 77 | + | root := t.TempDir() | |||
| 78 | + | cwd := t.TempDir() | |||
| 79 | + | s, err := NewSession(root, cwd, "provider", "model", "test") | |||
| 80 | + | if err != nil { | |||
| 81 | + | t.Fatal(err) | |||
| 82 | + | } | |||
| 83 | + | if err := s.AppendMessage(provider.Message{ | |||
| 84 | + | Role: provider.RoleUser, | |||
| 85 | + | Content: []provider.Content{provider.TextBlock{Text: "hi"}}, | |||
| 86 | + | }); err != nil { | |||
| 87 | + | t.Fatal(err) | |||
| 88 | + | } | |||
| 89 | + | appendPhasedAssistant(t, s) | |||
| 90 | + | parentPath := s.Path | |||
| 91 | + | if err := s.Close(); err != nil { | |||
| 92 | + | t.Fatal(err) | |||
| 93 | + | } | |||
| 94 | + | ||||
| 95 | + | // Resume: reopening the same file preserves phases. | |||
| 96 | + | reopened, msgs, err := OpenSession(parentPath) | |||
| 97 | + | if err != nil { | |||
| 98 | + | t.Fatalf("OpenSession: %v", err) | |||
| 99 | + | } | |||
| 100 | + | reopened.Close() | |||
| 101 | + | if len(msgs) != 2 || len(msgs[1].Content) != 3 { | |||
| 102 | + | t.Fatalf("resume msgs = %#v, want 2 messages", msgs) | |||
| 103 | + | } | |||
| 104 | + | if got := phaseOf(msgs[1].Content[0]); got != provider.TextPhaseCommentary { | |||
| 105 | + | t.Fatalf("resume phase = %q, want commentary", got) | |||
| 106 | + | } | |||
| 107 | + | ||||
| 108 | + | // Fork: branching preserves phases. | |||
| 109 | + | branchPath, err := BranchSession(parentPath, root, cwd, "test", 2) | |||
| 110 | + | if err != nil { | |||
| 111 | + | t.Fatalf("BranchSession: %v", err) | |||
| 112 | + | } | |||
| 113 | + | branch, branchMsgs, err := OpenSession(branchPath) | |||
| 114 | + | if err != nil { | |||
| 115 | + | t.Fatalf("OpenSession(branch): %v", err) | |||
| 116 | + | } | |||
| 117 | + | defer branch.Close() | |||
| 118 | + | if len(branchMsgs) != 2 || len(branchMsgs[1].Content) != 3 { | |||
| 119 | + | t.Fatalf("fork msgs = %#v, want 2 messages", branchMsgs) | |||
| 120 | + | } | |||
| 121 | + | want := []string{provider.TextPhaseCommentary, provider.TextPhaseFinal, ""} | |||
| 122 | + | for i, w := range want { | |||
| 123 | + | if got := phaseOf(branchMsgs[1].Content[i]); got != w { | |||
| 124 | + | t.Fatalf("fork block %d phase = %q, want %q", i, got, w) | |||
| 125 | + | } | |||
| 126 | + | } | |||
| 127 | + | } | |||
| 128 | + | ||||
| 129 | + | func TestSessionPhasePortableImportExport(t *testing.T) { | |||
| 130 | + | root := t.TempDir() | |||
| 131 | + | cwd := t.TempDir() | |||
| 132 | + | s, err := NewSession(root, cwd, "provider", "model", "test") | |||
| 133 | + | if err != nil { | |||
| 134 | + | t.Fatal(err) | |||
| 135 | + | } | |||
| 136 | + | appendPhasedAssistant(t, s) | |||
| 137 | + | srcPath := s.Path | |||
| 138 | + | if err := s.Close(); err != nil { | |||
| 139 | + | t.Fatal(err) | |||
| 140 | + | } | |||
| 141 | + | exportPath := filepath.Join(t.TempDir(), "export.json") | |||
| 142 | + | out, err := ExportSession(srcPath, exportPath) | |||
| 143 | + | if err != nil { | |||
| 144 | + | t.Fatalf("ExportSession: %v", err) | |||
| 145 | + | } | |||
| 146 | + | importedPath, err := ImportSession(out, root, cwd, "test") | |||
| 147 | + | if err != nil { | |||
| 148 | + | t.Fatalf("ImportSession: %v", err) | |||
| 149 | + | } | |||
| 150 | + | imported, msgs, err := OpenSession(importedPath) | |||
| 151 | + | if err != nil { | |||
| 152 | + | t.Fatalf("OpenSession(imported): %v", err) | |||
| 153 | + | } | |||
| 154 | + | defer imported.Close() | |||
| 155 | + | if len(msgs) != 1 || len(msgs[0].Content) != 3 { | |||
| 156 | + | t.Fatalf("imported msgs = %#v, want 1 message with 3 blocks", msgs) | |||
| 157 | + | } | |||
| 158 | + | want := []string{provider.TextPhaseCommentary, provider.TextPhaseFinal, ""} | |||
| 159 | + | for i, w := range want { | |||
| 160 | + | if got := phaseOf(msgs[0].Content[i]); got != w { | |||
| 161 | + | t.Fatalf("imported block %d phase = %q, want %q", i, got, w) | |||
| 162 | + | } | |||
| 163 | + | } | |||
| 164 | + | } | |||
| 165 | + | ||||
| 166 | + | func TestSessionPhaseRetainedProjection(t *testing.T) { | |||
| 167 | + | msgs := []provider.Message{ | |||
| 168 | + | {Role: provider.RoleAssistant, Content: []provider.Content{ | |||
| 169 | + | provider.TextBlock{Text: "progress", Phase: provider.TextPhaseCommentary}, | |||
| 170 | + | provider.TextBlock{Text: "answer", Phase: provider.TextPhaseFinal}, | |||
| 171 | + | }}, | |||
| 172 | + | } | |||
| 173 | + | projected := projectProviderMessages(msgs) | |||
| 174 | + | if len(projected) != 1 || len(projected[0].Content) != 2 { | |||
| 175 | + | t.Fatalf("projected = %#v, want 2 retained blocks", projected) | |||
| 176 | + | } | |||
| 177 | + | if got := phaseOf(projected[0].Content[0]); got != provider.TextPhaseCommentary { | |||
| 178 | + | t.Fatalf("projected[0] phase = %q, want commentary", got) | |||
| 179 | + | } | |||
| 180 | + | if got := phaseOf(projected[0].Content[1]); got != provider.TextPhaseFinal { | |||
| 181 | + | t.Fatalf("projected[1] phase = %q, want final_answer", got) | |||
| 182 | + | } | |||
| 183 | + | // Original must not be mutated by projection. | |||
| 184 | + | if got := phaseOf(msgs[0].Content[0]); got != provider.TextPhaseCommentary { | |||
| 185 | + | t.Fatalf("original mutated: phase = %q", got) | |||
| 186 | + | } | |||
| 187 | + | } | |||
| 188 | + | ||||
| 189 | + | func TestSessionPhaseUnknownPreserved(t *testing.T) { | |||
| 190 | + | root := t.TempDir() | |||
| 191 | + | cwd := t.TempDir() | |||
| 192 | + | s, err := NewSession(root, cwd, "provider", "model", "test") | |||
| 193 | + | if err != nil { | |||
| 194 | + | t.Fatal(err) | |||
| 195 | + | } | |||
| 196 | + | if err := s.AppendMessage(provider.Message{ | |||
| 197 | + | Role: provider.RoleAssistant, | |||
| 198 | + | Content: []provider.Content{provider.TextBlock{Text: "odd", Phase: "mystery"}}, | |||
| 199 | + | }); err != nil { | |||
| 200 | + | t.Fatal(err) | |||
| 201 | + | } | |||
| 202 | + | path := s.Path | |||
| 203 | + | if err := s.Close(); err != nil { | |||
| 204 | + | t.Fatal(err) | |||
| 205 | + | } | |||
| 206 | + | // Raw file must carry the unrecognized phase verbatim. | |||
| 207 | + | data, err := os.ReadFile(path) | |||
| 208 | + | if err != nil { | |||
| 209 | + | t.Fatal(err) | |||
| 210 | + | } | |||
| 211 | + | var found bool | |||
| 212 | + | f, err := os.Open(path) | |||
| 213 | + | if err != nil { | |||
| 214 | + | t.Fatal(err) | |||
| 215 | + | } | |||
| 216 | + | defer f.Close() | |||
| 217 | + | if err := forEachJSONLLine(f, func(line []byte) error { | |||
| 218 | + | var row struct { | |||
| 219 | + | Message *json.RawMessage `json:"message"` | |||
| 220 | + | } | |||
| 221 | + | if err := json.Unmarshal(line, &row); err != nil || row.Message == nil { | |||
| 222 | + | return nil | |||
| 223 | + | } | |||
| 224 | + | var msg struct { | |||
| 225 | + | Content []map[string]any `json:"content"` | |||
| 226 | + | } | |||
| 227 | + | if err := json.Unmarshal(*row.Message, &msg); err != nil { | |||
| 228 | + | return nil | |||
| 229 | + | } | |||
| 230 | + | for _, c := range msg.Content { | |||
| 231 | + | if c["phase"] == "mystery" { | |||
| 232 | + | found = true | |||
| 233 | + | } | |||
| 234 | + | } | |||
| 235 | + | return nil | |||
| 236 | + | }); err != nil { | |||
| 237 | + | t.Fatal(err) | |||
| 238 | + | } | |||
| 239 | + | if !found { | |||
| 240 | + | t.Fatalf("unrecognized phase not persisted in %s", data) | |||
| 241 | + | } | |||
| 242 | + | reopened, msgs, err := OpenSession(path) | |||
| 243 | + | if err != nil { | |||
| 244 | + | t.Fatalf("OpenSession: %v", err) | |||
| 245 | + | } | |||
| 246 | + | defer reopened.Close() | |||
| 247 | + | if len(msgs) != 1 || phaseOf(msgs[0].Content[0]) != "mystery" { | |||
| 248 | + | t.Fatalf("msgs = %#v, want preserved mystery phase", msgs) | |||
| 249 | + | } | |||
| 250 | + | } | |||
A packages/core/turn_recovery.go +118 -0
| @@ -0,0 +1,118 @@ | |||
| 1 | + | package core | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "errors" | |
| 5 | + | "strings" | |
| 6 | + | ||
| 7 | + | "github.com/bnema/zut/packages/provider" | |
| 8 | + | ) | |
| 9 | + | ||
| 10 | + | // ErrIncompleteTurn reports that a normal turn ended without a final answer | |
| 11 | + | // twice in one top-level invocation, even after one bounded recovery | |
| 12 | + | // continuation. Hosts surface it as a visible incomplete outcome, never as | |
| 13 | + | // success. It matches with errors.Is. | |
| 14 | + | var ErrIncompleteTurn = errors.New("agent turn ended without a final answer") | |
| 15 | + | ||
| 16 | + | // Bounded normal-turn recovery reasons emitted on EvTurnRecovery. | |
| 17 | + | const ( | |
| 18 | + | TurnRecoveryReasonMissingAnswer = "missing_answer" | |
| 19 | + | TurnRecoveryReasonCommentaryOnly = "commentary_only" | |
| 20 | + | ) | |
| 21 | + | ||
| 22 | + | // turnRecoveryMetaKey marks the synthetic user-context message appended for | |
| 23 | + | // a recovery continuation, following the existing goal synthetic-context | |
| 24 | + | // pattern. | |
| 25 | + | const turnRecoveryMetaKey = "zut_turn_recovery" | |
| 26 | + | ||
| 27 | + | // turnRecoveryPrompt is the fixed bounded continuation appended on an | |
| 28 | + | // eligible incomplete normal stop. | |
| 29 | + | const turnRecoveryPrompt = "The request ended without a final answer. Continue the authorized work if it remains actionable. Otherwise provide the result, the concrete blocker, or the specific decision required. Do not merely promise to continue. Respect the user's scope, tool permissions, required confirmations, and any pending required work." | |
| 30 | + | ||
| 31 | + | // classifyTerminalAssistant classifies the current terminal assistant | |
| 32 | + | // message of a successful normal stop. It returns the recovery reason and | |
| 33 | + | // true when the message carries no final answer: empty, whitespace-only, or | |
| 34 | + | // reasoning-only content yields missing_answer; nonempty text exclusively | |
| 35 | + | // marked commentary yields commentary_only. | |
| 36 | + | // | |
| 37 | + | // A non-whitespace text block with empty, unknown, or final phase is an | |
| 38 | + | // answer, as is any user-facing non-text output such as an image. Reasoning | |
| 39 | + | // and tool-call blocks are never answers. A nonempty refusal, clarification | |
| 40 | + | // question, or short final answer arrives as ordinary text and therefore | |
| 41 | + | // counts as complete; no prose sniffing is performed here. | |
| 42 | + | func classifyTerminalAssistant(msg provider.Message) (reason string, incomplete bool) { | |
| 43 | + | commentaryOnly := false | |
| 44 | + | for _, c := range msg.Content { | |
| 45 | + | switch b := c.(type) { | |
| 46 | + | case provider.TextBlock: | |
| 47 | + | if strings.TrimSpace(b.Text) == "" { | |
| 48 | + | continue | |
| 49 | + | } | |
| 50 | + | if b.Phase == provider.TextPhaseCommentary { | |
| 51 | + | commentaryOnly = true | |
| 52 | + | continue | |
| 53 | + | } | |
| 54 | + | // Empty, unknown, and final phases all retain compatibility | |
| 55 | + | // behavior: the text counts as an answer. | |
| 56 | + | return "", false | |
| 57 | + | case provider.ImageBlock: | |
| 58 | + | return "", false | |
| 59 | + | default: | |
| 60 | + | // Reasoning, tool-call, and tool-result blocks are not | |
| 61 | + | // user-facing answers. | |
| 62 | + | continue | |
| 63 | + | } | |
| 64 | + | } | |
| 65 | + | if commentaryOnly { | |
| 66 | + | return TurnRecoveryReasonCommentaryOnly, true | |
| 67 | + | } | |
| 68 | + | return TurnRecoveryReasonMissingAnswer, true | |
| 69 | + | } | |
| 70 | + | ||
| 71 | + | // ToolDeniedError marks a tool execution refusal (permission scope, | |
| 72 | + | // allowlist, jail confinement, or admission gate) as distinct from an | |
| 73 | + | // ordinary tool failure. Tools return it through Execute; runOneTool | |
| 74 | + | // records the refusal so a later incomplete normal stop does not gain a | |
| 75 | + | // recovery prompt urging more actions. The Reason text is shown to the | |
| 76 | + | // model as the tool result, unchanged from the untyped errors it replaces. | |
| 77 | + | type ToolDeniedError struct { | |
| 78 | + | Reason string | |
| 79 | + | } | |
| 80 | + | ||
| 81 | + | func (e *ToolDeniedError) Error() string { | |
| 82 | + | if e == nil || e.Reason == "" { | |
| 83 | + | return "tool call denied" | |
| 84 | + | } | |
| 85 | + | return e.Reason | |
| 86 | + | } | |
| 87 | + | ||
| 88 | + | // asToolDeniedError reports whether err chains to a ToolDeniedError. | |
| 89 | + | func asToolDeniedError(err error) bool { | |
| 90 | + | var denied *ToolDeniedError | |
| 91 | + | return errors.As(err, &denied) | |
| 92 | + | } | |
| 93 | + | ||
| 94 | + | // markDeniedToolCall records that a tool invocation was refused at the | |
| 95 | + | // execution boundary (guard or confirmation denial, or a denied tool | |
| 96 | + | // execution). A denied invocation must not gain a recovery prompt urging | |
| 97 | + | // more actions. | |
| 98 | + | func (a *Agent) markDeniedToolCall() { | |
| 99 | + | a.mu.Lock() | |
| 100 | + | a.deniedToolCall = true | |
| 101 | + | a.mu.Unlock() | |
| 102 | + | } | |
| 103 | + | ||
| 104 | + | // sawDeniedToolCall reports whether any tool invocation was denied during | |
| 105 | + | // the current top-level runLoop invocation. | |
| 106 | + | func (a *Agent) sawDeniedToolCall() bool { | |
| 107 | + | a.mu.Lock() | |
| 108 | + | defer a.mu.Unlock() | |
| 109 | + | return a.deniedToolCall | |
| 110 | + | } | |
| 111 | + | ||
| 112 | + | // resetDeniedToolCalls clears the execution-local denial flag at the start | |
| 113 | + | // of each top-level Prompt/Continue invocation. | |
| 114 | + | func (a *Agent) resetDeniedToolCalls() { | |
| 115 | + | a.mu.Lock() | |
| 116 | + | a.deniedToolCall = false | |
| 117 | + | a.mu.Unlock() | |
| 118 | + | } | |
| @@ -0,0 +1,118 @@ | |||||
| 1 | + | package core | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "errors" | |||
| 5 | + | "strings" | |||
| 6 | + | ||||
| 7 | + | "github.com/bnema/zut/packages/provider" | |||
| 8 | + | ) | |||
| 9 | + | ||||
| 10 | + | // ErrIncompleteTurn reports that a normal turn ended without a final answer | |||
| 11 | + | // twice in one top-level invocation, even after one bounded recovery | |||
| 12 | + | // continuation. Hosts surface it as a visible incomplete outcome, never as | |||
| 13 | + | // success. It matches with errors.Is. | |||
| 14 | + | var ErrIncompleteTurn = errors.New("agent turn ended without a final answer") | |||
| 15 | + | ||||
| 16 | + | // Bounded normal-turn recovery reasons emitted on EvTurnRecovery. | |||
| 17 | + | const ( | |||
| 18 | + | TurnRecoveryReasonMissingAnswer = "missing_answer" | |||
| 19 | + | TurnRecoveryReasonCommentaryOnly = "commentary_only" | |||
| 20 | + | ) | |||
| 21 | + | ||||
| 22 | + | // turnRecoveryMetaKey marks the synthetic user-context message appended for | |||
| 23 | + | // a recovery continuation, following the existing goal synthetic-context | |||
| 24 | + | // pattern. | |||
| 25 | + | const turnRecoveryMetaKey = "zut_turn_recovery" | |||
| 26 | + | ||||
| 27 | + | // turnRecoveryPrompt is the fixed bounded continuation appended on an | |||
| 28 | + | // eligible incomplete normal stop. | |||
| 29 | + | const turnRecoveryPrompt = "The request ended without a final answer. Continue the authorized work if it remains actionable. Otherwise provide the result, the concrete blocker, or the specific decision required. Do not merely promise to continue. Respect the user's scope, tool permissions, required confirmations, and any pending required work." | |||
| 30 | + | ||||
| 31 | + | // classifyTerminalAssistant classifies the current terminal assistant | |||
| 32 | + | // message of a successful normal stop. It returns the recovery reason and | |||
| 33 | + | // true when the message carries no final answer: empty, whitespace-only, or | |||
| 34 | + | // reasoning-only content yields missing_answer; nonempty text exclusively | |||
| 35 | + | // marked commentary yields commentary_only. | |||
| 36 | + | // | |||
| 37 | + | // A non-whitespace text block with empty, unknown, or final phase is an | |||
| 38 | + | // answer, as is any user-facing non-text output such as an image. Reasoning | |||
| 39 | + | // and tool-call blocks are never answers. A nonempty refusal, clarification | |||
| 40 | + | // question, or short final answer arrives as ordinary text and therefore | |||
| 41 | + | // counts as complete; no prose sniffing is performed here. | |||
| 42 | + | func classifyTerminalAssistant(msg provider.Message) (reason string, incomplete bool) { | |||
| 43 | + | commentaryOnly := false | |||
| 44 | + | for _, c := range msg.Content { | |||
| 45 | + | switch b := c.(type) { | |||
| 46 | + | case provider.TextBlock: | |||
| 47 | + | if strings.TrimSpace(b.Text) == "" { | |||
| 48 | + | continue | |||
| 49 | + | } | |||
| 50 | + | if b.Phase == provider.TextPhaseCommentary { | |||
| 51 | + | commentaryOnly = true | |||
| 52 | + | continue | |||
| 53 | + | } | |||
| 54 | + | // Empty, unknown, and final phases all retain compatibility | |||
| 55 | + | // behavior: the text counts as an answer. | |||
| 56 | + | return "", false | |||
| 57 | + | case provider.ImageBlock: | |||
| 58 | + | return "", false | |||
| 59 | + | default: | |||
| 60 | + | // Reasoning, tool-call, and tool-result blocks are not | |||
| 61 | + | // user-facing answers. | |||
| 62 | + | continue | |||
| 63 | + | } | |||
| 64 | + | } | |||
| 65 | + | if commentaryOnly { | |||
| 66 | + | return TurnRecoveryReasonCommentaryOnly, true | |||
| 67 | + | } | |||
| 68 | + | return TurnRecoveryReasonMissingAnswer, true | |||
| 69 | + | } | |||
| 70 | + | ||||
| 71 | + | // ToolDeniedError marks a tool execution refusal (permission scope, | |||
| 72 | + | // allowlist, jail confinement, or admission gate) as distinct from an | |||
| 73 | + | // ordinary tool failure. Tools return it through Execute; runOneTool | |||
| 74 | + | // records the refusal so a later incomplete normal stop does not gain a | |||
| 75 | + | // recovery prompt urging more actions. The Reason text is shown to the | |||
| 76 | + | // model as the tool result, unchanged from the untyped errors it replaces. | |||
| 77 | + | type ToolDeniedError struct { | |||
| 78 | + | Reason string | |||
| 79 | + | } | |||
| 80 | + | ||||
| 81 | + | func (e *ToolDeniedError) Error() string { | |||
| 82 | + | if e == nil || e.Reason == "" { | |||
| 83 | + | return "tool call denied" | |||
| 84 | + | } | |||
| 85 | + | return e.Reason | |||
| 86 | + | } | |||
| 87 | + | ||||
| 88 | + | // asToolDeniedError reports whether err chains to a ToolDeniedError. | |||
| 89 | + | func asToolDeniedError(err error) bool { | |||
| 90 | + | var denied *ToolDeniedError | |||
| 91 | + | return errors.As(err, &denied) | |||
| 92 | + | } | |||
| 93 | + | ||||
| 94 | + | // markDeniedToolCall records that a tool invocation was refused at the | |||
| 95 | + | // execution boundary (guard or confirmation denial, or a denied tool | |||
| 96 | + | // execution). A denied invocation must not gain a recovery prompt urging | |||
| 97 | + | // more actions. | |||
| 98 | + | func (a *Agent) markDeniedToolCall() { | |||
| 99 | + | a.mu.Lock() | |||
| 100 | + | a.deniedToolCall = true | |||
| 101 | + | a.mu.Unlock() | |||
| 102 | + | } | |||
| 103 | + | ||||
| 104 | + | // sawDeniedToolCall reports whether any tool invocation was denied during | |||
| 105 | + | // the current top-level runLoop invocation. | |||
| 106 | + | func (a *Agent) sawDeniedToolCall() bool { | |||
| 107 | + | a.mu.Lock() | |||
| 108 | + | defer a.mu.Unlock() | |||
| 109 | + | return a.deniedToolCall | |||
| 110 | + | } | |||
| 111 | + | ||||
| 112 | + | // resetDeniedToolCalls clears the execution-local denial flag at the start | |||
| 113 | + | // of each top-level Prompt/Continue invocation. | |||
| 114 | + | func (a *Agent) resetDeniedToolCalls() { | |||
| 115 | + | a.mu.Lock() | |||
| 116 | + | a.deniedToolCall = false | |||
| 117 | + | a.mu.Unlock() | |||
| 118 | + | } | |||
A packages/core/turn_recovery_test.go +636 -0
| @@ -0,0 +1,636 @@ | |||
| 1 | + | package core | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "context" | |
| 5 | + | "encoding/json" | |
| 6 | + | "errors" | |
| 7 | + | "strings" | |
| 8 | + | "sync" | |
| 9 | + | "sync/atomic" | |
| 10 | + | "testing" | |
| 11 | + | "time" | |
| 12 | + | ||
| 13 | + | "github.com/bnema/zut/packages/provider" | |
| 14 | + | ) | |
| 15 | + | ||
| 16 | + | // turnRecoveryScriptClient replays a scripted sequence of provider turns, | |
| 17 | + | // one per Stream call. A nil-content entry emits an assistant message with | |
| 18 | + | // no content blocks; unknownStop entries exercise non-end stops. | |
| 19 | + | type turnRecoveryScriptClient struct { | |
| 20 | + | calls int32 | |
| 21 | + | turns []turnRecoveryTurn | |
| 22 | + | } | |
| 23 | + | ||
| 24 | + | type turnRecoveryTurn struct { | |
| 25 | + | content []provider.Content | |
| 26 | + | stop provider.StopReason | |
| 27 | + | err error | |
| 28 | + | } | |
| 29 | + | ||
| 30 | + | func (c *turnRecoveryScriptClient) Name() string { return "turn-recovery-script" } | |
| 31 | + | ||
| 32 | + | func (c *turnRecoveryScriptClient) Stream(_ context.Context, req provider.Request) (<-chan provider.Event, error) { | |
| 33 | + | call := int(atomic.AddInt32(&c.calls, 1)) - 1 | |
| 34 | + | turn := turnRecoveryTurn{stop: provider.StopEnd} | |
| 35 | + | if call < len(c.turns) { | |
| 36 | + | turn = c.turns[call] | |
| 37 | + | } | |
| 38 | + | out := make(chan provider.Event, 8) | |
| 39 | + | out <- provider.EventStart{Provider: c.Name(), Model: req.Model} | |
| 40 | + | out <- provider.EventUsage{Usage: provider.Usage{InputTokens: 1, OutputTokens: 1}} | |
| 41 | + | msg := provider.Message{Role: provider.RoleAssistant, Content: turn.content, Time: time.Now()} | |
| 42 | + | out <- provider.EventDone{Stop: turn.stop, Err: turn.err, Message: msg} | |
| 43 | + | close(out) | |
| 44 | + | return out, nil | |
| 45 | + | } | |
| 46 | + | ||
| 47 | + | func (c *turnRecoveryScriptClient) Calls() int32 { return atomic.LoadInt32(&c.calls) } | |
| 48 | + | ||
| 49 | + | func textContent(text, phase string) provider.Content { | |
| 50 | + | return provider.TextBlock{Text: text, Phase: phase} | |
| 51 | + | } | |
| 52 | + | ||
| 53 | + | func recoveryEvents(sink *[]AgentEvent) func(AgentEvent) { | |
| 54 | + | return func(ev AgentEvent) { | |
| 55 | + | *sink = append(*sink, ev) | |
| 56 | + | } | |
| 57 | + | } | |
| 58 | + | ||
| 59 | + | func countEvents(events []AgentEvent, want func(AgentEvent) bool) int { | |
| 60 | + | n := 0 | |
| 61 | + | for _, ev := range events { | |
| 62 | + | if want(ev) { | |
| 63 | + | n++ | |
| 64 | + | } | |
| 65 | + | } | |
| 66 | + | return n | |
| 67 | + | } | |
| 68 | + | ||
| 69 | + | func isEvDone(ev AgentEvent) bool { _, ok := ev.(EvDone); return ok } | |
| 70 | + | ||
| 71 | + | func isTurnEnd(stop provider.StopReason) func(AgentEvent) bool { | |
| 72 | + | return func(ev AgentEvent) bool { | |
| 73 | + | te, ok := ev.(EvTurnEnd) | |
| 74 | + | return ok && te.Stop == stop && te.Err == nil | |
| 75 | + | } | |
| 76 | + | } | |
| 77 | + | ||
| 78 | + | func lastAssistantMessage(t *testing.T, agent *Agent) provider.Message { | |
| 79 | + | t.Helper() | |
| 80 | + | msgs := agent.Messages() | |
| 81 | + | for i := len(msgs) - 1; i >= 0; i-- { | |
| 82 | + | if msgs[i].Role == provider.RoleAssistant { | |
| 83 | + | return msgs[i] | |
| 84 | + | } | |
| 85 | + | } | |
| 86 | + | t.Fatal("no assistant message in transcript") | |
| 87 | + | return provider.Message{} | |
| 88 | + | } | |
| 89 | + | ||
| 90 | + | func recoverySynthetic(t *testing.T, agent *Agent) provider.Message { | |
| 91 | + | t.Helper() | |
| 92 | + | for _, m := range agent.Messages() { | |
| 93 | + | if m.Role == provider.RoleUser && m.Meta[turnRecoveryMetaKey] == "true" { | |
| 94 | + | return m | |
| 95 | + | } | |
| 96 | + | } | |
| 97 | + | t.Fatal("no recovery synthetic user message in transcript") | |
| 98 | + | return provider.Message{} | |
| 99 | + | } | |
| 100 | + | ||
| 101 | + | func runPrompt(t *testing.T, agent *Agent, events *[]AgentEvent) error { | |
| 102 | + | t.Helper() | |
| 103 | + | ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) | |
| 104 | + | defer cancel() | |
| 105 | + | return agent.Prompt(ctx, "start", nil, recoveryEvents(events)) | |
| 106 | + | } | |
| 107 | + | ||
| 108 | + | // TestTurnRecoveryEmptyFinalAfterTool: a tool turn followed by an empty | |
| 109 | + | // normal terminal response triggers one recovery and can then finish. | |
| 110 | + | func TestTurnRecoveryEmptyFinalAfterTool(t *testing.T) { | |
| 111 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 112 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |
| 113 | + | {content: nil, stop: provider.StopEnd}, | |
| 114 | + | {content: []provider.Content{textContent("done", "")}, stop: provider.StopEnd}, | |
| 115 | + | }} | |
| 116 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |
| 117 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |
| 118 | + | }) | |
| 119 | + | var events []AgentEvent | |
| 120 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 121 | + | t.Fatalf("Prompt returned %v", err) | |
| 122 | + | } | |
| 123 | + | if got := client.Calls(); got != 3 { | |
| 124 | + | t.Fatalf("provider calls = %d, want 3", got) | |
| 125 | + | } | |
| 126 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 127 | + | r, ok := ev.(EvTurnRecovery) | |
| 128 | + | return ok && r.Reason == TurnRecoveryReasonMissingAnswer && r.Attempt == 1 | |
| 129 | + | }); got != 1 { | |
| 130 | + | t.Fatalf("EvTurnRecovery count = %d, want 1 missing_answer", got) | |
| 131 | + | } | |
| 132 | + | if got := lastAssistantMessage(t, agent).Content[0].(provider.TextBlock).Text; got != "done" { | |
| 133 | + | t.Fatalf("final text = %q, want done", got) | |
| 134 | + | } | |
| 135 | + | synth := recoverySynthetic(t, agent) | |
| 136 | + | if !strings.Contains(synth.Content[0].(provider.TextBlock).Text, "without a final answer") { | |
| 137 | + | t.Fatalf("recovery text = %q", synth.Content[0].(provider.TextBlock).Text) | |
| 138 | + | } | |
| 139 | + | // Tool-result pairing survives the recovery round trip. | |
| 140 | + | var toolResults int | |
| 141 | + | for _, m := range agent.Messages() { | |
| 142 | + | if m.Role == provider.RoleTool { | |
| 143 | + | toolResults++ | |
| 144 | + | } | |
| 145 | + | } | |
| 146 | + | if toolResults != 1 { | |
| 147 | + | t.Fatalf("tool messages = %d, want 1", toolResults) | |
| 148 | + | } | |
| 149 | + | } | |
| 150 | + | ||
| 151 | + | // TestTurnRecoveryCommentaryOnly: explicitly commentary-only normal | |
| 152 | + | // completion triggers one recovery; the commentary reason is reported. | |
| 153 | + | func TestTurnRecoveryCommentaryOnly(t *testing.T) { | |
| 154 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 155 | + | {content: []provider.Content{textContent("working on it", provider.TextPhaseCommentary)}, stop: provider.StopEnd}, | |
| 156 | + | {content: []provider.Content{textContent("result", "")}, stop: provider.StopEnd}, | |
| 157 | + | }} | |
| 158 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 159 | + | var events []AgentEvent | |
| 160 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 161 | + | t.Fatalf("Prompt returned %v", err) | |
| 162 | + | } | |
| 163 | + | if got := client.Calls(); got != 2 { | |
| 164 | + | t.Fatalf("provider calls = %d, want 2", got) | |
| 165 | + | } | |
| 166 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 167 | + | r, ok := ev.(EvTurnRecovery) | |
| 168 | + | return ok && r.Reason == TurnRecoveryReasonCommentaryOnly && r.Attempt == 1 | |
| 169 | + | }); got != 1 { | |
| 170 | + | t.Fatalf("commentary_only EvTurnRecovery count = %d, want 1", got) | |
| 171 | + | } | |
| 172 | + | if got := countEvents(events, isEvDone); got != 1 { | |
| 173 | + | t.Fatalf("EvDone count = %d, want exactly 1", got) | |
| 174 | + | } | |
| 175 | + | } | |
| 176 | + | ||
| 177 | + | // TestTurnRecoveryFinalAnswerNoRecovery covers the no-recovery cases: a | |
| 178 | + | // final answer, an unknown/absent phase, image output, and commentary- | |
| 179 | + | // then-final likewise complete without extra requests. | |
| 180 | + | func TestTurnRecoveryFinalAnswerNoRecovery(t *testing.T) { | |
| 181 | + | cases := map[string]turnRecoveryTurn{ | |
| 182 | + | "final answer": {content: []provider.Content{textContent("here is the result", provider.TextPhaseFinal)}}, | |
| 183 | + | "unknown phase": {content: []provider.Content{textContent("answer", "draft")}}, | |
| 184 | + | "absent phase": {content: []provider.Content{textContent("answer", "")}}, | |
| 185 | + | "refusal": {content: []provider.Content{textContent("I cannot do that.", "")}}, | |
| 186 | + | "clarification": {content: []provider.Content{textContent("Which file do you mean?", "")}}, | |
| 187 | + | "image output": {content: []provider.Content{provider.ImageBlock{MimeType: "image/png", Data: []byte{1, 2, 3}}}}, | |
| 188 | + | "commentary then fin": {content: []provider.Content{textContent("progress", provider.TextPhaseCommentary), textContent("result", provider.TextPhaseFinal)}}, | |
| 189 | + | "commentary then pla": {content: []provider.Content{textContent("progress", provider.TextPhaseCommentary), textContent("result", "")}}, | |
| 190 | + | } | |
| 191 | + | for name, turn := range cases { | |
| 192 | + | t.Run(name, func(t *testing.T) { | |
| 193 | + | turn.stop = provider.StopEnd | |
| 194 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{turn}} | |
| 195 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 196 | + | var events []AgentEvent | |
| 197 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 198 | + | t.Fatalf("Prompt returned %v", err) | |
| 199 | + | } | |
| 200 | + | if got := client.Calls(); got != 1 { | |
| 201 | + | t.Fatalf("provider calls = %d, want 1 (no recovery)", got) | |
| 202 | + | } | |
| 203 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 204 | + | _, ok := ev.(EvTurnRecovery) | |
| 205 | + | return ok | |
| 206 | + | }); got != 0 { | |
| 207 | + | t.Fatalf("EvTurnRecovery count = %d, want 0", got) | |
| 208 | + | } | |
| 209 | + | }) | |
| 210 | + | } | |
| 211 | + | } | |
| 212 | + | ||
| 213 | + | // TestAgentTurnRecoveryReasoningOnlyThenAnswer: a reasoning-only terminal | |
| 214 | + | // message carries no answer and recovers as missing_answer. | |
| 215 | + | func TestAgentTurnRecoveryReasoningOnlyThenAnswer(t *testing.T) { | |
| 216 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 217 | + | {content: []provider.Content{provider.ReasoningBlock{Summary: "thinking"}}, stop: provider.StopEnd}, | |
| 218 | + | {content: []provider.Content{textContent("result", "")}, stop: provider.StopEnd}, | |
| 219 | + | }} | |
| 220 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 221 | + | var events []AgentEvent | |
| 222 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 223 | + | t.Fatalf("Prompt returned %v", err) | |
| 224 | + | } | |
| 225 | + | if got := client.Calls(); got != 2 { | |
| 226 | + | t.Fatalf("provider calls = %d, want 2", got) | |
| 227 | + | } | |
| 228 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 229 | + | r, ok := ev.(EvTurnRecovery) | |
| 230 | + | return ok && r.Reason == TurnRecoveryReasonMissingAnswer | |
| 231 | + | }); got != 1 { | |
| 232 | + | t.Fatalf("missing_answer EvTurnRecovery count = %d, want 1", got) | |
| 233 | + | } | |
| 234 | + | } | |
| 235 | + | ||
| 236 | + | // TestAgentTurnRecoveryWhitespaceThenAnswer: a whitespace-only terminal | |
| 237 | + | // message recovers as missing_answer and can finish on the next turn. | |
| 238 | + | func TestAgentTurnRecoveryWhitespaceThenAnswer(t *testing.T) { | |
| 239 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 240 | + | {content: []provider.Content{textContent(" \n\t ", "")}, stop: provider.StopEnd}, | |
| 241 | + | {content: []provider.Content{textContent("result", "")}, stop: provider.StopEnd}, | |
| 242 | + | }} | |
| 243 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 244 | + | var events []AgentEvent | |
| 245 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 246 | + | t.Fatalf("Prompt returned %v", err) | |
| 247 | + | } | |
| 248 | + | if got := client.Calls(); got != 2 { | |
| 249 | + | t.Fatalf("provider calls = %d, want 2", got) | |
| 250 | + | } | |
| 251 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 252 | + | r, ok := ev.(EvTurnRecovery) | |
| 253 | + | return ok && r.Reason == TurnRecoveryReasonMissingAnswer | |
| 254 | + | }); got != 1 { | |
| 255 | + | t.Fatalf("missing_answer EvTurnRecovery count = %d, want 1", got) | |
| 256 | + | } | |
| 257 | + | } | |
| 258 | + | ||
| 259 | + | // TestAgentTurnRecoveryTwoIncompleteEndings: a second eligible incomplete | |
| 260 | + | // StopEnd returns sentinel ErrIncompleteTurn with exactly-once terminal | |
| 261 | + | // notification and no second model EvTurnEnd for the host decision. | |
| 262 | + | func TestAgentTurnRecoveryTwoIncompleteEndings(t *testing.T) { | |
| 263 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 264 | + | {content: nil, stop: provider.StopEnd}, | |
| 265 | + | {content: []provider.Content{textContent("still working", provider.TextPhaseCommentary)}, stop: provider.StopEnd}, | |
| 266 | + | }} | |
| 267 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 268 | + | var events []AgentEvent | |
| 269 | + | err := runPrompt(t, agent, &events) | |
| 270 | + | if !errors.Is(err, ErrIncompleteTurn) { | |
| 271 | + | t.Fatalf("Prompt error = %v, want ErrIncompleteTurn", err) | |
| 272 | + | } | |
| 273 | + | if got := client.Calls(); got != 2 { | |
| 274 | + | t.Fatalf("provider calls = %d, want 2 (bounded)", got) | |
| 275 | + | } | |
| 276 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 277 | + | _, ok := ev.(EvTurnRecovery) | |
| 278 | + | return ok | |
| 279 | + | }); got != 1 { | |
| 280 | + | t.Fatalf("EvTurnRecovery count = %d, want exactly 1", got) | |
| 281 | + | } | |
| 282 | + | if got := countEvents(events, isEvDone); got != 1 { | |
| 283 | + | t.Fatalf("EvDone count = %d, want exactly 1", got) | |
| 284 | + | } | |
| 285 | + | if got := countEvents(events, isTurnEnd(provider.StopEnd)); got != 2 { | |
| 286 | + | t.Fatalf("model StopEnd EvTurnEnd count = %d, want 2 (no second model turn for the host decision)", got) | |
| 287 | + | } | |
| 288 | + | } | |
| 289 | + | ||
| 290 | + | // TestAgentTurnRecoveryToolsBetweenIncomplete: calls stay bounded even when | |
| 291 | + | // tools execute between the two incomplete endings. | |
| 292 | + | func TestAgentTurnRecoveryToolsBetweenIncomplete(t *testing.T) { | |
| 293 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 294 | + | {content: nil, stop: provider.StopEnd}, | |
| 295 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |
| 296 | + | {content: nil, stop: provider.StopEnd}, | |
| 297 | + | }} | |
| 298 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |
| 299 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |
| 300 | + | }) | |
| 301 | + | var events []AgentEvent | |
| 302 | + | err := runPrompt(t, agent, &events) | |
| 303 | + | if !errors.Is(err, ErrIncompleteTurn) { | |
| 304 | + | t.Fatalf("Prompt error = %v, want ErrIncompleteTurn", err) | |
| 305 | + | } | |
| 306 | + | if got := client.Calls(); got != 3 { | |
| 307 | + | t.Fatalf("provider calls = %d, want 3", got) | |
| 308 | + | } | |
| 309 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 310 | + | _, ok := ev.(EvTurnRecovery) | |
| 311 | + | return ok | |
| 312 | + | }); got != 1 { | |
| 313 | + | t.Fatalf("EvTurnRecovery count = %d, want 1 (no reset by tool use)", got) | |
| 314 | + | } | |
| 315 | + | if got := countEvents(events, isEvDone); got != 1 { | |
| 316 | + | t.Fatalf("EvDone count = %d, want 1", got) | |
| 317 | + | } | |
| 318 | + | } | |
| 319 | + | ||
| 320 | + | // TestAgentTurnRecoveryQueuedInputPriority: user input queued during an | |
| 321 | + | // incomplete turn is answered first; recovery must not create an unbounded | |
| 322 | + | // loop for later queued input. | |
| 323 | + | func TestAgentTurnRecoveryQueuedInputPriority(t *testing.T) { | |
| 324 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 325 | + | {content: nil, stop: provider.StopEnd}, | |
| 326 | + | {content: []provider.Content{textContent("queued answer", "")}, stop: provider.StopEnd}, | |
| 327 | + | }} | |
| 328 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 329 | + | var events []AgentEvent | |
| 330 | + | ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) | |
| 331 | + | defer cancel() | |
| 332 | + | var once sync.Once | |
| 333 | + | err := agent.Prompt(ctx, "start", nil, func(ev AgentEvent) { | |
| 334 | + | events = append(events, ev) | |
| 335 | + | if te, ok := ev.(EvTurnEnd); ok && te.Stop == provider.StopEnd && te.Err == nil { | |
| 336 | + | once.Do(func() { | |
| 337 | + | if !agent.QueueMessage("follow-up", nil) { | |
| 338 | + | t.Error("QueueMessage returned false") | |
| 339 | + | } | |
| 340 | + | }) | |
| 341 | + | } | |
| 342 | + | if _, ok := ev.(EvTurnRecovery); ok { | |
| 343 | + | t.Error("recovery must not fire while queued input is pending") | |
| 344 | + | } | |
| 345 | + | }) | |
| 346 | + | if err != nil { | |
| 347 | + | t.Fatalf("Prompt returned %v", err) | |
| 348 | + | } | |
| 349 | + | if got := client.Calls(); got != 2 { | |
| 350 | + | t.Fatalf("provider calls = %d, want 2", got) | |
| 351 | + | } | |
| 352 | + | var sawQueued bool | |
| 353 | + | for _, m := range agent.Messages() { | |
| 354 | + | if m.Role == provider.RoleUser && strings.Contains(extractText(m), "follow-up") && m.Meta[turnRecoveryMetaKey] != "true" { | |
| 355 | + | sawQueued = true | |
| 356 | + | } | |
| 357 | + | } | |
| 358 | + | if !sawQueued { | |
| 359 | + | t.Fatal("queued input was not delivered as a user message") | |
| 360 | + | } | |
| 361 | + | } | |
| 362 | + | ||
| 363 | + | // TestAgentTurnRecoveryCancellation: no recovery request follows a | |
| 364 | + | // cancellation; the cancellation error surfaces instead. | |
| 365 | + | func TestAgentTurnRecoveryCancellation(t *testing.T) { | |
| 366 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 367 | + | {content: nil, stop: provider.StopEnd}, | |
| 368 | + | }} | |
| 369 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 370 | + | ctx, cancel := context.WithCancel(context.Background()) | |
| 371 | + | var events []AgentEvent | |
| 372 | + | cancel() | |
| 373 | + | err := agent.Prompt(ctx, "start", nil, recoveryEvents(&events)) | |
| 374 | + | if !errors.Is(err, context.Canceled) { | |
| 375 | + | t.Fatalf("Prompt error = %v, want context.Canceled", err) | |
| 376 | + | } | |
| 377 | + | if got := client.Calls(); got != 1 { | |
| 378 | + | t.Fatalf("provider calls = %d, want 1 (no recovery after cancel)", got) | |
| 379 | + | } | |
| 380 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 381 | + | _, ok := ev.(EvTurnRecovery) | |
| 382 | + | return ok | |
| 383 | + | }); got != 0 { | |
| 384 | + | t.Fatalf("EvTurnRecovery count = %d, want 0", got) | |
| 385 | + | } | |
| 386 | + | } | |
| 387 | + | ||
| 388 | + | // TestAgentTurnRecoveryDeniedToolNoRecovery: a denied tool invocation must | |
| 389 | + | // not gain a recovery prompt urging more actions. | |
| 390 | + | func TestAgentTurnRecoveryDeniedToolNoRecovery(t *testing.T) { | |
| 391 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 392 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |
| 393 | + | {content: nil, stop: provider.StopEnd}, | |
| 394 | + | }} | |
| 395 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |
| 396 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |
| 397 | + | }) | |
| 398 | + | agent.BeforeToolExecute = func(provider.ToolCallBlock) (bool, string, json.RawMessage) { | |
| 399 | + | return false, "tool call refused by user", nil | |
| 400 | + | } | |
| 401 | + | var events []AgentEvent | |
| 402 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 403 | + | t.Fatalf("Prompt returned %v", err) | |
| 404 | + | } | |
| 405 | + | if got := client.Calls(); got != 2 { | |
| 406 | + | t.Fatalf("provider calls = %d, want 2", got) | |
| 407 | + | } | |
| 408 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 409 | + | _, ok := ev.(EvTurnRecovery) | |
| 410 | + | return ok | |
| 411 | + | }); got != 0 { | |
| 412 | + | t.Fatalf("EvTurnRecovery count = %d, want 0 after denial", got) | |
| 413 | + | } | |
| 414 | + | var sawRecoverySynthetic bool | |
| 415 | + | for _, m := range agent.Messages() { | |
| 416 | + | if m.Meta[turnRecoveryMetaKey] == "true" { | |
| 417 | + | sawRecoverySynthetic = true | |
| 418 | + | } | |
| 419 | + | } | |
| 420 | + | if sawRecoverySynthetic { | |
| 421 | + | t.Fatal("recovery synthetic message appended after denial") | |
| 422 | + | } | |
| 423 | + | } | |
| 424 | + | ||
| 425 | + | // TestAgentTurnRecoveryDeniedExecuteNoRecovery: a permission refusal | |
| 426 | + | // returned from tool Execute (not the BeforeToolExecute hook) is also a | |
| 427 | + | // denial. The model sees the refusal text as the tool result, and a later | |
| 428 | + | // incomplete normal stop must not gain a recovery prompt urging more work. | |
| 429 | + | func TestAgentTurnRecoveryDeniedExecuteNoRecovery(t *testing.T) { | |
| 430 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 431 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |
| 432 | + | {content: nil, stop: provider.StopEnd}, | |
| 433 | + | }} | |
| 434 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |
| 435 | + | "result": &resultTool{err: &ToolDeniedError{Reason: "permission denied: bash command \"rm\" is not in allowlist"}}, | |
| 436 | + | }) | |
| 437 | + | var events []AgentEvent | |
| 438 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 439 | + | t.Fatalf("Prompt returned %v", err) | |
| 440 | + | } | |
| 441 | + | if got := client.Calls(); got != 2 { | |
| 442 | + | t.Fatalf("provider calls = %d, want 2", got) | |
| 443 | + | } | |
| 444 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 445 | + | _, ok := ev.(EvTurnRecovery) | |
| 446 | + | return ok | |
| 447 | + | }); got != 0 { | |
| 448 | + | t.Fatalf("EvTurnRecovery count = %d, want 0 after Execute refusal", got) | |
| 449 | + | } | |
| 450 | + | for _, m := range agent.Messages() { | |
| 451 | + | if m.Meta[turnRecoveryMetaKey] == "true" { | |
| 452 | + | t.Fatal("recovery synthetic message appended after Execute refusal") | |
| 453 | + | } | |
| 454 | + | } | |
| 455 | + | // The refusal text itself must still reach the model as the tool | |
| 456 | + | // result so it can propose a different action. | |
| 457 | + | found := false | |
| 458 | + | for _, m := range agent.Messages() { | |
| 459 | + | if m.Role != provider.RoleTool { | |
| 460 | + | continue | |
| 461 | + | } | |
| 462 | + | for _, c := range m.Content { | |
| 463 | + | if tr, ok := c.(provider.ToolResultBlock); ok && tr.IsError { | |
| 464 | + | for _, inner := range tr.Content { | |
| 465 | + | if tb, ok := inner.(provider.TextBlock); ok && strings.Contains(tb.Text, "permission denied") { | |
| 466 | + | found = true | |
| 467 | + | } | |
| 468 | + | } | |
| 469 | + | } | |
| 470 | + | } | |
| 471 | + | } | |
| 472 | + | if !found { | |
| 473 | + | t.Fatal("refusal text missing from tool result") | |
| 474 | + | } | |
| 475 | + | } | |
| 476 | + | ||
| 477 | + | // TestAgentTurnRecoveryDenialLatchIsSticky: the denial latch is | |
| 478 | + | // intentionally fail-closed for the whole top-level invocation. An early | |
| 479 | + | // unrelated denial suppresses later recovery too, so a silent ending | |
| 480 | + | // after a denial succeeds without urging more actions rather than | |
| 481 | + | // recovering or erroring. This pins the conservative semantic. | |
| 482 | + | func TestAgentTurnRecoveryDenialLatchIsSticky(t *testing.T) { | |
| 483 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 484 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |
| 485 | + | {content: nil, stop: provider.StopEnd}, | |
| 486 | + | }} | |
| 487 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |
| 488 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |
| 489 | + | }) | |
| 490 | + | agent.BeforeToolExecute = func(provider.ToolCallBlock) (bool, string, json.RawMessage) { | |
| 491 | + | return false, "tool call refused by user", nil | |
| 492 | + | } | |
| 493 | + | var events []AgentEvent | |
| 494 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 495 | + | t.Fatalf("Prompt returned %v", err) | |
| 496 | + | } | |
| 497 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 498 | + | _, ok := ev.(EvTurnRecovery) | |
| 499 | + | return ok | |
| 500 | + | }); got != 0 { | |
| 501 | + | t.Fatalf("EvTurnRecovery count = %d, want 0: early denial suppresses later recovery", got) | |
| 502 | + | } | |
| 503 | + | } | |
| 504 | + | ||
| 505 | + | // TestAgentTurnRecoveryGuardDenial likewise covers the BeforeTurn guard: | |
| 506 | + | // a guard-blocked turn is not a missing answer. | |
| 507 | + | func TestAgentTurnRecoveryGuardDenial(t *testing.T) { | |
| 508 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 509 | + | {content: []provider.Content{textContent("answer", "")}, stop: provider.StopEnd}, | |
| 510 | + | }} | |
| 511 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 512 | + | agent.BeforeTurn = func(step int) (bool, string) { | |
| 513 | + | if step > 1 { | |
| 514 | + | return false, "blocked at step 2" | |
| 515 | + | } | |
| 516 | + | return true, "" | |
| 517 | + | } | |
| 518 | + | var events []AgentEvent | |
| 519 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 520 | + | t.Fatalf("Prompt returned %v", err) | |
| 521 | + | } | |
| 522 | + | if got := client.Calls(); got != 1 { | |
| 523 | + | t.Fatalf("provider calls = %d, want 1", got) | |
| 524 | + | } | |
| 525 | + | } | |
| 526 | + | ||
| 527 | + | // TestAgentTurnRecoveryLimitsNoRecovery: output and context limits never | |
| 528 | + | // recover. | |
| 529 | + | func TestAgentTurnRecoveryLimitsNoRecovery(t *testing.T) { | |
| 530 | + | for _, stop := range []provider.StopReason{provider.StopLength, provider.StopAborted, provider.StopError} { | |
| 531 | + | t.Run(string(stop), func(t *testing.T) { | |
| 532 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 533 | + | {content: nil, stop: stop}, | |
| 534 | + | }} | |
| 535 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 536 | + | var events []AgentEvent | |
| 537 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 538 | + | t.Fatalf("Prompt returned %v", err) | |
| 539 | + | } | |
| 540 | + | if got := client.Calls(); got != 1 { | |
| 541 | + | t.Fatalf("provider calls = %d, want 1", got) | |
| 542 | + | } | |
| 543 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |
| 544 | + | _, ok := ev.(EvTurnRecovery) | |
| 545 | + | return ok | |
| 546 | + | }); got != 0 { | |
| 547 | + | t.Fatalf("EvTurnRecovery count = %d, want 0 for stop %q", got, stop) | |
| 548 | + | } | |
| 549 | + | }) | |
| 550 | + | } | |
| 551 | + | } | |
| 552 | + | ||
| 553 | + | // TestAgentTurnRecoveryRespectsMaxSteps: the recovery continuation runs | |
| 554 | + | // inside the existing step budget and does not reset it. With MaxSteps=1 | |
| 555 | + | // the recovery is marked but the budget is exhausted, so the existing | |
| 556 | + | // typed limit error surfaces; with MaxSteps=2 two incomplete endings | |
| 557 | + | // still terminate with ErrIncompleteTurn. | |
| 558 | + | func TestAgentTurnRecoveryRespectsMaxSteps(t *testing.T) { | |
| 559 | + | t.Run("budget exhausted", func(t *testing.T) { | |
| 560 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 561 | + | {content: nil, stop: provider.StopEnd}, | |
| 562 | + | }} | |
| 563 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 564 | + | agent.MaxSteps = 1 | |
| 565 | + | var events []AgentEvent | |
| 566 | + | err := runPrompt(t, agent, &events) | |
| 567 | + | if !errors.Is(err, ErrMaxSteps) { | |
| 568 | + | t.Fatalf("Prompt error = %v, want ErrMaxSteps", err) | |
| 569 | + | } | |
| 570 | + | if got := client.Calls(); got != 1 { | |
| 571 | + | t.Fatalf("provider calls = %d, want 1 (no reset)", got) | |
| 572 | + | } | |
| 573 | + | }) | |
| 574 | + | t.Run("two incomplete within budget", func(t *testing.T) { | |
| 575 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 576 | + | {content: nil, stop: provider.StopEnd}, | |
| 577 | + | {content: nil, stop: provider.StopEnd}, | |
| 578 | + | }} | |
| 579 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |
| 580 | + | agent.MaxSteps = 2 | |
| 581 | + | var events []AgentEvent | |
| 582 | + | err := runPrompt(t, agent, &events) | |
| 583 | + | if !errors.Is(err, ErrIncompleteTurn) { | |
| 584 | + | t.Fatalf("Prompt error = %v, want ErrIncompleteTurn", err) | |
| 585 | + | } | |
| 586 | + | if got := client.Calls(); got != 2 { | |
| 587 | + | t.Fatalf("provider calls = %d, want 2", got) | |
| 588 | + | } | |
| 589 | + | }) | |
| 590 | + | } | |
| 591 | + | ||
| 592 | + | // TestAgentTurnRecoverySyntheticPersistence: the recovery message is | |
| 593 | + | // appended through AppendUserContext (firing OnMessageAppended for session | |
| 594 | + | // durability) and replays to the provider on the next request. | |
| 595 | + | func TestAgentTurnRecoverySyntheticPersistence(t *testing.T) { | |
| 596 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |
| 597 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |
| 598 | + | {content: nil, stop: provider.StopEnd}, | |
| 599 | + | {content: []provider.Content{textContent("done", "")}, stop: provider.StopEnd}, | |
| 600 | + | }} | |
| 601 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |
| 602 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |
| 603 | + | }) | |
| 604 | + | var appended []provider.Message | |
| 605 | + | agent.OnMessageAppended = func(m provider.Message) { appended = append(appended, m) } | |
| 606 | + | var events []AgentEvent | |
| 607 | + | if err := runPrompt(t, agent, &events); err != nil { | |
| 608 | + | t.Fatalf("Prompt returned %v", err) | |
| 609 | + | } | |
| 610 | + | var sawSynthetic bool | |
| 611 | + | for _, m := range appended { | |
| 612 | + | if m.Meta[turnRecoveryMetaKey] == "true" { | |
| 613 | + | sawSynthetic = true | |
| 614 | + | if m.Role != provider.RoleUser || len(m.Content) != 1 { | |
| 615 | + | t.Fatalf("recovery appended message = %#v, want single-block user message", m) | |
| 616 | + | } | |
| 617 | + | } | |
| 618 | + | } | |
| 619 | + | if !sawSynthetic { | |
| 620 | + | t.Fatal("OnMessageAppended never fired for the recovery message") | |
| 621 | + | } | |
| 622 | + | // The transcript keeps tool-result pairing and the recovery marker in | |
| 623 | + | // order. An empty terminal message carries no blocks, so oneTurn does | |
| 624 | + | // not append it: user, assistant tool call, tool result, synthetic, | |
| 625 | + | // final answer. | |
| 626 | + | msgs := agent.Messages() | |
| 627 | + | if len(msgs) != 5 { | |
| 628 | + | t.Fatalf("transcript messages = %d, want 5", len(msgs)) | |
| 629 | + | } | |
| 630 | + | if msgs[1].Role != provider.RoleAssistant || msgs[2].Role != provider.RoleTool { | |
| 631 | + | t.Fatalf("tool pairing broken: roles = %q, %q", msgs[1].Role, msgs[2].Role) | |
| 632 | + | } | |
| 633 | + | if msgs[3].Meta[turnRecoveryMetaKey] != "true" { | |
| 634 | + | t.Fatalf("message 3 meta = %#v, want recovery marker", msgs[3].Meta) | |
| 635 | + | } | |
| 636 | + | } | |
| @@ -0,0 +1,636 @@ | |||||
| 1 | + | package core | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "context" | |||
| 5 | + | "encoding/json" | |||
| 6 | + | "errors" | |||
| 7 | + | "strings" | |||
| 8 | + | "sync" | |||
| 9 | + | "sync/atomic" | |||
| 10 | + | "testing" | |||
| 11 | + | "time" | |||
| 12 | + | ||||
| 13 | + | "github.com/bnema/zut/packages/provider" | |||
| 14 | + | ) | |||
| 15 | + | ||||
| 16 | + | // turnRecoveryScriptClient replays a scripted sequence of provider turns, | |||
| 17 | + | // one per Stream call. A nil-content entry emits an assistant message with | |||
| 18 | + | // no content blocks; unknownStop entries exercise non-end stops. | |||
| 19 | + | type turnRecoveryScriptClient struct { | |||
| 20 | + | calls int32 | |||
| 21 | + | turns []turnRecoveryTurn | |||
| 22 | + | } | |||
| 23 | + | ||||
| 24 | + | type turnRecoveryTurn struct { | |||
| 25 | + | content []provider.Content | |||
| 26 | + | stop provider.StopReason | |||
| 27 | + | err error | |||
| 28 | + | } | |||
| 29 | + | ||||
| 30 | + | func (c *turnRecoveryScriptClient) Name() string { return "turn-recovery-script" } | |||
| 31 | + | ||||
| 32 | + | func (c *turnRecoveryScriptClient) Stream(_ context.Context, req provider.Request) (<-chan provider.Event, error) { | |||
| 33 | + | call := int(atomic.AddInt32(&c.calls, 1)) - 1 | |||
| 34 | + | turn := turnRecoveryTurn{stop: provider.StopEnd} | |||
| 35 | + | if call < len(c.turns) { | |||
| 36 | + | turn = c.turns[call] | |||
| 37 | + | } | |||
| 38 | + | out := make(chan provider.Event, 8) | |||
| 39 | + | out <- provider.EventStart{Provider: c.Name(), Model: req.Model} | |||
| 40 | + | out <- provider.EventUsage{Usage: provider.Usage{InputTokens: 1, OutputTokens: 1}} | |||
| 41 | + | msg := provider.Message{Role: provider.RoleAssistant, Content: turn.content, Time: time.Now()} | |||
| 42 | + | out <- provider.EventDone{Stop: turn.stop, Err: turn.err, Message: msg} | |||
| 43 | + | close(out) | |||
| 44 | + | return out, nil | |||
| 45 | + | } | |||
| 46 | + | ||||
| 47 | + | func (c *turnRecoveryScriptClient) Calls() int32 { return atomic.LoadInt32(&c.calls) } | |||
| 48 | + | ||||
| 49 | + | func textContent(text, phase string) provider.Content { | |||
| 50 | + | return provider.TextBlock{Text: text, Phase: phase} | |||
| 51 | + | } | |||
| 52 | + | ||||
| 53 | + | func recoveryEvents(sink *[]AgentEvent) func(AgentEvent) { | |||
| 54 | + | return func(ev AgentEvent) { | |||
| 55 | + | *sink = append(*sink, ev) | |||
| 56 | + | } | |||
| 57 | + | } | |||
| 58 | + | ||||
| 59 | + | func countEvents(events []AgentEvent, want func(AgentEvent) bool) int { | |||
| 60 | + | n := 0 | |||
| 61 | + | for _, ev := range events { | |||
| 62 | + | if want(ev) { | |||
| 63 | + | n++ | |||
| 64 | + | } | |||
| 65 | + | } | |||
| 66 | + | return n | |||
| 67 | + | } | |||
| 68 | + | ||||
| 69 | + | func isEvDone(ev AgentEvent) bool { _, ok := ev.(EvDone); return ok } | |||
| 70 | + | ||||
| 71 | + | func isTurnEnd(stop provider.StopReason) func(AgentEvent) bool { | |||
| 72 | + | return func(ev AgentEvent) bool { | |||
| 73 | + | te, ok := ev.(EvTurnEnd) | |||
| 74 | + | return ok && te.Stop == stop && te.Err == nil | |||
| 75 | + | } | |||
| 76 | + | } | |||
| 77 | + | ||||
| 78 | + | func lastAssistantMessage(t *testing.T, agent *Agent) provider.Message { | |||
| 79 | + | t.Helper() | |||
| 80 | + | msgs := agent.Messages() | |||
| 81 | + | for i := len(msgs) - 1; i >= 0; i-- { | |||
| 82 | + | if msgs[i].Role == provider.RoleAssistant { | |||
| 83 | + | return msgs[i] | |||
| 84 | + | } | |||
| 85 | + | } | |||
| 86 | + | t.Fatal("no assistant message in transcript") | |||
| 87 | + | return provider.Message{} | |||
| 88 | + | } | |||
| 89 | + | ||||
| 90 | + | func recoverySynthetic(t *testing.T, agent *Agent) provider.Message { | |||
| 91 | + | t.Helper() | |||
| 92 | + | for _, m := range agent.Messages() { | |||
| 93 | + | if m.Role == provider.RoleUser && m.Meta[turnRecoveryMetaKey] == "true" { | |||
| 94 | + | return m | |||
| 95 | + | } | |||
| 96 | + | } | |||
| 97 | + | t.Fatal("no recovery synthetic user message in transcript") | |||
| 98 | + | return provider.Message{} | |||
| 99 | + | } | |||
| 100 | + | ||||
| 101 | + | func runPrompt(t *testing.T, agent *Agent, events *[]AgentEvent) error { | |||
| 102 | + | t.Helper() | |||
| 103 | + | ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) | |||
| 104 | + | defer cancel() | |||
| 105 | + | return agent.Prompt(ctx, "start", nil, recoveryEvents(events)) | |||
| 106 | + | } | |||
| 107 | + | ||||
| 108 | + | // TestTurnRecoveryEmptyFinalAfterTool: a tool turn followed by an empty | |||
| 109 | + | // normal terminal response triggers one recovery and can then finish. | |||
| 110 | + | func TestTurnRecoveryEmptyFinalAfterTool(t *testing.T) { | |||
| 111 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 112 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |||
| 113 | + | {content: nil, stop: provider.StopEnd}, | |||
| 114 | + | {content: []provider.Content{textContent("done", "")}, stop: provider.StopEnd}, | |||
| 115 | + | }} | |||
| 116 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |||
| 117 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |||
| 118 | + | }) | |||
| 119 | + | var events []AgentEvent | |||
| 120 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 121 | + | t.Fatalf("Prompt returned %v", err) | |||
| 122 | + | } | |||
| 123 | + | if got := client.Calls(); got != 3 { | |||
| 124 | + | t.Fatalf("provider calls = %d, want 3", got) | |||
| 125 | + | } | |||
| 126 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 127 | + | r, ok := ev.(EvTurnRecovery) | |||
| 128 | + | return ok && r.Reason == TurnRecoveryReasonMissingAnswer && r.Attempt == 1 | |||
| 129 | + | }); got != 1 { | |||
| 130 | + | t.Fatalf("EvTurnRecovery count = %d, want 1 missing_answer", got) | |||
| 131 | + | } | |||
| 132 | + | if got := lastAssistantMessage(t, agent).Content[0].(provider.TextBlock).Text; got != "done" { | |||
| 133 | + | t.Fatalf("final text = %q, want done", got) | |||
| 134 | + | } | |||
| 135 | + | synth := recoverySynthetic(t, agent) | |||
| 136 | + | if !strings.Contains(synth.Content[0].(provider.TextBlock).Text, "without a final answer") { | |||
| 137 | + | t.Fatalf("recovery text = %q", synth.Content[0].(provider.TextBlock).Text) | |||
| 138 | + | } | |||
| 139 | + | // Tool-result pairing survives the recovery round trip. | |||
| 140 | + | var toolResults int | |||
| 141 | + | for _, m := range agent.Messages() { | |||
| 142 | + | if m.Role == provider.RoleTool { | |||
| 143 | + | toolResults++ | |||
| 144 | + | } | |||
| 145 | + | } | |||
| 146 | + | if toolResults != 1 { | |||
| 147 | + | t.Fatalf("tool messages = %d, want 1", toolResults) | |||
| 148 | + | } | |||
| 149 | + | } | |||
| 150 | + | ||||
| 151 | + | // TestTurnRecoveryCommentaryOnly: explicitly commentary-only normal | |||
| 152 | + | // completion triggers one recovery; the commentary reason is reported. | |||
| 153 | + | func TestTurnRecoveryCommentaryOnly(t *testing.T) { | |||
| 154 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 155 | + | {content: []provider.Content{textContent("working on it", provider.TextPhaseCommentary)}, stop: provider.StopEnd}, | |||
| 156 | + | {content: []provider.Content{textContent("result", "")}, stop: provider.StopEnd}, | |||
| 157 | + | }} | |||
| 158 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 159 | + | var events []AgentEvent | |||
| 160 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 161 | + | t.Fatalf("Prompt returned %v", err) | |||
| 162 | + | } | |||
| 163 | + | if got := client.Calls(); got != 2 { | |||
| 164 | + | t.Fatalf("provider calls = %d, want 2", got) | |||
| 165 | + | } | |||
| 166 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 167 | + | r, ok := ev.(EvTurnRecovery) | |||
| 168 | + | return ok && r.Reason == TurnRecoveryReasonCommentaryOnly && r.Attempt == 1 | |||
| 169 | + | }); got != 1 { | |||
| 170 | + | t.Fatalf("commentary_only EvTurnRecovery count = %d, want 1", got) | |||
| 171 | + | } | |||
| 172 | + | if got := countEvents(events, isEvDone); got != 1 { | |||
| 173 | + | t.Fatalf("EvDone count = %d, want exactly 1", got) | |||
| 174 | + | } | |||
| 175 | + | } | |||
| 176 | + | ||||
| 177 | + | // TestTurnRecoveryFinalAnswerNoRecovery covers the no-recovery cases: a | |||
| 178 | + | // final answer, an unknown/absent phase, image output, and commentary- | |||
| 179 | + | // then-final likewise complete without extra requests. | |||
| 180 | + | func TestTurnRecoveryFinalAnswerNoRecovery(t *testing.T) { | |||
| 181 | + | cases := map[string]turnRecoveryTurn{ | |||
| 182 | + | "final answer": {content: []provider.Content{textContent("here is the result", provider.TextPhaseFinal)}}, | |||
| 183 | + | "unknown phase": {content: []provider.Content{textContent("answer", "draft")}}, | |||
| 184 | + | "absent phase": {content: []provider.Content{textContent("answer", "")}}, | |||
| 185 | + | "refusal": {content: []provider.Content{textContent("I cannot do that.", "")}}, | |||
| 186 | + | "clarification": {content: []provider.Content{textContent("Which file do you mean?", "")}}, | |||
| 187 | + | "image output": {content: []provider.Content{provider.ImageBlock{MimeType: "image/png", Data: []byte{1, 2, 3}}}}, | |||
| 188 | + | "commentary then fin": {content: []provider.Content{textContent("progress", provider.TextPhaseCommentary), textContent("result", provider.TextPhaseFinal)}}, | |||
| 189 | + | "commentary then pla": {content: []provider.Content{textContent("progress", provider.TextPhaseCommentary), textContent("result", "")}}, | |||
| 190 | + | } | |||
| 191 | + | for name, turn := range cases { | |||
| 192 | + | t.Run(name, func(t *testing.T) { | |||
| 193 | + | turn.stop = provider.StopEnd | |||
| 194 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{turn}} | |||
| 195 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 196 | + | var events []AgentEvent | |||
| 197 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 198 | + | t.Fatalf("Prompt returned %v", err) | |||
| 199 | + | } | |||
| 200 | + | if got := client.Calls(); got != 1 { | |||
| 201 | + | t.Fatalf("provider calls = %d, want 1 (no recovery)", got) | |||
| 202 | + | } | |||
| 203 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 204 | + | _, ok := ev.(EvTurnRecovery) | |||
| 205 | + | return ok | |||
| 206 | + | }); got != 0 { | |||
| 207 | + | t.Fatalf("EvTurnRecovery count = %d, want 0", got) | |||
| 208 | + | } | |||
| 209 | + | }) | |||
| 210 | + | } | |||
| 211 | + | } | |||
| 212 | + | ||||
| 213 | + | // TestAgentTurnRecoveryReasoningOnlyThenAnswer: a reasoning-only terminal | |||
| 214 | + | // message carries no answer and recovers as missing_answer. | |||
| 215 | + | func TestAgentTurnRecoveryReasoningOnlyThenAnswer(t *testing.T) { | |||
| 216 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 217 | + | {content: []provider.Content{provider.ReasoningBlock{Summary: "thinking"}}, stop: provider.StopEnd}, | |||
| 218 | + | {content: []provider.Content{textContent("result", "")}, stop: provider.StopEnd}, | |||
| 219 | + | }} | |||
| 220 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 221 | + | var events []AgentEvent | |||
| 222 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 223 | + | t.Fatalf("Prompt returned %v", err) | |||
| 224 | + | } | |||
| 225 | + | if got := client.Calls(); got != 2 { | |||
| 226 | + | t.Fatalf("provider calls = %d, want 2", got) | |||
| 227 | + | } | |||
| 228 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 229 | + | r, ok := ev.(EvTurnRecovery) | |||
| 230 | + | return ok && r.Reason == TurnRecoveryReasonMissingAnswer | |||
| 231 | + | }); got != 1 { | |||
| 232 | + | t.Fatalf("missing_answer EvTurnRecovery count = %d, want 1", got) | |||
| 233 | + | } | |||
| 234 | + | } | |||
| 235 | + | ||||
| 236 | + | // TestAgentTurnRecoveryWhitespaceThenAnswer: a whitespace-only terminal | |||
| 237 | + | // message recovers as missing_answer and can finish on the next turn. | |||
| 238 | + | func TestAgentTurnRecoveryWhitespaceThenAnswer(t *testing.T) { | |||
| 239 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 240 | + | {content: []provider.Content{textContent(" \n\t ", "")}, stop: provider.StopEnd}, | |||
| 241 | + | {content: []provider.Content{textContent("result", "")}, stop: provider.StopEnd}, | |||
| 242 | + | }} | |||
| 243 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 244 | + | var events []AgentEvent | |||
| 245 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 246 | + | t.Fatalf("Prompt returned %v", err) | |||
| 247 | + | } | |||
| 248 | + | if got := client.Calls(); got != 2 { | |||
| 249 | + | t.Fatalf("provider calls = %d, want 2", got) | |||
| 250 | + | } | |||
| 251 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 252 | + | r, ok := ev.(EvTurnRecovery) | |||
| 253 | + | return ok && r.Reason == TurnRecoveryReasonMissingAnswer | |||
| 254 | + | }); got != 1 { | |||
| 255 | + | t.Fatalf("missing_answer EvTurnRecovery count = %d, want 1", got) | |||
| 256 | + | } | |||
| 257 | + | } | |||
| 258 | + | ||||
| 259 | + | // TestAgentTurnRecoveryTwoIncompleteEndings: a second eligible incomplete | |||
| 260 | + | // StopEnd returns sentinel ErrIncompleteTurn with exactly-once terminal | |||
| 261 | + | // notification and no second model EvTurnEnd for the host decision. | |||
| 262 | + | func TestAgentTurnRecoveryTwoIncompleteEndings(t *testing.T) { | |||
| 263 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 264 | + | {content: nil, stop: provider.StopEnd}, | |||
| 265 | + | {content: []provider.Content{textContent("still working", provider.TextPhaseCommentary)}, stop: provider.StopEnd}, | |||
| 266 | + | }} | |||
| 267 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 268 | + | var events []AgentEvent | |||
| 269 | + | err := runPrompt(t, agent, &events) | |||
| 270 | + | if !errors.Is(err, ErrIncompleteTurn) { | |||
| 271 | + | t.Fatalf("Prompt error = %v, want ErrIncompleteTurn", err) | |||
| 272 | + | } | |||
| 273 | + | if got := client.Calls(); got != 2 { | |||
| 274 | + | t.Fatalf("provider calls = %d, want 2 (bounded)", got) | |||
| 275 | + | } | |||
| 276 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 277 | + | _, ok := ev.(EvTurnRecovery) | |||
| 278 | + | return ok | |||
| 279 | + | }); got != 1 { | |||
| 280 | + | t.Fatalf("EvTurnRecovery count = %d, want exactly 1", got) | |||
| 281 | + | } | |||
| 282 | + | if got := countEvents(events, isEvDone); got != 1 { | |||
| 283 | + | t.Fatalf("EvDone count = %d, want exactly 1", got) | |||
| 284 | + | } | |||
| 285 | + | if got := countEvents(events, isTurnEnd(provider.StopEnd)); got != 2 { | |||
| 286 | + | t.Fatalf("model StopEnd EvTurnEnd count = %d, want 2 (no second model turn for the host decision)", got) | |||
| 287 | + | } | |||
| 288 | + | } | |||
| 289 | + | ||||
| 290 | + | // TestAgentTurnRecoveryToolsBetweenIncomplete: calls stay bounded even when | |||
| 291 | + | // tools execute between the two incomplete endings. | |||
| 292 | + | func TestAgentTurnRecoveryToolsBetweenIncomplete(t *testing.T) { | |||
| 293 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 294 | + | {content: nil, stop: provider.StopEnd}, | |||
| 295 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |||
| 296 | + | {content: nil, stop: provider.StopEnd}, | |||
| 297 | + | }} | |||
| 298 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |||
| 299 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |||
| 300 | + | }) | |||
| 301 | + | var events []AgentEvent | |||
| 302 | + | err := runPrompt(t, agent, &events) | |||
| 303 | + | if !errors.Is(err, ErrIncompleteTurn) { | |||
| 304 | + | t.Fatalf("Prompt error = %v, want ErrIncompleteTurn", err) | |||
| 305 | + | } | |||
| 306 | + | if got := client.Calls(); got != 3 { | |||
| 307 | + | t.Fatalf("provider calls = %d, want 3", got) | |||
| 308 | + | } | |||
| 309 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 310 | + | _, ok := ev.(EvTurnRecovery) | |||
| 311 | + | return ok | |||
| 312 | + | }); got != 1 { | |||
| 313 | + | t.Fatalf("EvTurnRecovery count = %d, want 1 (no reset by tool use)", got) | |||
| 314 | + | } | |||
| 315 | + | if got := countEvents(events, isEvDone); got != 1 { | |||
| 316 | + | t.Fatalf("EvDone count = %d, want 1", got) | |||
| 317 | + | } | |||
| 318 | + | } | |||
| 319 | + | ||||
| 320 | + | // TestAgentTurnRecoveryQueuedInputPriority: user input queued during an | |||
| 321 | + | // incomplete turn is answered first; recovery must not create an unbounded | |||
| 322 | + | // loop for later queued input. | |||
| 323 | + | func TestAgentTurnRecoveryQueuedInputPriority(t *testing.T) { | |||
| 324 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 325 | + | {content: nil, stop: provider.StopEnd}, | |||
| 326 | + | {content: []provider.Content{textContent("queued answer", "")}, stop: provider.StopEnd}, | |||
| 327 | + | }} | |||
| 328 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 329 | + | var events []AgentEvent | |||
| 330 | + | ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) | |||
| 331 | + | defer cancel() | |||
| 332 | + | var once sync.Once | |||
| 333 | + | err := agent.Prompt(ctx, "start", nil, func(ev AgentEvent) { | |||
| 334 | + | events = append(events, ev) | |||
| 335 | + | if te, ok := ev.(EvTurnEnd); ok && te.Stop == provider.StopEnd && te.Err == nil { | |||
| 336 | + | once.Do(func() { | |||
| 337 | + | if !agent.QueueMessage("follow-up", nil) { | |||
| 338 | + | t.Error("QueueMessage returned false") | |||
| 339 | + | } | |||
| 340 | + | }) | |||
| 341 | + | } | |||
| 342 | + | if _, ok := ev.(EvTurnRecovery); ok { | |||
| 343 | + | t.Error("recovery must not fire while queued input is pending") | |||
| 344 | + | } | |||
| 345 | + | }) | |||
| 346 | + | if err != nil { | |||
| 347 | + | t.Fatalf("Prompt returned %v", err) | |||
| 348 | + | } | |||
| 349 | + | if got := client.Calls(); got != 2 { | |||
| 350 | + | t.Fatalf("provider calls = %d, want 2", got) | |||
| 351 | + | } | |||
| 352 | + | var sawQueued bool | |||
| 353 | + | for _, m := range agent.Messages() { | |||
| 354 | + | if m.Role == provider.RoleUser && strings.Contains(extractText(m), "follow-up") && m.Meta[turnRecoveryMetaKey] != "true" { | |||
| 355 | + | sawQueued = true | |||
| 356 | + | } | |||
| 357 | + | } | |||
| 358 | + | if !sawQueued { | |||
| 359 | + | t.Fatal("queued input was not delivered as a user message") | |||
| 360 | + | } | |||
| 361 | + | } | |||
| 362 | + | ||||
| 363 | + | // TestAgentTurnRecoveryCancellation: no recovery request follows a | |||
| 364 | + | // cancellation; the cancellation error surfaces instead. | |||
| 365 | + | func TestAgentTurnRecoveryCancellation(t *testing.T) { | |||
| 366 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 367 | + | {content: nil, stop: provider.StopEnd}, | |||
| 368 | + | }} | |||
| 369 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 370 | + | ctx, cancel := context.WithCancel(context.Background()) | |||
| 371 | + | var events []AgentEvent | |||
| 372 | + | cancel() | |||
| 373 | + | err := agent.Prompt(ctx, "start", nil, recoveryEvents(&events)) | |||
| 374 | + | if !errors.Is(err, context.Canceled) { | |||
| 375 | + | t.Fatalf("Prompt error = %v, want context.Canceled", err) | |||
| 376 | + | } | |||
| 377 | + | if got := client.Calls(); got != 1 { | |||
| 378 | + | t.Fatalf("provider calls = %d, want 1 (no recovery after cancel)", got) | |||
| 379 | + | } | |||
| 380 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 381 | + | _, ok := ev.(EvTurnRecovery) | |||
| 382 | + | return ok | |||
| 383 | + | }); got != 0 { | |||
| 384 | + | t.Fatalf("EvTurnRecovery count = %d, want 0", got) | |||
| 385 | + | } | |||
| 386 | + | } | |||
| 387 | + | ||||
| 388 | + | // TestAgentTurnRecoveryDeniedToolNoRecovery: a denied tool invocation must | |||
| 389 | + | // not gain a recovery prompt urging more actions. | |||
| 390 | + | func TestAgentTurnRecoveryDeniedToolNoRecovery(t *testing.T) { | |||
| 391 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 392 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |||
| 393 | + | {content: nil, stop: provider.StopEnd}, | |||
| 394 | + | }} | |||
| 395 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |||
| 396 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |||
| 397 | + | }) | |||
| 398 | + | agent.BeforeToolExecute = func(provider.ToolCallBlock) (bool, string, json.RawMessage) { | |||
| 399 | + | return false, "tool call refused by user", nil | |||
| 400 | + | } | |||
| 401 | + | var events []AgentEvent | |||
| 402 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 403 | + | t.Fatalf("Prompt returned %v", err) | |||
| 404 | + | } | |||
| 405 | + | if got := client.Calls(); got != 2 { | |||
| 406 | + | t.Fatalf("provider calls = %d, want 2", got) | |||
| 407 | + | } | |||
| 408 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 409 | + | _, ok := ev.(EvTurnRecovery) | |||
| 410 | + | return ok | |||
| 411 | + | }); got != 0 { | |||
| 412 | + | t.Fatalf("EvTurnRecovery count = %d, want 0 after denial", got) | |||
| 413 | + | } | |||
| 414 | + | var sawRecoverySynthetic bool | |||
| 415 | + | for _, m := range agent.Messages() { | |||
| 416 | + | if m.Meta[turnRecoveryMetaKey] == "true" { | |||
| 417 | + | sawRecoverySynthetic = true | |||
| 418 | + | } | |||
| 419 | + | } | |||
| 420 | + | if sawRecoverySynthetic { | |||
| 421 | + | t.Fatal("recovery synthetic message appended after denial") | |||
| 422 | + | } | |||
| 423 | + | } | |||
| 424 | + | ||||
| 425 | + | // TestAgentTurnRecoveryDeniedExecuteNoRecovery: a permission refusal | |||
| 426 | + | // returned from tool Execute (not the BeforeToolExecute hook) is also a | |||
| 427 | + | // denial. The model sees the refusal text as the tool result, and a later | |||
| 428 | + | // incomplete normal stop must not gain a recovery prompt urging more work. | |||
| 429 | + | func TestAgentTurnRecoveryDeniedExecuteNoRecovery(t *testing.T) { | |||
| 430 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 431 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |||
| 432 | + | {content: nil, stop: provider.StopEnd}, | |||
| 433 | + | }} | |||
| 434 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |||
| 435 | + | "result": &resultTool{err: &ToolDeniedError{Reason: "permission denied: bash command \"rm\" is not in allowlist"}}, | |||
| 436 | + | }) | |||
| 437 | + | var events []AgentEvent | |||
| 438 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 439 | + | t.Fatalf("Prompt returned %v", err) | |||
| 440 | + | } | |||
| 441 | + | if got := client.Calls(); got != 2 { | |||
| 442 | + | t.Fatalf("provider calls = %d, want 2", got) | |||
| 443 | + | } | |||
| 444 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 445 | + | _, ok := ev.(EvTurnRecovery) | |||
| 446 | + | return ok | |||
| 447 | + | }); got != 0 { | |||
| 448 | + | t.Fatalf("EvTurnRecovery count = %d, want 0 after Execute refusal", got) | |||
| 449 | + | } | |||
| 450 | + | for _, m := range agent.Messages() { | |||
| 451 | + | if m.Meta[turnRecoveryMetaKey] == "true" { | |||
| 452 | + | t.Fatal("recovery synthetic message appended after Execute refusal") | |||
| 453 | + | } | |||
| 454 | + | } | |||
| 455 | + | // The refusal text itself must still reach the model as the tool | |||
| 456 | + | // result so it can propose a different action. | |||
| 457 | + | found := false | |||
| 458 | + | for _, m := range agent.Messages() { | |||
| 459 | + | if m.Role != provider.RoleTool { | |||
| 460 | + | continue | |||
| 461 | + | } | |||
| 462 | + | for _, c := range m.Content { | |||
| 463 | + | if tr, ok := c.(provider.ToolResultBlock); ok && tr.IsError { | |||
| 464 | + | for _, inner := range tr.Content { | |||
| 465 | + | if tb, ok := inner.(provider.TextBlock); ok && strings.Contains(tb.Text, "permission denied") { | |||
| 466 | + | found = true | |||
| 467 | + | } | |||
| 468 | + | } | |||
| 469 | + | } | |||
| 470 | + | } | |||
| 471 | + | } | |||
| 472 | + | if !found { | |||
| 473 | + | t.Fatal("refusal text missing from tool result") | |||
| 474 | + | } | |||
| 475 | + | } | |||
| 476 | + | ||||
| 477 | + | // TestAgentTurnRecoveryDenialLatchIsSticky: the denial latch is | |||
| 478 | + | // intentionally fail-closed for the whole top-level invocation. An early | |||
| 479 | + | // unrelated denial suppresses later recovery too, so a silent ending | |||
| 480 | + | // after a denial succeeds without urging more actions rather than | |||
| 481 | + | // recovering or erroring. This pins the conservative semantic. | |||
| 482 | + | func TestAgentTurnRecoveryDenialLatchIsSticky(t *testing.T) { | |||
| 483 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 484 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |||
| 485 | + | {content: nil, stop: provider.StopEnd}, | |||
| 486 | + | }} | |||
| 487 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |||
| 488 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |||
| 489 | + | }) | |||
| 490 | + | agent.BeforeToolExecute = func(provider.ToolCallBlock) (bool, string, json.RawMessage) { | |||
| 491 | + | return false, "tool call refused by user", nil | |||
| 492 | + | } | |||
| 493 | + | var events []AgentEvent | |||
| 494 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 495 | + | t.Fatalf("Prompt returned %v", err) | |||
| 496 | + | } | |||
| 497 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 498 | + | _, ok := ev.(EvTurnRecovery) | |||
| 499 | + | return ok | |||
| 500 | + | }); got != 0 { | |||
| 501 | + | t.Fatalf("EvTurnRecovery count = %d, want 0: early denial suppresses later recovery", got) | |||
| 502 | + | } | |||
| 503 | + | } | |||
| 504 | + | ||||
| 505 | + | // TestAgentTurnRecoveryGuardDenial likewise covers the BeforeTurn guard: | |||
| 506 | + | // a guard-blocked turn is not a missing answer. | |||
| 507 | + | func TestAgentTurnRecoveryGuardDenial(t *testing.T) { | |||
| 508 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 509 | + | {content: []provider.Content{textContent("answer", "")}, stop: provider.StopEnd}, | |||
| 510 | + | }} | |||
| 511 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 512 | + | agent.BeforeTurn = func(step int) (bool, string) { | |||
| 513 | + | if step > 1 { | |||
| 514 | + | return false, "blocked at step 2" | |||
| 515 | + | } | |||
| 516 | + | return true, "" | |||
| 517 | + | } | |||
| 518 | + | var events []AgentEvent | |||
| 519 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 520 | + | t.Fatalf("Prompt returned %v", err) | |||
| 521 | + | } | |||
| 522 | + | if got := client.Calls(); got != 1 { | |||
| 523 | + | t.Fatalf("provider calls = %d, want 1", got) | |||
| 524 | + | } | |||
| 525 | + | } | |||
| 526 | + | ||||
| 527 | + | // TestAgentTurnRecoveryLimitsNoRecovery: output and context limits never | |||
| 528 | + | // recover. | |||
| 529 | + | func TestAgentTurnRecoveryLimitsNoRecovery(t *testing.T) { | |||
| 530 | + | for _, stop := range []provider.StopReason{provider.StopLength, provider.StopAborted, provider.StopError} { | |||
| 531 | + | t.Run(string(stop), func(t *testing.T) { | |||
| 532 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 533 | + | {content: nil, stop: stop}, | |||
| 534 | + | }} | |||
| 535 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 536 | + | var events []AgentEvent | |||
| 537 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 538 | + | t.Fatalf("Prompt returned %v", err) | |||
| 539 | + | } | |||
| 540 | + | if got := client.Calls(); got != 1 { | |||
| 541 | + | t.Fatalf("provider calls = %d, want 1", got) | |||
| 542 | + | } | |||
| 543 | + | if got := countEvents(events, func(ev AgentEvent) bool { | |||
| 544 | + | _, ok := ev.(EvTurnRecovery) | |||
| 545 | + | return ok | |||
| 546 | + | }); got != 0 { | |||
| 547 | + | t.Fatalf("EvTurnRecovery count = %d, want 0 for stop %q", got, stop) | |||
| 548 | + | } | |||
| 549 | + | }) | |||
| 550 | + | } | |||
| 551 | + | } | |||
| 552 | + | ||||
| 553 | + | // TestAgentTurnRecoveryRespectsMaxSteps: the recovery continuation runs | |||
| 554 | + | // inside the existing step budget and does not reset it. With MaxSteps=1 | |||
| 555 | + | // the recovery is marked but the budget is exhausted, so the existing | |||
| 556 | + | // typed limit error surfaces; with MaxSteps=2 two incomplete endings | |||
| 557 | + | // still terminate with ErrIncompleteTurn. | |||
| 558 | + | func TestAgentTurnRecoveryRespectsMaxSteps(t *testing.T) { | |||
| 559 | + | t.Run("budget exhausted", func(t *testing.T) { | |||
| 560 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 561 | + | {content: nil, stop: provider.StopEnd}, | |||
| 562 | + | }} | |||
| 563 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 564 | + | agent.MaxSteps = 1 | |||
| 565 | + | var events []AgentEvent | |||
| 566 | + | err := runPrompt(t, agent, &events) | |||
| 567 | + | if !errors.Is(err, ErrMaxSteps) { | |||
| 568 | + | t.Fatalf("Prompt error = %v, want ErrMaxSteps", err) | |||
| 569 | + | } | |||
| 570 | + | if got := client.Calls(); got != 1 { | |||
| 571 | + | t.Fatalf("provider calls = %d, want 1 (no reset)", got) | |||
| 572 | + | } | |||
| 573 | + | }) | |||
| 574 | + | t.Run("two incomplete within budget", func(t *testing.T) { | |||
| 575 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 576 | + | {content: nil, stop: provider.StopEnd}, | |||
| 577 | + | {content: nil, stop: provider.StopEnd}, | |||
| 578 | + | }} | |||
| 579 | + | agent := NewAgent(client, "test-model", "system", Registry{}) | |||
| 580 | + | agent.MaxSteps = 2 | |||
| 581 | + | var events []AgentEvent | |||
| 582 | + | err := runPrompt(t, agent, &events) | |||
| 583 | + | if !errors.Is(err, ErrIncompleteTurn) { | |||
| 584 | + | t.Fatalf("Prompt error = %v, want ErrIncompleteTurn", err) | |||
| 585 | + | } | |||
| 586 | + | if got := client.Calls(); got != 2 { | |||
| 587 | + | t.Fatalf("provider calls = %d, want 2", got) | |||
| 588 | + | } | |||
| 589 | + | }) | |||
| 590 | + | } | |||
| 591 | + | ||||
| 592 | + | // TestAgentTurnRecoverySyntheticPersistence: the recovery message is | |||
| 593 | + | // appended through AppendUserContext (firing OnMessageAppended for session | |||
| 594 | + | // durability) and replays to the provider on the next request. | |||
| 595 | + | func TestAgentTurnRecoverySyntheticPersistence(t *testing.T) { | |||
| 596 | + | client := &turnRecoveryScriptClient{turns: []turnRecoveryTurn{ | |||
| 597 | + | {content: []provider.Content{provider.ToolCallBlock{ID: "c1", Name: "result", Arguments: json.RawMessage(`{}`)}}, stop: provider.StopToolUse}, | |||
| 598 | + | {content: nil, stop: provider.StopEnd}, | |||
| 599 | + | {content: []provider.Content{textContent("done", "")}, stop: provider.StopEnd}, | |||
| 600 | + | }} | |||
| 601 | + | agent := NewAgent(client, "test-model", "system", Registry{ | |||
| 602 | + | "result": &resultTool{result: ToolResult{Content: []provider.Content{provider.TextBlock{Text: "ok"}}}}, | |||
| 603 | + | }) | |||
| 604 | + | var appended []provider.Message | |||
| 605 | + | agent.OnMessageAppended = func(m provider.Message) { appended = append(appended, m) } | |||
| 606 | + | var events []AgentEvent | |||
| 607 | + | if err := runPrompt(t, agent, &events); err != nil { | |||
| 608 | + | t.Fatalf("Prompt returned %v", err) | |||
| 609 | + | } | |||
| 610 | + | var sawSynthetic bool | |||
| 611 | + | for _, m := range appended { | |||
| 612 | + | if m.Meta[turnRecoveryMetaKey] == "true" { | |||
| 613 | + | sawSynthetic = true | |||
| 614 | + | if m.Role != provider.RoleUser || len(m.Content) != 1 { | |||
| 615 | + | t.Fatalf("recovery appended message = %#v, want single-block user message", m) | |||
| 616 | + | } | |||
| 617 | + | } | |||
| 618 | + | } | |||
| 619 | + | if !sawSynthetic { | |||
| 620 | + | t.Fatal("OnMessageAppended never fired for the recovery message") | |||
| 621 | + | } | |||
| 622 | + | // The transcript keeps tool-result pairing and the recovery marker in | |||
| 623 | + | // order. An empty terminal message carries no blocks, so oneTurn does | |||
| 624 | + | // not append it: user, assistant tool call, tool result, synthetic, | |||
| 625 | + | // final answer. | |||
| 626 | + | msgs := agent.Messages() | |||
| 627 | + | if len(msgs) != 5 { | |||
| 628 | + | t.Fatalf("transcript messages = %d, want 5", len(msgs)) | |||
| 629 | + | } | |||
| 630 | + | if msgs[1].Role != provider.RoleAssistant || msgs[2].Role != provider.RoleTool { | |||
| 631 | + | t.Fatalf("tool pairing broken: roles = %q, %q", msgs[1].Role, msgs[2].Role) | |||
| 632 | + | } | |||
| 633 | + | if msgs[3].Meta[turnRecoveryMetaKey] != "true" { | |||
| 634 | + | t.Fatalf("message 3 meta = %#v, want recovery marker", msgs[3].Meta) | |||
| 635 | + | } | |||
| 636 | + | } | |||
M packages/provider/openai_codex.go +90 -11
| @@ -126,6 +126,7 @@ type codexOutputText struct { | |||
| 126 | 126 | Type string `json:"type"` // "output_text" | |
| 127 | 127 | Text string `json:"text"` | |
| 128 | 128 | Annotations []any `json:"annotations"` | |
| 129 | + | Phase string `json:"phase,omitempty"` | |
| 129 | 130 | } | |
| 130 | 131 | ||
| 131 | 132 | type codexPromptCacheOptions struct { | |
| @@ -368,14 +369,16 @@ func (c *codexClient) buildRequest(req Request) (*codexRequest, error) { | |||
| 368 | 369 | continue | |
| 369 | 370 | } | |
| 370 | 371 | msgIdx++ | |
| 372 | + | out := codexOutputText{Type: "output_text", Text: v.Text, Annotations: []any{}} | |
| 373 | + | if IsRecognizedTextPhase(v.Phase) { | |
| 374 | + | out.Phase = v.Phase | |
| 375 | + | } | |
| 371 | 376 | body.Input = append(body.Input, codexOutputMessage{ | |
| 372 | - | Type: "message", | |
| 373 | - | Role: "assistant", | |
| 374 | - | Status: "completed", | |
| 375 | - | ID: fmt.Sprintf("msg_%d", msgIdx), | |
| 376 | - | Content: []codexOutputText{ | |
| 377 | - | {Type: "output_text", Text: v.Text, Annotations: []any{}}, | |
| 378 | - | }, | |
| 377 | + | Type: "message", | |
| 378 | + | Role: "assistant", | |
| 379 | + | Status: "completed", | |
| 380 | + | ID: fmt.Sprintf("msg_%d", msgIdx), | |
| 381 | + | Content: []codexOutputText{out}, | |
| 379 | 382 | }) | |
| 380 | 383 | case ToolCallBlock: | |
| 381 | 384 | args := string(v.Arguments) | |
| @@ -476,6 +479,28 @@ func splitCallID(id string) (string, string) { | |||
| 476 | 479 | return id, "" | |
| 477 | 480 | } | |
| 478 | 481 | ||
| 482 | + | // messageItemPhase extracts the Responses message phase from either the | |
| 483 | + | // item-level phase field or the first nonempty phase on its output_text | |
| 484 | + | // content parts. The return is fill-only input for itemState.phase: empty | |
| 485 | + | // means absent, never a reason to erase an observed value. Unrecognized | |
| 486 | + | // values are preserved so storage stays faithful; IsRecognizedTextPhase | |
| 487 | + | // gates interpretation and replay. | |
| 488 | + | func messageItemPhase(itemPhase string, content []struct { | |
| 489 | + | Type string `json:"type"` | |
| 490 | + | Text string `json:"text"` | |
| 491 | + | Phase string `json:"phase"` | |
| 492 | + | }) string { | |
| 493 | + | if itemPhase != "" { | |
| 494 | + | return itemPhase | |
| 495 | + | } | |
| 496 | + | for _, part := range content { | |
| 497 | + | if part.Phase != "" { | |
| 498 | + | return part.Phase | |
| 499 | + | } | |
| 500 | + | } | |
| 501 | + | return "" | |
| 502 | + | } | |
| 503 | + | ||
| 479 | 504 | const minOpenAIPromptCacheTokens = 1024 | |
| 480 | 505 | ||
| 481 | 506 | func responsesCacheDiagnostics(wire *codexRequest, transport, continuation string) CacheDiagnostics { | |
| @@ -646,6 +671,11 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||
| 646 | 671 | summary strings.Builder | |
| 647 | 672 | rawID string | |
| 648 | 673 | encrypted string | |
| 674 | + | // phase is the Responses output phase observed for a message item. | |
| 675 | + | // It is fill-only: an absent later field must never erase an | |
| 676 | + | // observed value. Any observed value (including unrecognized ones) | |
| 677 | + | // is retained in storage; only recognized values are replayed. | |
| 678 | + | phase string | |
| 649 | 679 | announced bool | |
| 650 | 680 | } | |
| 651 | 681 | var ( | |
| @@ -663,7 +693,7 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||
| 663 | 693 | switch it.kind { | |
| 664 | 694 | case "message": | |
| 665 | 695 | if it.textBuf.Len() > 0 { | |
| 666 | - | content = append(content, TextBlock{Text: it.textBuf.String()}) | |
| 696 | + | content = append(content, TextBlock{Text: it.textBuf.String(), Phase: it.phase}) | |
| 667 | 697 | } | |
| 668 | 698 | case "function_call": | |
| 669 | 699 | args := it.argsBuf.String() | |
| @@ -736,6 +766,12 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||
| 736 | 766 | CallID string `json:"call_id"` | |
| 737 | 767 | Name string `json:"name"` | |
| 738 | 768 | EncryptedContent string `json:"encrypted_content"` | |
| 769 | + | Phase string `json:"phase"` | |
| 770 | + | Content []struct { | |
| 771 | + | Type string `json:"type"` | |
| 772 | + | Text string `json:"text"` | |
| 773 | + | Phase string `json:"phase"` | |
| 774 | + | } `json:"content"` | |
| 739 | 775 | } `json:"item"` | |
| 740 | 776 | } | |
| 741 | 777 | _ = json.Unmarshal([]byte(ev.Data), &p) | |
| @@ -743,6 +779,7 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||
| 743 | 779 | switch p.Item.Type { | |
| 744 | 780 | case "message": | |
| 745 | 781 | it.kind = "message" | |
| 782 | + | it.phase = messageItemPhase(p.Item.Phase, p.Item.Content) | |
| 746 | 783 | case "function_call": | |
| 747 | 784 | it.kind = "function_call" | |
| 748 | 785 | it.callID = p.Item.CallID | |
| @@ -798,7 +835,13 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||
| 798 | 835 | Type string `json:"type"` | |
| 799 | 836 | ID string `json:"id"` | |
| 800 | 837 | EncryptedContent string `json:"encrypted_content"` | |
| 801 | - | Summary []struct { | |
| 838 | + | Phase string `json:"phase"` | |
| 839 | + | Content []struct { | |
| 840 | + | Type string `json:"type"` | |
| 841 | + | Text string `json:"text"` | |
| 842 | + | Phase string `json:"phase"` | |
| 843 | + | } `json:"content"` | |
| 844 | + | Summary []struct { | |
| 802 | 845 | Type string `json:"type"` | |
| 803 | 846 | Text string `json:"text"` | |
| 804 | 847 | } `json:"summary"` | |
| @@ -809,6 +852,12 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||
| 809 | 852 | switch it.kind { | |
| 810 | 853 | case "function_call": | |
| 811 | 854 | out <- EventToolEnd{ID: it.callID} | |
| 855 | + | case "message": | |
| 856 | + | // Fill-only: an absent phase on item-done must not | |
| 857 | + | // erase a phase already observed on item-added. | |
| 858 | + | if it.phase == "" { | |
| 859 | + | it.phase = messageItemPhase(p.Item.Phase, p.Item.Content) | |
| 860 | + | } | |
| 812 | 861 | case "reasoning": | |
| 813 | 862 | // Fill-only: earlier streamed payloads and summaries win | |
| 814 | 863 | // over later duplicates from output_item.done. | |
| @@ -849,7 +898,13 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||
| 849 | 898 | Type string `json:"type"` | |
| 850 | 899 | ID string `json:"id"` | |
| 851 | 900 | EncryptedContent string `json:"encrypted_content"` | |
| 852 | - | Summary []struct { | |
| 901 | + | Phase string `json:"phase"` | |
| 902 | + | Content []struct { | |
| 903 | + | Type string `json:"type"` | |
| 904 | + | Text string `json:"text"` | |
| 905 | + | Phase string `json:"phase"` | |
| 906 | + | } `json:"content"` | |
| 907 | + | Summary []struct { | |
| 853 | 908 | Type string `json:"type"` | |
| 854 | 909 | Text string `json:"text"` | |
| 855 | 910 | } `json:"summary"` | |
| @@ -861,13 +916,20 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||
| 861 | 916 | // response output. Match by nonempty provider-issued item ID, | |
| 862 | 917 | // never by output-array position. Fill a missing payload from | |
| 863 | 918 | // a matching terminal item without overwriting an earlier | |
| 864 | - | // payload or duplicating streamed summaries. | |
| 919 | + | // payload or duplicating streamed summaries. Message phase is | |
| 920 | + | // reconciled the same fill-only way, but by output-array | |
| 921 | + | // position: terminal text content without a phase must not | |
| 922 | + | // erase a phase observed during streaming. | |
| 865 | 923 | if len(p.Response.Output) > 0 { | |
| 866 | 924 | terminal := make(map[string]struct { | |
| 867 | 925 | encrypted string | |
| 868 | 926 | summary string | |
| 869 | 927 | }, len(p.Response.Output)) | |
| 928 | + | terminalPhases := make([]string, 0, len(p.Response.Output)) | |
| 870 | 929 | for _, out := range p.Response.Output { | |
| 930 | + | if out.Type == "message" { | |
| 931 | + | terminalPhases = append(terminalPhases, messageItemPhase(out.Phase, out.Content)) | |
| 932 | + | } | |
| 871 | 933 | if out.Type != "reasoning" || out.ID == "" { | |
| 872 | 934 | continue | |
| 873 | 935 | } | |
| @@ -904,6 +966,23 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||
| 904 | 966 | it.summary.WriteString(match.summary) | |
| 905 | 967 | } | |
| 906 | 968 | } | |
| 969 | + | // Reconcile message phases by output-array position: the | |
| 970 | + | // n-th terminal message corresponds to the n-th streamed | |
| 971 | + | // message item. Fill only; never erase streamed phases. | |
| 972 | + | pos := 0 | |
| 973 | + | for _, idx := range order { | |
| 974 | + | it := items[idx] | |
| 975 | + | if it == nil || it.kind != "message" { | |
| 976 | + | continue | |
| 977 | + | } | |
| 978 | + | if pos >= len(terminalPhases) { | |
| 979 | + | break | |
| 980 | + | } | |
| 981 | + | if it.phase == "" { | |
| 982 | + | it.phase = terminalPhases[pos] | |
| 983 | + | } | |
| 984 | + | pos++ | |
| 985 | + | } | |
| 907 | 986 | } | |
| 908 | 987 | usage = normalizeOpenAIUsage(p.Response.Usage.InputTokens, p.Response.Usage.OutputTokens, p.Response.Usage.InputTokensDetails) | |
| 909 | 988 | if details := p.Response.Usage.OutputTokensDetails; details != nil { | |
| @@ -126,6 +126,7 @@ type codexOutputText struct { | |||||
| 126 | Type string `json:"type"` // "output_text" | 126 | Type string `json:"type"` // "output_text" | ||
| 127 | Text string `json:"text"` | 127 | Text string `json:"text"` | ||
| 128 | Annotations []any `json:"annotations"` | 128 | Annotations []any `json:"annotations"` | ||
| 129 | + | Phase string `json:"phase,omitempty"` | |||
| 129 | } | 130 | } | ||
| 130 | 131 | ||||
| 131 | type codexPromptCacheOptions struct { | 132 | type codexPromptCacheOptions struct { | ||
| @@ -368,14 +369,16 @@ func (c *codexClient) buildRequest(req Request) (*codexRequest, error) { | |||||
| 368 | continue | 369 | continue | ||
| 369 | } | 370 | } | ||
| 370 | msgIdx++ | 371 | msgIdx++ | ||
| 372 | + | out := codexOutputText{Type: "output_text", Text: v.Text, Annotations: []any{}} | |||
| 373 | + | if IsRecognizedTextPhase(v.Phase) { | |||
| 374 | + | out.Phase = v.Phase | |||
| 375 | + | } | |||
| 371 | body.Input = append(body.Input, codexOutputMessage{ | 376 | body.Input = append(body.Input, codexOutputMessage{ | ||
| 372 | - | Type: "message", | 377 | + | Type: "message", |
| 373 | - | Role: "assistant", | 378 | + | Role: "assistant", |
| 374 | - | Status: "completed", | 379 | + | Status: "completed", |
| 375 | - | ID: fmt.Sprintf("msg_%d", msgIdx), | 380 | + | ID: fmt.Sprintf("msg_%d", msgIdx), |
| 376 | - | Content: []codexOutputText{ | 381 | + | Content: []codexOutputText{out}, |
| 377 | - | {Type: "output_text", Text: v.Text, Annotations: []any{}}, | |||
| 378 | - | }, | |||
| 379 | }) | 382 | }) | ||
| 380 | case ToolCallBlock: | 383 | case ToolCallBlock: | ||
| 381 | args := string(v.Arguments) | 384 | args := string(v.Arguments) | ||
| @@ -476,6 +479,28 @@ func splitCallID(id string) (string, string) { | |||||
| 476 | return id, "" | 479 | return id, "" | ||
| 477 | } | 480 | } | ||
| 478 | 481 | ||||
| 482 | + | // messageItemPhase extracts the Responses message phase from either the | |||
| 483 | + | // item-level phase field or the first nonempty phase on its output_text | |||
| 484 | + | // content parts. The return is fill-only input for itemState.phase: empty | |||
| 485 | + | // means absent, never a reason to erase an observed value. Unrecognized | |||
| 486 | + | // values are preserved so storage stays faithful; IsRecognizedTextPhase | |||
| 487 | + | // gates interpretation and replay. | |||
| 488 | + | func messageItemPhase(itemPhase string, content []struct { | |||
| 489 | + | Type string `json:"type"` | |||
| 490 | + | Text string `json:"text"` | |||
| 491 | + | Phase string `json:"phase"` | |||
| 492 | + | }) string { | |||
| 493 | + | if itemPhase != "" { | |||
| 494 | + | return itemPhase | |||
| 495 | + | } | |||
| 496 | + | for _, part := range content { | |||
| 497 | + | if part.Phase != "" { | |||
| 498 | + | return part.Phase | |||
| 499 | + | } | |||
| 500 | + | } | |||
| 501 | + | return "" | |||
| 502 | + | } | |||
| 503 | + | ||||
| 479 | const minOpenAIPromptCacheTokens = 1024 | 504 | const minOpenAIPromptCacheTokens = 1024 | ||
| 480 | 505 | ||||
| 481 | func responsesCacheDiagnostics(wire *codexRequest, transport, continuation string) CacheDiagnostics { | 506 | func responsesCacheDiagnostics(wire *codexRequest, transport, continuation string) CacheDiagnostics { | ||
| @@ -646,6 +671,11 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||||
| 646 | summary strings.Builder | 671 | summary strings.Builder | ||
| 647 | rawID string | 672 | rawID string | ||
| 648 | encrypted string | 673 | encrypted string | ||
| 674 | + | // phase is the Responses output phase observed for a message item. | |||
| 675 | + | // It is fill-only: an absent later field must never erase an | |||
| 676 | + | // observed value. Any observed value (including unrecognized ones) | |||
| 677 | + | // is retained in storage; only recognized values are replayed. | |||
| 678 | + | phase string | |||
| 649 | announced bool | 679 | announced bool | ||
| 650 | } | 680 | } | ||
| 651 | var ( | 681 | var ( | ||
| @@ -663,7 +693,7 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||||
| 663 | switch it.kind { | 693 | switch it.kind { | ||
| 664 | case "message": | 694 | case "message": | ||
| 665 | if it.textBuf.Len() > 0 { | 695 | if it.textBuf.Len() > 0 { | ||
| 666 | - | content = append(content, TextBlock{Text: it.textBuf.String()}) | 696 | + | content = append(content, TextBlock{Text: it.textBuf.String(), Phase: it.phase}) |
| 667 | } | 697 | } | ||
| 668 | case "function_call": | 698 | case "function_call": | ||
| 669 | args := it.argsBuf.String() | 699 | args := it.argsBuf.String() | ||
| @@ -736,6 +766,12 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||||
| 736 | CallID string `json:"call_id"` | 766 | CallID string `json:"call_id"` | ||
| 737 | Name string `json:"name"` | 767 | Name string `json:"name"` | ||
| 738 | EncryptedContent string `json:"encrypted_content"` | 768 | EncryptedContent string `json:"encrypted_content"` | ||
| 769 | + | Phase string `json:"phase"` | |||
| 770 | + | Content []struct { | |||
| 771 | + | Type string `json:"type"` | |||
| 772 | + | Text string `json:"text"` | |||
| 773 | + | Phase string `json:"phase"` | |||
| 774 | + | } `json:"content"` | |||
| 739 | } `json:"item"` | 775 | } `json:"item"` | ||
| 740 | } | 776 | } | ||
| 741 | _ = json.Unmarshal([]byte(ev.Data), &p) | 777 | _ = json.Unmarshal([]byte(ev.Data), &p) | ||
| @@ -743,6 +779,7 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||||
| 743 | switch p.Item.Type { | 779 | switch p.Item.Type { | ||
| 744 | case "message": | 780 | case "message": | ||
| 745 | it.kind = "message" | 781 | it.kind = "message" | ||
| 782 | + | it.phase = messageItemPhase(p.Item.Phase, p.Item.Content) | |||
| 746 | case "function_call": | 783 | case "function_call": | ||
| 747 | it.kind = "function_call" | 784 | it.kind = "function_call" | ||
| 748 | it.callID = p.Item.CallID | 785 | it.callID = p.Item.CallID | ||
| @@ -798,7 +835,13 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||||
| 798 | Type string `json:"type"` | 835 | Type string `json:"type"` | ||
| 799 | ID string `json:"id"` | 836 | ID string `json:"id"` | ||
| 800 | EncryptedContent string `json:"encrypted_content"` | 837 | EncryptedContent string `json:"encrypted_content"` | ||
| 801 | - | Summary []struct { | 838 | + | Phase string `json:"phase"` |
| 839 | + | Content []struct { | |||
| 840 | + | Type string `json:"type"` | |||
| 841 | + | Text string `json:"text"` | |||
| 842 | + | Phase string `json:"phase"` | |||
| 843 | + | } `json:"content"` | |||
| 844 | + | Summary []struct { | |||
| 802 | Type string `json:"type"` | 845 | Type string `json:"type"` | ||
| 803 | Text string `json:"text"` | 846 | Text string `json:"text"` | ||
| 804 | } `json:"summary"` | 847 | } `json:"summary"` | ||
| @@ -809,6 +852,12 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||||
| 809 | switch it.kind { | 852 | switch it.kind { | ||
| 810 | case "function_call": | 853 | case "function_call": | ||
| 811 | out <- EventToolEnd{ID: it.callID} | 854 | out <- EventToolEnd{ID: it.callID} | ||
| 855 | + | case "message": | |||
| 856 | + | // Fill-only: an absent phase on item-done must not | |||
| 857 | + | // erase a phase already observed on item-added. | |||
| 858 | + | if it.phase == "" { | |||
| 859 | + | it.phase = messageItemPhase(p.Item.Phase, p.Item.Content) | |||
| 860 | + | } | |||
| 812 | case "reasoning": | 861 | case "reasoning": | ||
| 813 | // Fill-only: earlier streamed payloads and summaries win | 862 | // Fill-only: earlier streamed payloads and summaries win | ||
| 814 | // over later duplicates from output_item.done. | 863 | // over later duplicates from output_item.done. | ||
| @@ -849,7 +898,13 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||||
| 849 | Type string `json:"type"` | 898 | Type string `json:"type"` | ||
| 850 | ID string `json:"id"` | 899 | ID string `json:"id"` | ||
| 851 | EncryptedContent string `json:"encrypted_content"` | 900 | EncryptedContent string `json:"encrypted_content"` | ||
| 852 | - | Summary []struct { | 901 | + | Phase string `json:"phase"` |
| 902 | + | Content []struct { | |||
| 903 | + | Type string `json:"type"` | |||
| 904 | + | Text string `json:"text"` | |||
| 905 | + | Phase string `json:"phase"` | |||
| 906 | + | } `json:"content"` | |||
| 907 | + | Summary []struct { | |||
| 853 | Type string `json:"type"` | 908 | Type string `json:"type"` | ||
| 854 | Text string `json:"text"` | 909 | Text string `json:"text"` | ||
| 855 | } `json:"summary"` | 910 | } `json:"summary"` | ||
| @@ -861,13 +916,20 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||||
| 861 | // response output. Match by nonempty provider-issued item ID, | 916 | // response output. Match by nonempty provider-issued item ID, | ||
| 862 | // never by output-array position. Fill a missing payload from | 917 | // never by output-array position. Fill a missing payload from | ||
| 863 | // a matching terminal item without overwriting an earlier | 918 | // a matching terminal item without overwriting an earlier | ||
| 864 | - | // payload or duplicating streamed summaries. | 919 | + | // payload or duplicating streamed summaries. Message phase is |
| 920 | + | // reconciled the same fill-only way, but by output-array | |||
| 921 | + | // position: terminal text content without a phase must not | |||
| 922 | + | // erase a phase observed during streaming. | |||
| 865 | if len(p.Response.Output) > 0 { | 923 | if len(p.Response.Output) > 0 { | ||
| 866 | terminal := make(map[string]struct { | 924 | terminal := make(map[string]struct { | ||
| 867 | encrypted string | 925 | encrypted string | ||
| 868 | summary string | 926 | summary string | ||
| 869 | }, len(p.Response.Output)) | 927 | }, len(p.Response.Output)) | ||
| 928 | + | terminalPhases := make([]string, 0, len(p.Response.Output)) | |||
| 870 | for _, out := range p.Response.Output { | 929 | for _, out := range p.Response.Output { | ||
| 930 | + | if out.Type == "message" { | |||
| 931 | + | terminalPhases = append(terminalPhases, messageItemPhase(out.Phase, out.Content)) | |||
| 932 | + | } | |||
| 871 | if out.Type != "reasoning" || out.ID == "" { | 933 | if out.Type != "reasoning" || out.ID == "" { | ||
| 872 | continue | 934 | continue | ||
| 873 | } | 935 | } | ||
| @@ -904,6 +966,23 @@ func (c *codexClient) runResponseEventsWithFirst(ctx context.Context, req Reques | |||||
| 904 | it.summary.WriteString(match.summary) | 966 | it.summary.WriteString(match.summary) | ||
| 905 | } | 967 | } | ||
| 906 | } | 968 | } | ||
| 969 | + | // Reconcile message phases by output-array position: the | |||
| 970 | + | // n-th terminal message corresponds to the n-th streamed | |||
| 971 | + | // message item. Fill only; never erase streamed phases. | |||
| 972 | + | pos := 0 | |||
| 973 | + | for _, idx := range order { | |||
| 974 | + | it := items[idx] | |||
| 975 | + | if it == nil || it.kind != "message" { | |||
| 976 | + | continue | |||
| 977 | + | } | |||
| 978 | + | if pos >= len(terminalPhases) { | |||
| 979 | + | break | |||
| 980 | + | } | |||
| 981 | + | if it.phase == "" { | |||
| 982 | + | it.phase = terminalPhases[pos] | |||
| 983 | + | } | |||
| 984 | + | pos++ | |||
| 985 | + | } | |||
| 907 | } | 986 | } | ||
| 908 | usage = normalizeOpenAIUsage(p.Response.Usage.InputTokens, p.Response.Usage.OutputTokens, p.Response.Usage.InputTokensDetails) | 987 | usage = normalizeOpenAIUsage(p.Response.Usage.InputTokens, p.Response.Usage.OutputTokens, p.Response.Usage.InputTokensDetails) | ||
| 909 | if details := p.Response.Usage.OutputTokensDetails; details != nil { | 988 | if details := p.Response.Usage.OutputTokensDetails; details != nil { | ||
A packages/provider/openai_responses_phase_test.go +222 -0
| @@ -0,0 +1,222 @@ | |||
| 1 | + | package provider | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "context" | |
| 5 | + | "encoding/json" | |
| 6 | + | "strings" | |
| 7 | + | "testing" | |
| 8 | + | ) | |
| 9 | + | ||
| 10 | + | // phaseTexts returns (text, phase) pairs for assistant text blocks. | |
| 11 | + | func phaseTexts(t *testing.T, done EventDone) [][2]string { | |
| 12 | + | t.Helper() | |
| 13 | + | var out [][2]string | |
| 14 | + | for _, c := range done.Message.Content { | |
| 15 | + | if tb, ok := c.(TextBlock); ok { | |
| 16 | + | out = append(out, [2]string{tb.Text, tb.Phase}) | |
| 17 | + | } | |
| 18 | + | } | |
| 19 | + | return out | |
| 20 | + | } | |
| 21 | + | ||
| 22 | + | func TestResponsesPhaseCommentaryThenFinal(t *testing.T) { | |
| 23 | + | done, _ := runResponsesCompletionFixture(t, | |
| 24 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message","phase":"commentary"}}`, | |
| 25 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"working on it"}`, | |
| 26 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |
| 27 | + | `{"type":"response.output_item.added","output_index":1,"item":{"type":"message","phase":"final_answer"}}`, | |
| 28 | + | `{"type":"response.output_text.delta","output_index":1,"delta":"done"}`, | |
| 29 | + | `{"type":"response.output_item.done","output_index":1,"item":{"type":"message"}}`, | |
| 30 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |
| 31 | + | ) | |
| 32 | + | got := phaseTexts(t, done) | |
| 33 | + | want := [][2]string{{"working on it", "commentary"}, {"done", "final_answer"}} | |
| 34 | + | if len(got) != len(want) { | |
| 35 | + | t.Fatalf("blocks = %#v, want %#v", got, want) | |
| 36 | + | } | |
| 37 | + | for i := range want { | |
| 38 | + | if got[i] != want[i] { | |
| 39 | + | t.Fatalf("blocks = %#v, want %#v", got, want) | |
| 40 | + | } | |
| 41 | + | } | |
| 42 | + | } | |
| 43 | + | ||
| 44 | + | func TestResponsesPhaseOnItemCompletion(t *testing.T) { | |
| 45 | + | // Phase first appears on item-done rather than item-added. | |
| 46 | + | done, _ := runResponsesCompletionFixture(t, | |
| 47 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message"}}`, | |
| 48 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"late phase"}`, | |
| 49 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message","phase":"commentary"}}`, | |
| 50 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |
| 51 | + | ) | |
| 52 | + | got := phaseTexts(t, done) | |
| 53 | + | if len(got) != 1 || got[0] != [2]string{"late phase", "commentary"} { | |
| 54 | + | t.Fatalf("blocks = %#v, want [{late phase commentary}]", got) | |
| 55 | + | } | |
| 56 | + | } | |
| 57 | + | ||
| 58 | + | func TestResponsesPhaseAddedNotErasedByDone(t *testing.T) { | |
| 59 | + | // Phase observed on item-added must survive a phaseless item-done. | |
| 60 | + | done, _ := runResponsesCompletionFixture(t, | |
| 61 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message","phase":"commentary"}}`, | |
| 62 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"keep me"}`, | |
| 63 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |
| 64 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |
| 65 | + | ) | |
| 66 | + | got := phaseTexts(t, done) | |
| 67 | + | if len(got) != 1 || got[0] != [2]string{"keep me", "commentary"} { | |
| 68 | + | t.Fatalf("blocks = %#v, want [{keep me commentary}]", got) | |
| 69 | + | } | |
| 70 | + | } | |
| 71 | + | ||
| 72 | + | func TestResponsesPhaseTerminalReconciliation(t *testing.T) { | |
| 73 | + | // Phase only present in the terminal response.output array. | |
| 74 | + | done, _ := runResponsesCompletionFixture(t, | |
| 75 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message"}}`, | |
| 76 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"terminal phase"}`, | |
| 77 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |
| 78 | + | `{"type":"response.completed","response":{"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1},"output":[{"type":"message","content":[{"type":"output_text","text":"terminal phase","phase":"final_answer"}]}]}}`, | |
| 79 | + | ) | |
| 80 | + | got := phaseTexts(t, done) | |
| 81 | + | if len(got) != 1 || got[0] != [2]string{"terminal phase", "final_answer"} { | |
| 82 | + | t.Fatalf("blocks = %#v, want [{terminal phase final_answer}]", got) | |
| 83 | + | } | |
| 84 | + | } | |
| 85 | + | ||
| 86 | + | func TestResponsesPhaseAbsentAndUnknown(t *testing.T) { | |
| 87 | + | done, _ := runResponsesCompletionFixture(t, | |
| 88 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message"}}`, | |
| 89 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"plain"}`, | |
| 90 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |
| 91 | + | `{"type":"response.output_item.added","output_index":1,"item":{"type":"message","phase":"mystery"}}`, | |
| 92 | + | `{"type":"response.output_text.delta","output_index":1,"delta":"weird"}`, | |
| 93 | + | `{"type":"response.output_item.done","output_index":1,"item":{"type":"message"}}`, | |
| 94 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |
| 95 | + | ) | |
| 96 | + | if done.Stop != StopEnd { | |
| 97 | + | t.Fatalf("stop = %q, want %q", done.Stop, StopEnd) | |
| 98 | + | } | |
| 99 | + | got := phaseTexts(t, done) | |
| 100 | + | want := [][2]string{{"plain", ""}, {"weird", "mystery"}} | |
| 101 | + | if len(got) != len(want) { | |
| 102 | + | t.Fatalf("blocks = %#v, want %#v", got, want) | |
| 103 | + | } | |
| 104 | + | for i := range want { | |
| 105 | + | if got[i] != want[i] { | |
| 106 | + | t.Fatalf("blocks = %#v, want %#v", got, want) | |
| 107 | + | } | |
| 108 | + | } | |
| 109 | + | // Unknown phases are stored but not interpreted as recognized. | |
| 110 | + | for _, c := range done.Message.Content { | |
| 111 | + | if tb, ok := c.(TextBlock); ok && tb.Phase == "mystery" && IsRecognizedTextPhase(tb.Phase) { | |
| 112 | + | t.Fatalf("unknown phase treated as recognized: %q", tb.Phase) | |
| 113 | + | } | |
| 114 | + | } | |
| 115 | + | } | |
| 116 | + | ||
| 117 | + | func TestResponsesPhaseToolInterleaving(t *testing.T) { | |
| 118 | + | done, _ := runResponsesCompletionFixture(t, | |
| 119 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message","phase":"commentary"}}`, | |
| 120 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"checking"}`, | |
| 121 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |
| 122 | + | `{"type":"response.output_item.added","output_index":1,"item":{"type":"function_call","call_id":"call-1","name":"echo"}}`, | |
| 123 | + | `{"type":"response.function_call_arguments.delta","output_index":1,"delta":"{}"}`, | |
| 124 | + | `{"type":"response.output_item.done","output_index":1,"item":{"type":"function_call"}}`, | |
| 125 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |
| 126 | + | ) | |
| 127 | + | if done.Stop != StopToolUse { | |
| 128 | + | t.Fatalf("stop = %q, want %q", done.Stop, StopToolUse) | |
| 129 | + | } | |
| 130 | + | if len(done.Message.Content) != 2 { | |
| 131 | + | t.Fatalf("content = %#v, want 2 blocks", done.Message.Content) | |
| 132 | + | } | |
| 133 | + | tb, ok := done.Message.Content[0].(TextBlock) | |
| 134 | + | if !ok || tb.Text != "checking" || tb.Phase != "commentary" { | |
| 135 | + | t.Fatalf("first block = %#v, want commentary text", done.Message.Content[0]) | |
| 136 | + | } | |
| 137 | + | if tc, ok := done.Message.Content[1].(ToolCallBlock); !ok || tc.ID != "call-1" { | |
| 138 | + | t.Fatalf("second block = %#v, want tool call", done.Message.Content[1]) | |
| 139 | + | } | |
| 140 | + | } | |
| 141 | + | ||
| 142 | + | func TestResponsesPhaseRequestReplay(t *testing.T) { | |
| 143 | + | c := newOpenAICodexClient("token", "acct", "") | |
| 144 | + | req := Request{ | |
| 145 | + | Model: "gpt-5.2", | |
| 146 | + | Messages: []Message{ | |
| 147 | + | {Role: RoleAssistant, Content: []Content{ | |
| 148 | + | TextBlock{Text: "progress", Phase: "commentary"}, | |
| 149 | + | TextBlock{Text: "result", Phase: "final_answer"}, | |
| 150 | + | TextBlock{Text: "plain"}, | |
| 151 | + | TextBlock{Text: "odd", Phase: "mystery"}, | |
| 152 | + | }}, | |
| 153 | + | }, | |
| 154 | + | } | |
| 155 | + | wire, err := c.buildRequest(req) | |
| 156 | + | if err != nil { | |
| 157 | + | t.Fatalf("buildRequest: %v", err) | |
| 158 | + | } | |
| 159 | + | raw, err := json.Marshal(wire) | |
| 160 | + | if err != nil { | |
| 161 | + | t.Fatalf("marshal: %v", err) | |
| 162 | + | } | |
| 163 | + | var decoded struct { | |
| 164 | + | Input []struct { | |
| 165 | + | Type string `json:"type"` | |
| 166 | + | Role string `json:"role"` | |
| 167 | + | Content []struct { | |
| 168 | + | Type string `json:"type"` | |
| 169 | + | Text string `json:"text"` | |
| 170 | + | Phase string `json:"phase"` | |
| 171 | + | } `json:"content"` | |
| 172 | + | } `json:"input"` | |
| 173 | + | } | |
| 174 | + | if err := json.Unmarshal(raw, &decoded); err != nil { | |
| 175 | + | t.Fatalf("unmarshal: %v", err) | |
| 176 | + | } | |
| 177 | + | var phases []string | |
| 178 | + | for _, item := range decoded.Input { | |
| 179 | + | if item.Role != "assistant" { | |
| 180 | + | continue | |
| 181 | + | } | |
| 182 | + | for _, part := range item.Content { | |
| 183 | + | phases = append(phases, part.Text+":"+part.Phase) | |
| 184 | + | } | |
| 185 | + | } | |
| 186 | + | want := []string{"progress:commentary", "result:final_answer", "plain:", "odd:"} | |
| 187 | + | if len(phases) != len(want) { | |
| 188 | + | t.Fatalf("replay = %v, want %v (raw %s)", phases, want, raw) | |
| 189 | + | } | |
| 190 | + | for i := range want { | |
| 191 | + | if phases[i] != want[i] { | |
| 192 | + | t.Fatalf("replay = %v, want %v (raw %s)", phases, want, raw) | |
| 193 | + | } | |
| 194 | + | } | |
| 195 | + | if strings.Contains(string(raw), "mystery") { | |
| 196 | + | t.Fatalf("unrecognized phase leaked to wire: %s", raw) | |
| 197 | + | } | |
| 198 | + | } | |
| 199 | + | ||
| 200 | + | func TestResponsesPhaseWebSocketSharedStateMachine(t *testing.T) { | |
| 201 | + | // The WebSocket transport feeds the same runResponseEvents parser, so | |
| 202 | + | // exercise it through the shared entry point used by the socket path. | |
| 203 | + | c := newOpenAICodexClient("token", "acct", "") | |
| 204 | + | first := sseEvent{Data: `{"type":"response.output_item.added","output_index":0,"item":{"type":"message","phase":"commentary"}}`} | |
| 205 | + | raw := make(chan sseEvent, 4) | |
| 206 | + | raw <- sseEvent{Data: `{"type":"response.output_text.delta","output_index":0,"delta":"ws text"}`} | |
| 207 | + | raw <- sseEvent{Data: `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`} | |
| 208 | + | raw <- sseEvent{Data: completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`)} | |
| 209 | + | close(raw) | |
| 210 | + | out := make(chan Event, 32) | |
| 211 | + | go c.runResponseEventsWithFirst(context.Background(), Request{Model: "gpt-5.2"}, out, raw, &first, nil) | |
| 212 | + | for ev := range out { | |
| 213 | + | if done, ok := ev.(EventDone); ok { | |
| 214 | + | got := phaseTexts(t, done) | |
| 215 | + | if len(got) != 1 || got[0] != [2]string{"ws text", "commentary"} { | |
| 216 | + | t.Fatalf("blocks = %#v, want [{ws text commentary}]", got) | |
| 217 | + | } | |
| 218 | + | return | |
| 219 | + | } | |
| 220 | + | } | |
| 221 | + | t.Fatal("stream ended without EventDone") | |
| 222 | + | } | |
| @@ -0,0 +1,222 @@ | |||||
| 1 | + | package provider | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "context" | |||
| 5 | + | "encoding/json" | |||
| 6 | + | "strings" | |||
| 7 | + | "testing" | |||
| 8 | + | ) | |||
| 9 | + | ||||
| 10 | + | // phaseTexts returns (text, phase) pairs for assistant text blocks. | |||
| 11 | + | func phaseTexts(t *testing.T, done EventDone) [][2]string { | |||
| 12 | + | t.Helper() | |||
| 13 | + | var out [][2]string | |||
| 14 | + | for _, c := range done.Message.Content { | |||
| 15 | + | if tb, ok := c.(TextBlock); ok { | |||
| 16 | + | out = append(out, [2]string{tb.Text, tb.Phase}) | |||
| 17 | + | } | |||
| 18 | + | } | |||
| 19 | + | return out | |||
| 20 | + | } | |||
| 21 | + | ||||
| 22 | + | func TestResponsesPhaseCommentaryThenFinal(t *testing.T) { | |||
| 23 | + | done, _ := runResponsesCompletionFixture(t, | |||
| 24 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message","phase":"commentary"}}`, | |||
| 25 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"working on it"}`, | |||
| 26 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |||
| 27 | + | `{"type":"response.output_item.added","output_index":1,"item":{"type":"message","phase":"final_answer"}}`, | |||
| 28 | + | `{"type":"response.output_text.delta","output_index":1,"delta":"done"}`, | |||
| 29 | + | `{"type":"response.output_item.done","output_index":1,"item":{"type":"message"}}`, | |||
| 30 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |||
| 31 | + | ) | |||
| 32 | + | got := phaseTexts(t, done) | |||
| 33 | + | want := [][2]string{{"working on it", "commentary"}, {"done", "final_answer"}} | |||
| 34 | + | if len(got) != len(want) { | |||
| 35 | + | t.Fatalf("blocks = %#v, want %#v", got, want) | |||
| 36 | + | } | |||
| 37 | + | for i := range want { | |||
| 38 | + | if got[i] != want[i] { | |||
| 39 | + | t.Fatalf("blocks = %#v, want %#v", got, want) | |||
| 40 | + | } | |||
| 41 | + | } | |||
| 42 | + | } | |||
| 43 | + | ||||
| 44 | + | func TestResponsesPhaseOnItemCompletion(t *testing.T) { | |||
| 45 | + | // Phase first appears on item-done rather than item-added. | |||
| 46 | + | done, _ := runResponsesCompletionFixture(t, | |||
| 47 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message"}}`, | |||
| 48 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"late phase"}`, | |||
| 49 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message","phase":"commentary"}}`, | |||
| 50 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |||
| 51 | + | ) | |||
| 52 | + | got := phaseTexts(t, done) | |||
| 53 | + | if len(got) != 1 || got[0] != [2]string{"late phase", "commentary"} { | |||
| 54 | + | t.Fatalf("blocks = %#v, want [{late phase commentary}]", got) | |||
| 55 | + | } | |||
| 56 | + | } | |||
| 57 | + | ||||
| 58 | + | func TestResponsesPhaseAddedNotErasedByDone(t *testing.T) { | |||
| 59 | + | // Phase observed on item-added must survive a phaseless item-done. | |||
| 60 | + | done, _ := runResponsesCompletionFixture(t, | |||
| 61 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message","phase":"commentary"}}`, | |||
| 62 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"keep me"}`, | |||
| 63 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |||
| 64 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |||
| 65 | + | ) | |||
| 66 | + | got := phaseTexts(t, done) | |||
| 67 | + | if len(got) != 1 || got[0] != [2]string{"keep me", "commentary"} { | |||
| 68 | + | t.Fatalf("blocks = %#v, want [{keep me commentary}]", got) | |||
| 69 | + | } | |||
| 70 | + | } | |||
| 71 | + | ||||
| 72 | + | func TestResponsesPhaseTerminalReconciliation(t *testing.T) { | |||
| 73 | + | // Phase only present in the terminal response.output array. | |||
| 74 | + | done, _ := runResponsesCompletionFixture(t, | |||
| 75 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message"}}`, | |||
| 76 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"terminal phase"}`, | |||
| 77 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |||
| 78 | + | `{"type":"response.completed","response":{"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1},"output":[{"type":"message","content":[{"type":"output_text","text":"terminal phase","phase":"final_answer"}]}]}}`, | |||
| 79 | + | ) | |||
| 80 | + | got := phaseTexts(t, done) | |||
| 81 | + | if len(got) != 1 || got[0] != [2]string{"terminal phase", "final_answer"} { | |||
| 82 | + | t.Fatalf("blocks = %#v, want [{terminal phase final_answer}]", got) | |||
| 83 | + | } | |||
| 84 | + | } | |||
| 85 | + | ||||
| 86 | + | func TestResponsesPhaseAbsentAndUnknown(t *testing.T) { | |||
| 87 | + | done, _ := runResponsesCompletionFixture(t, | |||
| 88 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message"}}`, | |||
| 89 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"plain"}`, | |||
| 90 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |||
| 91 | + | `{"type":"response.output_item.added","output_index":1,"item":{"type":"message","phase":"mystery"}}`, | |||
| 92 | + | `{"type":"response.output_text.delta","output_index":1,"delta":"weird"}`, | |||
| 93 | + | `{"type":"response.output_item.done","output_index":1,"item":{"type":"message"}}`, | |||
| 94 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |||
| 95 | + | ) | |||
| 96 | + | if done.Stop != StopEnd { | |||
| 97 | + | t.Fatalf("stop = %q, want %q", done.Stop, StopEnd) | |||
| 98 | + | } | |||
| 99 | + | got := phaseTexts(t, done) | |||
| 100 | + | want := [][2]string{{"plain", ""}, {"weird", "mystery"}} | |||
| 101 | + | if len(got) != len(want) { | |||
| 102 | + | t.Fatalf("blocks = %#v, want %#v", got, want) | |||
| 103 | + | } | |||
| 104 | + | for i := range want { | |||
| 105 | + | if got[i] != want[i] { | |||
| 106 | + | t.Fatalf("blocks = %#v, want %#v", got, want) | |||
| 107 | + | } | |||
| 108 | + | } | |||
| 109 | + | // Unknown phases are stored but not interpreted as recognized. | |||
| 110 | + | for _, c := range done.Message.Content { | |||
| 111 | + | if tb, ok := c.(TextBlock); ok && tb.Phase == "mystery" && IsRecognizedTextPhase(tb.Phase) { | |||
| 112 | + | t.Fatalf("unknown phase treated as recognized: %q", tb.Phase) | |||
| 113 | + | } | |||
| 114 | + | } | |||
| 115 | + | } | |||
| 116 | + | ||||
| 117 | + | func TestResponsesPhaseToolInterleaving(t *testing.T) { | |||
| 118 | + | done, _ := runResponsesCompletionFixture(t, | |||
| 119 | + | `{"type":"response.output_item.added","output_index":0,"item":{"type":"message","phase":"commentary"}}`, | |||
| 120 | + | `{"type":"response.output_text.delta","output_index":0,"delta":"checking"}`, | |||
| 121 | + | `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`, | |||
| 122 | + | `{"type":"response.output_item.added","output_index":1,"item":{"type":"function_call","call_id":"call-1","name":"echo"}}`, | |||
| 123 | + | `{"type":"response.function_call_arguments.delta","output_index":1,"delta":"{}"}`, | |||
| 124 | + | `{"type":"response.output_item.done","output_index":1,"item":{"type":"function_call"}}`, | |||
| 125 | + | completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`), | |||
| 126 | + | ) | |||
| 127 | + | if done.Stop != StopToolUse { | |||
| 128 | + | t.Fatalf("stop = %q, want %q", done.Stop, StopToolUse) | |||
| 129 | + | } | |||
| 130 | + | if len(done.Message.Content) != 2 { | |||
| 131 | + | t.Fatalf("content = %#v, want 2 blocks", done.Message.Content) | |||
| 132 | + | } | |||
| 133 | + | tb, ok := done.Message.Content[0].(TextBlock) | |||
| 134 | + | if !ok || tb.Text != "checking" || tb.Phase != "commentary" { | |||
| 135 | + | t.Fatalf("first block = %#v, want commentary text", done.Message.Content[0]) | |||
| 136 | + | } | |||
| 137 | + | if tc, ok := done.Message.Content[1].(ToolCallBlock); !ok || tc.ID != "call-1" { | |||
| 138 | + | t.Fatalf("second block = %#v, want tool call", done.Message.Content[1]) | |||
| 139 | + | } | |||
| 140 | + | } | |||
| 141 | + | ||||
| 142 | + | func TestResponsesPhaseRequestReplay(t *testing.T) { | |||
| 143 | + | c := newOpenAICodexClient("token", "acct", "") | |||
| 144 | + | req := Request{ | |||
| 145 | + | Model: "gpt-5.2", | |||
| 146 | + | Messages: []Message{ | |||
| 147 | + | {Role: RoleAssistant, Content: []Content{ | |||
| 148 | + | TextBlock{Text: "progress", Phase: "commentary"}, | |||
| 149 | + | TextBlock{Text: "result", Phase: "final_answer"}, | |||
| 150 | + | TextBlock{Text: "plain"}, | |||
| 151 | + | TextBlock{Text: "odd", Phase: "mystery"}, | |||
| 152 | + | }}, | |||
| 153 | + | }, | |||
| 154 | + | } | |||
| 155 | + | wire, err := c.buildRequest(req) | |||
| 156 | + | if err != nil { | |||
| 157 | + | t.Fatalf("buildRequest: %v", err) | |||
| 158 | + | } | |||
| 159 | + | raw, err := json.Marshal(wire) | |||
| 160 | + | if err != nil { | |||
| 161 | + | t.Fatalf("marshal: %v", err) | |||
| 162 | + | } | |||
| 163 | + | var decoded struct { | |||
| 164 | + | Input []struct { | |||
| 165 | + | Type string `json:"type"` | |||
| 166 | + | Role string `json:"role"` | |||
| 167 | + | Content []struct { | |||
| 168 | + | Type string `json:"type"` | |||
| 169 | + | Text string `json:"text"` | |||
| 170 | + | Phase string `json:"phase"` | |||
| 171 | + | } `json:"content"` | |||
| 172 | + | } `json:"input"` | |||
| 173 | + | } | |||
| 174 | + | if err := json.Unmarshal(raw, &decoded); err != nil { | |||
| 175 | + | t.Fatalf("unmarshal: %v", err) | |||
| 176 | + | } | |||
| 177 | + | var phases []string | |||
| 178 | + | for _, item := range decoded.Input { | |||
| 179 | + | if item.Role != "assistant" { | |||
| 180 | + | continue | |||
| 181 | + | } | |||
| 182 | + | for _, part := range item.Content { | |||
| 183 | + | phases = append(phases, part.Text+":"+part.Phase) | |||
| 184 | + | } | |||
| 185 | + | } | |||
| 186 | + | want := []string{"progress:commentary", "result:final_answer", "plain:", "odd:"} | |||
| 187 | + | if len(phases) != len(want) { | |||
| 188 | + | t.Fatalf("replay = %v, want %v (raw %s)", phases, want, raw) | |||
| 189 | + | } | |||
| 190 | + | for i := range want { | |||
| 191 | + | if phases[i] != want[i] { | |||
| 192 | + | t.Fatalf("replay = %v, want %v (raw %s)", phases, want, raw) | |||
| 193 | + | } | |||
| 194 | + | } | |||
| 195 | + | if strings.Contains(string(raw), "mystery") { | |||
| 196 | + | t.Fatalf("unrecognized phase leaked to wire: %s", raw) | |||
| 197 | + | } | |||
| 198 | + | } | |||
| 199 | + | ||||
| 200 | + | func TestResponsesPhaseWebSocketSharedStateMachine(t *testing.T) { | |||
| 201 | + | // The WebSocket transport feeds the same runResponseEvents parser, so | |||
| 202 | + | // exercise it through the shared entry point used by the socket path. | |||
| 203 | + | c := newOpenAICodexClient("token", "acct", "") | |||
| 204 | + | first := sseEvent{Data: `{"type":"response.output_item.added","output_index":0,"item":{"type":"message","phase":"commentary"}}`} | |||
| 205 | + | raw := make(chan sseEvent, 4) | |||
| 206 | + | raw <- sseEvent{Data: `{"type":"response.output_text.delta","output_index":0,"delta":"ws text"}`} | |||
| 207 | + | raw <- sseEvent{Data: `{"type":"response.output_item.done","output_index":0,"item":{"type":"message"}}`} | |||
| 208 | + | raw <- sseEvent{Data: completedPayload("response.completed", `"id":"resp-1","usage":{"input_tokens":1,"output_tokens":1}`)} | |||
| 209 | + | close(raw) | |||
| 210 | + | out := make(chan Event, 32) | |||
| 211 | + | go c.runResponseEventsWithFirst(context.Background(), Request{Model: "gpt-5.2"}, out, raw, &first, nil) | |||
| 212 | + | for ev := range out { | |||
| 213 | + | if done, ok := ev.(EventDone); ok { | |||
| 214 | + | got := phaseTexts(t, done) | |||
| 215 | + | if len(got) != 1 || got[0] != [2]string{"ws text", "commentary"} { | |||
| 216 | + | t.Fatalf("blocks = %#v, want [{ws text commentary}]", got) | |||
| 217 | + | } | |||
| 218 | + | return | |||
| 219 | + | } | |||
| 220 | + | } | |||
| 221 | + | t.Fatal("stream ended without EventDone") | |||
| 222 | + | } | |||
M packages/provider/provider.go +18 -0
| @@ -35,10 +35,28 @@ type Content interface { | |||
| 35 | 35 | type TextBlock struct { | |
| 36 | 36 | Text string `json:"text"` | |
| 37 | 37 | ThoughtSignature string `json:"thought_signature,omitempty"` | |
| 38 | + | // Phase is the Responses output phase of assistant text, when the | |
| 39 | + | // provider marks it. Recognized values are "commentary" (intermediate | |
| 40 | + | // progress) and "final_answer". Empty and unrecognized values retain | |
| 41 | + | // compatibility behavior; unrecognized values are preserved in storage | |
| 42 | + | // but never interpreted or replayed as Responses phase values. | |
| 43 | + | Phase string `json:"phase,omitempty"` | |
| 38 | 44 | } | |
| 39 | 45 | ||
| 40 | 46 | func (TextBlock) isContent() {} | |
| 41 | 47 | ||
| 48 | + | // Recognized TextBlock phase values from the Responses API. | |
| 49 | + | const ( | |
| 50 | + | TextPhaseCommentary = "commentary" | |
| 51 | + | TextPhaseFinal = "final_answer" | |
| 52 | + | ) | |
| 53 | + | ||
| 54 | + | // IsRecognizedTextPhase reports whether phase is a Responses phase value | |
| 55 | + | // safe to interpret and replay on the wire. | |
| 56 | + | func IsRecognizedTextPhase(phase string) bool { | |
| 57 | + | return phase == TextPhaseCommentary || phase == TextPhaseFinal | |
| 58 | + | } | |
| 59 | + | ||
| 42 | 60 | // ImageBlock is an inline image (PNG/JPEG/GIF/WebP). | |
| 43 | 61 | type ImageBlock struct { | |
| 44 | 62 | MimeType string `json:"mime_type"` | |
| @@ -35,10 +35,28 @@ type Content interface { | |||||
| 35 | type TextBlock struct { | 35 | type TextBlock struct { | ||
| 36 | Text string `json:"text"` | 36 | Text string `json:"text"` | ||
| 37 | ThoughtSignature string `json:"thought_signature,omitempty"` | 37 | ThoughtSignature string `json:"thought_signature,omitempty"` | ||
| 38 | + | // Phase is the Responses output phase of assistant text, when the | |||
| 39 | + | // provider marks it. Recognized values are "commentary" (intermediate | |||
| 40 | + | // progress) and "final_answer". Empty and unrecognized values retain | |||
| 41 | + | // compatibility behavior; unrecognized values are preserved in storage | |||
| 42 | + | // but never interpreted or replayed as Responses phase values. | |||
| 43 | + | Phase string `json:"phase,omitempty"` | |||
| 38 | } | 44 | } | ||
| 39 | 45 | ||||
| 40 | func (TextBlock) isContent() {} | 46 | func (TextBlock) isContent() {} | ||
| 41 | 47 | ||||
| 48 | + | // Recognized TextBlock phase values from the Responses API. | |||
| 49 | + | const ( | |||
| 50 | + | TextPhaseCommentary = "commentary" | |||
| 51 | + | TextPhaseFinal = "final_answer" | |||
| 52 | + | ) | |||
| 53 | + | ||||
| 54 | + | // IsRecognizedTextPhase reports whether phase is a Responses phase value | |||
| 55 | + | // safe to interpret and replay on the wire. | |||
| 56 | + | func IsRecognizedTextPhase(phase string) bool { | |||
| 57 | + | return phase == TextPhaseCommentary || phase == TextPhaseFinal | |||
| 58 | + | } | |||
| 59 | + | ||||
| 42 | // ImageBlock is an inline image (PNG/JPEG/GIF/WebP). | 60 | // ImageBlock is an inline image (PNG/JPEG/GIF/WebP). | ||
| 43 | type ImageBlock struct { | 61 | type ImageBlock struct { | ||
| 44 | MimeType string `json:"mime_type"` | 62 | MimeType string `json:"mime_type"` | ||