feat(subagents): bound resident children by context, not model steps (#202)
* feat(subagents): bound resident children by context, not model steps A resident child inherits the parent's resolved step limit instead of a hardcoded 50-step fallback, so an unlimited parent no longer caps a child and an explicit --max-steps is still enforced. The context reminder re-arms once per band at 85, 90, and 95 percent instead of firing once per runner, and a provider context-overflow error now recovers once per accepted turn: the child compacts its transcript, keeps the two most recent messages, journals the replacement as a child.compacted record, and continues the same turn. Recovery preserves an explicit step allowance, and a compaction or persistence failure restores the pre-compaction transcript and fails the turn. Resume replay starts from the newest checkpoint; the history pager keeps the append-only log. Reconciliation restarts tool bookkeeping from the checkpoint so a call dropped by compaction cannot gain a synthetic result. * fix(subagents): keep compaction checkpoints free of stale host context Review of the context-bounded child change found four correctness gaps: - The summarization request's usage became the per-turn gauge, so a successful recovery could inject a reminder claiming the compacted transcript sat at the top of the window. Recovery now clears that gauge. - The checkpoint carried host-authored internal context, so a stale reminder was replayed verbatim on resume. core.IsInternalContextMessage is exported and the runner excludes those messages from the durable replacement transcript. - A checkpoint could be attributed to a queued follow-up instead of the executing turn. RecordTurnStarted is now the only writer of the active turn ID. - Reconciliation treated a checkpoint result without its call as repairable work and rewrote the lifecycle time on every pass. Dangling calls are now the exact repair trigger. appendSync also applies the reader's newline-inclusive record bound, so the checkpoint writer no longer duplicates that check, and the reminder bands use a fixed array. New regression tests cover the gauge reset, the checkpoint contents, turn attribution, reconciliation idempotence, a partial step allowance, and an overflow without enough history to compact. * test(subagents): cover the checkpoint record bound and reminder recovery Close the coverage gaps left after the compaction-checkpoint fixes: - appendSync now has a boundary test for the largest record the readers accept and for the one-byte-larger record it must reject; - a dangling call kept by a checkpoint is repaired once and stays a stable no-op on later passes; - residentCheckpointMessages keeps non-internal developer messages; - the gauge-reset test also proves a truthful reminder still fires on a later continuation step. The reminder documentation now states what the code does: the per-turn gauge resets, while the band ladder advances at most once per child runner.
- commit
- 0d89d165e1a01771405514d4fb1b3f703b89af80
- author
- bnema · · 2026-09-11 05:45 UTC
- committer
- GitHub · · 2026-09-11 05:45 UTC
- parent
- 21873e8
- signature
- GPG Unverified
B5690EEEBB952194The key is unknown
9 files · +1449 -146
M README.md +1 -1
| @@ -442,7 +442,7 @@ Resident subagents run alongside the main session as independent in-process agen | |||
| 442 | 442 | ||
| 443 | 443 | The fair global scheduler defaults to six concurrent turns, one active turn per child. `subagents.max_concurrent` overrides that limit. Queueing has no default deadline; a positive `subagents.queue_timeout` durably fails a prompt that waits too long for a slot. Required child work remains an obligation until an explicit successful follow-up. | |
| 444 | 444 | ||
| 445 | - | Each child runs without a cumulative token budget; execution stays bounded through a finite per-turn step default. A child whose last-turn prompt reaches 85% of its model context window receives one non-blocking reminder to finish up, injected into its next provider request. Retrieve a terminal result without a model call using `subagent_status` with `agent_id` and `include_result: true`. After checking the saved progress and side effects, an explicit resume continues the retained session. Required work stays unresolved until success. See [lifecycle and persistence](docs/subagents.md#lifecycle-and-persistence) for details. | |
| 445 | + | Each child runs without a cumulative token budget or step cap: it inherits the parent's resolved `--max-steps`, unlimited by default, and each turn is bounded by the model context window with one compaction recovery. A child whose last-turn prompt reaches 85%, 90%, or 95% of its model context window receives one non-blocking reminder per band to finish up, injected into its next provider request. Retrieve a terminal result without a model call using `subagent_status` with `agent_id` and `include_result: true`. After checking the saved progress and side effects, an explicit resume continues the retained session. Required work stays unresolved until success. See [lifecycle and persistence](docs/subagents.md#lifecycle-and-persistence) for details. | |
| 446 | 446 | ||
| 447 | 447 | Worktree isolation captures a patch and changed-file list without applying it. Shared children use the host checkout. The available logical references are `subagent://<id>`, `subagent://<id>/history`, `subagent://<id>/result`, and `subagent://<id>/patch`. | |
| 448 | 448 | ||
| @@ -442,7 +442,7 @@ Resident subagents run alongside the main session as independent in-process agen | |||||
| 442 | 442 | ||||
| 443 | The fair global scheduler defaults to six concurrent turns, one active turn per child. `subagents.max_concurrent` overrides that limit. Queueing has no default deadline; a positive `subagents.queue_timeout` durably fails a prompt that waits too long for a slot. Required child work remains an obligation until an explicit successful follow-up. | 443 | The fair global scheduler defaults to six concurrent turns, one active turn per child. `subagents.max_concurrent` overrides that limit. Queueing has no default deadline; a positive `subagents.queue_timeout` durably fails a prompt that waits too long for a slot. Required child work remains an obligation until an explicit successful follow-up. | ||
| 444 | 444 | ||||
| 445 | - | Each child runs without a cumulative token budget; execution stays bounded through a finite per-turn step default. A child whose last-turn prompt reaches 85% of its model context window receives one non-blocking reminder to finish up, injected into its next provider request. Retrieve a terminal result without a model call using `subagent_status` with `agent_id` and `include_result: true`. After checking the saved progress and side effects, an explicit resume continues the retained session. Required work stays unresolved until success. See [lifecycle and persistence](docs/subagents.md#lifecycle-and-persistence) for details. | 445 | + | Each child runs without a cumulative token budget or step cap: it inherits the parent's resolved `--max-steps`, unlimited by default, and each turn is bounded by the model context window with one compaction recovery. A child whose last-turn prompt reaches 85%, 90%, or 95% of its model context window receives one non-blocking reminder per band to finish up, injected into its next provider request. Retrieve a terminal result without a model call using `subagent_status` with `agent_id` and `include_result: true`. After checking the saved progress and side effects, an explicit resume continues the retained session. Required work stays unresolved until success. See [lifecycle and persistence](docs/subagents.md#lifecycle-and-persistence) for details. |
| 446 | 446 | ||||
| 447 | Worktree isolation captures a patch and changed-file list without applying it. Shared children use the host checkout. The available logical references are `subagent://<id>`, `subagent://<id>/history`, `subagent://<id>/result`, and `subagent://<id>/patch`. | 447 | Worktree isolation captures a patch and changed-file list without applying it. Shared children use the host checkout. The available logical references are `subagent://<id>`, `subagent://<id>/history`, `subagent://<id>/result`, and `subagent://<id>/patch`. | ||
| 448 | 448 | ||||
M docs/subagents.md +46 -14
| @@ -52,7 +52,7 @@ The manager records each accepted child under its managed state root: | |||
| 52 | 52 | ||
| 53 | 53 | ```text | |
| 54 | 54 | subagents/<child-id>/ | |
| 55 | - | transcript.jsonl # authoritative accepted turns and finalized messages | |
| 55 | + | transcript.jsonl # accepted turns, finalized messages, and compaction checkpoints | |
| 56 | 56 | owner.lock # host-ownership coordination; do not modify or remove | |
| 57 | 57 | .transcript-backup-* # retained private pre-repair transcript, when repaired | |
| 58 | 58 | metadata.json # rebuildable bounded state | |
| @@ -63,7 +63,10 @@ subagents/<child-id>/ | |||
| 63 | 63 | Acceptance is durable before `subagent_spawn` reports success. Finalized user, | |
| 64 | 64 | assistant, tool-call, and tool-result messages are journaled; streamed deltas | |
| 65 | 65 | and hidden reasoning are not exposed as ordinary history. A tool call is stored | |
| 66 | - | before execution and reconciliation repairs incomplete call/result pairs. | |
| 66 | + | before execution and reconciliation repairs incomplete call/result pairs. A | |
| 67 | + | context-overflow recovery appends one `child.compacted` checkpoint carrying the | |
| 68 | + | replacement transcript; resume starts from the newest checkpoint, while the | |
| 69 | + | paged history view still shows everything appended before it. | |
| 67 | 70 | ||
| 68 | 71 | All children stop when the host exits. On the next start, queued or running | |
| 69 | 72 | turns are marked `interrupted`; zut never replays their task. Resume only with | |
| @@ -90,9 +93,10 @@ unlimited by default; a positive `subagents.queue_timeout` cancels an accepted | |||
| 90 | 93 | prompt that has not received a slot and records its terminal failure durably. | |
| 91 | 94 | `subagents.allowed_tools` is an allowlist for child-visible tools and | |
| 92 | 95 | `subagents.allowed_roots` limits eligible child workspaces. Resident children | |
| 93 | - | have no cumulative token budget. Execution stays bounded through a finite | |
| 94 | - | per-turn step default: a child whose parent resolution is unlimited runs with | |
| 95 | - | `MaxSteps=50`; an explicit parent limit is preserved. Removed legacy | |
| 96 | + | have no cumulative token budget and no child-only step cap: a child inherits | |
| 97 | + | the parent's resolved limit, including the unlimited default. A turn is bounded | |
| 98 | + | by the provider context window, with one compaction recovery, and by an | |
| 99 | + | explicit parent `--max-steps`. Removed legacy | |
| 96 | 100 | settings, including `tui_subagent_position` and `subagents.budget_ratio`, are | |
| 97 | 101 | ignored. Resident journals written by older versions are read with explicit | |
| 98 | 102 | translation (an old `budget_exhausted` outcome reconciles as failure) or, for | |
| @@ -109,21 +113,49 @@ no longer exposes `budget` or `budget_source` fields. | |||
| 109 | 113 | ||
| 110 | 114 | ### Context reminders | |
| 111 | 115 | ||
| 112 | - | When a child's last-turn prompt reaches 85% of its model context window, the | |
| 113 | - | runner injects one non-blocking reminder into the next provider request: | |
| 116 | + | When a child's last-turn prompt reaches 85%, 90%, or 95% of its model context | |
| 117 | + | window, the runner injects a non-blocking reminder into the next provider | |
| 118 | + | request: | |
| 114 | 119 | ||
| 115 | 120 | ```text | |
| 116 | 121 | Context usage is at NN% of the model window. Finish the current task, report | |
| 117 | 122 | results, limits, and remaining verifications, and do not start broad new work. | |
| 118 | 123 | ``` | |
| 119 | 124 | ||
| 120 | - | The reminder fires at most once per child runner and never blocks a turn. | |
| 121 | - | It travels as developer context in the live request; the durable transcript | |
| 122 | - | keeps only finalized history. Usage is the last-turn prompt size (input plus | |
| 123 | - | cache read/write tokens) against the resolved context window; an unknown | |
| 124 | - | window never triggers a reminder. The per-turn gauge resets when the host | |
| 125 | - | restarts, so a restarted child is reminded again only once fresh usage past | |
| 126 | - | the threshold arrives. | |
| 125 | + | Each band fires at most once per child runner and reminders never block a turn. | |
| 126 | + | A jump straight to a higher band delivers one reminder for that band, and usage | |
| 127 | + | that falls does not re-deliver a band. Reminders travel as developer context in | |
| 128 | + | the live request; the durable transcript keeps only finalized history. Usage is | |
| 129 | + | the last-turn prompt size (input plus cache read/write tokens) against the | |
| 130 | + | resolved context window; an unknown window never triggers a reminder. The | |
| 131 | + | per-turn gauge resets when the host restarts and when a recovery compacts the | |
| 132 | + | context, so neither event reports a stale percentage. The band ladder itself | |
| 133 | + | advances at most once per child runner: a restart starts a fresh ladder, while a | |
| 134 | + | compacted child is reminded again only when usage reaches a higher band. | |
| 135 | + | ||
| 136 | + | ### Context-overflow recovery | |
| 137 | + | ||
| 138 | + | A provider request that exceeds the model context window ends the turn. When | |
| 139 | + | the transcript holds at least four messages, the child recovers once per turn: | |
| 140 | + | it compacts the transcript, keeps the two most recent messages verbatim, | |
| 141 | + | journals the replacement as a `child.compacted` record, and continues the same | |
| 142 | + | turn. A compaction or persistence failure restores the pre-compaction | |
| 143 | + | transcript and fails the turn, and a second overflow stays terminal. Recovery | |
| 144 | + | never replenishes an explicit parent `--max-steps`: when the allowance is | |
| 145 | + | already spent, the checkpoint is still written for a later resume, but the turn | |
| 146 | + | is not continued. | |
| 147 | + | ||
| 148 | + | Explicit resume replays the newest checkpoint instead of the full append-only | |
| 149 | + | log. The history views (`/subagents logs`, `HistoryPage`, `subagent://` refs) | |
| 150 | + | still show the append-only log, including pre-compaction records. A build that | |
| 151 | + | predates checkpoint support cannot resume such a child correctly; downgrading | |
| 152 | + | is unsupported. | |
| 153 | + | ||
| 154 | + | The summarization request itself can be rejected when a single oversized | |
| 155 | + | message leaves nothing to summarize away. The turn then fails with the original | |
| 156 | + | provider error and no checkpoint is written, so the child keeps failing on | |
| 157 | + | resume until its transcript shrinks. Recovery is a best effort, not a | |
| 158 | + | guarantee. | |
| 127 | 159 | ||
| 128 | 160 | Retrieve the saved result without executing a model: | |
| 129 | 161 | ||
| @@ -52,7 +52,7 @@ The manager records each accepted child under its managed state root: | |||||
| 52 | 52 | ||||
| 53 | ```text | 53 | ```text | ||
| 54 | subagents/<child-id>/ | 54 | subagents/<child-id>/ | ||
| 55 | - | transcript.jsonl # authoritative accepted turns and finalized messages | 55 | + | transcript.jsonl # accepted turns, finalized messages, and compaction checkpoints |
| 56 | owner.lock # host-ownership coordination; do not modify or remove | 56 | owner.lock # host-ownership coordination; do not modify or remove | ||
| 57 | .transcript-backup-* # retained private pre-repair transcript, when repaired | 57 | .transcript-backup-* # retained private pre-repair transcript, when repaired | ||
| 58 | metadata.json # rebuildable bounded state | 58 | metadata.json # rebuildable bounded state | ||
| @@ -63,7 +63,10 @@ subagents/<child-id>/ | |||||
| 63 | Acceptance is durable before `subagent_spawn` reports success. Finalized user, | 63 | Acceptance is durable before `subagent_spawn` reports success. Finalized user, | ||
| 64 | assistant, tool-call, and tool-result messages are journaled; streamed deltas | 64 | assistant, tool-call, and tool-result messages are journaled; streamed deltas | ||
| 65 | and hidden reasoning are not exposed as ordinary history. A tool call is stored | 65 | and hidden reasoning are not exposed as ordinary history. A tool call is stored | ||
| 66 | - | before execution and reconciliation repairs incomplete call/result pairs. | 66 | + | before execution and reconciliation repairs incomplete call/result pairs. A |
| 67 | + | context-overflow recovery appends one `child.compacted` checkpoint carrying the | |||
| 68 | + | replacement transcript; resume starts from the newest checkpoint, while the | |||
| 69 | + | paged history view still shows everything appended before it. | |||
| 67 | 70 | ||||
| 68 | All children stop when the host exits. On the next start, queued or running | 71 | All children stop when the host exits. On the next start, queued or running | ||
| 69 | turns are marked `interrupted`; zut never replays their task. Resume only with | 72 | turns are marked `interrupted`; zut never replays their task. Resume only with | ||
| @@ -90,9 +93,10 @@ unlimited by default; a positive `subagents.queue_timeout` cancels an accepted | |||||
| 90 | prompt that has not received a slot and records its terminal failure durably. | 93 | prompt that has not received a slot and records its terminal failure durably. | ||
| 91 | `subagents.allowed_tools` is an allowlist for child-visible tools and | 94 | `subagents.allowed_tools` is an allowlist for child-visible tools and | ||
| 92 | `subagents.allowed_roots` limits eligible child workspaces. Resident children | 95 | `subagents.allowed_roots` limits eligible child workspaces. Resident children | ||
| 93 | - | have no cumulative token budget. Execution stays bounded through a finite | 96 | + | have no cumulative token budget and no child-only step cap: a child inherits |
| 94 | - | per-turn step default: a child whose parent resolution is unlimited runs with | 97 | + | the parent's resolved limit, including the unlimited default. A turn is bounded |
| 95 | - | `MaxSteps=50`; an explicit parent limit is preserved. Removed legacy | 98 | + | by the provider context window, with one compaction recovery, and by an |
| 99 | + | explicit parent `--max-steps`. Removed legacy | |||
| 96 | settings, including `tui_subagent_position` and `subagents.budget_ratio`, are | 100 | settings, including `tui_subagent_position` and `subagents.budget_ratio`, are | ||
| 97 | ignored. Resident journals written by older versions are read with explicit | 101 | ignored. Resident journals written by older versions are read with explicit | ||
| 98 | translation (an old `budget_exhausted` outcome reconciles as failure) or, for | 102 | translation (an old `budget_exhausted` outcome reconciles as failure) or, for | ||
| @@ -109,21 +113,49 @@ no longer exposes `budget` or `budget_source` fields. | |||||
| 109 | 113 | ||||
| 110 | ### Context reminders | 114 | ### Context reminders | ||
| 111 | 115 | ||||
| 112 | - | When a child's last-turn prompt reaches 85% of its model context window, the | 116 | + | When a child's last-turn prompt reaches 85%, 90%, or 95% of its model context |
| 113 | - | runner injects one non-blocking reminder into the next provider request: | 117 | + | window, the runner injects a non-blocking reminder into the next provider |
| 118 | + | request: | |||
| 114 | 119 | ||||
| 115 | ```text | 120 | ```text | ||
| 116 | Context usage is at NN% of the model window. Finish the current task, report | 121 | Context usage is at NN% of the model window. Finish the current task, report | ||
| 117 | results, limits, and remaining verifications, and do not start broad new work. | 122 | results, limits, and remaining verifications, and do not start broad new work. | ||
| 118 | ``` | 123 | ``` | ||
| 119 | 124 | ||||
| 120 | - | The reminder fires at most once per child runner and never blocks a turn. | 125 | + | Each band fires at most once per child runner and reminders never block a turn. |
| 121 | - | It travels as developer context in the live request; the durable transcript | 126 | + | A jump straight to a higher band delivers one reminder for that band, and usage |
| 122 | - | keeps only finalized history. Usage is the last-turn prompt size (input plus | 127 | + | that falls does not re-deliver a band. Reminders travel as developer context in |
| 123 | - | cache read/write tokens) against the resolved context window; an unknown | 128 | + | the live request; the durable transcript keeps only finalized history. Usage is |
| 124 | - | window never triggers a reminder. The per-turn gauge resets when the host | 129 | + | the last-turn prompt size (input plus cache read/write tokens) against the |
| 125 | - | restarts, so a restarted child is reminded again only once fresh usage past | 130 | + | resolved context window; an unknown window never triggers a reminder. The |
| 126 | - | the threshold arrives. | 131 | + | per-turn gauge resets when the host restarts and when a recovery compacts the |
| 132 | + | context, so neither event reports a stale percentage. The band ladder itself | |||
| 133 | + | advances at most once per child runner: a restart starts a fresh ladder, while a | |||
| 134 | + | compacted child is reminded again only when usage reaches a higher band. | |||
| 135 | + | ||||
| 136 | + | ### Context-overflow recovery | |||
| 137 | + | ||||
| 138 | + | A provider request that exceeds the model context window ends the turn. When | |||
| 139 | + | the transcript holds at least four messages, the child recovers once per turn: | |||
| 140 | + | it compacts the transcript, keeps the two most recent messages verbatim, | |||
| 141 | + | journals the replacement as a `child.compacted` record, and continues the same | |||
| 142 | + | turn. A compaction or persistence failure restores the pre-compaction | |||
| 143 | + | transcript and fails the turn, and a second overflow stays terminal. Recovery | |||
| 144 | + | never replenishes an explicit parent `--max-steps`: when the allowance is | |||
| 145 | + | already spent, the checkpoint is still written for a later resume, but the turn | |||
| 146 | + | is not continued. | |||
| 147 | + | ||||
| 148 | + | Explicit resume replays the newest checkpoint instead of the full append-only | |||
| 149 | + | log. The history views (`/subagents logs`, `HistoryPage`, `subagent://` refs) | |||
| 150 | + | still show the append-only log, including pre-compaction records. A build that | |||
| 151 | + | predates checkpoint support cannot resume such a child correctly; downgrading | |||
| 152 | + | is unsupported. | |||
| 153 | + | ||||
| 154 | + | The summarization request itself can be rejected when a single oversized | |||
| 155 | + | message leaves nothing to summarize away. The turn then fails with the original | |||
| 156 | + | provider error and no checkpoint is written, so the child keeps failing on | |||
| 157 | + | resume until its transcript shrinks. Recovery is a best effort, not a | |||
| 158 | + | guarantee. | |||
| 127 | 159 | ||||
| 128 | Retrieve the saved result without executing a model: | 160 | Retrieve the saved result without executing a model: | ||
| 129 | 161 | ||||
M packages/agent/subagent_factory.go +132 -26
| @@ -2,6 +2,7 @@ package agent | |||
| 2 | 2 | ||
| 3 | 3 | import ( | |
| 4 | 4 | "context" | |
| 5 | + | "errors" | |
| 5 | 6 | "fmt" | |
| 6 | 7 | "strings" | |
| 7 | 8 | "sync" | |
| @@ -9,6 +10,7 @@ import ( | |||
| 9 | 10 | "github.com/bnema/zut/packages/agent/subagents" | |
| 10 | 11 | "github.com/bnema/zut/packages/agent/tools" | |
| 11 | 12 | "github.com/bnema/zut/packages/core" | |
| 13 | + | "github.com/bnema/zut/packages/provider" | |
| 12 | 14 | ) | |
| 13 | 15 | ||
| 14 | 16 | // newResidentChildRunner is the host-owned construction boundary for one | |
| @@ -37,13 +39,11 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||
| 37 | 39 | } | |
| 38 | 40 | } | |
| 39 | 41 | agent := core.NewAgent(resolved.NewClient(), resolved.Model, system, registry) | |
| 42 | + | // A resident child inherits the parent's resolved step limit, including the | |
| 43 | + | // unlimited default. Its ceiling is the provider context window, with one | |
| 44 | + | // compaction recovery per accepted turn, plus any explicit parent | |
| 45 | + | // --max-steps. There is no child-only step budget. | |
| 40 | 46 | agent.MaxSteps = resolved.MaxSteps | |
| 41 | - | if agent.MaxSteps <= 0 { | |
| 42 | - | // Resident execution must stay bounded once the cumulative rollout | |
| 43 | - | // budget is gone. An unlimited parent default must not propagate to | |
| 44 | - | // children; explicit parent limits are preserved above. | |
| 45 | - | agent.MaxSteps = 50 | |
| 46 | - | } | |
| 47 | 47 | agent.ContextWindow = resolved.ContextWindow | |
| 48 | 48 | agent.MaxTokens = resolved.MaxOutput | |
| 49 | 49 | agent.Reasoning = resolved.Reasoning | |
| @@ -54,8 +54,8 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||
| 54 | 54 | baseline = journal.ConfigureUsage(resolved.ContextWindow, resolved.AuthMethod == "oauth") | |
| 55 | 55 | agent.SeedCost(baseline.Usage) | |
| 56 | 56 | } | |
| 57 | - | // The nudge reads the agent's own per-turn usage gauge. No cumulative | |
| 58 | - | // budget remains: execution stays bounded through the MaxSteps default. | |
| 57 | + | // The nudge reads the agent's own per-turn usage gauge. It stays advisory: | |
| 58 | + | // the provider context window is the hard ceiling on a child turn. | |
| 59 | 59 | configureResidentContextNudge(agent, resolved.ContextWindow) | |
| 60 | 60 | rootCacheID := strings.TrimSpace(spec.RootCacheID) | |
| 61 | 61 | if rootCacheID == "" { | |
| @@ -78,6 +78,7 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||
| 78 | 78 | agent.SetMessages(messages) | |
| 79 | 79 | } | |
| 80 | 80 | } | |
| 81 | + | limit := agent.MaxSteps | |
| 81 | 82 | return func(ctx context.Context, prompt string) error { | |
| 82 | 83 | if ctx == nil { | |
| 83 | 84 | ctx = context.Background() | |
| @@ -86,7 +87,23 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||
| 86 | 87 | defer cancel() | |
| 87 | 88 | var journalErr error | |
| 88 | 89 | var journalMu sync.Mutex | |
| 89 | - | err := agent.Prompt(turnCtx, prompt, nil, func(event core.AgentEvent) { | |
| 90 | + | checkJournal := func() error { | |
| 91 | + | journalMu.Lock() | |
| 92 | + | defer journalMu.Unlock() | |
| 93 | + | if journalErr == nil { | |
| 94 | + | return nil | |
| 95 | + | } | |
| 96 | + | return fmt.Errorf("persist resident child transcript: %w", journalErr) | |
| 97 | + | } | |
| 98 | + | // usedSteps is the step high-water mark of this accepted turn. The sink | |
| 99 | + | // runs on the runner goroutine, so it needs no lock; the mark bounds the | |
| 100 | + | // continuation after a compaction recovery so an explicit parent limit | |
| 101 | + | // is never replenished. | |
| 102 | + | usedSteps := 0 | |
| 103 | + | sink := func(event core.AgentEvent) { | |
| 104 | + | if turnStart, ok := event.(core.EvTurnStart); ok && turnStart.Step > usedSteps { | |
| 105 | + | usedSteps = turnStart.Step | |
| 106 | + | } | |
| 90 | 107 | if err := journal.RecordAgentEvent(event); err != nil { | |
| 91 | 108 | journalMu.Lock() | |
| 92 | 109 | if journalErr == nil { | |
| @@ -95,44 +112,133 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||
| 95 | 112 | } | |
| 96 | 113 | journalMu.Unlock() | |
| 97 | 114 | } | |
| 98 | - | }) | |
| 99 | - | journalMu.Lock() | |
| 100 | - | defer journalMu.Unlock() | |
| 101 | - | if journalErr != nil { | |
| 102 | - | return fmt.Errorf("persist resident child transcript: %w", journalErr) | |
| 115 | + | } | |
| 116 | + | err := agent.Prompt(turnCtx, prompt, nil, sink) | |
| 117 | + | if journalFailure := checkJournal(); journalFailure != nil { | |
| 118 | + | return journalFailure | |
| 119 | + | } | |
| 120 | + | if err != nil && turnCtx.Err() == nil && provider.IsContextOverflowError(err) { | |
| 121 | + | err = recoverResidentContextOverflow(turnCtx, agent, journal, sink, limit, usedSteps, err) | |
| 122 | + | if journalFailure := checkJournal(); journalFailure != nil { | |
| 123 | + | return journalFailure | |
| 124 | + | } | |
| 103 | 125 | } | |
| 104 | 126 | return err | |
| 105 | 127 | }, nil | |
| 106 | 128 | } | |
| 107 | 129 | ||
| 108 | - | // residentContextNudgePercent is the single context-usage threshold for the | |
| 109 | - | // resident reminder. It is a completion aid, not a cost guard: execution | |
| 110 | - | // stays bounded through the finite MaxSteps default. | |
| 111 | - | const residentContextNudgePercent = 85 | |
| 130 | + | // A compaction recovery never rewrites a transcript with nothing worth | |
| 131 | + | // summarizing: the child task and the current exchange must survive verbatim. | |
| 132 | + | const ( | |
| 133 | + | residentContextRecoveryMinimumMessages = 4 | |
| 134 | + | residentContextRecoveryKeepTail = 2 | |
| 135 | + | ) | |
| 136 | + | ||
| 137 | + | // recoverResidentContextOverflow gives one accepted resident turn a second | |
| 138 | + | // chance after the provider rejected it for exceeding the model context | |
| 139 | + | // window. It compacts the transcript, journals the replacement transcript, and | |
| 140 | + | // continues the same turn. Recovery happens at most once per call, and any | |
| 141 | + | // compaction or persistence failure restores the exact pre-compaction | |
| 142 | + | // transcript before returning the original error. | |
| 143 | + | func recoverResidentContextOverflow(ctx context.Context, agent *core.Agent, journal *subagents.ResidentJournal, sink func(core.AgentEvent), limit, usedSteps int, overflowErr error) error { | |
| 144 | + | if agent == nil || journal == nil { | |
| 145 | + | return overflowErr | |
| 146 | + | } | |
| 147 | + | before := agent.Messages() | |
| 148 | + | if len(before) < residentContextRecoveryMinimumMessages { | |
| 149 | + | // Summarizing a transcript this short would replace the instruction | |
| 150 | + | // rather than shrink prior work. | |
| 151 | + | return overflowErr | |
| 152 | + | } | |
| 153 | + | if _, err := agent.CompactWithEvents(ctx, residentContextRecoveryKeepTail, sink); err != nil { | |
| 154 | + | agent.SetMessages(before) | |
| 155 | + | return errors.Join(overflowErr, fmt.Errorf("compact resident child transcript: %w", err)) | |
| 156 | + | } | |
| 157 | + | if err := journal.RecordCompacted(residentCheckpointMessages(agent.Messages())); err != nil { | |
| 158 | + | // Memory must not run ahead of durable history: a later resume would | |
| 159 | + | // otherwise replay the pre-compaction transcript. | |
| 160 | + | agent.SetMessages(before) | |
| 161 | + | return errors.Join(overflowErr, fmt.Errorf("persist resident child compaction: %w", err)) | |
| 162 | + | } | |
| 163 | + | // The summarization request's usage describes the pre-compaction | |
| 164 | + | // transcript. It must not arm the next reminder, which would report a | |
| 165 | + | // false percentage for the compacted transcript the continuation sends. | |
| 166 | + | agent.SeedLastTurnUsage(provider.Usage{}) | |
| 167 | + | if limit > 0 { | |
| 168 | + | remaining := limit - usedSteps | |
| 169 | + | if remaining <= 0 { | |
| 170 | + | // An explicit parent limit is already spent. The checkpoint stays | |
| 171 | + | // durable so an explicit resume starts from the compacted | |
| 172 | + | // transcript, but this turn is not continued. | |
| 173 | + | return overflowErr | |
| 174 | + | } | |
| 175 | + | agent.MaxSteps = remaining | |
| 176 | + | defer func() { agent.MaxSteps = limit }() | |
| 177 | + | } | |
| 178 | + | return agent.Continue(ctx, sink) | |
| 179 | + | } | |
| 180 | + | ||
| 181 | + | // residentCheckpointMessages builds the durable replacement transcript of a | |
| 182 | + | // compaction. Host-authored internal context is dropped: it belongs to the | |
| 183 | + | // live request, and a replayed reminder would survive as a stale instruction | |
| 184 | + | // that the resumed child would still obey. | |
| 185 | + | func residentCheckpointMessages(messages []provider.Message) []provider.Message { | |
| 186 | + | checkpoint := make([]provider.Message, 0, len(messages)) | |
| 187 | + | for _, message := range messages { | |
| 188 | + | if core.IsInternalContextMessage(message) { | |
| 189 | + | continue | |
| 190 | + | } | |
| 191 | + | checkpoint = append(checkpoint, message) | |
| 192 | + | } | |
| 193 | + | return checkpoint | |
| 194 | + | } | |
| 195 | + | ||
| 196 | + | // residentContextReminderBands are the context-usage percentages that arm the | |
| 197 | + | // resident reminder. Each band fires at most once per child runner, so a child | |
| 198 | + | // whose prompt keeps growing toward the window is told to wrap up again | |
| 199 | + | // without turning the reminder into a budget. | |
| 200 | + | var residentContextReminderBands = [...]int{85, 90, 95} | |
| 112 | 201 | ||
| 113 | 202 | func configureResidentContextNudge(agent *core.Agent, contextMax int) { | |
| 114 | 203 | if agent == nil { | |
| 115 | 204 | return | |
| 116 | 205 | } | |
| 117 | - | // notified is owned by the runner closure and never reset from Resume | |
| 118 | - | // or journal paths. The injected developer-context message persists via | |
| 119 | - | // appendDynamicContext and is not replayed verbatim from the child | |
| 120 | - | // journal after restart; re-nudging after restart is acceptable. | |
| 121 | - | var notified bool | |
| 206 | + | // deliveredBand is owned by the runner closure and never reset from | |
| 207 | + | // Resume or journal paths. The injected developer-context message persists | |
| 208 | + | // via appendDynamicContext and is not replayed verbatim from the child | |
| 209 | + | // journal after restart; restarting a child rests the ladder. | |
| 210 | + | deliveredBand := -1 | |
| 122 | 211 | agent.BeforeTurnContext = func(_ context.Context, _ int) (bool, string, string) { | |
| 123 | - | if notified || contextMax <= 0 { | |
| 212 | + | if contextMax <= 0 { | |
| 124 | 213 | return true, "", "" | |
| 125 | 214 | } | |
| 126 | 215 | usage := agent.LastTurnUsage() | |
| 127 | 216 | used := usage.InputTokens + usage.CacheReadTokens + usage.CacheWriteTokens | |
| 128 | - | if used*100 < contextMax*residentContextNudgePercent { | |
| 217 | + | band := residentContextReminderBand(used, contextMax) | |
| 218 | + | if band <= deliveredBand { | |
| 129 | 219 | return true, "", "" | |
| 130 | 220 | } | |
| 131 | - | notified = true | |
| 221 | + | deliveredBand = band | |
| 132 | 222 | return true, "", fmt.Sprintf("Context usage is at %d%% of the model window. Finish the current task, report results, limits, and remaining verifications, and do not start broad new work.", used*100/contextMax) | |
| 133 | 223 | } | |
| 134 | 224 | } | |
| 135 | 225 | ||
| 226 | + | // residentContextReminderBand returns the highest armed band crossed by the | |
| 227 | + | // current prompt size, or -1 while usage stays below the first band. A direct | |
| 228 | + | // jump to a higher band delivers one reminder for that band. | |
| 229 | + | func residentContextReminderBand(used, contextMax int) int { | |
| 230 | + | if contextMax <= 0 { | |
| 231 | + | return -1 | |
| 232 | + | } | |
| 233 | + | band := -1 | |
| 234 | + | for _, candidate := range residentContextReminderBands { | |
| 235 | + | if used*100 >= contextMax*candidate { | |
| 236 | + | band = candidate | |
| 237 | + | } | |
| 238 | + | } | |
| 239 | + | return band | |
| 240 | + | } | |
| 241 | + | ||
| 136 | 242 | // residentChildArgs applies the complete durable child specification without | |
| 137 | 243 | // rediscovering a mutable profile file. That preserves the profile inheritance | |
| 138 | 244 | // decision accepted at spawn time across explicit resume and restart. | |
| @@ -2,6 +2,7 @@ package agent | |||||
| 2 | 2 | ||||
| 3 | import ( | 3 | import ( | ||
| 4 | "context" | 4 | "context" | ||
| 5 | + | "errors" | |||
| 5 | "fmt" | 6 | "fmt" | ||
| 6 | "strings" | 7 | "strings" | ||
| 7 | "sync" | 8 | "sync" | ||
| @@ -9,6 +10,7 @@ import ( | |||||
| 9 | "github.com/bnema/zut/packages/agent/subagents" | 10 | "github.com/bnema/zut/packages/agent/subagents" | ||
| 10 | "github.com/bnema/zut/packages/agent/tools" | 11 | "github.com/bnema/zut/packages/agent/tools" | ||
| 11 | "github.com/bnema/zut/packages/core" | 12 | "github.com/bnema/zut/packages/core" | ||
| 13 | + | "github.com/bnema/zut/packages/provider" | |||
| 12 | ) | 14 | ) | ||
| 13 | 15 | ||||
| 14 | // newResidentChildRunner is the host-owned construction boundary for one | 16 | // newResidentChildRunner is the host-owned construction boundary for one | ||
| @@ -37,13 +39,11 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||||
| 37 | } | 39 | } | ||
| 38 | } | 40 | } | ||
| 39 | agent := core.NewAgent(resolved.NewClient(), resolved.Model, system, registry) | 41 | agent := core.NewAgent(resolved.NewClient(), resolved.Model, system, registry) | ||
| 42 | + | // A resident child inherits the parent's resolved step limit, including the | |||
| 43 | + | // unlimited default. Its ceiling is the provider context window, with one | |||
| 44 | + | // compaction recovery per accepted turn, plus any explicit parent | |||
| 45 | + | // --max-steps. There is no child-only step budget. | |||
| 40 | agent.MaxSteps = resolved.MaxSteps | 46 | agent.MaxSteps = resolved.MaxSteps | ||
| 41 | - | if agent.MaxSteps <= 0 { | |||
| 42 | - | // Resident execution must stay bounded once the cumulative rollout | |||
| 43 | - | // budget is gone. An unlimited parent default must not propagate to | |||
| 44 | - | // children; explicit parent limits are preserved above. | |||
| 45 | - | agent.MaxSteps = 50 | |||
| 46 | - | } | |||
| 47 | agent.ContextWindow = resolved.ContextWindow | 47 | agent.ContextWindow = resolved.ContextWindow | ||
| 48 | agent.MaxTokens = resolved.MaxOutput | 48 | agent.MaxTokens = resolved.MaxOutput | ||
| 49 | agent.Reasoning = resolved.Reasoning | 49 | agent.Reasoning = resolved.Reasoning | ||
| @@ -54,8 +54,8 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||||
| 54 | baseline = journal.ConfigureUsage(resolved.ContextWindow, resolved.AuthMethod == "oauth") | 54 | baseline = journal.ConfigureUsage(resolved.ContextWindow, resolved.AuthMethod == "oauth") | ||
| 55 | agent.SeedCost(baseline.Usage) | 55 | agent.SeedCost(baseline.Usage) | ||
| 56 | } | 56 | } | ||
| 57 | - | // The nudge reads the agent's own per-turn usage gauge. No cumulative | 57 | + | // The nudge reads the agent's own per-turn usage gauge. It stays advisory: |
| 58 | - | // budget remains: execution stays bounded through the MaxSteps default. | 58 | + | // the provider context window is the hard ceiling on a child turn. |
| 59 | configureResidentContextNudge(agent, resolved.ContextWindow) | 59 | configureResidentContextNudge(agent, resolved.ContextWindow) | ||
| 60 | rootCacheID := strings.TrimSpace(spec.RootCacheID) | 60 | rootCacheID := strings.TrimSpace(spec.RootCacheID) | ||
| 61 | if rootCacheID == "" { | 61 | if rootCacheID == "" { | ||
| @@ -78,6 +78,7 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||||
| 78 | agent.SetMessages(messages) | 78 | agent.SetMessages(messages) | ||
| 79 | } | 79 | } | ||
| 80 | } | 80 | } | ||
| 81 | + | limit := agent.MaxSteps | |||
| 81 | return func(ctx context.Context, prompt string) error { | 82 | return func(ctx context.Context, prompt string) error { | ||
| 82 | if ctx == nil { | 83 | if ctx == nil { | ||
| 83 | ctx = context.Background() | 84 | ctx = context.Background() | ||
| @@ -86,7 +87,23 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||||
| 86 | defer cancel() | 87 | defer cancel() | ||
| 87 | var journalErr error | 88 | var journalErr error | ||
| 88 | var journalMu sync.Mutex | 89 | var journalMu sync.Mutex | ||
| 89 | - | err := agent.Prompt(turnCtx, prompt, nil, func(event core.AgentEvent) { | 90 | + | checkJournal := func() error { |
| 91 | + | journalMu.Lock() | |||
| 92 | + | defer journalMu.Unlock() | |||
| 93 | + | if journalErr == nil { | |||
| 94 | + | return nil | |||
| 95 | + | } | |||
| 96 | + | return fmt.Errorf("persist resident child transcript: %w", journalErr) | |||
| 97 | + | } | |||
| 98 | + | // usedSteps is the step high-water mark of this accepted turn. The sink | |||
| 99 | + | // runs on the runner goroutine, so it needs no lock; the mark bounds the | |||
| 100 | + | // continuation after a compaction recovery so an explicit parent limit | |||
| 101 | + | // is never replenished. | |||
| 102 | + | usedSteps := 0 | |||
| 103 | + | sink := func(event core.AgentEvent) { | |||
| 104 | + | if turnStart, ok := event.(core.EvTurnStart); ok && turnStart.Step > usedSteps { | |||
| 105 | + | usedSteps = turnStart.Step | |||
| 106 | + | } | |||
| 90 | if err := journal.RecordAgentEvent(event); err != nil { | 107 | if err := journal.RecordAgentEvent(event); err != nil { | ||
| 91 | journalMu.Lock() | 108 | journalMu.Lock() | ||
| 92 | if journalErr == nil { | 109 | if journalErr == nil { | ||
| @@ -95,44 +112,133 @@ func newResidentChildRunner(args Args, spec subagents.ResidentChildSpec, journal | |||||
| 95 | } | 112 | } | ||
| 96 | journalMu.Unlock() | 113 | journalMu.Unlock() | ||
| 97 | } | 114 | } | ||
| 98 | - | }) | 115 | + | } |
| 99 | - | journalMu.Lock() | 116 | + | err := agent.Prompt(turnCtx, prompt, nil, sink) |
| 100 | - | defer journalMu.Unlock() | 117 | + | if journalFailure := checkJournal(); journalFailure != nil { |
| 101 | - | if journalErr != nil { | 118 | + | return journalFailure |
| 102 | - | return fmt.Errorf("persist resident child transcript: %w", journalErr) | 119 | + | } |
| 120 | + | if err != nil && turnCtx.Err() == nil && provider.IsContextOverflowError(err) { | |||
| 121 | + | err = recoverResidentContextOverflow(turnCtx, agent, journal, sink, limit, usedSteps, err) | |||
| 122 | + | if journalFailure := checkJournal(); journalFailure != nil { | |||
| 123 | + | return journalFailure | |||
| 124 | + | } | |||
| 103 | } | 125 | } | ||
| 104 | return err | 126 | return err | ||
| 105 | }, nil | 127 | }, nil | ||
| 106 | } | 128 | } | ||
| 107 | 129 | ||||
| 108 | - | // residentContextNudgePercent is the single context-usage threshold for the | 130 | + | // A compaction recovery never rewrites a transcript with nothing worth |
| 109 | - | // resident reminder. It is a completion aid, not a cost guard: execution | 131 | + | // summarizing: the child task and the current exchange must survive verbatim. |
| 110 | - | // stays bounded through the finite MaxSteps default. | 132 | + | const ( |
| 111 | - | const residentContextNudgePercent = 85 | 133 | + | residentContextRecoveryMinimumMessages = 4 |
| 134 | + | residentContextRecoveryKeepTail = 2 | |||
| 135 | + | ) | |||
| 136 | + | ||||
| 137 | + | // recoverResidentContextOverflow gives one accepted resident turn a second | |||
| 138 | + | // chance after the provider rejected it for exceeding the model context | |||
| 139 | + | // window. It compacts the transcript, journals the replacement transcript, and | |||
| 140 | + | // continues the same turn. Recovery happens at most once per call, and any | |||
| 141 | + | // compaction or persistence failure restores the exact pre-compaction | |||
| 142 | + | // transcript before returning the original error. | |||
| 143 | + | func recoverResidentContextOverflow(ctx context.Context, agent *core.Agent, journal *subagents.ResidentJournal, sink func(core.AgentEvent), limit, usedSteps int, overflowErr error) error { | |||
| 144 | + | if agent == nil || journal == nil { | |||
| 145 | + | return overflowErr | |||
| 146 | + | } | |||
| 147 | + | before := agent.Messages() | |||
| 148 | + | if len(before) < residentContextRecoveryMinimumMessages { | |||
| 149 | + | // Summarizing a transcript this short would replace the instruction | |||
| 150 | + | // rather than shrink prior work. | |||
| 151 | + | return overflowErr | |||
| 152 | + | } | |||
| 153 | + | if _, err := agent.CompactWithEvents(ctx, residentContextRecoveryKeepTail, sink); err != nil { | |||
| 154 | + | agent.SetMessages(before) | |||
| 155 | + | return errors.Join(overflowErr, fmt.Errorf("compact resident child transcript: %w", err)) | |||
| 156 | + | } | |||
| 157 | + | if err := journal.RecordCompacted(residentCheckpointMessages(agent.Messages())); err != nil { | |||
| 158 | + | // Memory must not run ahead of durable history: a later resume would | |||
| 159 | + | // otherwise replay the pre-compaction transcript. | |||
| 160 | + | agent.SetMessages(before) | |||
| 161 | + | return errors.Join(overflowErr, fmt.Errorf("persist resident child compaction: %w", err)) | |||
| 162 | + | } | |||
| 163 | + | // The summarization request's usage describes the pre-compaction | |||
| 164 | + | // transcript. It must not arm the next reminder, which would report a | |||
| 165 | + | // false percentage for the compacted transcript the continuation sends. | |||
| 166 | + | agent.SeedLastTurnUsage(provider.Usage{}) | |||
| 167 | + | if limit > 0 { | |||
| 168 | + | remaining := limit - usedSteps | |||
| 169 | + | if remaining <= 0 { | |||
| 170 | + | // An explicit parent limit is already spent. The checkpoint stays | |||
| 171 | + | // durable so an explicit resume starts from the compacted | |||
| 172 | + | // transcript, but this turn is not continued. | |||
| 173 | + | return overflowErr | |||
| 174 | + | } | |||
| 175 | + | agent.MaxSteps = remaining | |||
| 176 | + | defer func() { agent.MaxSteps = limit }() | |||
| 177 | + | } | |||
| 178 | + | return agent.Continue(ctx, sink) | |||
| 179 | + | } | |||
| 180 | + | ||||
| 181 | + | // residentCheckpointMessages builds the durable replacement transcript of a | |||
| 182 | + | // compaction. Host-authored internal context is dropped: it belongs to the | |||
| 183 | + | // live request, and a replayed reminder would survive as a stale instruction | |||
| 184 | + | // that the resumed child would still obey. | |||
| 185 | + | func residentCheckpointMessages(messages []provider.Message) []provider.Message { | |||
| 186 | + | checkpoint := make([]provider.Message, 0, len(messages)) | |||
| 187 | + | for _, message := range messages { | |||
| 188 | + | if core.IsInternalContextMessage(message) { | |||
| 189 | + | continue | |||
| 190 | + | } | |||
| 191 | + | checkpoint = append(checkpoint, message) | |||
| 192 | + | } | |||
| 193 | + | return checkpoint | |||
| 194 | + | } | |||
| 195 | + | ||||
| 196 | + | // residentContextReminderBands are the context-usage percentages that arm the | |||
| 197 | + | // resident reminder. Each band fires at most once per child runner, so a child | |||
| 198 | + | // whose prompt keeps growing toward the window is told to wrap up again | |||
| 199 | + | // without turning the reminder into a budget. | |||
| 200 | + | var residentContextReminderBands = [...]int{85, 90, 95} | |||
| 112 | 201 | ||||
| 113 | func configureResidentContextNudge(agent *core.Agent, contextMax int) { | 202 | func configureResidentContextNudge(agent *core.Agent, contextMax int) { | ||
| 114 | if agent == nil { | 203 | if agent == nil { | ||
| 115 | return | 204 | return | ||
| 116 | } | 205 | } | ||
| 117 | - | // notified is owned by the runner closure and never reset from Resume | 206 | + | // deliveredBand is owned by the runner closure and never reset from |
| 118 | - | // or journal paths. The injected developer-context message persists via | 207 | + | // Resume or journal paths. The injected developer-context message persists |
| 119 | - | // appendDynamicContext and is not replayed verbatim from the child | 208 | + | // via appendDynamicContext and is not replayed verbatim from the child |
| 120 | - | // journal after restart; re-nudging after restart is acceptable. | 209 | + | // journal after restart; restarting a child rests the ladder. |
| 121 | - | var notified bool | 210 | + | deliveredBand := -1 |
| 122 | agent.BeforeTurnContext = func(_ context.Context, _ int) (bool, string, string) { | 211 | agent.BeforeTurnContext = func(_ context.Context, _ int) (bool, string, string) { | ||
| 123 | - | if notified || contextMax <= 0 { | 212 | + | if contextMax <= 0 { |
| 124 | return true, "", "" | 213 | return true, "", "" | ||
| 125 | } | 214 | } | ||
| 126 | usage := agent.LastTurnUsage() | 215 | usage := agent.LastTurnUsage() | ||
| 127 | used := usage.InputTokens + usage.CacheReadTokens + usage.CacheWriteTokens | 216 | used := usage.InputTokens + usage.CacheReadTokens + usage.CacheWriteTokens | ||
| 128 | - | if used*100 < contextMax*residentContextNudgePercent { | 217 | + | band := residentContextReminderBand(used, contextMax) |
| 218 | + | if band <= deliveredBand { | |||
| 129 | return true, "", "" | 219 | return true, "", "" | ||
| 130 | } | 220 | } | ||
| 131 | - | notified = true | 221 | + | deliveredBand = band |
| 132 | return true, "", fmt.Sprintf("Context usage is at %d%% of the model window. Finish the current task, report results, limits, and remaining verifications, and do not start broad new work.", used*100/contextMax) | 222 | return true, "", fmt.Sprintf("Context usage is at %d%% of the model window. Finish the current task, report results, limits, and remaining verifications, and do not start broad new work.", used*100/contextMax) | ||
| 133 | } | 223 | } | ||
| 134 | } | 224 | } | ||
| 135 | 225 | ||||
| 226 | + | // residentContextReminderBand returns the highest armed band crossed by the | |||
| 227 | + | // current prompt size, or -1 while usage stays below the first band. A direct | |||
| 228 | + | // jump to a higher band delivers one reminder for that band. | |||
| 229 | + | func residentContextReminderBand(used, contextMax int) int { | |||
| 230 | + | if contextMax <= 0 { | |||
| 231 | + | return -1 | |||
| 232 | + | } | |||
| 233 | + | band := -1 | |||
| 234 | + | for _, candidate := range residentContextReminderBands { | |||
| 235 | + | if used*100 >= contextMax*candidate { | |||
| 236 | + | band = candidate | |||
| 237 | + | } | |||
| 238 | + | } | |||
| 239 | + | return band | |||
| 240 | + | } | |||
| 241 | + | ||||
| 136 | // residentChildArgs applies the complete durable child specification without | 242 | // residentChildArgs applies the complete durable child specification without | ||
| 137 | // rediscovering a mutable profile file. That preserves the profile inheritance | 243 | // rediscovering a mutable profile file. That preserves the profile inheritance | ||
| 138 | // decision accepted at spawn time across explicit resume and restart. | 244 | // decision accepted at spawn time across explicit resume and restart. | ||
M packages/agent/subagent_factory_test.go +596 -79
| @@ -7,6 +7,9 @@ import ( | |||
| 7 | 7 | "fmt" | |
| 8 | 8 | "net/http" | |
| 9 | 9 | "net/http/httptest" | |
| 10 | + | "os" | |
| 11 | + | "path/filepath" | |
| 12 | + | "reflect" | |
| 10 | 13 | "strings" | |
| 11 | 14 | "sync" | |
| 12 | 15 | "sync/atomic" | |
| @@ -18,65 +21,87 @@ import ( | |||
| 18 | 21 | "github.com/bnema/zut/packages/provider" | |
| 19 | 22 | ) | |
| 20 | 23 | ||
| 21 | - | func TestResidentChildRunnerDefaultsToFiniteMaxSteps(t *testing.T) { | |
| 22 | - | for _, tc := range []struct { | |
| 23 | - | name string | |
| 24 | - | maxSteps int | |
| 25 | - | want int | |
| 26 | - | }{ | |
| 27 | - | {name: "unlimited parent defaults to bounded child", maxSteps: 0, want: 50}, | |
| 28 | - | {name: "explicit parent limit preserved", maxSteps: 30, want: 30}, | |
| 29 | - | } { | |
| 30 | - | t.Run(tc.name, func(t *testing.T) { | |
| 31 | - | var requests atomic.Int32 | |
| 32 | - | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | |
| 33 | - | requests.Add(1) | |
| 34 | - | w.Header().Set("content-type", "text/event-stream") | |
| 35 | - | writeOpenAIChunk(t, w, map[string]any{ | |
| 36 | - | "choices": []any{map[string]any{ | |
| 37 | - | "index": 0, | |
| 38 | - | "delta": map[string]any{"tool_calls": []any{map[string]any{ | |
| 39 | - | "index": 0, "id": "call-loop", "type": "function", | |
| 40 | - | "function": map[string]string{"name": "read", "arguments": `{"path":"x"}`}, | |
| 41 | - | }}}, | |
| 42 | - | "finish_reason": "tool_calls", | |
| 43 | - | }}, | |
| 44 | - | "usage": map[string]int{"prompt_tokens": 10, "completion_tokens": 1}, | |
| 45 | - | }) | |
| 46 | - | })) | |
| 47 | - | defer server.Close() | |
| 48 | - | ||
| 49 | - | spec := subagents.ResidentChildSpec{ | |
| 50 | - | ID: "maxsteps-child", SessionID: "maxsteps-session", Provider: "openai", Model: "gpt-4o", | |
| 51 | - | Tools: []string{"read"}, | |
| 52 | - | } | |
| 53 | - | journal, err := subagents.OpenResidentJournal(t.TempDir(), spec.ID) | |
| 54 | - | if err != nil { | |
| 55 | - | t.Fatal(err) | |
| 56 | - | } | |
| 57 | - | t.Cleanup(func() { _ = journal.Close() }) | |
| 58 | - | if err := journal.Accept(spec, "review"); err != nil { | |
| 59 | - | t.Fatal(err) | |
| 60 | - | } | |
| 61 | - | runner, err := newResidentChildRunner(Args{ | |
| 62 | - | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: server.URL, | |
| 63 | - | MaxSteps: tc.maxSteps, CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, | |
| 64 | - | }, spec, journal) | |
| 65 | - | if err != nil { | |
| 66 | - | t.Fatal(err) | |
| 67 | - | } | |
| 68 | - | err = runner(t.Context(), "review") | |
| 69 | - | if !errors.Is(err, core.ErrMaxSteps) { | |
| 70 | - | t.Fatalf("runner error = %v, want step-limit stop", err) | |
| 71 | - | } | |
| 72 | - | if !strings.Contains(err.Error(), fmt.Sprintf("%d", tc.want)) { | |
| 73 | - | t.Fatalf("runner error = %v, want limit %d", err, tc.want) | |
| 74 | - | } | |
| 75 | - | if got := requests.Load(); got != int32(tc.want) { | |
| 76 | - | t.Fatalf("provider requests = %d, want %d", got, tc.want) | |
| 77 | - | } | |
| 24 | + | func TestResidentChildRunnerWithoutStepLimitRunsPastFormerCap(t *testing.T) { | |
| 25 | + | server, requests := residentToolLoopServer(t, 51) | |
| 26 | + | runner, _, _ := newResidentChildTestRunner(t, "unlimited-child", Args{ | |
| 27 | + | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: server.URL, | |
| 28 | + | CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, | |
| 29 | + | }) | |
| 30 | + | if err := runner(t.Context(), "review"); err != nil { | |
| 31 | + | t.Fatalf("runner error = %v, want success past the former 50-step cap", err) | |
| 32 | + | } | |
| 33 | + | if got := requests.Load(); got != 52 { | |
| 34 | + | t.Fatalf("provider requests = %d, want 51 tool turns and one answer", got) | |
| 35 | + | } | |
| 36 | + | } | |
| 37 | + | ||
| 38 | + | func TestResidentChildRunnerEnforcesExplicitParentStepLimit(t *testing.T) { | |
| 39 | + | server, requests := residentToolLoopServer(t, 200) | |
| 40 | + | runner, _, _ := newResidentChildTestRunner(t, "limited-child", Args{ | |
| 41 | + | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: server.URL, | |
| 42 | + | MaxSteps: 3, CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, | |
| 43 | + | }) | |
| 44 | + | err := runner(t.Context(), "review") | |
| 45 | + | if !errors.Is(err, core.ErrMaxSteps) || !strings.Contains(err.Error(), "3") { | |
| 46 | + | t.Fatalf("runner error = %v, want step limit 3", err) | |
| 47 | + | } | |
| 48 | + | if got := requests.Load(); got != 3 { | |
| 49 | + | t.Fatalf("provider requests = %d, want 3", got) | |
| 50 | + | } | |
| 51 | + | } | |
| 52 | + | ||
| 53 | + | // residentToolLoopServer answers the first toolCalls requests with a distinct | |
| 54 | + | // read call and then returns a substantive final answer. | |
| 55 | + | func residentToolLoopServer(t *testing.T, toolCalls int) (*httptest.Server, *atomic.Int32) { | |
| 56 | + | t.Helper() | |
| 57 | + | var requests atomic.Int32 | |
| 58 | + | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | |
| 59 | + | number := requests.Add(1) | |
| 60 | + | w.Header().Set("content-type", "text/event-stream") | |
| 61 | + | if int(number) <= toolCalls { | |
| 62 | + | writeOpenAIChunk(t, w, map[string]any{ | |
| 63 | + | "choices": []any{map[string]any{ | |
| 64 | + | "index": 0, | |
| 65 | + | "delta": map[string]any{"tool_calls": []any{map[string]any{ | |
| 66 | + | "index": 0, "id": fmt.Sprintf("call-%d", number), "type": "function", | |
| 67 | + | "function": map[string]string{"name": "read", "arguments": `{"path":"x"}`}, | |
| 68 | + | }}}, | |
| 69 | + | "finish_reason": "tool_calls", | |
| 70 | + | }}, | |
| 71 | + | "usage": map[string]int{"prompt_tokens": 10, "completion_tokens": 1}, | |
| 72 | + | }) | |
| 73 | + | return | |
| 74 | + | } | |
| 75 | + | writeOpenAIChunk(t, w, map[string]any{ | |
| 76 | + | "choices": []any{map[string]any{"index": 0, "delta": map[string]string{"content": "final result"}, "finish_reason": "stop"}}, | |
| 77 | + | "usage": map[string]int{"prompt_tokens": 10, "completion_tokens": 1}, | |
| 78 | 78 | }) | |
| 79 | + | })) | |
| 80 | + | t.Cleanup(server.Close) | |
| 81 | + | return server, &requests | |
| 82 | + | } | |
| 83 | + | ||
| 84 | + | // newResidentChildTestRunner builds one journaled resident child runner with | |
| 85 | + | // its own accepted spec. | |
| 86 | + | func newResidentChildTestRunner(t *testing.T, childID string, args Args) (subagents.ResidentTurnRunner, *subagents.ResidentJournal, subagents.ResidentChildSpec) { | |
| 87 | + | t.Helper() | |
| 88 | + | spec := subagents.ResidentChildSpec{ | |
| 89 | + | ID: childID, SessionID: childID + "-session", Provider: "openai", Model: "gpt-4o", | |
| 90 | + | Tools: []string{"read"}, | |
| 91 | + | } | |
| 92 | + | journal, err := subagents.OpenResidentJournal(t.TempDir(), spec.ID) | |
| 93 | + | if err != nil { | |
| 94 | + | t.Fatal(err) | |
| 79 | 95 | } | |
| 96 | + | t.Cleanup(func() { _ = journal.Close() }) | |
| 97 | + | if err := journal.Accept(spec, "review"); err != nil { | |
| 98 | + | t.Fatal(err) | |
| 99 | + | } | |
| 100 | + | runner, err := newResidentChildRunner(args, spec, journal) | |
| 101 | + | if err != nil { | |
| 102 | + | t.Fatal(err) | |
| 103 | + | } | |
| 104 | + | return runner, journal, spec | |
| 80 | 105 | } | |
| 81 | 106 | ||
| 82 | 107 | func TestResidentChildRunnerInjectsContextReminderIntoNextRequest(t *testing.T) { | |
| @@ -202,7 +227,7 @@ func TestResidentChildRegistryUsesExactToolListAndForbidsDelegation(t *testing.T | |||
| 202 | 227 | } | |
| 203 | 228 | } | |
| 204 | 229 | ||
| 205 | - | func TestConfigureResidentContextNudgeRemindsOnceAtThreshold(t *testing.T) { | |
| 230 | + | func TestConfigureResidentContextNudgeRemindsOncePerBand(t *testing.T) { | |
| 206 | 231 | agent := core.NewAgent(nil, "model", "system", core.Registry{"read": nil}) | |
| 207 | 232 | configureResidentContextNudge(agent, 1_000) | |
| 208 | 233 | ||
| @@ -222,16 +247,52 @@ func TestConfigureResidentContextNudgeRemindsOnceAtThreshold(t *testing.T) { | |||
| 222 | 247 | return contextText | |
| 223 | 248 | } | |
| 224 | 249 | ||
| 225 | - | // Below the threshold: silent, including cache-heavy usage. | |
| 250 | + | // Below the first band: silent, including cache-heavy usage. | |
| 226 | 251 | check(provider.Usage{InputTokens: 849}, false) | |
| 227 | 252 | check(provider.Usage{InputTokens: 400, CacheReadTokens: 400, CacheWriteTokens: 49}, false) | |
| 228 | - | // First crossing: exactly one reminder carrying the computed percent. | |
| 253 | + | // Each band fires once, carrying the computed percent of the crossing turn. | |
| 229 | 254 | first := check(provider.Usage{InputTokens: 850}, true) | |
| 230 | 255 | if !strings.Contains(first, "85%") { | |
| 231 | 256 | t.Fatalf("reminder percent = %q, want 85%%", first) | |
| 232 | 257 | } | |
| 233 | - | // Later turns stay silent even as usage grows. | |
| 234 | - | check(provider.Usage{InputTokens: 970}, false) | |
| 258 | + | check(provider.Usage{InputTokens: 860}, false) | |
| 259 | + | second := check(provider.Usage{InputTokens: 900}, true) | |
| 260 | + | if !strings.Contains(second, "90%") { | |
| 261 | + | t.Fatalf("reminder percent = %q, want 90%%", second) | |
| 262 | + | } | |
| 263 | + | check(provider.Usage{InputTokens: 920}, false) | |
| 264 | + | third := check(provider.Usage{InputTokens: 950}, true) | |
| 265 | + | if !strings.Contains(third, "95%") { | |
| 266 | + | t.Fatalf("reminder percent = %q, want 95%%", third) | |
| 267 | + | } | |
| 268 | + | // Past the last band the ladder is spent. | |
| 269 | + | check(provider.Usage{InputTokens: 999}, false) | |
| 270 | + | } | |
| 271 | + | ||
| 272 | + | func TestConfigureResidentContextNudgeIgnoresShrinkingUsage(t *testing.T) { | |
| 273 | + | agent := core.NewAgent(nil, "model", "system", core.Registry{"read": nil}) | |
| 274 | + | configureResidentContextNudge(agent, 1_000) | |
| 275 | + | ||
| 276 | + | check := func(usage provider.Usage, wantReminder bool) string { | |
| 277 | + | t.Helper() | |
| 278 | + | agent.SeedLastTurnUsage(usage) | |
| 279 | + | allowed, reason, contextText := agent.BeforeTurnContext(t.Context(), 1) | |
| 280 | + | if !allowed || reason != "" { | |
| 281 | + | t.Fatalf("nudge blocked turn: allowed=%t reason=%q", allowed, reason) | |
| 282 | + | } | |
| 283 | + | if wantReminder != (contextText != "") { | |
| 284 | + | t.Fatalf("reminder for usage %#v = %q, want reminder=%t", usage, contextText, wantReminder) | |
| 285 | + | } | |
| 286 | + | return contextText | |
| 287 | + | } | |
| 288 | + | ||
| 289 | + | // A shrinking prompt never delivers a band it did not cross. | |
| 290 | + | check(provider.Usage{InputTokens: 700}, false) | |
| 291 | + | check(provider.Usage{InputTokens: 500}, false) | |
| 292 | + | jumped := check(provider.Usage{InputTokens: 910}, true) | |
| 293 | + | if !strings.Contains(jumped, "91%") { | |
| 294 | + | t.Fatalf("reminder percent = %q, want 91%%", jumped) | |
| 295 | + | } | |
| 235 | 296 | } | |
| 236 | 297 | ||
| 237 | 298 | func TestConfigureResidentContextNudgeHandlesDirectJumpAndUnknownWindow(t *testing.T) { | |
| @@ -400,25 +461,10 @@ func TestResidentChildRunnerResumesWithRetainedHistory(t *testing.T) { | |||
| 400 | 461 | ||
| 401 | 462 | func newResidentTestRunner(t *testing.T, baseURL string) (subagents.ResidentTurnRunner, *subagents.ResidentJournal) { | |
| 402 | 463 | t.Helper() | |
| 403 | - | spec := subagents.ResidentChildSpec{ | |
| 404 | - | ID: "resident-child", SessionID: "resident-session", Provider: "openai", Model: "gpt-4o", | |
| 405 | - | Tools: []string{"read"}, | |
| 406 | - | } | |
| 407 | - | journal, err := subagents.OpenResidentJournal(t.TempDir(), spec.ID) | |
| 408 | - | if err != nil { | |
| 409 | - | t.Fatal(err) | |
| 410 | - | } | |
| 411 | - | t.Cleanup(func() { _ = journal.Close() }) | |
| 412 | - | if err := journal.Accept(spec, "review"); err != nil { | |
| 413 | - | t.Fatal(err) | |
| 414 | - | } | |
| 415 | - | runner, err := newResidentChildRunner(Args{ | |
| 464 | + | runner, journal, _ := newResidentChildTestRunner(t, "resident-child", Args{ | |
| 416 | 465 | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: baseURL, | |
| 417 | 466 | CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, | |
| 418 | - | }, spec, journal) | |
| 419 | - | if err != nil { | |
| 420 | - | t.Fatal(err) | |
| 421 | - | } | |
| 467 | + | }) | |
| 422 | 468 | return runner, journal | |
| 423 | 469 | } | |
| 424 | 470 | ||
| @@ -484,3 +530,474 @@ func TestResidentChildArgsDoNotForwardCLIKeyAcrossProviders(t *testing.T) { | |||
| 484 | 530 | t.Fatalf("same-provider child API key = %q, want parent key", next.APIKey) | |
| 485 | 531 | } | |
| 486 | 532 | } | |
| 533 | + | ||
| 534 | + | // residentOverflowScript scripts the provider responses of a child that hits | |
| 535 | + | // the context window: leading tool-call turns, one overflow, one compaction | |
| 536 | + | // summary, then continued tool turns or the final text. | |
| 537 | + | type residentOverflowScript struct { | |
| 538 | + | toolCallRequests int | |
| 539 | + | continuationToolCalls int | |
| 540 | + | secondOverflow bool | |
| 541 | + | finalText string | |
| 542 | + | onOverflow func() | |
| 543 | + | // toolUsageTokens is the prompt size reported by tool-call responses; zero | |
| 544 | + | // reports a small prompt. | |
| 545 | + | toolUsageTokens int | |
| 546 | + | // summaryUsageTokens, when positive, is the prompt size reported by the | |
| 547 | + | // compaction summary response. | |
| 548 | + | summaryUsageTokens int | |
| 549 | + | // continuationUsageTokens is the prompt size reported by tool turns after a | |
| 550 | + | // successful compaction; zero reuses toolUsageTokens. | |
| 551 | + | continuationUsageTokens int | |
| 552 | + | ||
| 553 | + | requests atomic.Int32 | |
| 554 | + | mu sync.Mutex | |
| 555 | + | bodies []string | |
| 556 | + | } | |
| 557 | + | ||
| 558 | + | func (s *residentOverflowScript) start(t *testing.T) *httptest.Server { | |
| 559 | + | t.Helper() | |
| 560 | + | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | |
| 561 | + | number := int(s.requests.Add(1)) | |
| 562 | + | var body map[string]any | |
| 563 | + | if err := json.NewDecoder(r.Body).Decode(&body); err != nil { | |
| 564 | + | t.Errorf("decode request: %v", err) | |
| 565 | + | } | |
| 566 | + | encoded, _ := json.Marshal(body["messages"]) | |
| 567 | + | s.mu.Lock() | |
| 568 | + | s.bodies = append(s.bodies, string(encoded)) | |
| 569 | + | s.mu.Unlock() | |
| 570 | + | ||
| 571 | + | overflowAt := s.toolCallRequests + 1 | |
| 572 | + | switch { | |
| 573 | + | case number <= s.toolCallRequests: | |
| 574 | + | w.Header().Set("content-type", "text/event-stream") | |
| 575 | + | writeOpenAIChunk(t, w, residentToolCallChunk(number, s.toolUsageTokens)) | |
| 576 | + | case number == overflowAt: | |
| 577 | + | if s.onOverflow != nil { | |
| 578 | + | s.onOverflow() | |
| 579 | + | } | |
| 580 | + | w.WriteHeader(http.StatusRequestEntityTooLarge) | |
| 581 | + | _, _ = w.Write([]byte(`{"error":{"message":"input exceeds the context window","code":"context_length_exceeded"}}`)) | |
| 582 | + | case number == overflowAt+1: | |
| 583 | + | w.Header().Set("content-type", "text/event-stream") | |
| 584 | + | writeOpenAIChunk(t, w, residentSummaryChunk(s.summaryUsageTokens)) | |
| 585 | + | case s.secondOverflow: | |
| 586 | + | w.WriteHeader(http.StatusRequestEntityTooLarge) | |
| 587 | + | _, _ = w.Write([]byte(`{"error":{"message":"input exceeds the context window","code":"context_length_exceeded"}}`)) | |
| 588 | + | case number < overflowAt+2+s.continuationToolCalls: | |
| 589 | + | w.Header().Set("content-type", "text/event-stream") | |
| 590 | + | continuationUsage := s.continuationUsageTokens | |
| 591 | + | if continuationUsage <= 0 { | |
| 592 | + | continuationUsage = s.toolUsageTokens | |
| 593 | + | } | |
| 594 | + | writeOpenAIChunk(t, w, residentToolCallChunk(number, continuationUsage)) | |
| 595 | + | default: | |
| 596 | + | w.Header().Set("content-type", "text/event-stream") | |
| 597 | + | writeOpenAIChunk(t, w, residentFinalChunk(s.finalText)) | |
| 598 | + | } | |
| 599 | + | })) | |
| 600 | + | t.Cleanup(server.Close) | |
| 601 | + | return server | |
| 602 | + | } | |
| 603 | + | ||
| 604 | + | func residentToolCallChunk(number, usageTokens int) map[string]any { | |
| 605 | + | if usageTokens <= 0 { | |
| 606 | + | usageTokens = 10 | |
| 607 | + | } | |
| 608 | + | return map[string]any{ | |
| 609 | + | "choices": []any{map[string]any{ | |
| 610 | + | "index": 0, | |
| 611 | + | "delta": map[string]any{"tool_calls": []any{map[string]any{ | |
| 612 | + | "index": 0, "id": fmt.Sprintf("call-%d", number), "type": "function", | |
| 613 | + | "function": map[string]string{"name": "read", "arguments": `{"path":"x"}`}, | |
| 614 | + | }}}, | |
| 615 | + | "finish_reason": "tool_calls", | |
| 616 | + | }}, | |
| 617 | + | "usage": map[string]int{"prompt_tokens": usageTokens, "completion_tokens": 1}, | |
| 618 | + | } | |
| 619 | + | } | |
| 620 | + | ||
| 621 | + | func residentSummaryChunk(usageTokens int) map[string]any { | |
| 622 | + | chunk := map[string]any{ | |
| 623 | + | "choices": []any{map[string]any{"index": 0, "delta": map[string]string{"content": "compacted digest"}, "finish_reason": "stop"}}, | |
| 624 | + | } | |
| 625 | + | if usageTokens > 0 { | |
| 626 | + | chunk["usage"] = map[string]int{"prompt_tokens": usageTokens, "completion_tokens": 2} | |
| 627 | + | } | |
| 628 | + | return chunk | |
| 629 | + | } | |
| 630 | + | ||
| 631 | + | func residentFinalChunk(text string) map[string]any { | |
| 632 | + | return map[string]any{ | |
| 633 | + | "choices": []any{map[string]any{"index": 0, "delta": map[string]string{"content": text}, "finish_reason": "stop"}}, | |
| 634 | + | "usage": map[string]int{"prompt_tokens": 12, "completion_tokens": 5}, | |
| 635 | + | } | |
| 636 | + | } | |
| 637 | + | ||
| 638 | + | func (s *residentOverflowScript) capturedBodies() []string { | |
| 639 | + | s.mu.Lock() | |
| 640 | + | defer s.mu.Unlock() | |
| 641 | + | return append([]string(nil), s.bodies...) | |
| 642 | + | } | |
| 643 | + | ||
| 644 | + | func residentArguments(t *testing.T, baseURL string) Args { | |
| 645 | + | t.Helper() | |
| 646 | + | return Args{ | |
| 647 | + | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: baseURL, | |
| 648 | + | CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, | |
| 649 | + | } | |
| 650 | + | } | |
| 651 | + | ||
| 652 | + | func countResidentCheckpoints(t *testing.T, journal *subagents.ResidentJournal) int { | |
| 653 | + | t.Helper() | |
| 654 | + | records, err := subagents.ReadResidentJournal(filepath.Join(journal.Dir(), "transcript.jsonl")) | |
| 655 | + | if err != nil { | |
| 656 | + | t.Fatal(err) | |
| 657 | + | } | |
| 658 | + | checkpoints := 0 | |
| 659 | + | for _, record := range records { | |
| 660 | + | if record.Type == "child.compacted" { | |
| 661 | + | checkpoints++ | |
| 662 | + | } | |
| 663 | + | } | |
| 664 | + | return checkpoints | |
| 665 | + | } | |
| 666 | + | ||
| 667 | + | func TestResidentChildRunnerRecoversFromContextOverflowOnce(t *testing.T) { | |
| 668 | + | script := &residentOverflowScript{toolCallRequests: 2, finalText: "overflow recovered result"} | |
| 669 | + | server := script.start(t) | |
| 670 | + | runner, journal, spec := newResidentChildTestRunner(t, "overflow-child", residentArguments(t, server.URL)) | |
| 671 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |
| 672 | + | t.Fatal(err) | |
| 673 | + | } | |
| 674 | + | runErr := runner(t.Context(), "review") | |
| 675 | + | if err := journal.RecordTurnFinished(spec, "turn-1", runErr); err != nil { | |
| 676 | + | t.Fatal(err) | |
| 677 | + | } | |
| 678 | + | if runErr != nil { | |
| 679 | + | t.Fatalf("runner error = %v, want one recovered turn", runErr) | |
| 680 | + | } | |
| 681 | + | if got := script.requests.Load(); got != 5 { | |
| 682 | + | t.Fatalf("provider requests = %d, want 3 prompt, 1 compaction, and 1 continuation", got) | |
| 683 | + | } | |
| 684 | + | result, err := journal.Result() | |
| 685 | + | if err != nil { | |
| 686 | + | t.Fatal(err) | |
| 687 | + | } | |
| 688 | + | if result.State != subagents.ResidentIdle || result.Summary != "overflow recovered result" { | |
| 689 | + | t.Fatalf("result = %#v, want the recovered final text", result) | |
| 690 | + | } | |
| 691 | + | records, err := subagents.ReadResidentJournal(filepath.Join(journal.Dir(), "transcript.jsonl")) | |
| 692 | + | if err != nil { | |
| 693 | + | t.Fatal(err) | |
| 694 | + | } | |
| 695 | + | checkpoints := 0 | |
| 696 | + | for _, record := range records { | |
| 697 | + | if record.Type != "child.compacted" { | |
| 698 | + | continue | |
| 699 | + | } | |
| 700 | + | checkpoints++ | |
| 701 | + | if record.TurnID != "turn-1" { | |
| 702 | + | t.Fatalf("checkpoint turn ID = %q, want the active accepted turn", record.TurnID) | |
| 703 | + | } | |
| 704 | + | } | |
| 705 | + | if checkpoints != 1 { | |
| 706 | + | t.Fatalf("compaction checkpoints = %d, want 1", checkpoints) | |
| 707 | + | } | |
| 708 | + | // Resume replay starts from the checkpoint: the digest replaces the | |
| 709 | + | // pre-overflow tool exchange and the continuation request carried it. | |
| 710 | + | messages, err := subagents.ReadResidentTranscriptMessages(journal.Dir()) | |
| 711 | + | if err != nil { | |
| 712 | + | t.Fatal(err) | |
| 713 | + | } | |
| 714 | + | if len(messages) != 3 || !strings.HasPrefix(residentMessageText(messages[0]), "## Context Summary (compacted)") { | |
| 715 | + | t.Fatalf("resumed transcript = %#v, want the checkpoint plus the kept tail", messages) | |
| 716 | + | } | |
| 717 | + | bodies := script.capturedBodies() | |
| 718 | + | if len(bodies) != 5 || !strings.Contains(bodies[4], "compacted digest") { | |
| 719 | + | t.Fatalf("continuation request = %q, want the compacted transcript", bodies[len(bodies)-1]) | |
| 720 | + | } | |
| 721 | + | if strings.Contains(bodies[4], "call-1") { | |
| 722 | + | t.Fatalf("continuation request kept the dropped tool exchange: %q", bodies[4]) | |
| 723 | + | } | |
| 724 | + | } | |
| 725 | + | ||
| 726 | + | func TestResidentChildRunnerDoesNotRecoverTwicePerTurn(t *testing.T) { | |
| 727 | + | script := &residentOverflowScript{toolCallRequests: 2, secondOverflow: true, finalText: "unreached"} | |
| 728 | + | server := script.start(t) | |
| 729 | + | runner, journal, _ := newResidentChildTestRunner(t, "double-overflow", residentArguments(t, server.URL)) | |
| 730 | + | err := runner(t.Context(), "review") | |
| 731 | + | if err == nil || !provider.IsContextOverflowError(err) { | |
| 732 | + | t.Fatalf("runner error = %v, want the second overflow to stay terminal", err) | |
| 733 | + | } | |
| 734 | + | if got := script.requests.Load(); got != 5 { | |
| 735 | + | t.Fatalf("provider requests = %d, want no second compaction", got) | |
| 736 | + | } | |
| 737 | + | if checkpoints := countResidentCheckpoints(t, journal); checkpoints != 1 { | |
| 738 | + | t.Fatalf("compaction checkpoints = %d, want 1", checkpoints) | |
| 739 | + | } | |
| 740 | + | } | |
| 741 | + | ||
| 742 | + | func TestResidentChildRunnerDoesNotReplenishExplicitStepLimit(t *testing.T) { | |
| 743 | + | script := &residentOverflowScript{toolCallRequests: 3, finalText: "unreached"} | |
| 744 | + | server := script.start(t) | |
| 745 | + | args := residentArguments(t, server.URL) | |
| 746 | + | args.MaxSteps = 4 | |
| 747 | + | runner, journal, _ := newResidentChildTestRunner(t, "limited-overflow", args) | |
| 748 | + | err := runner(t.Context(), "review") | |
| 749 | + | if err == nil || !provider.IsContextOverflowError(err) { | |
| 750 | + | t.Fatalf("runner error = %v, want the overflow", err) | |
| 751 | + | } | |
| 752 | + | if got := script.requests.Load(); got != 5 { | |
| 753 | + | t.Fatalf("provider requests = %d, want the compaction attempt only", got) | |
| 754 | + | } | |
| 755 | + | // The checkpoint is still journaled so an explicit resume starts small, | |
| 756 | + | // but the spent allowance forbids continuing this turn. | |
| 757 | + | if checkpoints := countResidentCheckpoints(t, journal); checkpoints != 1 { | |
| 758 | + | t.Fatalf("compaction checkpoints = %d, want 1", checkpoints) | |
| 759 | + | } | |
| 760 | + | for _, body := range script.capturedBodies() { | |
| 761 | + | if strings.Contains(body, "compacted digest") { | |
| 762 | + | t.Fatalf("recovery continued after the step allowance was spent: %q", body) | |
| 763 | + | } | |
| 764 | + | } | |
| 765 | + | } | |
| 766 | + | ||
| 767 | + | func TestResidentChildRunnerFailsTurnWhenCheckpointCannotPersist(t *testing.T) { | |
| 768 | + | root := t.TempDir() | |
| 769 | + | spec := subagents.ResidentChildSpec{ | |
| 770 | + | ID: "persist-failure", SessionID: "persist-failure-session", Provider: "openai", Model: "gpt-4o", | |
| 771 | + | Tools: []string{"read"}, | |
| 772 | + | } | |
| 773 | + | journal, err := subagents.OpenResidentJournal(root, spec.ID) | |
| 774 | + | if err != nil { | |
| 775 | + | t.Fatal(err) | |
| 776 | + | } | |
| 777 | + | t.Cleanup(func() { _ = journal.Close() }) | |
| 778 | + | if err := journal.Accept(spec, "review"); err != nil { | |
| 779 | + | t.Fatal(err) | |
| 780 | + | } | |
| 781 | + | // The journal is closed at the exact overflow boundary, so the recovery | |
| 782 | + | // cannot persist its checkpoint. | |
| 783 | + | script := &residentOverflowScript{ | |
| 784 | + | toolCallRequests: 2, | |
| 785 | + | finalText: "unreached", | |
| 786 | + | onOverflow: func() { _ = journal.Close() }, | |
| 787 | + | } | |
| 788 | + | server := script.start(t) | |
| 789 | + | runner, err := newResidentChildRunner(residentArguments(t, server.URL), spec, journal) | |
| 790 | + | if err != nil { | |
| 791 | + | t.Fatal(err) | |
| 792 | + | } | |
| 793 | + | if err := runner(t.Context(), "review"); err == nil { | |
| 794 | + | t.Fatal("runner recovered without persisting its checkpoint") | |
| 795 | + | } | |
| 796 | + | if got := script.requests.Load(); got != 4 { | |
| 797 | + | t.Fatalf("provider requests = %d, want the compaction attempt only", got) | |
| 798 | + | } | |
| 799 | + | data, readErr := os.ReadFile(filepath.Join(journal.Dir(), "transcript.jsonl")) | |
| 800 | + | if readErr != nil { | |
| 801 | + | t.Fatal(readErr) | |
| 802 | + | } | |
| 803 | + | if strings.Contains(string(data), `"type":"child.compacted"`) { | |
| 804 | + | t.Fatal("a failed recovery recorded a checkpoint") | |
| 805 | + | } | |
| 806 | + | ||
| 807 | + | // A later reconstructed child resumes the pre-compaction transcript. | |
| 808 | + | reopened, err := subagents.OpenResidentJournal(root, spec.ID) | |
| 809 | + | if err != nil { | |
| 810 | + | t.Fatal(err) | |
| 811 | + | } | |
| 812 | + | t.Cleanup(func() { _ = reopened.Close() }) | |
| 813 | + | runner, err = newResidentChildRunner(residentArguments(t, server.URL), spec, reopened) | |
| 814 | + | if err != nil { | |
| 815 | + | t.Fatal(err) | |
| 816 | + | } | |
| 817 | + | if err := runner(t.Context(), "verify"); err != nil { | |
| 818 | + | t.Fatalf("follow-up error = %v", err) | |
| 819 | + | } | |
| 820 | + | bodies := script.capturedBodies() | |
| 821 | + | if last := bodies[len(bodies)-1]; !strings.Contains(last, "call-1") { | |
| 822 | + | t.Fatalf("follow-up request = %q, want the pre-compaction transcript", last) | |
| 823 | + | } | |
| 824 | + | } | |
| 825 | + | ||
| 826 | + | func TestRecoverResidentContextOverflowRestoresTranscript(t *testing.T) { | |
| 827 | + | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | |
| 828 | + | w.Header().Set("content-type", "text/event-stream") | |
| 829 | + | writeOpenAIChunk(t, w, map[string]any{ | |
| 830 | + | "choices": []any{map[string]any{"index": 0, "delta": map[string]string{"content": "compacted digest"}, "finish_reason": "stop"}}, | |
| 831 | + | "usage": map[string]int{"prompt_tokens": 5, "completion_tokens": 2}, | |
| 832 | + | }) | |
| 833 | + | })) | |
| 834 | + | defer server.Close() | |
| 835 | + | resolved, err := Resolve(residentArguments(t, server.URL), true) | |
| 836 | + | if err != nil { | |
| 837 | + | t.Fatal(err) | |
| 838 | + | } | |
| 839 | + | agent := core.NewAgent(resolved.NewClient(), resolved.Model, "", resolved.ToolRegistry) | |
| 840 | + | before := []provider.Message{ | |
| 841 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "task"}}}, | |
| 842 | + | {Role: provider.RoleAssistant, Content: []provider.Content{provider.ToolCallBlock{ID: "call-1", Name: "read", Arguments: json.RawMessage(`{"path":"x"}`)}}}, | |
| 843 | + | {Role: provider.RoleTool, Content: []provider.Content{provider.ToolResultBlock{CallID: "call-1", Content: []provider.Content{provider.TextBlock{Text: "missing"}}}}}, | |
| 844 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "keep working"}}}, | |
| 845 | + | } | |
| 846 | + | agent.SetMessages(before) | |
| 847 | + | ||
| 848 | + | journal, err := subagents.OpenResidentJournal(t.TempDir(), "restore-child") | |
| 849 | + | if err != nil { | |
| 850 | + | t.Fatal(err) | |
| 851 | + | } | |
| 852 | + | if err := journal.Accept(subagents.ResidentChildSpec{ID: "restore-child", SessionID: "restore-session", Provider: "openai", Model: "gpt-4o"}, "task"); err != nil { | |
| 853 | + | t.Fatal(err) | |
| 854 | + | } | |
| 855 | + | if err := journal.Close(); err != nil { | |
| 856 | + | t.Fatal(err) | |
| 857 | + | } | |
| 858 | + | ||
| 859 | + | overflow := errors.New("openai: http 413: context_length_exceeded") | |
| 860 | + | err = recoverResidentContextOverflow(t.Context(), agent, journal, func(core.AgentEvent) {}, 0, 4, overflow) | |
| 861 | + | if !errors.Is(err, overflow) || !strings.Contains(err.Error(), "persist resident child compaction") { | |
| 862 | + | t.Fatalf("recovery error = %v, want a checkpoint persistence failure", err) | |
| 863 | + | } | |
| 864 | + | if !reflect.DeepEqual(agent.Messages(), before) { | |
| 865 | + | t.Fatalf("transcript = %#v, want the pre-compaction transcript restored", agent.Messages()) | |
| 866 | + | } | |
| 867 | + | } | |
| 868 | + | ||
| 869 | + | func residentMessageText(message provider.Message) string { | |
| 870 | + | var text strings.Builder | |
| 871 | + | for _, content := range message.Content { | |
| 872 | + | if block, ok := content.(provider.TextBlock); ok { | |
| 873 | + | text.WriteString(block.Text) | |
| 874 | + | } | |
| 875 | + | } | |
| 876 | + | return text.String() | |
| 877 | + | } | |
| 878 | + | ||
| 879 | + | func TestResidentChildRunnerIgnoresCompactionUsageInReminder(t *testing.T) { | |
| 880 | + | // The summarization request reports the pre-compaction prompt size. If that | |
| 881 | + | // gauge armed the next reminder, the first continuation step would claim the | |
| 882 | + | // compacted transcript sits at the top of the window. The continuation's own | |
| 883 | + | // usage must still be able to arm a truthful reminder. | |
| 884 | + | script := &residentOverflowScript{ | |
| 885 | + | toolCallRequests: 2, | |
| 886 | + | continuationToolCalls: 1, | |
| 887 | + | finalText: "overflow recovered result", | |
| 888 | + | summaryUsageTokens: 1 << 20, | |
| 889 | + | continuationUsageTokens: 1 << 20, | |
| 890 | + | } | |
| 891 | + | server := script.start(t) | |
| 892 | + | runner, _, _ := newResidentChildTestRunner(t, "usage-child", residentArguments(t, server.URL)) | |
| 893 | + | if err := runner(t.Context(), "review"); err != nil { | |
| 894 | + | t.Fatalf("runner error = %v, want one recovered turn", err) | |
| 895 | + | } | |
| 896 | + | bodies := script.capturedBodies() | |
| 897 | + | if len(bodies) != 6 { | |
| 898 | + | t.Fatalf("provider requests = %d, want 3 prompt, 1 compaction, and 2 continued steps", len(bodies)) | |
| 899 | + | } | |
| 900 | + | if strings.Contains(bodies[4], "Context usage is at") { | |
| 901 | + | t.Fatalf("first continuation step carries a reminder from the compaction request: %q", bodies[4]) | |
| 902 | + | } | |
| 903 | + | if !strings.Contains(bodies[5], "Context usage is at") { | |
| 904 | + | t.Fatalf("second continuation step lost the reminder: %q", bodies[5]) | |
| 905 | + | } | |
| 906 | + | } | |
| 907 | + | ||
| 908 | + | func TestResidentCheckpointMessagesDropsOnlyHostContext(t *testing.T) { | |
| 909 | + | internal := provider.Message{ | |
| 910 | + | Role: provider.RoleDeveloper, | |
| 911 | + | Meta: map[string]string{"internal_context": "true"}, | |
| 912 | + | Content: []provider.Content{provider.TextBlock{Text: "Context usage is at 96% of the model window."}}, | |
| 913 | + | } | |
| 914 | + | importedNote := provider.Message{ | |
| 915 | + | Role: provider.RoleDeveloper, | |
| 916 | + | Content: []provider.Content{provider.TextBlock{Text: "imported developer note"}}, | |
| 917 | + | } | |
| 918 | + | task := provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "task"}}} | |
| 919 | + | kept := residentCheckpointMessages([]provider.Message{internal, importedNote, task}) | |
| 920 | + | if len(kept) != 2 || kept[0].Role != provider.RoleDeveloper || kept[1].Role != provider.RoleUser { | |
| 921 | + | t.Fatalf("checkpoint messages = %#v, want host context dropped and other messages kept", kept) | |
| 922 | + | } | |
| 923 | + | if !core.IsInternalContextMessage(internal) || core.IsInternalContextMessage(importedNote) { | |
| 924 | + | t.Fatal("IsInternalContextMessage misclassified a developer message") | |
| 925 | + | } | |
| 926 | + | } | |
| 927 | + | ||
| 928 | + | func TestResidentChildRunnerKeepsHostContextOutOfCheckpoint(t *testing.T) { | |
| 929 | + | // Tool turns report a prompt size past the top band, so the child holds a | |
| 930 | + | // host reminder when the overflow arrives. | |
| 931 | + | script := &residentOverflowScript{ | |
| 932 | + | toolCallRequests: 2, | |
| 933 | + | finalText: "overflow recovered result", | |
| 934 | + | toolUsageTokens: 1 << 20, | |
| 935 | + | } | |
| 936 | + | server := script.start(t) | |
| 937 | + | runner, journal, _ := newResidentChildTestRunner(t, "checkpoint-context", residentArguments(t, server.URL)) | |
| 938 | + | if err := runner(t.Context(), "review"); err != nil { | |
| 939 | + | t.Fatalf("runner error = %v, want one recovered turn", err) | |
| 940 | + | } | |
| 941 | + | messages, err := subagents.ReadResidentTranscriptMessages(journal.Dir()) | |
| 942 | + | if err != nil { | |
| 943 | + | t.Fatal(err) | |
| 944 | + | } | |
| 945 | + | joined := joinedResidentText(messages) | |
| 946 | + | if !strings.Contains(joined, "compacted digest") { | |
| 947 | + | t.Fatalf("resumed transcript = %q, want the checkpoint digest", joined) | |
| 948 | + | } | |
| 949 | + | if strings.Contains(joined, "Context usage is at") { | |
| 950 | + | t.Fatalf("checkpoint replayed a stale host reminder: %q", joined) | |
| 951 | + | } | |
| 952 | + | } | |
| 953 | + | ||
| 954 | + | func TestResidentChildRunnerKeepsRemainingStepAllowanceAcrossRecovery(t *testing.T) { | |
| 955 | + | // Two tool turns and the overflow leave two of the five allowed steps. The | |
| 956 | + | // continuation must receive exactly those, not a fresh limit. | |
| 957 | + | script := &residentOverflowScript{ | |
| 958 | + | toolCallRequests: 2, | |
| 959 | + | continuationToolCalls: 10, | |
| 960 | + | finalText: "unreached", | |
| 961 | + | } | |
| 962 | + | server := script.start(t) | |
| 963 | + | args := residentArguments(t, server.URL) | |
| 964 | + | args.MaxSteps = 5 | |
| 965 | + | runner, journal, _ := newResidentChildTestRunner(t, "partial-allowance", args) | |
| 966 | + | err := runner(t.Context(), "review") | |
| 967 | + | if !errors.Is(err, core.ErrMaxSteps) { | |
| 968 | + | t.Fatalf("runner error = %v, want the continuation to exhaust the remaining allowance", err) | |
| 969 | + | } | |
| 970 | + | if got := script.requests.Load(); got != 6 { | |
| 971 | + | t.Fatalf("provider requests = %d, want 2 tool turns, the overflow, the compaction, and 2 more steps", got) | |
| 972 | + | } | |
| 973 | + | if checkpoints := countResidentCheckpoints(t, journal); checkpoints != 1 { | |
| 974 | + | t.Fatalf("compaction checkpoints = %d, want 1", checkpoints) | |
| 975 | + | } | |
| 976 | + | } | |
| 977 | + | ||
| 978 | + | func TestResidentChildRunnerDoesNotCompactWithoutHistory(t *testing.T) { | |
| 979 | + | // The overflow arrives on the first request, so the transcript is too short | |
| 980 | + | // to summarize without replacing the task itself. | |
| 981 | + | script := &residentOverflowScript{toolCallRequests: 0, finalText: "unreached"} | |
| 982 | + | server := script.start(t) | |
| 983 | + | runner, journal, _ := newResidentChildTestRunner(t, "short-overflow", residentArguments(t, server.URL)) | |
| 984 | + | err := runner(t.Context(), "review") | |
| 985 | + | if err == nil || !provider.IsContextOverflowError(err) || !strings.Contains(err.Error(), "context_length_exceeded") { | |
| 986 | + | t.Fatalf("runner error = %v, want the original overflow", err) | |
| 987 | + | } | |
| 988 | + | if got := script.requests.Load(); got != 1 { | |
| 989 | + | t.Fatalf("provider requests = %d, want no compaction attempt", got) | |
| 990 | + | } | |
| 991 | + | if checkpoints := countResidentCheckpoints(t, journal); checkpoints != 0 { | |
| 992 | + | t.Fatalf("compaction checkpoints = %d, want none without history", checkpoints) | |
| 993 | + | } | |
| 994 | + | } | |
| 995 | + | ||
| 996 | + | func joinedResidentText(messages []provider.Message) string { | |
| 997 | + | var text strings.Builder | |
| 998 | + | for _, message := range messages { | |
| 999 | + | text.WriteString(residentMessageText(message)) | |
| 1000 | + | text.WriteString("\n") | |
| 1001 | + | } | |
| 1002 | + | return text.String() | |
| 1003 | + | } | |
| @@ -7,6 +7,9 @@ import ( | |||||
| 7 | "fmt" | 7 | "fmt" | ||
| 8 | "net/http" | 8 | "net/http" | ||
| 9 | "net/http/httptest" | 9 | "net/http/httptest" | ||
| 10 | + | "os" | |||
| 11 | + | "path/filepath" | |||
| 12 | + | "reflect" | |||
| 10 | "strings" | 13 | "strings" | ||
| 11 | "sync" | 14 | "sync" | ||
| 12 | "sync/atomic" | 15 | "sync/atomic" | ||
| @@ -18,65 +21,87 @@ import ( | |||||
| 18 | "github.com/bnema/zut/packages/provider" | 21 | "github.com/bnema/zut/packages/provider" | ||
| 19 | ) | 22 | ) | ||
| 20 | 23 | ||||
| 21 | - | func TestResidentChildRunnerDefaultsToFiniteMaxSteps(t *testing.T) { | 24 | + | func TestResidentChildRunnerWithoutStepLimitRunsPastFormerCap(t *testing.T) { |
| 22 | - | for _, tc := range []struct { | 25 | + | server, requests := residentToolLoopServer(t, 51) |
| 23 | - | name string | 26 | + | runner, _, _ := newResidentChildTestRunner(t, "unlimited-child", Args{ |
| 24 | - | maxSteps int | 27 | + | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: server.URL, |
| 25 | - | want int | 28 | + | CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, |
| 26 | - | }{ | 29 | + | }) |
| 27 | - | {name: "unlimited parent defaults to bounded child", maxSteps: 0, want: 50}, | 30 | + | if err := runner(t.Context(), "review"); err != nil { |
| 28 | - | {name: "explicit parent limit preserved", maxSteps: 30, want: 30}, | 31 | + | t.Fatalf("runner error = %v, want success past the former 50-step cap", err) |
| 29 | - | } { | 32 | + | } |
| 30 | - | t.Run(tc.name, func(t *testing.T) { | 33 | + | if got := requests.Load(); got != 52 { |
| 31 | - | var requests atomic.Int32 | 34 | + | t.Fatalf("provider requests = %d, want 51 tool turns and one answer", got) |
| 32 | - | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | 35 | + | } |
| 33 | - | requests.Add(1) | 36 | + | } |
| 34 | - | w.Header().Set("content-type", "text/event-stream") | 37 | + | |
| 35 | - | writeOpenAIChunk(t, w, map[string]any{ | 38 | + | func TestResidentChildRunnerEnforcesExplicitParentStepLimit(t *testing.T) { |
| 36 | - | "choices": []any{map[string]any{ | 39 | + | server, requests := residentToolLoopServer(t, 200) |
| 37 | - | "index": 0, | 40 | + | runner, _, _ := newResidentChildTestRunner(t, "limited-child", Args{ |
| 38 | - | "delta": map[string]any{"tool_calls": []any{map[string]any{ | 41 | + | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: server.URL, |
| 39 | - | "index": 0, "id": "call-loop", "type": "function", | 42 | + | MaxSteps: 3, CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, |
| 40 | - | "function": map[string]string{"name": "read", "arguments": `{"path":"x"}`}, | 43 | + | }) |
| 41 | - | }}}, | 44 | + | err := runner(t.Context(), "review") |
| 42 | - | "finish_reason": "tool_calls", | 45 | + | if !errors.Is(err, core.ErrMaxSteps) || !strings.Contains(err.Error(), "3") { |
| 43 | - | }}, | 46 | + | t.Fatalf("runner error = %v, want step limit 3", err) |
| 44 | - | "usage": map[string]int{"prompt_tokens": 10, "completion_tokens": 1}, | 47 | + | } |
| 45 | - | }) | 48 | + | if got := requests.Load(); got != 3 { |
| 46 | - | })) | 49 | + | t.Fatalf("provider requests = %d, want 3", got) |
| 47 | - | defer server.Close() | 50 | + | } |
| 48 | - | 51 | + | } | |
| 49 | - | spec := subagents.ResidentChildSpec{ | 52 | + | |
| 50 | - | ID: "maxsteps-child", SessionID: "maxsteps-session", Provider: "openai", Model: "gpt-4o", | 53 | + | // residentToolLoopServer answers the first toolCalls requests with a distinct |
| 51 | - | Tools: []string{"read"}, | 54 | + | // read call and then returns a substantive final answer. |
| 52 | - | } | 55 | + | func residentToolLoopServer(t *testing.T, toolCalls int) (*httptest.Server, *atomic.Int32) { |
| 53 | - | journal, err := subagents.OpenResidentJournal(t.TempDir(), spec.ID) | 56 | + | t.Helper() |
| 54 | - | if err != nil { | 57 | + | var requests atomic.Int32 |
| 55 | - | t.Fatal(err) | 58 | + | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| 56 | - | } | 59 | + | number := requests.Add(1) |
| 57 | - | t.Cleanup(func() { _ = journal.Close() }) | 60 | + | w.Header().Set("content-type", "text/event-stream") |
| 58 | - | if err := journal.Accept(spec, "review"); err != nil { | 61 | + | if int(number) <= toolCalls { |
| 59 | - | t.Fatal(err) | 62 | + | writeOpenAIChunk(t, w, map[string]any{ |
| 60 | - | } | 63 | + | "choices": []any{map[string]any{ |
| 61 | - | runner, err := newResidentChildRunner(Args{ | 64 | + | "index": 0, |
| 62 | - | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: server.URL, | 65 | + | "delta": map[string]any{"tool_calls": []any{map[string]any{ |
| 63 | - | MaxSteps: tc.maxSteps, CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, | 66 | + | "index": 0, "id": fmt.Sprintf("call-%d", number), "type": "function", |
| 64 | - | }, spec, journal) | 67 | + | "function": map[string]string{"name": "read", "arguments": `{"path":"x"}`}, |
| 65 | - | if err != nil { | 68 | + | }}}, |
| 66 | - | t.Fatal(err) | 69 | + | "finish_reason": "tool_calls", |
| 67 | - | } | 70 | + | }}, |
| 68 | - | err = runner(t.Context(), "review") | 71 | + | "usage": map[string]int{"prompt_tokens": 10, "completion_tokens": 1}, |
| 69 | - | if !errors.Is(err, core.ErrMaxSteps) { | 72 | + | }) |
| 70 | - | t.Fatalf("runner error = %v, want step-limit stop", err) | 73 | + | return |
| 71 | - | } | 74 | + | } |
| 72 | - | if !strings.Contains(err.Error(), fmt.Sprintf("%d", tc.want)) { | 75 | + | writeOpenAIChunk(t, w, map[string]any{ |
| 73 | - | t.Fatalf("runner error = %v, want limit %d", err, tc.want) | 76 | + | "choices": []any{map[string]any{"index": 0, "delta": map[string]string{"content": "final result"}, "finish_reason": "stop"}}, |
| 74 | - | } | 77 | + | "usage": map[string]int{"prompt_tokens": 10, "completion_tokens": 1}, |
| 75 | - | if got := requests.Load(); got != int32(tc.want) { | |||
| 76 | - | t.Fatalf("provider requests = %d, want %d", got, tc.want) | |||
| 77 | - | } | |||
| 78 | }) | 78 | }) | ||
| 79 | + | })) | |||
| 80 | + | t.Cleanup(server.Close) | |||
| 81 | + | return server, &requests | |||
| 82 | + | } | |||
| 83 | + | ||||
| 84 | + | // newResidentChildTestRunner builds one journaled resident child runner with | |||
| 85 | + | // its own accepted spec. | |||
| 86 | + | func newResidentChildTestRunner(t *testing.T, childID string, args Args) (subagents.ResidentTurnRunner, *subagents.ResidentJournal, subagents.ResidentChildSpec) { | |||
| 87 | + | t.Helper() | |||
| 88 | + | spec := subagents.ResidentChildSpec{ | |||
| 89 | + | ID: childID, SessionID: childID + "-session", Provider: "openai", Model: "gpt-4o", | |||
| 90 | + | Tools: []string{"read"}, | |||
| 91 | + | } | |||
| 92 | + | journal, err := subagents.OpenResidentJournal(t.TempDir(), spec.ID) | |||
| 93 | + | if err != nil { | |||
| 94 | + | t.Fatal(err) | |||
| 79 | } | 95 | } | ||
| 96 | + | t.Cleanup(func() { _ = journal.Close() }) | |||
| 97 | + | if err := journal.Accept(spec, "review"); err != nil { | |||
| 98 | + | t.Fatal(err) | |||
| 99 | + | } | |||
| 100 | + | runner, err := newResidentChildRunner(args, spec, journal) | |||
| 101 | + | if err != nil { | |||
| 102 | + | t.Fatal(err) | |||
| 103 | + | } | |||
| 104 | + | return runner, journal, spec | |||
| 80 | } | 105 | } | ||
| 81 | 106 | ||||
| 82 | func TestResidentChildRunnerInjectsContextReminderIntoNextRequest(t *testing.T) { | 107 | func TestResidentChildRunnerInjectsContextReminderIntoNextRequest(t *testing.T) { | ||
| @@ -202,7 +227,7 @@ func TestResidentChildRegistryUsesExactToolListAndForbidsDelegation(t *testing.T | |||||
| 202 | } | 227 | } | ||
| 203 | } | 228 | } | ||
| 204 | 229 | ||||
| 205 | - | func TestConfigureResidentContextNudgeRemindsOnceAtThreshold(t *testing.T) { | 230 | + | func TestConfigureResidentContextNudgeRemindsOncePerBand(t *testing.T) { |
| 206 | agent := core.NewAgent(nil, "model", "system", core.Registry{"read": nil}) | 231 | agent := core.NewAgent(nil, "model", "system", core.Registry{"read": nil}) | ||
| 207 | configureResidentContextNudge(agent, 1_000) | 232 | configureResidentContextNudge(agent, 1_000) | ||
| 208 | 233 | ||||
| @@ -222,16 +247,52 @@ func TestConfigureResidentContextNudgeRemindsOnceAtThreshold(t *testing.T) { | |||||
| 222 | return contextText | 247 | return contextText | ||
| 223 | } | 248 | } | ||
| 224 | 249 | ||||
| 225 | - | // Below the threshold: silent, including cache-heavy usage. | 250 | + | // Below the first band: silent, including cache-heavy usage. |
| 226 | check(provider.Usage{InputTokens: 849}, false) | 251 | check(provider.Usage{InputTokens: 849}, false) | ||
| 227 | check(provider.Usage{InputTokens: 400, CacheReadTokens: 400, CacheWriteTokens: 49}, false) | 252 | check(provider.Usage{InputTokens: 400, CacheReadTokens: 400, CacheWriteTokens: 49}, false) | ||
| 228 | - | // First crossing: exactly one reminder carrying the computed percent. | 253 | + | // Each band fires once, carrying the computed percent of the crossing turn. |
| 229 | first := check(provider.Usage{InputTokens: 850}, true) | 254 | first := check(provider.Usage{InputTokens: 850}, true) | ||
| 230 | if !strings.Contains(first, "85%") { | 255 | if !strings.Contains(first, "85%") { | ||
| 231 | t.Fatalf("reminder percent = %q, want 85%%", first) | 256 | t.Fatalf("reminder percent = %q, want 85%%", first) | ||
| 232 | } | 257 | } | ||
| 233 | - | // Later turns stay silent even as usage grows. | 258 | + | check(provider.Usage{InputTokens: 860}, false) |
| 234 | - | check(provider.Usage{InputTokens: 970}, false) | 259 | + | second := check(provider.Usage{InputTokens: 900}, true) |
| 260 | + | if !strings.Contains(second, "90%") { | |||
| 261 | + | t.Fatalf("reminder percent = %q, want 90%%", second) | |||
| 262 | + | } | |||
| 263 | + | check(provider.Usage{InputTokens: 920}, false) | |||
| 264 | + | third := check(provider.Usage{InputTokens: 950}, true) | |||
| 265 | + | if !strings.Contains(third, "95%") { | |||
| 266 | + | t.Fatalf("reminder percent = %q, want 95%%", third) | |||
| 267 | + | } | |||
| 268 | + | // Past the last band the ladder is spent. | |||
| 269 | + | check(provider.Usage{InputTokens: 999}, false) | |||
| 270 | + | } | |||
| 271 | + | ||||
| 272 | + | func TestConfigureResidentContextNudgeIgnoresShrinkingUsage(t *testing.T) { | |||
| 273 | + | agent := core.NewAgent(nil, "model", "system", core.Registry{"read": nil}) | |||
| 274 | + | configureResidentContextNudge(agent, 1_000) | |||
| 275 | + | ||||
| 276 | + | check := func(usage provider.Usage, wantReminder bool) string { | |||
| 277 | + | t.Helper() | |||
| 278 | + | agent.SeedLastTurnUsage(usage) | |||
| 279 | + | allowed, reason, contextText := agent.BeforeTurnContext(t.Context(), 1) | |||
| 280 | + | if !allowed || reason != "" { | |||
| 281 | + | t.Fatalf("nudge blocked turn: allowed=%t reason=%q", allowed, reason) | |||
| 282 | + | } | |||
| 283 | + | if wantReminder != (contextText != "") { | |||
| 284 | + | t.Fatalf("reminder for usage %#v = %q, want reminder=%t", usage, contextText, wantReminder) | |||
| 285 | + | } | |||
| 286 | + | return contextText | |||
| 287 | + | } | |||
| 288 | + | ||||
| 289 | + | // A shrinking prompt never delivers a band it did not cross. | |||
| 290 | + | check(provider.Usage{InputTokens: 700}, false) | |||
| 291 | + | check(provider.Usage{InputTokens: 500}, false) | |||
| 292 | + | jumped := check(provider.Usage{InputTokens: 910}, true) | |||
| 293 | + | if !strings.Contains(jumped, "91%") { | |||
| 294 | + | t.Fatalf("reminder percent = %q, want 91%%", jumped) | |||
| 295 | + | } | |||
| 235 | } | 296 | } | ||
| 236 | 297 | ||||
| 237 | func TestConfigureResidentContextNudgeHandlesDirectJumpAndUnknownWindow(t *testing.T) { | 298 | func TestConfigureResidentContextNudgeHandlesDirectJumpAndUnknownWindow(t *testing.T) { | ||
| @@ -400,25 +461,10 @@ func TestResidentChildRunnerResumesWithRetainedHistory(t *testing.T) { | |||||
| 400 | 461 | ||||
| 401 | func newResidentTestRunner(t *testing.T, baseURL string) (subagents.ResidentTurnRunner, *subagents.ResidentJournal) { | 462 | func newResidentTestRunner(t *testing.T, baseURL string) (subagents.ResidentTurnRunner, *subagents.ResidentJournal) { | ||
| 402 | t.Helper() | 463 | t.Helper() | ||
| 403 | - | spec := subagents.ResidentChildSpec{ | 464 | + | runner, journal, _ := newResidentChildTestRunner(t, "resident-child", Args{ |
| 404 | - | ID: "resident-child", SessionID: "resident-session", Provider: "openai", Model: "gpt-4o", | |||
| 405 | - | Tools: []string{"read"}, | |||
| 406 | - | } | |||
| 407 | - | journal, err := subagents.OpenResidentJournal(t.TempDir(), spec.ID) | |||
| 408 | - | if err != nil { | |||
| 409 | - | t.Fatal(err) | |||
| 410 | - | } | |||
| 411 | - | t.Cleanup(func() { _ = journal.Close() }) | |||
| 412 | - | if err := journal.Accept(spec, "review"); err != nil { | |||
| 413 | - | t.Fatal(err) | |||
| 414 | - | } | |||
| 415 | - | runner, err := newResidentChildRunner(Args{ | |||
| 416 | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: baseURL, | 465 | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: baseURL, | ||
| 417 | CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, | 466 | CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, | ||
| 418 | - | }, spec, journal) | 467 | + | }) |
| 419 | - | if err != nil { | |||
| 420 | - | t.Fatal(err) | |||
| 421 | - | } | |||
| 422 | return runner, journal | 468 | return runner, journal | ||
| 423 | } | 469 | } | ||
| 424 | 470 | ||||
| @@ -484,3 +530,474 @@ func TestResidentChildArgsDoNotForwardCLIKeyAcrossProviders(t *testing.T) { | |||||
| 484 | t.Fatalf("same-provider child API key = %q, want parent key", next.APIKey) | 530 | t.Fatalf("same-provider child API key = %q, want parent key", next.APIKey) | ||
| 485 | } | 531 | } | ||
| 486 | } | 532 | } | ||
| 533 | + | ||||
| 534 | + | // residentOverflowScript scripts the provider responses of a child that hits | |||
| 535 | + | // the context window: leading tool-call turns, one overflow, one compaction | |||
| 536 | + | // summary, then continued tool turns or the final text. | |||
| 537 | + | type residentOverflowScript struct { | |||
| 538 | + | toolCallRequests int | |||
| 539 | + | continuationToolCalls int | |||
| 540 | + | secondOverflow bool | |||
| 541 | + | finalText string | |||
| 542 | + | onOverflow func() | |||
| 543 | + | // toolUsageTokens is the prompt size reported by tool-call responses; zero | |||
| 544 | + | // reports a small prompt. | |||
| 545 | + | toolUsageTokens int | |||
| 546 | + | // summaryUsageTokens, when positive, is the prompt size reported by the | |||
| 547 | + | // compaction summary response. | |||
| 548 | + | summaryUsageTokens int | |||
| 549 | + | // continuationUsageTokens is the prompt size reported by tool turns after a | |||
| 550 | + | // successful compaction; zero reuses toolUsageTokens. | |||
| 551 | + | continuationUsageTokens int | |||
| 552 | + | ||||
| 553 | + | requests atomic.Int32 | |||
| 554 | + | mu sync.Mutex | |||
| 555 | + | bodies []string | |||
| 556 | + | } | |||
| 557 | + | ||||
| 558 | + | func (s *residentOverflowScript) start(t *testing.T) *httptest.Server { | |||
| 559 | + | t.Helper() | |||
| 560 | + | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | |||
| 561 | + | number := int(s.requests.Add(1)) | |||
| 562 | + | var body map[string]any | |||
| 563 | + | if err := json.NewDecoder(r.Body).Decode(&body); err != nil { | |||
| 564 | + | t.Errorf("decode request: %v", err) | |||
| 565 | + | } | |||
| 566 | + | encoded, _ := json.Marshal(body["messages"]) | |||
| 567 | + | s.mu.Lock() | |||
| 568 | + | s.bodies = append(s.bodies, string(encoded)) | |||
| 569 | + | s.mu.Unlock() | |||
| 570 | + | ||||
| 571 | + | overflowAt := s.toolCallRequests + 1 | |||
| 572 | + | switch { | |||
| 573 | + | case number <= s.toolCallRequests: | |||
| 574 | + | w.Header().Set("content-type", "text/event-stream") | |||
| 575 | + | writeOpenAIChunk(t, w, residentToolCallChunk(number, s.toolUsageTokens)) | |||
| 576 | + | case number == overflowAt: | |||
| 577 | + | if s.onOverflow != nil { | |||
| 578 | + | s.onOverflow() | |||
| 579 | + | } | |||
| 580 | + | w.WriteHeader(http.StatusRequestEntityTooLarge) | |||
| 581 | + | _, _ = w.Write([]byte(`{"error":{"message":"input exceeds the context window","code":"context_length_exceeded"}}`)) | |||
| 582 | + | case number == overflowAt+1: | |||
| 583 | + | w.Header().Set("content-type", "text/event-stream") | |||
| 584 | + | writeOpenAIChunk(t, w, residentSummaryChunk(s.summaryUsageTokens)) | |||
| 585 | + | case s.secondOverflow: | |||
| 586 | + | w.WriteHeader(http.StatusRequestEntityTooLarge) | |||
| 587 | + | _, _ = w.Write([]byte(`{"error":{"message":"input exceeds the context window","code":"context_length_exceeded"}}`)) | |||
| 588 | + | case number < overflowAt+2+s.continuationToolCalls: | |||
| 589 | + | w.Header().Set("content-type", "text/event-stream") | |||
| 590 | + | continuationUsage := s.continuationUsageTokens | |||
| 591 | + | if continuationUsage <= 0 { | |||
| 592 | + | continuationUsage = s.toolUsageTokens | |||
| 593 | + | } | |||
| 594 | + | writeOpenAIChunk(t, w, residentToolCallChunk(number, continuationUsage)) | |||
| 595 | + | default: | |||
| 596 | + | w.Header().Set("content-type", "text/event-stream") | |||
| 597 | + | writeOpenAIChunk(t, w, residentFinalChunk(s.finalText)) | |||
| 598 | + | } | |||
| 599 | + | })) | |||
| 600 | + | t.Cleanup(server.Close) | |||
| 601 | + | return server | |||
| 602 | + | } | |||
| 603 | + | ||||
| 604 | + | func residentToolCallChunk(number, usageTokens int) map[string]any { | |||
| 605 | + | if usageTokens <= 0 { | |||
| 606 | + | usageTokens = 10 | |||
| 607 | + | } | |||
| 608 | + | return map[string]any{ | |||
| 609 | + | "choices": []any{map[string]any{ | |||
| 610 | + | "index": 0, | |||
| 611 | + | "delta": map[string]any{"tool_calls": []any{map[string]any{ | |||
| 612 | + | "index": 0, "id": fmt.Sprintf("call-%d", number), "type": "function", | |||
| 613 | + | "function": map[string]string{"name": "read", "arguments": `{"path":"x"}`}, | |||
| 614 | + | }}}, | |||
| 615 | + | "finish_reason": "tool_calls", | |||
| 616 | + | }}, | |||
| 617 | + | "usage": map[string]int{"prompt_tokens": usageTokens, "completion_tokens": 1}, | |||
| 618 | + | } | |||
| 619 | + | } | |||
| 620 | + | ||||
| 621 | + | func residentSummaryChunk(usageTokens int) map[string]any { | |||
| 622 | + | chunk := map[string]any{ | |||
| 623 | + | "choices": []any{map[string]any{"index": 0, "delta": map[string]string{"content": "compacted digest"}, "finish_reason": "stop"}}, | |||
| 624 | + | } | |||
| 625 | + | if usageTokens > 0 { | |||
| 626 | + | chunk["usage"] = map[string]int{"prompt_tokens": usageTokens, "completion_tokens": 2} | |||
| 627 | + | } | |||
| 628 | + | return chunk | |||
| 629 | + | } | |||
| 630 | + | ||||
| 631 | + | func residentFinalChunk(text string) map[string]any { | |||
| 632 | + | return map[string]any{ | |||
| 633 | + | "choices": []any{map[string]any{"index": 0, "delta": map[string]string{"content": text}, "finish_reason": "stop"}}, | |||
| 634 | + | "usage": map[string]int{"prompt_tokens": 12, "completion_tokens": 5}, | |||
| 635 | + | } | |||
| 636 | + | } | |||
| 637 | + | ||||
| 638 | + | func (s *residentOverflowScript) capturedBodies() []string { | |||
| 639 | + | s.mu.Lock() | |||
| 640 | + | defer s.mu.Unlock() | |||
| 641 | + | return append([]string(nil), s.bodies...) | |||
| 642 | + | } | |||
| 643 | + | ||||
| 644 | + | func residentArguments(t *testing.T, baseURL string) Args { | |||
| 645 | + | t.Helper() | |||
| 646 | + | return Args{ | |||
| 647 | + | Provider: "openai", Model: "gpt-4o", APIKey: "synthetic", BaseURL: baseURL, | |||
| 648 | + | CWD: t.TempDir(), NoContextFiles: true, NoSkill: true, | |||
| 649 | + | } | |||
| 650 | + | } | |||
| 651 | + | ||||
| 652 | + | func countResidentCheckpoints(t *testing.T, journal *subagents.ResidentJournal) int { | |||
| 653 | + | t.Helper() | |||
| 654 | + | records, err := subagents.ReadResidentJournal(filepath.Join(journal.Dir(), "transcript.jsonl")) | |||
| 655 | + | if err != nil { | |||
| 656 | + | t.Fatal(err) | |||
| 657 | + | } | |||
| 658 | + | checkpoints := 0 | |||
| 659 | + | for _, record := range records { | |||
| 660 | + | if record.Type == "child.compacted" { | |||
| 661 | + | checkpoints++ | |||
| 662 | + | } | |||
| 663 | + | } | |||
| 664 | + | return checkpoints | |||
| 665 | + | } | |||
| 666 | + | ||||
| 667 | + | func TestResidentChildRunnerRecoversFromContextOverflowOnce(t *testing.T) { | |||
| 668 | + | script := &residentOverflowScript{toolCallRequests: 2, finalText: "overflow recovered result"} | |||
| 669 | + | server := script.start(t) | |||
| 670 | + | runner, journal, spec := newResidentChildTestRunner(t, "overflow-child", residentArguments(t, server.URL)) | |||
| 671 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |||
| 672 | + | t.Fatal(err) | |||
| 673 | + | } | |||
| 674 | + | runErr := runner(t.Context(), "review") | |||
| 675 | + | if err := journal.RecordTurnFinished(spec, "turn-1", runErr); err != nil { | |||
| 676 | + | t.Fatal(err) | |||
| 677 | + | } | |||
| 678 | + | if runErr != nil { | |||
| 679 | + | t.Fatalf("runner error = %v, want one recovered turn", runErr) | |||
| 680 | + | } | |||
| 681 | + | if got := script.requests.Load(); got != 5 { | |||
| 682 | + | t.Fatalf("provider requests = %d, want 3 prompt, 1 compaction, and 1 continuation", got) | |||
| 683 | + | } | |||
| 684 | + | result, err := journal.Result() | |||
| 685 | + | if err != nil { | |||
| 686 | + | t.Fatal(err) | |||
| 687 | + | } | |||
| 688 | + | if result.State != subagents.ResidentIdle || result.Summary != "overflow recovered result" { | |||
| 689 | + | t.Fatalf("result = %#v, want the recovered final text", result) | |||
| 690 | + | } | |||
| 691 | + | records, err := subagents.ReadResidentJournal(filepath.Join(journal.Dir(), "transcript.jsonl")) | |||
| 692 | + | if err != nil { | |||
| 693 | + | t.Fatal(err) | |||
| 694 | + | } | |||
| 695 | + | checkpoints := 0 | |||
| 696 | + | for _, record := range records { | |||
| 697 | + | if record.Type != "child.compacted" { | |||
| 698 | + | continue | |||
| 699 | + | } | |||
| 700 | + | checkpoints++ | |||
| 701 | + | if record.TurnID != "turn-1" { | |||
| 702 | + | t.Fatalf("checkpoint turn ID = %q, want the active accepted turn", record.TurnID) | |||
| 703 | + | } | |||
| 704 | + | } | |||
| 705 | + | if checkpoints != 1 { | |||
| 706 | + | t.Fatalf("compaction checkpoints = %d, want 1", checkpoints) | |||
| 707 | + | } | |||
| 708 | + | // Resume replay starts from the checkpoint: the digest replaces the | |||
| 709 | + | // pre-overflow tool exchange and the continuation request carried it. | |||
| 710 | + | messages, err := subagents.ReadResidentTranscriptMessages(journal.Dir()) | |||
| 711 | + | if err != nil { | |||
| 712 | + | t.Fatal(err) | |||
| 713 | + | } | |||
| 714 | + | if len(messages) != 3 || !strings.HasPrefix(residentMessageText(messages[0]), "## Context Summary (compacted)") { | |||
| 715 | + | t.Fatalf("resumed transcript = %#v, want the checkpoint plus the kept tail", messages) | |||
| 716 | + | } | |||
| 717 | + | bodies := script.capturedBodies() | |||
| 718 | + | if len(bodies) != 5 || !strings.Contains(bodies[4], "compacted digest") { | |||
| 719 | + | t.Fatalf("continuation request = %q, want the compacted transcript", bodies[len(bodies)-1]) | |||
| 720 | + | } | |||
| 721 | + | if strings.Contains(bodies[4], "call-1") { | |||
| 722 | + | t.Fatalf("continuation request kept the dropped tool exchange: %q", bodies[4]) | |||
| 723 | + | } | |||
| 724 | + | } | |||
| 725 | + | ||||
| 726 | + | func TestResidentChildRunnerDoesNotRecoverTwicePerTurn(t *testing.T) { | |||
| 727 | + | script := &residentOverflowScript{toolCallRequests: 2, secondOverflow: true, finalText: "unreached"} | |||
| 728 | + | server := script.start(t) | |||
| 729 | + | runner, journal, _ := newResidentChildTestRunner(t, "double-overflow", residentArguments(t, server.URL)) | |||
| 730 | + | err := runner(t.Context(), "review") | |||
| 731 | + | if err == nil || !provider.IsContextOverflowError(err) { | |||
| 732 | + | t.Fatalf("runner error = %v, want the second overflow to stay terminal", err) | |||
| 733 | + | } | |||
| 734 | + | if got := script.requests.Load(); got != 5 { | |||
| 735 | + | t.Fatalf("provider requests = %d, want no second compaction", got) | |||
| 736 | + | } | |||
| 737 | + | if checkpoints := countResidentCheckpoints(t, journal); checkpoints != 1 { | |||
| 738 | + | t.Fatalf("compaction checkpoints = %d, want 1", checkpoints) | |||
| 739 | + | } | |||
| 740 | + | } | |||
| 741 | + | ||||
| 742 | + | func TestResidentChildRunnerDoesNotReplenishExplicitStepLimit(t *testing.T) { | |||
| 743 | + | script := &residentOverflowScript{toolCallRequests: 3, finalText: "unreached"} | |||
| 744 | + | server := script.start(t) | |||
| 745 | + | args := residentArguments(t, server.URL) | |||
| 746 | + | args.MaxSteps = 4 | |||
| 747 | + | runner, journal, _ := newResidentChildTestRunner(t, "limited-overflow", args) | |||
| 748 | + | err := runner(t.Context(), "review") | |||
| 749 | + | if err == nil || !provider.IsContextOverflowError(err) { | |||
| 750 | + | t.Fatalf("runner error = %v, want the overflow", err) | |||
| 751 | + | } | |||
| 752 | + | if got := script.requests.Load(); got != 5 { | |||
| 753 | + | t.Fatalf("provider requests = %d, want the compaction attempt only", got) | |||
| 754 | + | } | |||
| 755 | + | // The checkpoint is still journaled so an explicit resume starts small, | |||
| 756 | + | // but the spent allowance forbids continuing this turn. | |||
| 757 | + | if checkpoints := countResidentCheckpoints(t, journal); checkpoints != 1 { | |||
| 758 | + | t.Fatalf("compaction checkpoints = %d, want 1", checkpoints) | |||
| 759 | + | } | |||
| 760 | + | for _, body := range script.capturedBodies() { | |||
| 761 | + | if strings.Contains(body, "compacted digest") { | |||
| 762 | + | t.Fatalf("recovery continued after the step allowance was spent: %q", body) | |||
| 763 | + | } | |||
| 764 | + | } | |||
| 765 | + | } | |||
| 766 | + | ||||
| 767 | + | func TestResidentChildRunnerFailsTurnWhenCheckpointCannotPersist(t *testing.T) { | |||
| 768 | + | root := t.TempDir() | |||
| 769 | + | spec := subagents.ResidentChildSpec{ | |||
| 770 | + | ID: "persist-failure", SessionID: "persist-failure-session", Provider: "openai", Model: "gpt-4o", | |||
| 771 | + | Tools: []string{"read"}, | |||
| 772 | + | } | |||
| 773 | + | journal, err := subagents.OpenResidentJournal(root, spec.ID) | |||
| 774 | + | if err != nil { | |||
| 775 | + | t.Fatal(err) | |||
| 776 | + | } | |||
| 777 | + | t.Cleanup(func() { _ = journal.Close() }) | |||
| 778 | + | if err := journal.Accept(spec, "review"); err != nil { | |||
| 779 | + | t.Fatal(err) | |||
| 780 | + | } | |||
| 781 | + | // The journal is closed at the exact overflow boundary, so the recovery | |||
| 782 | + | // cannot persist its checkpoint. | |||
| 783 | + | script := &residentOverflowScript{ | |||
| 784 | + | toolCallRequests: 2, | |||
| 785 | + | finalText: "unreached", | |||
| 786 | + | onOverflow: func() { _ = journal.Close() }, | |||
| 787 | + | } | |||
| 788 | + | server := script.start(t) | |||
| 789 | + | runner, err := newResidentChildRunner(residentArguments(t, server.URL), spec, journal) | |||
| 790 | + | if err != nil { | |||
| 791 | + | t.Fatal(err) | |||
| 792 | + | } | |||
| 793 | + | if err := runner(t.Context(), "review"); err == nil { | |||
| 794 | + | t.Fatal("runner recovered without persisting its checkpoint") | |||
| 795 | + | } | |||
| 796 | + | if got := script.requests.Load(); got != 4 { | |||
| 797 | + | t.Fatalf("provider requests = %d, want the compaction attempt only", got) | |||
| 798 | + | } | |||
| 799 | + | data, readErr := os.ReadFile(filepath.Join(journal.Dir(), "transcript.jsonl")) | |||
| 800 | + | if readErr != nil { | |||
| 801 | + | t.Fatal(readErr) | |||
| 802 | + | } | |||
| 803 | + | if strings.Contains(string(data), `"type":"child.compacted"`) { | |||
| 804 | + | t.Fatal("a failed recovery recorded a checkpoint") | |||
| 805 | + | } | |||
| 806 | + | ||||
| 807 | + | // A later reconstructed child resumes the pre-compaction transcript. | |||
| 808 | + | reopened, err := subagents.OpenResidentJournal(root, spec.ID) | |||
| 809 | + | if err != nil { | |||
| 810 | + | t.Fatal(err) | |||
| 811 | + | } | |||
| 812 | + | t.Cleanup(func() { _ = reopened.Close() }) | |||
| 813 | + | runner, err = newResidentChildRunner(residentArguments(t, server.URL), spec, reopened) | |||
| 814 | + | if err != nil { | |||
| 815 | + | t.Fatal(err) | |||
| 816 | + | } | |||
| 817 | + | if err := runner(t.Context(), "verify"); err != nil { | |||
| 818 | + | t.Fatalf("follow-up error = %v", err) | |||
| 819 | + | } | |||
| 820 | + | bodies := script.capturedBodies() | |||
| 821 | + | if last := bodies[len(bodies)-1]; !strings.Contains(last, "call-1") { | |||
| 822 | + | t.Fatalf("follow-up request = %q, want the pre-compaction transcript", last) | |||
| 823 | + | } | |||
| 824 | + | } | |||
| 825 | + | ||||
| 826 | + | func TestRecoverResidentContextOverflowRestoresTranscript(t *testing.T) { | |||
| 827 | + | server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | |||
| 828 | + | w.Header().Set("content-type", "text/event-stream") | |||
| 829 | + | writeOpenAIChunk(t, w, map[string]any{ | |||
| 830 | + | "choices": []any{map[string]any{"index": 0, "delta": map[string]string{"content": "compacted digest"}, "finish_reason": "stop"}}, | |||
| 831 | + | "usage": map[string]int{"prompt_tokens": 5, "completion_tokens": 2}, | |||
| 832 | + | }) | |||
| 833 | + | })) | |||
| 834 | + | defer server.Close() | |||
| 835 | + | resolved, err := Resolve(residentArguments(t, server.URL), true) | |||
| 836 | + | if err != nil { | |||
| 837 | + | t.Fatal(err) | |||
| 838 | + | } | |||
| 839 | + | agent := core.NewAgent(resolved.NewClient(), resolved.Model, "", resolved.ToolRegistry) | |||
| 840 | + | before := []provider.Message{ | |||
| 841 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "task"}}}, | |||
| 842 | + | {Role: provider.RoleAssistant, Content: []provider.Content{provider.ToolCallBlock{ID: "call-1", Name: "read", Arguments: json.RawMessage(`{"path":"x"}`)}}}, | |||
| 843 | + | {Role: provider.RoleTool, Content: []provider.Content{provider.ToolResultBlock{CallID: "call-1", Content: []provider.Content{provider.TextBlock{Text: "missing"}}}}}, | |||
| 844 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "keep working"}}}, | |||
| 845 | + | } | |||
| 846 | + | agent.SetMessages(before) | |||
| 847 | + | ||||
| 848 | + | journal, err := subagents.OpenResidentJournal(t.TempDir(), "restore-child") | |||
| 849 | + | if err != nil { | |||
| 850 | + | t.Fatal(err) | |||
| 851 | + | } | |||
| 852 | + | if err := journal.Accept(subagents.ResidentChildSpec{ID: "restore-child", SessionID: "restore-session", Provider: "openai", Model: "gpt-4o"}, "task"); err != nil { | |||
| 853 | + | t.Fatal(err) | |||
| 854 | + | } | |||
| 855 | + | if err := journal.Close(); err != nil { | |||
| 856 | + | t.Fatal(err) | |||
| 857 | + | } | |||
| 858 | + | ||||
| 859 | + | overflow := errors.New("openai: http 413: context_length_exceeded") | |||
| 860 | + | err = recoverResidentContextOverflow(t.Context(), agent, journal, func(core.AgentEvent) {}, 0, 4, overflow) | |||
| 861 | + | if !errors.Is(err, overflow) || !strings.Contains(err.Error(), "persist resident child compaction") { | |||
| 862 | + | t.Fatalf("recovery error = %v, want a checkpoint persistence failure", err) | |||
| 863 | + | } | |||
| 864 | + | if !reflect.DeepEqual(agent.Messages(), before) { | |||
| 865 | + | t.Fatalf("transcript = %#v, want the pre-compaction transcript restored", agent.Messages()) | |||
| 866 | + | } | |||
| 867 | + | } | |||
| 868 | + | ||||
| 869 | + | func residentMessageText(message provider.Message) string { | |||
| 870 | + | var text strings.Builder | |||
| 871 | + | for _, content := range message.Content { | |||
| 872 | + | if block, ok := content.(provider.TextBlock); ok { | |||
| 873 | + | text.WriteString(block.Text) | |||
| 874 | + | } | |||
| 875 | + | } | |||
| 876 | + | return text.String() | |||
| 877 | + | } | |||
| 878 | + | ||||
| 879 | + | func TestResidentChildRunnerIgnoresCompactionUsageInReminder(t *testing.T) { | |||
| 880 | + | // The summarization request reports the pre-compaction prompt size. If that | |||
| 881 | + | // gauge armed the next reminder, the first continuation step would claim the | |||
| 882 | + | // compacted transcript sits at the top of the window. The continuation's own | |||
| 883 | + | // usage must still be able to arm a truthful reminder. | |||
| 884 | + | script := &residentOverflowScript{ | |||
| 885 | + | toolCallRequests: 2, | |||
| 886 | + | continuationToolCalls: 1, | |||
| 887 | + | finalText: "overflow recovered result", | |||
| 888 | + | summaryUsageTokens: 1 << 20, | |||
| 889 | + | continuationUsageTokens: 1 << 20, | |||
| 890 | + | } | |||
| 891 | + | server := script.start(t) | |||
| 892 | + | runner, _, _ := newResidentChildTestRunner(t, "usage-child", residentArguments(t, server.URL)) | |||
| 893 | + | if err := runner(t.Context(), "review"); err != nil { | |||
| 894 | + | t.Fatalf("runner error = %v, want one recovered turn", err) | |||
| 895 | + | } | |||
| 896 | + | bodies := script.capturedBodies() | |||
| 897 | + | if len(bodies) != 6 { | |||
| 898 | + | t.Fatalf("provider requests = %d, want 3 prompt, 1 compaction, and 2 continued steps", len(bodies)) | |||
| 899 | + | } | |||
| 900 | + | if strings.Contains(bodies[4], "Context usage is at") { | |||
| 901 | + | t.Fatalf("first continuation step carries a reminder from the compaction request: %q", bodies[4]) | |||
| 902 | + | } | |||
| 903 | + | if !strings.Contains(bodies[5], "Context usage is at") { | |||
| 904 | + | t.Fatalf("second continuation step lost the reminder: %q", bodies[5]) | |||
| 905 | + | } | |||
| 906 | + | } | |||
| 907 | + | ||||
| 908 | + | func TestResidentCheckpointMessagesDropsOnlyHostContext(t *testing.T) { | |||
| 909 | + | internal := provider.Message{ | |||
| 910 | + | Role: provider.RoleDeveloper, | |||
| 911 | + | Meta: map[string]string{"internal_context": "true"}, | |||
| 912 | + | Content: []provider.Content{provider.TextBlock{Text: "Context usage is at 96% of the model window."}}, | |||
| 913 | + | } | |||
| 914 | + | importedNote := provider.Message{ | |||
| 915 | + | Role: provider.RoleDeveloper, | |||
| 916 | + | Content: []provider.Content{provider.TextBlock{Text: "imported developer note"}}, | |||
| 917 | + | } | |||
| 918 | + | task := provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "task"}}} | |||
| 919 | + | kept := residentCheckpointMessages([]provider.Message{internal, importedNote, task}) | |||
| 920 | + | if len(kept) != 2 || kept[0].Role != provider.RoleDeveloper || kept[1].Role != provider.RoleUser { | |||
| 921 | + | t.Fatalf("checkpoint messages = %#v, want host context dropped and other messages kept", kept) | |||
| 922 | + | } | |||
| 923 | + | if !core.IsInternalContextMessage(internal) || core.IsInternalContextMessage(importedNote) { | |||
| 924 | + | t.Fatal("IsInternalContextMessage misclassified a developer message") | |||
| 925 | + | } | |||
| 926 | + | } | |||
| 927 | + | ||||
| 928 | + | func TestResidentChildRunnerKeepsHostContextOutOfCheckpoint(t *testing.T) { | |||
| 929 | + | // Tool turns report a prompt size past the top band, so the child holds a | |||
| 930 | + | // host reminder when the overflow arrives. | |||
| 931 | + | script := &residentOverflowScript{ | |||
| 932 | + | toolCallRequests: 2, | |||
| 933 | + | finalText: "overflow recovered result", | |||
| 934 | + | toolUsageTokens: 1 << 20, | |||
| 935 | + | } | |||
| 936 | + | server := script.start(t) | |||
| 937 | + | runner, journal, _ := newResidentChildTestRunner(t, "checkpoint-context", residentArguments(t, server.URL)) | |||
| 938 | + | if err := runner(t.Context(), "review"); err != nil { | |||
| 939 | + | t.Fatalf("runner error = %v, want one recovered turn", err) | |||
| 940 | + | } | |||
| 941 | + | messages, err := subagents.ReadResidentTranscriptMessages(journal.Dir()) | |||
| 942 | + | if err != nil { | |||
| 943 | + | t.Fatal(err) | |||
| 944 | + | } | |||
| 945 | + | joined := joinedResidentText(messages) | |||
| 946 | + | if !strings.Contains(joined, "compacted digest") { | |||
| 947 | + | t.Fatalf("resumed transcript = %q, want the checkpoint digest", joined) | |||
| 948 | + | } | |||
| 949 | + | if strings.Contains(joined, "Context usage is at") { | |||
| 950 | + | t.Fatalf("checkpoint replayed a stale host reminder: %q", joined) | |||
| 951 | + | } | |||
| 952 | + | } | |||
| 953 | + | ||||
| 954 | + | func TestResidentChildRunnerKeepsRemainingStepAllowanceAcrossRecovery(t *testing.T) { | |||
| 955 | + | // Two tool turns and the overflow leave two of the five allowed steps. The | |||
| 956 | + | // continuation must receive exactly those, not a fresh limit. | |||
| 957 | + | script := &residentOverflowScript{ | |||
| 958 | + | toolCallRequests: 2, | |||
| 959 | + | continuationToolCalls: 10, | |||
| 960 | + | finalText: "unreached", | |||
| 961 | + | } | |||
| 962 | + | server := script.start(t) | |||
| 963 | + | args := residentArguments(t, server.URL) | |||
| 964 | + | args.MaxSteps = 5 | |||
| 965 | + | runner, journal, _ := newResidentChildTestRunner(t, "partial-allowance", args) | |||
| 966 | + | err := runner(t.Context(), "review") | |||
| 967 | + | if !errors.Is(err, core.ErrMaxSteps) { | |||
| 968 | + | t.Fatalf("runner error = %v, want the continuation to exhaust the remaining allowance", err) | |||
| 969 | + | } | |||
| 970 | + | if got := script.requests.Load(); got != 6 { | |||
| 971 | + | t.Fatalf("provider requests = %d, want 2 tool turns, the overflow, the compaction, and 2 more steps", got) | |||
| 972 | + | } | |||
| 973 | + | if checkpoints := countResidentCheckpoints(t, journal); checkpoints != 1 { | |||
| 974 | + | t.Fatalf("compaction checkpoints = %d, want 1", checkpoints) | |||
| 975 | + | } | |||
| 976 | + | } | |||
| 977 | + | ||||
| 978 | + | func TestResidentChildRunnerDoesNotCompactWithoutHistory(t *testing.T) { | |||
| 979 | + | // The overflow arrives on the first request, so the transcript is too short | |||
| 980 | + | // to summarize without replacing the task itself. | |||
| 981 | + | script := &residentOverflowScript{toolCallRequests: 0, finalText: "unreached"} | |||
| 982 | + | server := script.start(t) | |||
| 983 | + | runner, journal, _ := newResidentChildTestRunner(t, "short-overflow", residentArguments(t, server.URL)) | |||
| 984 | + | err := runner(t.Context(), "review") | |||
| 985 | + | if err == nil || !provider.IsContextOverflowError(err) || !strings.Contains(err.Error(), "context_length_exceeded") { | |||
| 986 | + | t.Fatalf("runner error = %v, want the original overflow", err) | |||
| 987 | + | } | |||
| 988 | + | if got := script.requests.Load(); got != 1 { | |||
| 989 | + | t.Fatalf("provider requests = %d, want no compaction attempt", got) | |||
| 990 | + | } | |||
| 991 | + | if checkpoints := countResidentCheckpoints(t, journal); checkpoints != 0 { | |||
| 992 | + | t.Fatalf("compaction checkpoints = %d, want none without history", checkpoints) | |||
| 993 | + | } | |||
| 994 | + | } | |||
| 995 | + | ||||
| 996 | + | func joinedResidentText(messages []provider.Message) string { | |||
| 997 | + | var text strings.Builder | |||
| 998 | + | for _, message := range messages { | |||
| 999 | + | text.WriteString(residentMessageText(message)) | |||
| 1000 | + | text.WriteString("\n") | |||
| 1001 | + | } | |||
| 1002 | + | return text.String() | |||
| 1003 | + | } | |||
M packages/agent/subagents/resident_history.go +72 -7
| @@ -29,13 +29,18 @@ const ( | |||
| 29 | 29 | // raw message and tool payloads are preserved so callers can render complete | |
| 30 | 30 | // arguments/results without relying on a lossy text projection. | |
| 31 | 31 | type ResidentHistoryItem struct { | |
| 32 | - | Type string `json:"type"` | |
| 33 | - | Time time.Time `json:"time"` | |
| 34 | - | Message json.RawMessage `json:"message,omitempty"` | |
| 35 | - | ToolID string `json:"tool_id,omitempty"` | |
| 36 | - | ToolName string `json:"tool_name,omitempty"` | |
| 37 | - | ToolArgs json.RawMessage `json:"tool_args,omitempty"` | |
| 38 | - | ToolResult json.RawMessage `json:"tool_result,omitempty"` | |
| 32 | + | Type string `json:"type"` | |
| 33 | + | Time time.Time `json:"time"` | |
| 34 | + | Message json.RawMessage `json:"message,omitempty"` | |
| 35 | + | // Messages carries a compaction checkpoint: the complete replacement | |
| 36 | + | // transcript of a child that recovered from a context overflow. It is | |
| 37 | + | // produced only for residentRecordCompacted items, which the UI pager does | |
| 38 | + | // not expose. | |
| 39 | + | Messages []json.RawMessage `json:"messages,omitempty"` | |
| 40 | + | ToolID string `json:"tool_id,omitempty"` | |
| 41 | + | ToolName string `json:"tool_name,omitempty"` | |
| 42 | + | ToolArgs json.RawMessage `json:"tool_args,omitempty"` | |
| 43 | + | ToolResult json.RawMessage `json:"tool_result,omitempty"` | |
| 39 | 44 | } | |
| 40 | 45 | ||
| 41 | 46 | // ResidentHistoryPage is a recent-first, bounded page of finalized history. | |
| @@ -273,6 +278,9 @@ func residentHistoryItem(record residentRecord) (ResidentHistoryItem, bool, erro | |||
| 273 | 278 | } | |
| 274 | 279 | item.ToolID, item.ToolResult = record.ToolID, append(json.RawMessage(nil), record.ToolResult...) | |
| 275 | 280 | default: | |
| 281 | + | // Lifecycle records and compaction checkpoints are not history items: | |
| 282 | + | // the pager keeps showing the append-only log, while resume replay | |
| 283 | + | // starts from the newest checkpoint (see ReadResidentTranscriptMessages). | |
| 276 | 284 | return ResidentHistoryItem{}, false, nil | |
| 277 | 285 | } | |
| 278 | 286 | return item, true, nil | |
| @@ -297,15 +305,49 @@ func residentHistoryPageStart(entries []residentHistoryEntry, limit int) int { | |||
| 297 | 305 | ||
| 298 | 306 | func cloneResidentHistoryItem(item ResidentHistoryItem) ResidentHistoryItem { | |
| 299 | 307 | item.Message = append(json.RawMessage(nil), item.Message...) | |
| 308 | + | item.Messages = cloneRawMessages(item.Messages) | |
| 300 | 309 | item.ToolArgs = append(json.RawMessage(nil), item.ToolArgs...) | |
| 301 | 310 | item.ToolResult = append(json.RawMessage(nil), item.ToolResult...) | |
| 302 | 311 | return item | |
| 303 | 312 | } | |
| 304 | 313 | ||
| 314 | + | func cloneRawMessages(messages []json.RawMessage) []json.RawMessage { | |
| 315 | + | if messages == nil { | |
| 316 | + | return nil | |
| 317 | + | } | |
| 318 | + | cloned := make([]json.RawMessage, len(messages)) | |
| 319 | + | for index, message := range messages { | |
| 320 | + | cloned[index] = append(json.RawMessage(nil), message...) | |
| 321 | + | } | |
| 322 | + | return cloned | |
| 323 | + | } | |
| 324 | + | ||
| 325 | + | // decodeResidentCompactedMessages decodes one compaction checkpoint. The | |
| 326 | + | // checkpoint is the complete replacement transcript, so an empty or malformed | |
| 327 | + | // payload is corruption rather than an empty transcript. | |
| 328 | + | func decodeResidentCompactedMessages(encoded []json.RawMessage) ([]provider.Message, error) { | |
| 329 | + | if len(encoded) == 0 { | |
| 330 | + | return nil, errors.New("empty compaction checkpoint") | |
| 331 | + | } | |
| 332 | + | messages := make([]provider.Message, 0, len(encoded)) | |
| 333 | + | for _, raw := range encoded { | |
| 334 | + | message, err := core.DecodeMessageJSON(raw) | |
| 335 | + | if err != nil { | |
| 336 | + | return nil, fmt.Errorf("decode compaction checkpoint message: %w", err) | |
| 337 | + | } | |
| 338 | + | messages = append(messages, message) | |
| 339 | + | } | |
| 340 | + | return messages, nil | |
| 341 | + | } | |
| 342 | + | ||
| 305 | 343 | // ResidentHistoryMessages converts finalized journal items into the same | |
| 306 | 344 | // provider-neutral message structures used by tui.View. Tool-call records | |
| 307 | 345 | // duplicate calls already embedded in an assistant message, so they are only | |
| 308 | 346 | // synthesized when the assistant record is absent (for recovery resilience). | |
| 347 | + | // | |
| 348 | + | // Items are one ordered log, not independent groups: a checkpoint item | |
| 349 | + | // replaces every message accumulated before it. A bounded history page never | |
| 350 | + | // contains one, because the pager excludes checkpoints. | |
| 309 | 351 | func ResidentHistoryMessages(items []ResidentHistoryItem) ([]provider.Message, error) { | |
| 310 | 352 | messages := make([]provider.Message, 0, len(items)) | |
| 311 | 353 | calls := make(map[string]struct{}) | |
| @@ -322,6 +364,24 @@ func ResidentHistoryMessages(items []ResidentHistoryItem) ([]provider.Message, e | |||
| 322 | 364 | } | |
| 323 | 365 | } | |
| 324 | 366 | messages = append(messages, message) | |
| 367 | + | case residentRecordCompacted: | |
| 368 | + | // A checkpoint replaces the transcript accumulated so far: the | |
| 369 | + | // resumed model must see the compacted view, not the full | |
| 370 | + | // append-only log. Tool-call bookkeeping starts over from it so a | |
| 371 | + | // call dropped by compaction is not re-synthesized below. | |
| 372 | + | checkpoint, err := decodeResidentCompactedMessages(item.Messages) | |
| 373 | + | if err != nil { | |
| 374 | + | return nil, fmt.Errorf("resident history: %w", err) | |
| 375 | + | } | |
| 376 | + | messages = checkpoint | |
| 377 | + | calls = make(map[string]struct{}) | |
| 378 | + | for _, message := range checkpoint { | |
| 379 | + | for _, content := range message.Content { | |
| 380 | + | if call, ok := content.(provider.ToolCallBlock); ok { | |
| 381 | + | calls[call.ID] = struct{}{} | |
| 382 | + | } | |
| 383 | + | } | |
| 384 | + | } | |
| 325 | 385 | case residentRecordToolCall: | |
| 326 | 386 | if _, exists := calls[item.ToolID]; exists { | |
| 327 | 387 | continue | |
| @@ -378,6 +438,11 @@ func ReadResidentTranscriptMessages(dir string) ([]provider.Message, error) { | |||
| 378 | 438 | } | |
| 379 | 439 | items := make([]ResidentHistoryItem, 0, len(records)) | |
| 380 | 440 | for _, record := range records { | |
| 441 | + | if record.Type == residentRecordCompacted { | |
| 442 | + | // The pager hides checkpoints, but resume replay starts from one. | |
| 443 | + | items = append(items, ResidentHistoryItem{Type: record.Type, Time: record.Time, Messages: cloneRawMessages(record.Messages)}) | |
| 444 | + | continue | |
| 445 | + | } | |
| 381 | 446 | item, include, err := residentHistoryItem(record) | |
| 382 | 447 | if err != nil { | |
| 383 | 448 | return nil, err | |
| @@ -29,13 +29,18 @@ const ( | |||||
| 29 | // raw message and tool payloads are preserved so callers can render complete | 29 | // raw message and tool payloads are preserved so callers can render complete | ||
| 30 | // arguments/results without relying on a lossy text projection. | 30 | // arguments/results without relying on a lossy text projection. | ||
| 31 | type ResidentHistoryItem struct { | 31 | type ResidentHistoryItem struct { | ||
| 32 | - | Type string `json:"type"` | 32 | + | Type string `json:"type"` |
| 33 | - | Time time.Time `json:"time"` | 33 | + | Time time.Time `json:"time"` |
| 34 | - | Message json.RawMessage `json:"message,omitempty"` | 34 | + | Message json.RawMessage `json:"message,omitempty"` |
| 35 | - | ToolID string `json:"tool_id,omitempty"` | 35 | + | // Messages carries a compaction checkpoint: the complete replacement |
| 36 | - | ToolName string `json:"tool_name,omitempty"` | 36 | + | // transcript of a child that recovered from a context overflow. It is |
| 37 | - | ToolArgs json.RawMessage `json:"tool_args,omitempty"` | 37 | + | // produced only for residentRecordCompacted items, which the UI pager does |
| 38 | - | ToolResult json.RawMessage `json:"tool_result,omitempty"` | 38 | + | // not expose. |
| 39 | + | Messages []json.RawMessage `json:"messages,omitempty"` | |||
| 40 | + | ToolID string `json:"tool_id,omitempty"` | |||
| 41 | + | ToolName string `json:"tool_name,omitempty"` | |||
| 42 | + | ToolArgs json.RawMessage `json:"tool_args,omitempty"` | |||
| 43 | + | ToolResult json.RawMessage `json:"tool_result,omitempty"` | |||
| 39 | } | 44 | } | ||
| 40 | 45 | ||||
| 41 | // ResidentHistoryPage is a recent-first, bounded page of finalized history. | 46 | // ResidentHistoryPage is a recent-first, bounded page of finalized history. | ||
| @@ -273,6 +278,9 @@ func residentHistoryItem(record residentRecord) (ResidentHistoryItem, bool, erro | |||||
| 273 | } | 278 | } | ||
| 274 | item.ToolID, item.ToolResult = record.ToolID, append(json.RawMessage(nil), record.ToolResult...) | 279 | item.ToolID, item.ToolResult = record.ToolID, append(json.RawMessage(nil), record.ToolResult...) | ||
| 275 | default: | 280 | default: | ||
| 281 | + | // Lifecycle records and compaction checkpoints are not history items: | |||
| 282 | + | // the pager keeps showing the append-only log, while resume replay | |||
| 283 | + | // starts from the newest checkpoint (see ReadResidentTranscriptMessages). | |||
| 276 | return ResidentHistoryItem{}, false, nil | 284 | return ResidentHistoryItem{}, false, nil | ||
| 277 | } | 285 | } | ||
| 278 | return item, true, nil | 286 | return item, true, nil | ||
| @@ -297,15 +305,49 @@ func residentHistoryPageStart(entries []residentHistoryEntry, limit int) int { | |||||
| 297 | 305 | ||||
| 298 | func cloneResidentHistoryItem(item ResidentHistoryItem) ResidentHistoryItem { | 306 | func cloneResidentHistoryItem(item ResidentHistoryItem) ResidentHistoryItem { | ||
| 299 | item.Message = append(json.RawMessage(nil), item.Message...) | 307 | item.Message = append(json.RawMessage(nil), item.Message...) | ||
| 308 | + | item.Messages = cloneRawMessages(item.Messages) | |||
| 300 | item.ToolArgs = append(json.RawMessage(nil), item.ToolArgs...) | 309 | item.ToolArgs = append(json.RawMessage(nil), item.ToolArgs...) | ||
| 301 | item.ToolResult = append(json.RawMessage(nil), item.ToolResult...) | 310 | item.ToolResult = append(json.RawMessage(nil), item.ToolResult...) | ||
| 302 | return item | 311 | return item | ||
| 303 | } | 312 | } | ||
| 304 | 313 | ||||
| 314 | + | func cloneRawMessages(messages []json.RawMessage) []json.RawMessage { | |||
| 315 | + | if messages == nil { | |||
| 316 | + | return nil | |||
| 317 | + | } | |||
| 318 | + | cloned := make([]json.RawMessage, len(messages)) | |||
| 319 | + | for index, message := range messages { | |||
| 320 | + | cloned[index] = append(json.RawMessage(nil), message...) | |||
| 321 | + | } | |||
| 322 | + | return cloned | |||
| 323 | + | } | |||
| 324 | + | ||||
| 325 | + | // decodeResidentCompactedMessages decodes one compaction checkpoint. The | |||
| 326 | + | // checkpoint is the complete replacement transcript, so an empty or malformed | |||
| 327 | + | // payload is corruption rather than an empty transcript. | |||
| 328 | + | func decodeResidentCompactedMessages(encoded []json.RawMessage) ([]provider.Message, error) { | |||
| 329 | + | if len(encoded) == 0 { | |||
| 330 | + | return nil, errors.New("empty compaction checkpoint") | |||
| 331 | + | } | |||
| 332 | + | messages := make([]provider.Message, 0, len(encoded)) | |||
| 333 | + | for _, raw := range encoded { | |||
| 334 | + | message, err := core.DecodeMessageJSON(raw) | |||
| 335 | + | if err != nil { | |||
| 336 | + | return nil, fmt.Errorf("decode compaction checkpoint message: %w", err) | |||
| 337 | + | } | |||
| 338 | + | messages = append(messages, message) | |||
| 339 | + | } | |||
| 340 | + | return messages, nil | |||
| 341 | + | } | |||
| 342 | + | ||||
| 305 | // ResidentHistoryMessages converts finalized journal items into the same | 343 | // ResidentHistoryMessages converts finalized journal items into the same | ||
| 306 | // provider-neutral message structures used by tui.View. Tool-call records | 344 | // provider-neutral message structures used by tui.View. Tool-call records | ||
| 307 | // duplicate calls already embedded in an assistant message, so they are only | 345 | // duplicate calls already embedded in an assistant message, so they are only | ||
| 308 | // synthesized when the assistant record is absent (for recovery resilience). | 346 | // synthesized when the assistant record is absent (for recovery resilience). | ||
| 347 | + | // | |||
| 348 | + | // Items are one ordered log, not independent groups: a checkpoint item | |||
| 349 | + | // replaces every message accumulated before it. A bounded history page never | |||
| 350 | + | // contains one, because the pager excludes checkpoints. | |||
| 309 | func ResidentHistoryMessages(items []ResidentHistoryItem) ([]provider.Message, error) { | 351 | func ResidentHistoryMessages(items []ResidentHistoryItem) ([]provider.Message, error) { | ||
| 310 | messages := make([]provider.Message, 0, len(items)) | 352 | messages := make([]provider.Message, 0, len(items)) | ||
| 311 | calls := make(map[string]struct{}) | 353 | calls := make(map[string]struct{}) | ||
| @@ -322,6 +364,24 @@ func ResidentHistoryMessages(items []ResidentHistoryItem) ([]provider.Message, e | |||||
| 322 | } | 364 | } | ||
| 323 | } | 365 | } | ||
| 324 | messages = append(messages, message) | 366 | messages = append(messages, message) | ||
| 367 | + | case residentRecordCompacted: | |||
| 368 | + | // A checkpoint replaces the transcript accumulated so far: the | |||
| 369 | + | // resumed model must see the compacted view, not the full | |||
| 370 | + | // append-only log. Tool-call bookkeeping starts over from it so a | |||
| 371 | + | // call dropped by compaction is not re-synthesized below. | |||
| 372 | + | checkpoint, err := decodeResidentCompactedMessages(item.Messages) | |||
| 373 | + | if err != nil { | |||
| 374 | + | return nil, fmt.Errorf("resident history: %w", err) | |||
| 375 | + | } | |||
| 376 | + | messages = checkpoint | |||
| 377 | + | calls = make(map[string]struct{}) | |||
| 378 | + | for _, message := range checkpoint { | |||
| 379 | + | for _, content := range message.Content { | |||
| 380 | + | if call, ok := content.(provider.ToolCallBlock); ok { | |||
| 381 | + | calls[call.ID] = struct{}{} | |||
| 382 | + | } | |||
| 383 | + | } | |||
| 384 | + | } | |||
| 325 | case residentRecordToolCall: | 385 | case residentRecordToolCall: | ||
| 326 | if _, exists := calls[item.ToolID]; exists { | 386 | if _, exists := calls[item.ToolID]; exists { | ||
| 327 | continue | 387 | continue | ||
| @@ -378,6 +438,11 @@ func ReadResidentTranscriptMessages(dir string) ([]provider.Message, error) { | |||||
| 378 | } | 438 | } | ||
| 379 | items := make([]ResidentHistoryItem, 0, len(records)) | 439 | items := make([]ResidentHistoryItem, 0, len(records)) | ||
| 380 | for _, record := range records { | 440 | for _, record := range records { | ||
| 441 | + | if record.Type == residentRecordCompacted { | |||
| 442 | + | // The pager hides checkpoints, but resume replay starts from one. | |||
| 443 | + | items = append(items, ResidentHistoryItem{Type: record.Type, Time: record.Time, Messages: cloneRawMessages(record.Messages)}) | |||
| 444 | + | continue | |||
| 445 | + | } | |||
| 381 | item, include, err := residentHistoryItem(record) | 446 | item, include, err := residentHistoryItem(record) | ||
| 382 | if err != nil { | 447 | if err != nil { | ||
| 383 | return nil, err | 448 | return nil, err | ||
M packages/agent/subagents/resident_history_test.go +154 -0
| @@ -5,6 +5,7 @@ import ( | |||
| 5 | 5 | "encoding/json" | |
| 6 | 6 | "os" | |
| 7 | 7 | "path/filepath" | |
| 8 | + | "strings" | |
| 8 | 9 | "testing" | |
| 9 | 10 | ||
| 10 | 11 | "github.com/bnema/zut/packages/core" | |
| @@ -244,3 +245,156 @@ func TestResidentManagerHistoryRejectsPathTraversal(t *testing.T) { | |||
| 244 | 245 | } | |
| 245 | 246 | } | |
| 246 | 247 | } | |
| 248 | + | ||
| 249 | + | func TestReadResidentTranscriptMessagesReplaysCompactionCheckpoint(t *testing.T) { | |
| 250 | + | root := t.TempDir() | |
| 251 | + | journal, err := OpenResidentJournal(root, "compacted") | |
| 252 | + | if err != nil { | |
| 253 | + | t.Fatal(err) | |
| 254 | + | } | |
| 255 | + | spec := ResidentChildSpec{ID: "compacted", SessionID: "compacted-session", Provider: "openai", Model: "gpt-5"} | |
| 256 | + | if err := journal.Accept(spec, "task"); err != nil { | |
| 257 | + | t.Fatal(err) | |
| 258 | + | } | |
| 259 | + | if err := journal.RecordAgentEvent(core.EvUserMessage{Message: provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "old question"}}}}); err != nil { | |
| 260 | + | t.Fatal(err) | |
| 261 | + | } | |
| 262 | + | if err := journal.RecordAgentEvent(core.EvToolCall{ID: "dropped-call", Name: "bash", Args: json.RawMessage(`{"command":"pwd"}`)}); err != nil { | |
| 263 | + | t.Fatal(err) | |
| 264 | + | } | |
| 265 | + | if err := journal.RecordAgentEvent(core.EvToolResult{ID: "dropped-call", Result: core.ToolResult{Content: []provider.Content{provider.TextBlock{Text: "/repo"}}}}); err != nil { | |
| 266 | + | t.Fatal(err) | |
| 267 | + | } | |
| 268 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |
| 269 | + | t.Fatal(err) | |
| 270 | + | } | |
| 271 | + | if err := journal.RecordCompacted([]provider.Message{ | |
| 272 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "## Context Summary (compacted)\n\ndigest"}}}, | |
| 273 | + | }); err != nil { | |
| 274 | + | t.Fatal(err) | |
| 275 | + | } | |
| 276 | + | if err := journal.RecordAgentEvent(core.EvAssistantMessage{Message: provider.Message{Role: provider.RoleAssistant, Content: []provider.Content{provider.TextBlock{Text: "after checkpoint"}}}}); err != nil { | |
| 277 | + | t.Fatal(err) | |
| 278 | + | } | |
| 279 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |
| 280 | + | if err := journal.Close(); err != nil { | |
| 281 | + | t.Fatal(err) | |
| 282 | + | } | |
| 283 | + | ||
| 284 | + | records, err := ReadResidentJournal(transcript) | |
| 285 | + | if err != nil { | |
| 286 | + | t.Fatal(err) | |
| 287 | + | } | |
| 288 | + | var checkpoints []residentRecord | |
| 289 | + | for _, record := range records { | |
| 290 | + | if record.Type == residentRecordCompacted { | |
| 291 | + | checkpoints = append(checkpoints, record) | |
| 292 | + | } | |
| 293 | + | } | |
| 294 | + | if len(checkpoints) != 1 || checkpoints[0].TurnID != "turn-1" || len(checkpoints[0].Messages) != 1 { | |
| 295 | + | t.Fatalf("checkpoints = %#v, want one attributed to the active turn", checkpoints) | |
| 296 | + | } | |
| 297 | + | ||
| 298 | + | dir := filepath.Join(root, spec.ID) | |
| 299 | + | messages, err := ReadResidentTranscriptMessages(dir) | |
| 300 | + | if err != nil { | |
| 301 | + | t.Fatal(err) | |
| 302 | + | } | |
| 303 | + | if len(messages) != 2 || messages[0].Role != provider.RoleUser || messages[1].Role != provider.RoleAssistant { | |
| 304 | + | t.Fatalf("resumed messages = %#v, want the checkpoint followed by the later reply", messages) | |
| 305 | + | } | |
| 306 | + | if text := joinedResidentMessageText(messages); strings.Contains(text, "old question") { | |
| 307 | + | t.Fatalf("resume replayed pre-checkpoint content: %q", text) | |
| 308 | + | } | |
| 309 | + | if !strings.Contains(joinedResidentMessageText(messages[:1]), "Context Summary (compacted)") { | |
| 310 | + | t.Fatalf("resume lost the checkpoint summary: %#v", messages[0]) | |
| 311 | + | } | |
| 312 | + | for _, message := range messages { | |
| 313 | + | for _, content := range message.Content { | |
| 314 | + | if call, ok := content.(provider.ToolCallBlock); ok { | |
| 315 | + | t.Fatalf("resume re-synthesized compacted tool call %q", call.ID) | |
| 316 | + | } | |
| 317 | + | } | |
| 318 | + | } | |
| 319 | + | ||
| 320 | + | // The UI pager keeps the append-only view: a checkpoint is not a history | |
| 321 | + | // item, so paging still reaches the pre-checkpoint log. | |
| 322 | + | page, err := ReadResidentHistoryPage(dir, "", 10) | |
| 323 | + | if err != nil { | |
| 324 | + | t.Fatal(err) | |
| 325 | + | } | |
| 326 | + | if page.OlderCursor == "" { | |
| 327 | + | t.Fatal("page has no cursor for the pre-checkpoint log") | |
| 328 | + | } | |
| 329 | + | older, err := ReadResidentHistoryPage(dir, page.OlderCursor, 10) | |
| 330 | + | if err != nil { | |
| 331 | + | t.Fatal(err) | |
| 332 | + | } | |
| 333 | + | if len(older.Items) != 3 || older.Items[0].Type != residentRecordUser || !strings.Contains(string(older.Items[0].Message), "old question") { | |
| 334 | + | t.Fatalf("older page = %#v, want the append-only log before the checkpoint", older.Items) | |
| 335 | + | } | |
| 336 | + | for _, item := range append(append([]ResidentHistoryItem(nil), page.Items...), older.Items...) { | |
| 337 | + | if item.Type == residentRecordCompacted { | |
| 338 | + | t.Fatalf("pager exposed a compaction checkpoint: %#v", item) | |
| 339 | + | } | |
| 340 | + | } | |
| 341 | + | } | |
| 342 | + | ||
| 343 | + | func TestReadResidentTranscriptMessagesRejectsInvalidCheckpoint(t *testing.T) { | |
| 344 | + | for _, tc := range []struct { | |
| 345 | + | name string | |
| 346 | + | line string | |
| 347 | + | }{ | |
| 348 | + | {name: "empty", line: `{"version":3,"type":"child.compacted","time":"2026-01-01T00:00:00Z"}`}, | |
| 349 | + | {name: "undecodable", line: `{"version":3,"type":"child.compacted","time":"2026-01-01T00:00:00Z","messages":["not a message"]}`}, | |
| 350 | + | } { | |
| 351 | + | t.Run(tc.name, func(t *testing.T) { | |
| 352 | + | root := t.TempDir() | |
| 353 | + | journal, err := OpenResidentJournal(root, "bad-checkpoint") | |
| 354 | + | if err != nil { | |
| 355 | + | t.Fatal(err) | |
| 356 | + | } | |
| 357 | + | spec := ResidentChildSpec{ID: "bad-checkpoint", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |
| 358 | + | if err := journal.Accept(spec, "task"); err != nil { | |
| 359 | + | t.Fatal(err) | |
| 360 | + | } | |
| 361 | + | if err := journal.RecordAgentEvent(core.EvUserMessage{Message: provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "question"}}}}); err != nil { | |
| 362 | + | t.Fatal(err) | |
| 363 | + | } | |
| 364 | + | dir := journal.Dir() | |
| 365 | + | if err := journal.Close(); err != nil { | |
| 366 | + | t.Fatal(err) | |
| 367 | + | } | |
| 368 | + | path := filepath.Join(dir, residentTranscriptName) | |
| 369 | + | file, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600) | |
| 370 | + | if err != nil { | |
| 371 | + | t.Fatal(err) | |
| 372 | + | } | |
| 373 | + | if _, err := file.WriteString(tc.line + "\n"); err != nil { | |
| 374 | + | t.Fatal(err) | |
| 375 | + | } | |
| 376 | + | if err := file.Close(); err != nil { | |
| 377 | + | t.Fatal(err) | |
| 378 | + | } | |
| 379 | + | if _, err := ReadResidentTranscriptMessages(dir); err == nil { | |
| 380 | + | t.Fatal("invalid checkpoint was accepted") | |
| 381 | + | } | |
| 382 | + | if _, err := ReconcileResidentJournal(dir); err == nil { | |
| 383 | + | t.Fatal("reconciliation accepted an invalid checkpoint") | |
| 384 | + | } | |
| 385 | + | }) | |
| 386 | + | } | |
| 387 | + | } | |
| 388 | + | ||
| 389 | + | func joinedResidentMessageText(messages []provider.Message) string { | |
| 390 | + | var text strings.Builder | |
| 391 | + | for _, message := range messages { | |
| 392 | + | for _, content := range message.Content { | |
| 393 | + | if block, ok := content.(provider.TextBlock); ok { | |
| 394 | + | text.WriteString(block.Text) | |
| 395 | + | text.WriteString("\n") | |
| 396 | + | } | |
| 397 | + | } | |
| 398 | + | } | |
| 399 | + | return text.String() | |
| 400 | + | } | |
| @@ -5,6 +5,7 @@ import ( | |||||
| 5 | "encoding/json" | 5 | "encoding/json" | ||
| 6 | "os" | 6 | "os" | ||
| 7 | "path/filepath" | 7 | "path/filepath" | ||
| 8 | + | "strings" | |||
| 8 | "testing" | 9 | "testing" | ||
| 9 | 10 | ||||
| 10 | "github.com/bnema/zut/packages/core" | 11 | "github.com/bnema/zut/packages/core" | ||
| @@ -244,3 +245,156 @@ func TestResidentManagerHistoryRejectsPathTraversal(t *testing.T) { | |||||
| 244 | } | 245 | } | ||
| 245 | } | 246 | } | ||
| 246 | } | 247 | } | ||
| 248 | + | ||||
| 249 | + | func TestReadResidentTranscriptMessagesReplaysCompactionCheckpoint(t *testing.T) { | |||
| 250 | + | root := t.TempDir() | |||
| 251 | + | journal, err := OpenResidentJournal(root, "compacted") | |||
| 252 | + | if err != nil { | |||
| 253 | + | t.Fatal(err) | |||
| 254 | + | } | |||
| 255 | + | spec := ResidentChildSpec{ID: "compacted", SessionID: "compacted-session", Provider: "openai", Model: "gpt-5"} | |||
| 256 | + | if err := journal.Accept(spec, "task"); err != nil { | |||
| 257 | + | t.Fatal(err) | |||
| 258 | + | } | |||
| 259 | + | if err := journal.RecordAgentEvent(core.EvUserMessage{Message: provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "old question"}}}}); err != nil { | |||
| 260 | + | t.Fatal(err) | |||
| 261 | + | } | |||
| 262 | + | if err := journal.RecordAgentEvent(core.EvToolCall{ID: "dropped-call", Name: "bash", Args: json.RawMessage(`{"command":"pwd"}`)}); err != nil { | |||
| 263 | + | t.Fatal(err) | |||
| 264 | + | } | |||
| 265 | + | if err := journal.RecordAgentEvent(core.EvToolResult{ID: "dropped-call", Result: core.ToolResult{Content: []provider.Content{provider.TextBlock{Text: "/repo"}}}}); err != nil { | |||
| 266 | + | t.Fatal(err) | |||
| 267 | + | } | |||
| 268 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |||
| 269 | + | t.Fatal(err) | |||
| 270 | + | } | |||
| 271 | + | if err := journal.RecordCompacted([]provider.Message{ | |||
| 272 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "## Context Summary (compacted)\n\ndigest"}}}, | |||
| 273 | + | }); err != nil { | |||
| 274 | + | t.Fatal(err) | |||
| 275 | + | } | |||
| 276 | + | if err := journal.RecordAgentEvent(core.EvAssistantMessage{Message: provider.Message{Role: provider.RoleAssistant, Content: []provider.Content{provider.TextBlock{Text: "after checkpoint"}}}}); err != nil { | |||
| 277 | + | t.Fatal(err) | |||
| 278 | + | } | |||
| 279 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |||
| 280 | + | if err := journal.Close(); err != nil { | |||
| 281 | + | t.Fatal(err) | |||
| 282 | + | } | |||
| 283 | + | ||||
| 284 | + | records, err := ReadResidentJournal(transcript) | |||
| 285 | + | if err != nil { | |||
| 286 | + | t.Fatal(err) | |||
| 287 | + | } | |||
| 288 | + | var checkpoints []residentRecord | |||
| 289 | + | for _, record := range records { | |||
| 290 | + | if record.Type == residentRecordCompacted { | |||
| 291 | + | checkpoints = append(checkpoints, record) | |||
| 292 | + | } | |||
| 293 | + | } | |||
| 294 | + | if len(checkpoints) != 1 || checkpoints[0].TurnID != "turn-1" || len(checkpoints[0].Messages) != 1 { | |||
| 295 | + | t.Fatalf("checkpoints = %#v, want one attributed to the active turn", checkpoints) | |||
| 296 | + | } | |||
| 297 | + | ||||
| 298 | + | dir := filepath.Join(root, spec.ID) | |||
| 299 | + | messages, err := ReadResidentTranscriptMessages(dir) | |||
| 300 | + | if err != nil { | |||
| 301 | + | t.Fatal(err) | |||
| 302 | + | } | |||
| 303 | + | if len(messages) != 2 || messages[0].Role != provider.RoleUser || messages[1].Role != provider.RoleAssistant { | |||
| 304 | + | t.Fatalf("resumed messages = %#v, want the checkpoint followed by the later reply", messages) | |||
| 305 | + | } | |||
| 306 | + | if text := joinedResidentMessageText(messages); strings.Contains(text, "old question") { | |||
| 307 | + | t.Fatalf("resume replayed pre-checkpoint content: %q", text) | |||
| 308 | + | } | |||
| 309 | + | if !strings.Contains(joinedResidentMessageText(messages[:1]), "Context Summary (compacted)") { | |||
| 310 | + | t.Fatalf("resume lost the checkpoint summary: %#v", messages[0]) | |||
| 311 | + | } | |||
| 312 | + | for _, message := range messages { | |||
| 313 | + | for _, content := range message.Content { | |||
| 314 | + | if call, ok := content.(provider.ToolCallBlock); ok { | |||
| 315 | + | t.Fatalf("resume re-synthesized compacted tool call %q", call.ID) | |||
| 316 | + | } | |||
| 317 | + | } | |||
| 318 | + | } | |||
| 319 | + | ||||
| 320 | + | // The UI pager keeps the append-only view: a checkpoint is not a history | |||
| 321 | + | // item, so paging still reaches the pre-checkpoint log. | |||
| 322 | + | page, err := ReadResidentHistoryPage(dir, "", 10) | |||
| 323 | + | if err != nil { | |||
| 324 | + | t.Fatal(err) | |||
| 325 | + | } | |||
| 326 | + | if page.OlderCursor == "" { | |||
| 327 | + | t.Fatal("page has no cursor for the pre-checkpoint log") | |||
| 328 | + | } | |||
| 329 | + | older, err := ReadResidentHistoryPage(dir, page.OlderCursor, 10) | |||
| 330 | + | if err != nil { | |||
| 331 | + | t.Fatal(err) | |||
| 332 | + | } | |||
| 333 | + | if len(older.Items) != 3 || older.Items[0].Type != residentRecordUser || !strings.Contains(string(older.Items[0].Message), "old question") { | |||
| 334 | + | t.Fatalf("older page = %#v, want the append-only log before the checkpoint", older.Items) | |||
| 335 | + | } | |||
| 336 | + | for _, item := range append(append([]ResidentHistoryItem(nil), page.Items...), older.Items...) { | |||
| 337 | + | if item.Type == residentRecordCompacted { | |||
| 338 | + | t.Fatalf("pager exposed a compaction checkpoint: %#v", item) | |||
| 339 | + | } | |||
| 340 | + | } | |||
| 341 | + | } | |||
| 342 | + | ||||
| 343 | + | func TestReadResidentTranscriptMessagesRejectsInvalidCheckpoint(t *testing.T) { | |||
| 344 | + | for _, tc := range []struct { | |||
| 345 | + | name string | |||
| 346 | + | line string | |||
| 347 | + | }{ | |||
| 348 | + | {name: "empty", line: `{"version":3,"type":"child.compacted","time":"2026-01-01T00:00:00Z"}`}, | |||
| 349 | + | {name: "undecodable", line: `{"version":3,"type":"child.compacted","time":"2026-01-01T00:00:00Z","messages":["not a message"]}`}, | |||
| 350 | + | } { | |||
| 351 | + | t.Run(tc.name, func(t *testing.T) { | |||
| 352 | + | root := t.TempDir() | |||
| 353 | + | journal, err := OpenResidentJournal(root, "bad-checkpoint") | |||
| 354 | + | if err != nil { | |||
| 355 | + | t.Fatal(err) | |||
| 356 | + | } | |||
| 357 | + | spec := ResidentChildSpec{ID: "bad-checkpoint", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |||
| 358 | + | if err := journal.Accept(spec, "task"); err != nil { | |||
| 359 | + | t.Fatal(err) | |||
| 360 | + | } | |||
| 361 | + | if err := journal.RecordAgentEvent(core.EvUserMessage{Message: provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "question"}}}}); err != nil { | |||
| 362 | + | t.Fatal(err) | |||
| 363 | + | } | |||
| 364 | + | dir := journal.Dir() | |||
| 365 | + | if err := journal.Close(); err != nil { | |||
| 366 | + | t.Fatal(err) | |||
| 367 | + | } | |||
| 368 | + | path := filepath.Join(dir, residentTranscriptName) | |||
| 369 | + | file, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0o600) | |||
| 370 | + | if err != nil { | |||
| 371 | + | t.Fatal(err) | |||
| 372 | + | } | |||
| 373 | + | if _, err := file.WriteString(tc.line + "\n"); err != nil { | |||
| 374 | + | t.Fatal(err) | |||
| 375 | + | } | |||
| 376 | + | if err := file.Close(); err != nil { | |||
| 377 | + | t.Fatal(err) | |||
| 378 | + | } | |||
| 379 | + | if _, err := ReadResidentTranscriptMessages(dir); err == nil { | |||
| 380 | + | t.Fatal("invalid checkpoint was accepted") | |||
| 381 | + | } | |||
| 382 | + | if _, err := ReconcileResidentJournal(dir); err == nil { | |||
| 383 | + | t.Fatal("reconciliation accepted an invalid checkpoint") | |||
| 384 | + | } | |||
| 385 | + | }) | |||
| 386 | + | } | |||
| 387 | + | } | |||
| 388 | + | ||||
| 389 | + | func joinedResidentMessageText(messages []provider.Message) string { | |||
| 390 | + | var text strings.Builder | |||
| 391 | + | for _, message := range messages { | |||
| 392 | + | for _, content := range message.Content { | |||
| 393 | + | if block, ok := content.(provider.TextBlock); ok { | |||
| 394 | + | text.WriteString(block.Text) | |||
| 395 | + | text.WriteString("\n") | |||
| 396 | + | } | |||
| 397 | + | } | |||
| 398 | + | } | |||
| 399 | + | return text.String() | |||
| 400 | + | } | |||
M packages/agent/subagents/resident_journal.go +105 -19
| @@ -36,6 +36,7 @@ const ( | |||
| 36 | 36 | residentRecordAssistant = "message.assistant" | |
| 37 | 37 | residentRecordToolCall = "tool.call" | |
| 38 | 38 | residentRecordToolResult = "tool.result" | |
| 39 | + | residentRecordCompacted = "child.compacted" | |
| 39 | 40 | residentRecordUsage = "usage" | |
| 40 | 41 | residentJournalVersion = 3 | |
| 41 | 42 | residentJournalVersionV2 = 2 | |
| @@ -121,15 +122,19 @@ type residentRecord struct { | |||
| 121 | 122 | PatchRef string `json:"patch_ref,omitempty"` | |
| 122 | 123 | ChangedFiles []string `json:"changed_files,omitempty"` | |
| 123 | 124 | Message json.RawMessage `json:"message,omitempty"` | |
| 124 | - | ToolID string `json:"tool_id,omitempty"` | |
| 125 | - | ToolName string `json:"tool_name,omitempty"` | |
| 126 | - | ToolArgs json.RawMessage `json:"tool_args,omitempty"` | |
| 127 | - | ToolResult json.RawMessage `json:"tool_result,omitempty"` | |
| 128 | - | Usage *provider.Usage `json:"usage,omitempty"` | |
| 129 | - | ContextUsed int `json:"context_used,omitempty"` | |
| 130 | - | ContextMax int `json:"context_max,omitempty"` | |
| 131 | - | Subscription bool `json:"subscription,omitempty"` | |
| 132 | - | raw json.RawMessage `json:"-"` | |
| 125 | + | // Messages carries a full replacement transcript. It is used only by | |
| 126 | + | // child.compacted, the durable checkpoint written after an overflow | |
| 127 | + | // recovery compacts the child transcript. | |
| 128 | + | Messages []json.RawMessage `json:"messages,omitempty"` | |
| 129 | + | ToolID string `json:"tool_id,omitempty"` | |
| 130 | + | ToolName string `json:"tool_name,omitempty"` | |
| 131 | + | ToolArgs json.RawMessage `json:"tool_args,omitempty"` | |
| 132 | + | ToolResult json.RawMessage `json:"tool_result,omitempty"` | |
| 133 | + | Usage *provider.Usage `json:"usage,omitempty"` | |
| 134 | + | ContextUsed int `json:"context_used,omitempty"` | |
| 135 | + | ContextMax int `json:"context_max,omitempty"` | |
| 136 | + | Subscription bool `json:"subscription,omitempty"` | |
| 137 | + | raw json.RawMessage `json:"-"` | |
| 133 | 138 | } | |
| 134 | 139 | ||
| 135 | 140 | // RecordAgentEvent persists finalized provider-neutral transcript events. It | |
| @@ -231,6 +236,40 @@ func (j *ResidentJournal) RecordAgentEvent(event core.AgentEvent) error { | |||
| 231 | 236 | return err | |
| 232 | 237 | } | |
| 233 | 238 | ||
| 239 | + | // RecordCompacted persists the replacement transcript produced by a | |
| 240 | + | // context-overflow recovery. The checkpoint is additive: journals without one | |
| 241 | + | // replay exactly as before, and resume replay starts from the newest | |
| 242 | + | // checkpoint instead of the full append-only log. | |
| 243 | + | func (j *ResidentJournal) RecordCompacted(messages []provider.Message) error { | |
| 244 | + | if j == nil { | |
| 245 | + | return errors.New("resident journal: unavailable") | |
| 246 | + | } | |
| 247 | + | if len(messages) == 0 { | |
| 248 | + | return errors.New("resident journal: empty compaction checkpoint") | |
| 249 | + | } | |
| 250 | + | encoded := make([]json.RawMessage, 0, len(messages)) | |
| 251 | + | for _, message := range messages { | |
| 252 | + | data, err := json.Marshal(message) | |
| 253 | + | if err != nil { | |
| 254 | + | return fmt.Errorf("resident journal encode compacted message: %w", err) | |
| 255 | + | } | |
| 256 | + | encoded = append(encoded, data) | |
| 257 | + | } | |
| 258 | + | j.mu.Lock() | |
| 259 | + | defer j.mu.Unlock() | |
| 260 | + | if j.file == nil { | |
| 261 | + | return errors.New("resident journal: closed") | |
| 262 | + | } | |
| 263 | + | record := residentRecord{ | |
| 264 | + | Version: residentJournalVersion, | |
| 265 | + | Type: residentRecordCompacted, | |
| 266 | + | Time: time.Now().UTC(), | |
| 267 | + | TurnID: j.activeTurnID, | |
| 268 | + | Messages: encoded, | |
| 269 | + | } | |
| 270 | + | return j.appendSync(record) | |
| 271 | + | } | |
| 272 | + | ||
| 234 | 273 | // ResidentMetadata is a bounded, rebuildable projection. The transcript is | |
| 235 | 274 | // the authority for acceptance; callers must never infer acceptance solely | |
| 236 | 275 | // from this file. | |
| @@ -285,6 +324,12 @@ type ResidentJournal struct { | |||
| 285 | 324 | dir string | |
| 286 | 325 | file *os.File | |
| 287 | 326 | lease *residentLease | |
| 327 | + | // activeTurnID is the resident turn currently executing. It attributes | |
| 328 | + | // records that the runner writes without a turn parameter, such as a | |
| 329 | + | // compaction checkpoint, and is diagnostics only: no reader consumes a | |
| 330 | + | // checkpoint's turn ID. RecordTurnStarted owns the value and a terminal | |
| 331 | + | // boundary clears it. It is guarded by mu. | |
| 332 | + | activeTurnID string | |
| 288 | 333 | } | |
| 289 | 334 | ||
| 290 | 335 | // ResidentUsageSnapshot is the bounded usage projection shared by durable and | |
| @@ -581,6 +626,18 @@ func (j *ResidentJournal) recordTurnBoundary(spec ResidentChildSpec, state Resid | |||
| 581 | 626 | if err := j.appendSync(record); err != nil { | |
| 582 | 627 | return err | |
| 583 | 628 | } | |
| 629 | + | // Track the accepted turn that is currently executing so records written | |
| 630 | + | // without an explicit turn parameter, such as a compaction checkpoint, stay | |
| 631 | + | // attributable. A terminal boundary clears it; a late checkpoint from a | |
| 632 | + | // finished turn can therefore not be attributed to the next one. | |
| 633 | + | switch record.Type { | |
| 634 | + | case residentRecordTurnStarted: | |
| 635 | + | j.activeTurnID = record.TurnID | |
| 636 | + | case residentRecordTurnFinished, residentRecordInterrupted: | |
| 637 | + | if j.activeTurnID == record.TurnID { | |
| 638 | + | j.activeTurnID = "" | |
| 639 | + | } | |
| 640 | + | } | |
| 584 | 641 | return writeResidentMetadata(j.dir, j.metadata(spec, state, record.Time)) | |
| 585 | 642 | } | |
| 586 | 643 | ||
| @@ -617,7 +674,9 @@ func (j *ResidentJournal) appendSync(record residentRecord) error { | |||
| 617 | 674 | if err != nil { | |
| 618 | 675 | return fmt.Errorf("resident journal encode: %w", err) | |
| 619 | 676 | } | |
| 620 | - | if len(data) > residentMaxRecordBytes { | |
| 677 | + | if len(data)+1 > residentMaxRecordBytes { | |
| 678 | + | // Readers reject the whole record line, newline included, so the writer | |
| 679 | + | // applies the same bound instead of emitting a line it cannot read back. | |
| 621 | 680 | return errors.New("resident journal: record too large") | |
| 622 | 681 | } | |
| 623 | 682 | data = append(data, '\n') | |
| @@ -912,6 +971,32 @@ func reconcileOwnedResidentJournal(journal *ResidentJournal) (ResidentMetadata, | |||
| 912 | 971 | return ResidentMetadata{}, errors.New("resident journal: duplicate tool result") | |
| 913 | 972 | } | |
| 914 | 973 | toolResults[record.ToolID] = struct{}{} | |
| 974 | + | case residentRecordCompacted: | |
| 975 | + | checkpoint, err := decodeResidentCompactedMessages(record.Messages) | |
| 976 | + | if err != nil { | |
| 977 | + | return ResidentMetadata{}, fmt.Errorf("resident journal: invalid compaction checkpoint: %w", err) | |
| 978 | + | } | |
| 979 | + | // The checkpoint replaces everything before it, so tool bookkeeping | |
| 980 | + | // restarts from the compacted transcript. Without this, a call that | |
| 981 | + | // compaction dropped would look dangling and gain a synthetic | |
| 982 | + | // interrupted result appended after the checkpoint, which resume | |
| 983 | + | // replay would then surface as an orphan. | |
| 984 | + | toolCalls = make(map[string]struct{}) | |
| 985 | + | toolResults = make(map[string]struct{}) | |
| 986 | + | for _, message := range checkpoint { | |
| 987 | + | for _, content := range message.Content { | |
| 988 | + | switch block := content.(type) { | |
| 989 | + | case provider.ToolCallBlock: | |
| 990 | + | if block.ID != "" { | |
| 991 | + | toolCalls[block.ID] = struct{}{} | |
| 992 | + | } | |
| 993 | + | case provider.ToolResultBlock: | |
| 994 | + | if block.CallID != "" { | |
| 995 | + | toolResults[block.CallID] = struct{}{} | |
| 996 | + | } | |
| 997 | + | } | |
| 998 | + | } | |
| 999 | + | } | |
| 915 | 1000 | case residentRecordUsage: | |
| 916 | 1001 | if record.Usage == nil { | |
| 917 | 1002 | return ResidentMetadata{}, errors.New("resident journal: invalid usage record") | |
| @@ -928,18 +1013,19 @@ func reconcileOwnedResidentJournal(journal *ResidentJournal) (ResidentMetadata, | |||
| 928 | 1013 | Usage: usage.Usage, ContextUsed: usage.ContextUsed, ContextMax: usage.ContextMax, Subscription: usage.Subscription, | |
| 929 | 1014 | } | |
| 930 | 1015 | needsInterruption := state == ResidentQueued || state == ResidentRunning | |
| 931 | - | needsToolRepair := len(toolCalls) != len(toolResults) | |
| 932 | - | if needsInterruption || needsToolRepair { | |
| 1016 | + | // Dangling calls, not a tool-count mismatch, are the repair trigger: a | |
| 1017 | + | // checkpoint may legitimately carry a result without repeating its call. | |
| 1018 | + | dangling := make([]string, 0, len(toolCalls)) | |
| 1019 | + | for toolID := range toolCalls { | |
| 1020 | + | if _, finished := toolResults[toolID]; !finished { | |
| 1021 | + | dangling = append(dangling, toolID) | |
| 1022 | + | } | |
| 1023 | + | } | |
| 1024 | + | sort.Strings(dangling) | |
| 1025 | + | if needsInterruption || len(dangling) > 0 { | |
| 933 | 1026 | // A recorded repair is new observable work. A no-op reconciliation, | |
| 934 | 1027 | // however, must retain the last durable lifecycle time. | |
| 935 | 1028 | metadata.UpdatedAt = time.Now().UTC() | |
| 936 | - | dangling := make([]string, 0, len(toolCalls)) | |
| 937 | - | for toolID := range toolCalls { | |
| 938 | - | if _, finished := toolResults[toolID]; !finished { | |
| 939 | - | dangling = append(dangling, toolID) | |
| 940 | - | } | |
| 941 | - | } | |
| 942 | - | sort.Strings(dangling) | |
| 943 | 1029 | for _, toolID := range dangling { | |
| 944 | 1030 | result, marshalErr := json.Marshal(core.ToolResult{IsError: true, Content: []provider.Content{provider.TextBlock{Text: residentInterruptedText}}}) | |
| 945 | 1031 | if marshalErr != nil { | |
| @@ -36,6 +36,7 @@ const ( | |||||
| 36 | residentRecordAssistant = "message.assistant" | 36 | residentRecordAssistant = "message.assistant" | ||
| 37 | residentRecordToolCall = "tool.call" | 37 | residentRecordToolCall = "tool.call" | ||
| 38 | residentRecordToolResult = "tool.result" | 38 | residentRecordToolResult = "tool.result" | ||
| 39 | + | residentRecordCompacted = "child.compacted" | |||
| 39 | residentRecordUsage = "usage" | 40 | residentRecordUsage = "usage" | ||
| 40 | residentJournalVersion = 3 | 41 | residentJournalVersion = 3 | ||
| 41 | residentJournalVersionV2 = 2 | 42 | residentJournalVersionV2 = 2 | ||
| @@ -121,15 +122,19 @@ type residentRecord struct { | |||||
| 121 | PatchRef string `json:"patch_ref,omitempty"` | 122 | PatchRef string `json:"patch_ref,omitempty"` | ||
| 122 | ChangedFiles []string `json:"changed_files,omitempty"` | 123 | ChangedFiles []string `json:"changed_files,omitempty"` | ||
| 123 | Message json.RawMessage `json:"message,omitempty"` | 124 | Message json.RawMessage `json:"message,omitempty"` | ||
| 124 | - | ToolID string `json:"tool_id,omitempty"` | 125 | + | // Messages carries a full replacement transcript. It is used only by |
| 125 | - | ToolName string `json:"tool_name,omitempty"` | 126 | + | // child.compacted, the durable checkpoint written after an overflow |
| 126 | - | ToolArgs json.RawMessage `json:"tool_args,omitempty"` | 127 | + | // recovery compacts the child transcript. |
| 127 | - | ToolResult json.RawMessage `json:"tool_result,omitempty"` | 128 | + | Messages []json.RawMessage `json:"messages,omitempty"` |
| 128 | - | Usage *provider.Usage `json:"usage,omitempty"` | 129 | + | ToolID string `json:"tool_id,omitempty"` |
| 129 | - | ContextUsed int `json:"context_used,omitempty"` | 130 | + | ToolName string `json:"tool_name,omitempty"` |
| 130 | - | ContextMax int `json:"context_max,omitempty"` | 131 | + | ToolArgs json.RawMessage `json:"tool_args,omitempty"` |
| 131 | - | Subscription bool `json:"subscription,omitempty"` | 132 | + | ToolResult json.RawMessage `json:"tool_result,omitempty"` |
| 132 | - | raw json.RawMessage `json:"-"` | 133 | + | Usage *provider.Usage `json:"usage,omitempty"` |
| 134 | + | ContextUsed int `json:"context_used,omitempty"` | |||
| 135 | + | ContextMax int `json:"context_max,omitempty"` | |||
| 136 | + | Subscription bool `json:"subscription,omitempty"` | |||
| 137 | + | raw json.RawMessage `json:"-"` | |||
| 133 | } | 138 | } | ||
| 134 | 139 | ||||
| 135 | // RecordAgentEvent persists finalized provider-neutral transcript events. It | 140 | // RecordAgentEvent persists finalized provider-neutral transcript events. It | ||
| @@ -231,6 +236,40 @@ func (j *ResidentJournal) RecordAgentEvent(event core.AgentEvent) error { | |||||
| 231 | return err | 236 | return err | ||
| 232 | } | 237 | } | ||
| 233 | 238 | ||||
| 239 | + | // RecordCompacted persists the replacement transcript produced by a | |||
| 240 | + | // context-overflow recovery. The checkpoint is additive: journals without one | |||
| 241 | + | // replay exactly as before, and resume replay starts from the newest | |||
| 242 | + | // checkpoint instead of the full append-only log. | |||
| 243 | + | func (j *ResidentJournal) RecordCompacted(messages []provider.Message) error { | |||
| 244 | + | if j == nil { | |||
| 245 | + | return errors.New("resident journal: unavailable") | |||
| 246 | + | } | |||
| 247 | + | if len(messages) == 0 { | |||
| 248 | + | return errors.New("resident journal: empty compaction checkpoint") | |||
| 249 | + | } | |||
| 250 | + | encoded := make([]json.RawMessage, 0, len(messages)) | |||
| 251 | + | for _, message := range messages { | |||
| 252 | + | data, err := json.Marshal(message) | |||
| 253 | + | if err != nil { | |||
| 254 | + | return fmt.Errorf("resident journal encode compacted message: %w", err) | |||
| 255 | + | } | |||
| 256 | + | encoded = append(encoded, data) | |||
| 257 | + | } | |||
| 258 | + | j.mu.Lock() | |||
| 259 | + | defer j.mu.Unlock() | |||
| 260 | + | if j.file == nil { | |||
| 261 | + | return errors.New("resident journal: closed") | |||
| 262 | + | } | |||
| 263 | + | record := residentRecord{ | |||
| 264 | + | Version: residentJournalVersion, | |||
| 265 | + | Type: residentRecordCompacted, | |||
| 266 | + | Time: time.Now().UTC(), | |||
| 267 | + | TurnID: j.activeTurnID, | |||
| 268 | + | Messages: encoded, | |||
| 269 | + | } | |||
| 270 | + | return j.appendSync(record) | |||
| 271 | + | } | |||
| 272 | + | ||||
| 234 | // ResidentMetadata is a bounded, rebuildable projection. The transcript is | 273 | // ResidentMetadata is a bounded, rebuildable projection. The transcript is | ||
| 235 | // the authority for acceptance; callers must never infer acceptance solely | 274 | // the authority for acceptance; callers must never infer acceptance solely | ||
| 236 | // from this file. | 275 | // from this file. | ||
| @@ -285,6 +324,12 @@ type ResidentJournal struct { | |||||
| 285 | dir string | 324 | dir string | ||
| 286 | file *os.File | 325 | file *os.File | ||
| 287 | lease *residentLease | 326 | lease *residentLease | ||
| 327 | + | // activeTurnID is the resident turn currently executing. It attributes | |||
| 328 | + | // records that the runner writes without a turn parameter, such as a | |||
| 329 | + | // compaction checkpoint, and is diagnostics only: no reader consumes a | |||
| 330 | + | // checkpoint's turn ID. RecordTurnStarted owns the value and a terminal | |||
| 331 | + | // boundary clears it. It is guarded by mu. | |||
| 332 | + | activeTurnID string | |||
| 288 | } | 333 | } | ||
| 289 | 334 | ||||
| 290 | // ResidentUsageSnapshot is the bounded usage projection shared by durable and | 335 | // ResidentUsageSnapshot is the bounded usage projection shared by durable and | ||
| @@ -581,6 +626,18 @@ func (j *ResidentJournal) recordTurnBoundary(spec ResidentChildSpec, state Resid | |||||
| 581 | if err := j.appendSync(record); err != nil { | 626 | if err := j.appendSync(record); err != nil { | ||
| 582 | return err | 627 | return err | ||
| 583 | } | 628 | } | ||
| 629 | + | // Track the accepted turn that is currently executing so records written | |||
| 630 | + | // without an explicit turn parameter, such as a compaction checkpoint, stay | |||
| 631 | + | // attributable. A terminal boundary clears it; a late checkpoint from a | |||
| 632 | + | // finished turn can therefore not be attributed to the next one. | |||
| 633 | + | switch record.Type { | |||
| 634 | + | case residentRecordTurnStarted: | |||
| 635 | + | j.activeTurnID = record.TurnID | |||
| 636 | + | case residentRecordTurnFinished, residentRecordInterrupted: | |||
| 637 | + | if j.activeTurnID == record.TurnID { | |||
| 638 | + | j.activeTurnID = "" | |||
| 639 | + | } | |||
| 640 | + | } | |||
| 584 | return writeResidentMetadata(j.dir, j.metadata(spec, state, record.Time)) | 641 | return writeResidentMetadata(j.dir, j.metadata(spec, state, record.Time)) | ||
| 585 | } | 642 | } | ||
| 586 | 643 | ||||
| @@ -617,7 +674,9 @@ func (j *ResidentJournal) appendSync(record residentRecord) error { | |||||
| 617 | if err != nil { | 674 | if err != nil { | ||
| 618 | return fmt.Errorf("resident journal encode: %w", err) | 675 | return fmt.Errorf("resident journal encode: %w", err) | ||
| 619 | } | 676 | } | ||
| 620 | - | if len(data) > residentMaxRecordBytes { | 677 | + | if len(data)+1 > residentMaxRecordBytes { |
| 678 | + | // Readers reject the whole record line, newline included, so the writer | |||
| 679 | + | // applies the same bound instead of emitting a line it cannot read back. | |||
| 621 | return errors.New("resident journal: record too large") | 680 | return errors.New("resident journal: record too large") | ||
| 622 | } | 681 | } | ||
| 623 | data = append(data, '\n') | 682 | data = append(data, '\n') | ||
| @@ -912,6 +971,32 @@ func reconcileOwnedResidentJournal(journal *ResidentJournal) (ResidentMetadata, | |||||
| 912 | return ResidentMetadata{}, errors.New("resident journal: duplicate tool result") | 971 | return ResidentMetadata{}, errors.New("resident journal: duplicate tool result") | ||
| 913 | } | 972 | } | ||
| 914 | toolResults[record.ToolID] = struct{}{} | 973 | toolResults[record.ToolID] = struct{}{} | ||
| 974 | + | case residentRecordCompacted: | |||
| 975 | + | checkpoint, err := decodeResidentCompactedMessages(record.Messages) | |||
| 976 | + | if err != nil { | |||
| 977 | + | return ResidentMetadata{}, fmt.Errorf("resident journal: invalid compaction checkpoint: %w", err) | |||
| 978 | + | } | |||
| 979 | + | // The checkpoint replaces everything before it, so tool bookkeeping | |||
| 980 | + | // restarts from the compacted transcript. Without this, a call that | |||
| 981 | + | // compaction dropped would look dangling and gain a synthetic | |||
| 982 | + | // interrupted result appended after the checkpoint, which resume | |||
| 983 | + | // replay would then surface as an orphan. | |||
| 984 | + | toolCalls = make(map[string]struct{}) | |||
| 985 | + | toolResults = make(map[string]struct{}) | |||
| 986 | + | for _, message := range checkpoint { | |||
| 987 | + | for _, content := range message.Content { | |||
| 988 | + | switch block := content.(type) { | |||
| 989 | + | case provider.ToolCallBlock: | |||
| 990 | + | if block.ID != "" { | |||
| 991 | + | toolCalls[block.ID] = struct{}{} | |||
| 992 | + | } | |||
| 993 | + | case provider.ToolResultBlock: | |||
| 994 | + | if block.CallID != "" { | |||
| 995 | + | toolResults[block.CallID] = struct{}{} | |||
| 996 | + | } | |||
| 997 | + | } | |||
| 998 | + | } | |||
| 999 | + | } | |||
| 915 | case residentRecordUsage: | 1000 | case residentRecordUsage: | ||
| 916 | if record.Usage == nil { | 1001 | if record.Usage == nil { | ||
| 917 | return ResidentMetadata{}, errors.New("resident journal: invalid usage record") | 1002 | return ResidentMetadata{}, errors.New("resident journal: invalid usage record") | ||
| @@ -928,18 +1013,19 @@ func reconcileOwnedResidentJournal(journal *ResidentJournal) (ResidentMetadata, | |||||
| 928 | Usage: usage.Usage, ContextUsed: usage.ContextUsed, ContextMax: usage.ContextMax, Subscription: usage.Subscription, | 1013 | Usage: usage.Usage, ContextUsed: usage.ContextUsed, ContextMax: usage.ContextMax, Subscription: usage.Subscription, | ||
| 929 | } | 1014 | } | ||
| 930 | needsInterruption := state == ResidentQueued || state == ResidentRunning | 1015 | needsInterruption := state == ResidentQueued || state == ResidentRunning | ||
| 931 | - | needsToolRepair := len(toolCalls) != len(toolResults) | 1016 | + | // Dangling calls, not a tool-count mismatch, are the repair trigger: a |
| 932 | - | if needsInterruption || needsToolRepair { | 1017 | + | // checkpoint may legitimately carry a result without repeating its call. |
| 1018 | + | dangling := make([]string, 0, len(toolCalls)) | |||
| 1019 | + | for toolID := range toolCalls { | |||
| 1020 | + | if _, finished := toolResults[toolID]; !finished { | |||
| 1021 | + | dangling = append(dangling, toolID) | |||
| 1022 | + | } | |||
| 1023 | + | } | |||
| 1024 | + | sort.Strings(dangling) | |||
| 1025 | + | if needsInterruption || len(dangling) > 0 { | |||
| 933 | // A recorded repair is new observable work. A no-op reconciliation, | 1026 | // A recorded repair is new observable work. A no-op reconciliation, | ||
| 934 | // however, must retain the last durable lifecycle time. | 1027 | // however, must retain the last durable lifecycle time. | ||
| 935 | metadata.UpdatedAt = time.Now().UTC() | 1028 | metadata.UpdatedAt = time.Now().UTC() | ||
| 936 | - | dangling := make([]string, 0, len(toolCalls)) | |||
| 937 | - | for toolID := range toolCalls { | |||
| 938 | - | if _, finished := toolResults[toolID]; !finished { | |||
| 939 | - | dangling = append(dangling, toolID) | |||
| 940 | - | } | |||
| 941 | - | } | |||
| 942 | - | sort.Strings(dangling) | |||
| 943 | for _, toolID := range dangling { | 1029 | for _, toolID := range dangling { | ||
| 944 | result, marshalErr := json.Marshal(core.ToolResult{IsError: true, Content: []provider.Content{provider.TextBlock{Text: residentInterruptedText}}}) | 1030 | result, marshalErr := json.Marshal(core.ToolResult{IsError: true, Content: []provider.Content{provider.TextBlock{Text: residentInterruptedText}}}) | ||
| 945 | if marshalErr != nil { | 1031 | if marshalErr != nil { | ||
M packages/agent/subagents/resident_journal_test.go +334 -0
| @@ -1055,3 +1055,337 @@ func TestReconcileResidentJournalRejectsConflictingToolPairs(t *testing.T) { | |||
| 1055 | 1055 | }) | |
| 1056 | 1056 | } | |
| 1057 | 1057 | } | |
| 1058 | + | ||
| 1059 | + | func TestRecordCompactedRejectsOversizedCheckpoint(t *testing.T) { | |
| 1060 | + | root := t.TempDir() | |
| 1061 | + | journal, err := OpenResidentJournal(root, "oversized") | |
| 1062 | + | if err != nil { | |
| 1063 | + | t.Fatal(err) | |
| 1064 | + | } | |
| 1065 | + | spec := ResidentChildSpec{ID: "oversized", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |
| 1066 | + | if err := journal.Accept(spec, "task"); err != nil { | |
| 1067 | + | t.Fatal(err) | |
| 1068 | + | } | |
| 1069 | + | if err := journal.RecordAgentEvent(core.EvUserMessage{Message: provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "question"}}}}); err != nil { | |
| 1070 | + | t.Fatal(err) | |
| 1071 | + | } | |
| 1072 | + | oversized := []provider.Message{{ | |
| 1073 | + | Role: provider.RoleUser, | |
| 1074 | + | Content: []provider.Content{provider.TextBlock{Text: strings.Repeat("x", residentMaxRecordBytes)}}, | |
| 1075 | + | }} | |
| 1076 | + | if err := journal.RecordCompacted(oversized); err == nil || !strings.Contains(err.Error(), "too large") { | |
| 1077 | + | t.Fatalf("RecordCompacted error = %v, want oversize rejection", err) | |
| 1078 | + | } | |
| 1079 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |
| 1080 | + | if err := journal.Close(); err != nil { | |
| 1081 | + | t.Fatal(err) | |
| 1082 | + | } | |
| 1083 | + | records, err := ReadResidentJournal(transcript) | |
| 1084 | + | if err != nil { | |
| 1085 | + | t.Fatalf("journal unreadable after an oversized checkpoint: %v", err) | |
| 1086 | + | } | |
| 1087 | + | for _, record := range records { | |
| 1088 | + | if record.Type == residentRecordCompacted { | |
| 1089 | + | t.Fatal("oversized checkpoint was appended") | |
| 1090 | + | } | |
| 1091 | + | } | |
| 1092 | + | if _, err := ReadResidentTranscriptMessages(filepath.Join(root, spec.ID)); err != nil { | |
| 1093 | + | t.Fatalf("transcript unreadable after an oversized checkpoint: %v", err) | |
| 1094 | + | } | |
| 1095 | + | } | |
| 1096 | + | ||
| 1097 | + | func TestReconcileResidentJournalKeepsToolBookkeepingAcrossCheckpoint(t *testing.T) { | |
| 1098 | + | root := t.TempDir() | |
| 1099 | + | journal, err := OpenResidentJournal(root, "checkpoint-repair") | |
| 1100 | + | if err != nil { | |
| 1101 | + | t.Fatal(err) | |
| 1102 | + | } | |
| 1103 | + | spec := ResidentChildSpec{ID: "checkpoint-repair", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |
| 1104 | + | if err := journal.Accept(spec, "task"); err != nil { | |
| 1105 | + | t.Fatal(err) | |
| 1106 | + | } | |
| 1107 | + | if err := journal.RecordAgentEvent(core.EvUserMessage{Message: provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "question"}}}}); err != nil { | |
| 1108 | + | t.Fatal(err) | |
| 1109 | + | } | |
| 1110 | + | // The call never produced a result and compaction dropped it from the | |
| 1111 | + | // replacement transcript. | |
| 1112 | + | if err := journal.RecordAgentEvent(core.EvToolCall{ID: "dropped-call", Name: "bash", Args: json.RawMessage(`{"command":"pwd"}`)}); err != nil { | |
| 1113 | + | t.Fatal(err) | |
| 1114 | + | } | |
| 1115 | + | if err := journal.AcceptFollowUp(spec, "turn-1", "follow up"); err != nil { | |
| 1116 | + | t.Fatal(err) | |
| 1117 | + | } | |
| 1118 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |
| 1119 | + | t.Fatal(err) | |
| 1120 | + | } | |
| 1121 | + | if err := journal.RecordCompacted([]provider.Message{ | |
| 1122 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "## Context Summary (compacted)\n\ndigest"}}}, | |
| 1123 | + | }); err != nil { | |
| 1124 | + | t.Fatal(err) | |
| 1125 | + | } | |
| 1126 | + | if err := journal.RecordTurnFinished(spec, "turn-1", nil); err != nil { | |
| 1127 | + | t.Fatal(err) | |
| 1128 | + | } | |
| 1129 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |
| 1130 | + | if err := journal.Close(); err != nil { | |
| 1131 | + | t.Fatal(err) | |
| 1132 | + | } | |
| 1133 | + | ||
| 1134 | + | metadata, err := ReconcileResidentJournal(filepath.Join(root, spec.ID)) | |
| 1135 | + | if err != nil { | |
| 1136 | + | t.Fatal(err) | |
| 1137 | + | } | |
| 1138 | + | if metadata.State != ResidentIdle { | |
| 1139 | + | t.Fatalf("reconciled state = %q, want idle", metadata.State) | |
| 1140 | + | } | |
| 1141 | + | records, err := ReadResidentJournal(transcript) | |
| 1142 | + | if err != nil { | |
| 1143 | + | t.Fatal(err) | |
| 1144 | + | } | |
| 1145 | + | for _, record := range records { | |
| 1146 | + | if record.Type == residentRecordToolResult { | |
| 1147 | + | t.Fatalf("reconciliation repaired a call dropped by compaction: %#v", record) | |
| 1148 | + | } | |
| 1149 | + | } | |
| 1150 | + | messages, err := ReadResidentTranscriptMessages(filepath.Join(root, spec.ID)) | |
| 1151 | + | if err != nil { | |
| 1152 | + | t.Fatal(err) | |
| 1153 | + | } | |
| 1154 | + | if len(messages) != 1 || messages[0].Role != provider.RoleUser { | |
| 1155 | + | t.Fatalf("resumed messages = %#v, want only the checkpoint summary", messages) | |
| 1156 | + | } | |
| 1157 | + | } | |
| 1158 | + | ||
| 1159 | + | func TestRecordCompactedAttributesTheStartedTurn(t *testing.T) { | |
| 1160 | + | root := t.TempDir() | |
| 1161 | + | journal, err := OpenResidentJournal(root, "turn-attribution") | |
| 1162 | + | if err != nil { | |
| 1163 | + | t.Fatal(err) | |
| 1164 | + | } | |
| 1165 | + | spec := ResidentChildSpec{ID: "turn-attribution", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |
| 1166 | + | if err := journal.Accept(spec, "task"); err != nil { | |
| 1167 | + | t.Fatal(err) | |
| 1168 | + | } | |
| 1169 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |
| 1170 | + | t.Fatal(err) | |
| 1171 | + | } | |
| 1172 | + | // A follow-up accepted while turn-1 is still running must not steal the | |
| 1173 | + | // checkpoint's turn attribution. | |
| 1174 | + | if err := journal.AcceptFollowUp(spec, "turn-2", "queued follow up"); err != nil { | |
| 1175 | + | t.Fatal(err) | |
| 1176 | + | } | |
| 1177 | + | if err := journal.RecordCompacted([]provider.Message{ | |
| 1178 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "## Context Summary (compacted)\n\ndigest"}}}, | |
| 1179 | + | }); err != nil { | |
| 1180 | + | t.Fatal(err) | |
| 1181 | + | } | |
| 1182 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |
| 1183 | + | if err := journal.Close(); err != nil { | |
| 1184 | + | t.Fatal(err) | |
| 1185 | + | } | |
| 1186 | + | records, err := ReadResidentJournal(transcript) | |
| 1187 | + | if err != nil { | |
| 1188 | + | t.Fatal(err) | |
| 1189 | + | } | |
| 1190 | + | for _, record := range records { | |
| 1191 | + | if record.Type != residentRecordCompacted { | |
| 1192 | + | continue | |
| 1193 | + | } | |
| 1194 | + | if record.TurnID != "turn-1" { | |
| 1195 | + | t.Fatalf("checkpoint turn ID = %q, want the executing turn", record.TurnID) | |
| 1196 | + | } | |
| 1197 | + | return | |
| 1198 | + | } | |
| 1199 | + | t.Fatal("journal has no compaction checkpoint") | |
| 1200 | + | } | |
| 1201 | + | ||
| 1202 | + | func TestReconcileResidentJournalIsIdempotentForCheckpointResults(t *testing.T) { | |
| 1203 | + | root := t.TempDir() | |
| 1204 | + | journal, err := OpenResidentJournal(root, "idempotent-checkpoint") | |
| 1205 | + | if err != nil { | |
| 1206 | + | t.Fatal(err) | |
| 1207 | + | } | |
| 1208 | + | spec := ResidentChildSpec{ID: "idempotent-checkpoint", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |
| 1209 | + | if err := journal.Accept(spec, "task"); err != nil { | |
| 1210 | + | t.Fatal(err) | |
| 1211 | + | } | |
| 1212 | + | if err := journal.AcceptFollowUp(spec, "turn-1", "follow up"); err != nil { | |
| 1213 | + | t.Fatal(err) | |
| 1214 | + | } | |
| 1215 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |
| 1216 | + | t.Fatal(err) | |
| 1217 | + | } | |
| 1218 | + | // A checkpoint can carry a result whose call was compacted away. That is | |
| 1219 | + | // not repairable work, so reconciliation must stay a no-op. | |
| 1220 | + | if err := journal.RecordCompacted([]provider.Message{{ | |
| 1221 | + | Role: provider.RoleTool, | |
| 1222 | + | Content: []provider.Content{provider.ToolResultBlock{ | |
| 1223 | + | CallID: "compacted-call", | |
| 1224 | + | Content: []provider.Content{provider.TextBlock{Text: "kept result"}}, | |
| 1225 | + | }}, | |
| 1226 | + | }}); err != nil { | |
| 1227 | + | t.Fatal(err) | |
| 1228 | + | } | |
| 1229 | + | if err := journal.RecordTurnFinished(spec, "turn-1", nil); err != nil { | |
| 1230 | + | t.Fatal(err) | |
| 1231 | + | } | |
| 1232 | + | if err := journal.Close(); err != nil { | |
| 1233 | + | t.Fatal(err) | |
| 1234 | + | } | |
| 1235 | + | first, err := ReconcileResidentJournal(filepath.Join(root, spec.ID)) | |
| 1236 | + | if err != nil { | |
| 1237 | + | t.Fatal(err) | |
| 1238 | + | } | |
| 1239 | + | second, err := ReconcileResidentJournal(filepath.Join(root, spec.ID)) | |
| 1240 | + | if err != nil { | |
| 1241 | + | t.Fatal(err) | |
| 1242 | + | } | |
| 1243 | + | if !first.UpdatedAt.Equal(second.UpdatedAt) { | |
| 1244 | + | t.Fatalf("reconciliation rewrote the lifecycle time: %v then %v", first.UpdatedAt, second.UpdatedAt) | |
| 1245 | + | } | |
| 1246 | + | if second.State != ResidentIdle { | |
| 1247 | + | t.Fatalf("reconciled state = %q, want idle", second.State) | |
| 1248 | + | } | |
| 1249 | + | } | |
| 1250 | + | ||
| 1251 | + | func TestAppendSyncEnforcesTheReaderRecordBound(t *testing.T) { | |
| 1252 | + | root := t.TempDir() | |
| 1253 | + | journal, err := OpenResidentJournal(root, "record-bound") | |
| 1254 | + | if err != nil { | |
| 1255 | + | t.Fatal(err) | |
| 1256 | + | } | |
| 1257 | + | spec := ResidentChildSpec{ID: "record-bound", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |
| 1258 | + | if err := journal.Accept(spec, "task"); err != nil { | |
| 1259 | + | t.Fatal(err) | |
| 1260 | + | } | |
| 1261 | + | _, base := residentAssistantRecordSize(t, "") | |
| 1262 | + | if base >= residentMaxRecordBytes-1 { | |
| 1263 | + | t.Fatalf("empty assistant record = %d bytes, too large for a boundary check", base) | |
| 1264 | + | } | |
| 1265 | + | // A JSON record of max-1 bytes produces a max-byte line with its newline, | |
| 1266 | + | // which is exactly what the readers accept. | |
| 1267 | + | padding := residentMaxRecordBytes - 1 - base | |
| 1268 | + | borderline, borderlineSize := residentAssistantRecordSize(t, strings.Repeat("x", padding)) | |
| 1269 | + | if borderlineSize != residentMaxRecordBytes-1 { | |
| 1270 | + | t.Fatalf("borderline record = %d bytes, want %d", borderlineSize, residentMaxRecordBytes-1) | |
| 1271 | + | } | |
| 1272 | + | if err := journal.appendSync(borderline); err != nil { | |
| 1273 | + | t.Fatalf("appendSync rejected a reader-legal record: %v", err) | |
| 1274 | + | } | |
| 1275 | + | oversized, oversizedSize := residentAssistantRecordSize(t, strings.Repeat("x", padding+1)) | |
| 1276 | + | if oversizedSize != residentMaxRecordBytes { | |
| 1277 | + | t.Fatalf("oversized record = %d bytes, want %d", oversizedSize, residentMaxRecordBytes) | |
| 1278 | + | } | |
| 1279 | + | if err := journal.appendSync(oversized); err == nil || !strings.Contains(err.Error(), "too large") { | |
| 1280 | + | t.Fatalf("appendSync error = %v, want the reader record bound", err) | |
| 1281 | + | } | |
| 1282 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |
| 1283 | + | if err := journal.Close(); err != nil { | |
| 1284 | + | t.Fatal(err) | |
| 1285 | + | } | |
| 1286 | + | records, err := ReadResidentJournal(transcript) | |
| 1287 | + | if err != nil { | |
| 1288 | + | t.Fatalf("journal unreadable after a boundary record: %v", err) | |
| 1289 | + | } | |
| 1290 | + | if len(records) != 2 { | |
| 1291 | + | t.Fatalf("records = %d, want the accepted child and the borderline message", len(records)) | |
| 1292 | + | } | |
| 1293 | + | } | |
| 1294 | + | ||
| 1295 | + | func residentAssistantRecordSize(t *testing.T, text string) (residentRecord, int) { | |
| 1296 | + | t.Helper() | |
| 1297 | + | message, err := json.Marshal(provider.Message{ | |
| 1298 | + | Role: provider.RoleAssistant, | |
| 1299 | + | Content: []provider.Content{provider.TextBlock{Text: text}}, | |
| 1300 | + | }) | |
| 1301 | + | if err != nil { | |
| 1302 | + | t.Fatal(err) | |
| 1303 | + | } | |
| 1304 | + | record := residentRecord{Version: residentJournalVersion, Type: residentRecordAssistant, Message: message} | |
| 1305 | + | data, err := json.Marshal(record) | |
| 1306 | + | if err != nil { | |
| 1307 | + | t.Fatal(err) | |
| 1308 | + | } | |
| 1309 | + | return record, len(data) | |
| 1310 | + | } | |
| 1311 | + | ||
| 1312 | + | func TestReconcileResidentJournalRepairsDanglingCheckpointCall(t *testing.T) { | |
| 1313 | + | root := t.TempDir() | |
| 1314 | + | journal, err := OpenResidentJournal(root, "checkpoint-dangling") | |
| 1315 | + | if err != nil { | |
| 1316 | + | t.Fatal(err) | |
| 1317 | + | } | |
| 1318 | + | spec := ResidentChildSpec{ID: "checkpoint-dangling", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |
| 1319 | + | if err := journal.Accept(spec, "task"); err != nil { | |
| 1320 | + | t.Fatal(err) | |
| 1321 | + | } | |
| 1322 | + | if err := journal.AcceptFollowUp(spec, "turn-1", "follow up"); err != nil { | |
| 1323 | + | t.Fatal(err) | |
| 1324 | + | } | |
| 1325 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |
| 1326 | + | t.Fatal(err) | |
| 1327 | + | } | |
| 1328 | + | // The checkpoint kept a call whose result never arrived. | |
| 1329 | + | if err := journal.RecordCompacted([]provider.Message{{ | |
| 1330 | + | Role: provider.RoleAssistant, | |
| 1331 | + | Content: []provider.Content{provider.ToolCallBlock{ | |
| 1332 | + | ID: "compacted-call", | |
| 1333 | + | Name: "bash", | |
| 1334 | + | Arguments: json.RawMessage(`{"command":"pwd"}`), | |
| 1335 | + | }}, | |
| 1336 | + | }}); err != nil { | |
| 1337 | + | t.Fatal(err) | |
| 1338 | + | } | |
| 1339 | + | if err := journal.RecordTurnFinished(spec, "turn-1", nil); err != nil { | |
| 1340 | + | t.Fatal(err) | |
| 1341 | + | } | |
| 1342 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |
| 1343 | + | if err := journal.Close(); err != nil { | |
| 1344 | + | t.Fatal(err) | |
| 1345 | + | } | |
| 1346 | + | dir := filepath.Join(root, spec.ID) | |
| 1347 | + | if _, err := ReconcileResidentJournal(dir); err != nil { | |
| 1348 | + | t.Fatal(err) | |
| 1349 | + | } | |
| 1350 | + | records, err := ReadResidentJournal(transcript) | |
| 1351 | + | if err != nil { | |
| 1352 | + | t.Fatal(err) | |
| 1353 | + | } | |
| 1354 | + | repaired := 0 | |
| 1355 | + | for _, record := range records { | |
| 1356 | + | if record.Type != residentRecordToolResult || record.ToolID != "compacted-call" { | |
| 1357 | + | continue | |
| 1358 | + | } | |
| 1359 | + | if !isSyntheticInterruption(record.ToolResult) { | |
| 1360 | + | t.Fatalf("repaired result = %s, want the interruption result", record.ToolResult) | |
| 1361 | + | } | |
| 1362 | + | repaired++ | |
| 1363 | + | } | |
| 1364 | + | if repaired != 1 { | |
| 1365 | + | t.Fatalf("repaired results = %d, want exactly one synthetic result", repaired) | |
| 1366 | + | } | |
| 1367 | + | // Once the repair is durable, later reconciliations must be stable no-ops. | |
| 1368 | + | second, err := ReconcileResidentJournal(dir) | |
| 1369 | + | if err != nil { | |
| 1370 | + | t.Fatal(err) | |
| 1371 | + | } | |
| 1372 | + | third, err := ReconcileResidentJournal(dir) | |
| 1373 | + | if err != nil { | |
| 1374 | + | t.Fatal(err) | |
| 1375 | + | } | |
| 1376 | + | if !second.UpdatedAt.Equal(third.UpdatedAt) { | |
| 1377 | + | t.Fatalf("reconciliation kept rewriting the lifecycle time: %v then %v", second.UpdatedAt, third.UpdatedAt) | |
| 1378 | + | } | |
| 1379 | + | if records, err = ReadResidentJournal(transcript); err != nil { | |
| 1380 | + | t.Fatal(err) | |
| 1381 | + | } | |
| 1382 | + | results := 0 | |
| 1383 | + | for _, record := range records { | |
| 1384 | + | if record.Type == residentRecordToolResult && record.ToolID == "compacted-call" { | |
| 1385 | + | results++ | |
| 1386 | + | } | |
| 1387 | + | } | |
| 1388 | + | if results != 1 { | |
| 1389 | + | t.Fatalf("tool results = %d, want the single repaired result", results) | |
| 1390 | + | } | |
| 1391 | + | } | |
| @@ -1055,3 +1055,337 @@ func TestReconcileResidentJournalRejectsConflictingToolPairs(t *testing.T) { | |||||
| 1055 | }) | 1055 | }) | ||
| 1056 | } | 1056 | } | ||
| 1057 | } | 1057 | } | ||
| 1058 | + | ||||
| 1059 | + | func TestRecordCompactedRejectsOversizedCheckpoint(t *testing.T) { | |||
| 1060 | + | root := t.TempDir() | |||
| 1061 | + | journal, err := OpenResidentJournal(root, "oversized") | |||
| 1062 | + | if err != nil { | |||
| 1063 | + | t.Fatal(err) | |||
| 1064 | + | } | |||
| 1065 | + | spec := ResidentChildSpec{ID: "oversized", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |||
| 1066 | + | if err := journal.Accept(spec, "task"); err != nil { | |||
| 1067 | + | t.Fatal(err) | |||
| 1068 | + | } | |||
| 1069 | + | if err := journal.RecordAgentEvent(core.EvUserMessage{Message: provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "question"}}}}); err != nil { | |||
| 1070 | + | t.Fatal(err) | |||
| 1071 | + | } | |||
| 1072 | + | oversized := []provider.Message{{ | |||
| 1073 | + | Role: provider.RoleUser, | |||
| 1074 | + | Content: []provider.Content{provider.TextBlock{Text: strings.Repeat("x", residentMaxRecordBytes)}}, | |||
| 1075 | + | }} | |||
| 1076 | + | if err := journal.RecordCompacted(oversized); err == nil || !strings.Contains(err.Error(), "too large") { | |||
| 1077 | + | t.Fatalf("RecordCompacted error = %v, want oversize rejection", err) | |||
| 1078 | + | } | |||
| 1079 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |||
| 1080 | + | if err := journal.Close(); err != nil { | |||
| 1081 | + | t.Fatal(err) | |||
| 1082 | + | } | |||
| 1083 | + | records, err := ReadResidentJournal(transcript) | |||
| 1084 | + | if err != nil { | |||
| 1085 | + | t.Fatalf("journal unreadable after an oversized checkpoint: %v", err) | |||
| 1086 | + | } | |||
| 1087 | + | for _, record := range records { | |||
| 1088 | + | if record.Type == residentRecordCompacted { | |||
| 1089 | + | t.Fatal("oversized checkpoint was appended") | |||
| 1090 | + | } | |||
| 1091 | + | } | |||
| 1092 | + | if _, err := ReadResidentTranscriptMessages(filepath.Join(root, spec.ID)); err != nil { | |||
| 1093 | + | t.Fatalf("transcript unreadable after an oversized checkpoint: %v", err) | |||
| 1094 | + | } | |||
| 1095 | + | } | |||
| 1096 | + | ||||
| 1097 | + | func TestReconcileResidentJournalKeepsToolBookkeepingAcrossCheckpoint(t *testing.T) { | |||
| 1098 | + | root := t.TempDir() | |||
| 1099 | + | journal, err := OpenResidentJournal(root, "checkpoint-repair") | |||
| 1100 | + | if err != nil { | |||
| 1101 | + | t.Fatal(err) | |||
| 1102 | + | } | |||
| 1103 | + | spec := ResidentChildSpec{ID: "checkpoint-repair", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |||
| 1104 | + | if err := journal.Accept(spec, "task"); err != nil { | |||
| 1105 | + | t.Fatal(err) | |||
| 1106 | + | } | |||
| 1107 | + | if err := journal.RecordAgentEvent(core.EvUserMessage{Message: provider.Message{Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "question"}}}}); err != nil { | |||
| 1108 | + | t.Fatal(err) | |||
| 1109 | + | } | |||
| 1110 | + | // The call never produced a result and compaction dropped it from the | |||
| 1111 | + | // replacement transcript. | |||
| 1112 | + | if err := journal.RecordAgentEvent(core.EvToolCall{ID: "dropped-call", Name: "bash", Args: json.RawMessage(`{"command":"pwd"}`)}); err != nil { | |||
| 1113 | + | t.Fatal(err) | |||
| 1114 | + | } | |||
| 1115 | + | if err := journal.AcceptFollowUp(spec, "turn-1", "follow up"); err != nil { | |||
| 1116 | + | t.Fatal(err) | |||
| 1117 | + | } | |||
| 1118 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |||
| 1119 | + | t.Fatal(err) | |||
| 1120 | + | } | |||
| 1121 | + | if err := journal.RecordCompacted([]provider.Message{ | |||
| 1122 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "## Context Summary (compacted)\n\ndigest"}}}, | |||
| 1123 | + | }); err != nil { | |||
| 1124 | + | t.Fatal(err) | |||
| 1125 | + | } | |||
| 1126 | + | if err := journal.RecordTurnFinished(spec, "turn-1", nil); err != nil { | |||
| 1127 | + | t.Fatal(err) | |||
| 1128 | + | } | |||
| 1129 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |||
| 1130 | + | if err := journal.Close(); err != nil { | |||
| 1131 | + | t.Fatal(err) | |||
| 1132 | + | } | |||
| 1133 | + | ||||
| 1134 | + | metadata, err := ReconcileResidentJournal(filepath.Join(root, spec.ID)) | |||
| 1135 | + | if err != nil { | |||
| 1136 | + | t.Fatal(err) | |||
| 1137 | + | } | |||
| 1138 | + | if metadata.State != ResidentIdle { | |||
| 1139 | + | t.Fatalf("reconciled state = %q, want idle", metadata.State) | |||
| 1140 | + | } | |||
| 1141 | + | records, err := ReadResidentJournal(transcript) | |||
| 1142 | + | if err != nil { | |||
| 1143 | + | t.Fatal(err) | |||
| 1144 | + | } | |||
| 1145 | + | for _, record := range records { | |||
| 1146 | + | if record.Type == residentRecordToolResult { | |||
| 1147 | + | t.Fatalf("reconciliation repaired a call dropped by compaction: %#v", record) | |||
| 1148 | + | } | |||
| 1149 | + | } | |||
| 1150 | + | messages, err := ReadResidentTranscriptMessages(filepath.Join(root, spec.ID)) | |||
| 1151 | + | if err != nil { | |||
| 1152 | + | t.Fatal(err) | |||
| 1153 | + | } | |||
| 1154 | + | if len(messages) != 1 || messages[0].Role != provider.RoleUser { | |||
| 1155 | + | t.Fatalf("resumed messages = %#v, want only the checkpoint summary", messages) | |||
| 1156 | + | } | |||
| 1157 | + | } | |||
| 1158 | + | ||||
| 1159 | + | func TestRecordCompactedAttributesTheStartedTurn(t *testing.T) { | |||
| 1160 | + | root := t.TempDir() | |||
| 1161 | + | journal, err := OpenResidentJournal(root, "turn-attribution") | |||
| 1162 | + | if err != nil { | |||
| 1163 | + | t.Fatal(err) | |||
| 1164 | + | } | |||
| 1165 | + | spec := ResidentChildSpec{ID: "turn-attribution", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |||
| 1166 | + | if err := journal.Accept(spec, "task"); err != nil { | |||
| 1167 | + | t.Fatal(err) | |||
| 1168 | + | } | |||
| 1169 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |||
| 1170 | + | t.Fatal(err) | |||
| 1171 | + | } | |||
| 1172 | + | // A follow-up accepted while turn-1 is still running must not steal the | |||
| 1173 | + | // checkpoint's turn attribution. | |||
| 1174 | + | if err := journal.AcceptFollowUp(spec, "turn-2", "queued follow up"); err != nil { | |||
| 1175 | + | t.Fatal(err) | |||
| 1176 | + | } | |||
| 1177 | + | if err := journal.RecordCompacted([]provider.Message{ | |||
| 1178 | + | {Role: provider.RoleUser, Content: []provider.Content{provider.TextBlock{Text: "## Context Summary (compacted)\n\ndigest"}}}, | |||
| 1179 | + | }); err != nil { | |||
| 1180 | + | t.Fatal(err) | |||
| 1181 | + | } | |||
| 1182 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |||
| 1183 | + | if err := journal.Close(); err != nil { | |||
| 1184 | + | t.Fatal(err) | |||
| 1185 | + | } | |||
| 1186 | + | records, err := ReadResidentJournal(transcript) | |||
| 1187 | + | if err != nil { | |||
| 1188 | + | t.Fatal(err) | |||
| 1189 | + | } | |||
| 1190 | + | for _, record := range records { | |||
| 1191 | + | if record.Type != residentRecordCompacted { | |||
| 1192 | + | continue | |||
| 1193 | + | } | |||
| 1194 | + | if record.TurnID != "turn-1" { | |||
| 1195 | + | t.Fatalf("checkpoint turn ID = %q, want the executing turn", record.TurnID) | |||
| 1196 | + | } | |||
| 1197 | + | return | |||
| 1198 | + | } | |||
| 1199 | + | t.Fatal("journal has no compaction checkpoint") | |||
| 1200 | + | } | |||
| 1201 | + | ||||
| 1202 | + | func TestReconcileResidentJournalIsIdempotentForCheckpointResults(t *testing.T) { | |||
| 1203 | + | root := t.TempDir() | |||
| 1204 | + | journal, err := OpenResidentJournal(root, "idempotent-checkpoint") | |||
| 1205 | + | if err != nil { | |||
| 1206 | + | t.Fatal(err) | |||
| 1207 | + | } | |||
| 1208 | + | spec := ResidentChildSpec{ID: "idempotent-checkpoint", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |||
| 1209 | + | if err := journal.Accept(spec, "task"); err != nil { | |||
| 1210 | + | t.Fatal(err) | |||
| 1211 | + | } | |||
| 1212 | + | if err := journal.AcceptFollowUp(spec, "turn-1", "follow up"); err != nil { | |||
| 1213 | + | t.Fatal(err) | |||
| 1214 | + | } | |||
| 1215 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |||
| 1216 | + | t.Fatal(err) | |||
| 1217 | + | } | |||
| 1218 | + | // A checkpoint can carry a result whose call was compacted away. That is | |||
| 1219 | + | // not repairable work, so reconciliation must stay a no-op. | |||
| 1220 | + | if err := journal.RecordCompacted([]provider.Message{{ | |||
| 1221 | + | Role: provider.RoleTool, | |||
| 1222 | + | Content: []provider.Content{provider.ToolResultBlock{ | |||
| 1223 | + | CallID: "compacted-call", | |||
| 1224 | + | Content: []provider.Content{provider.TextBlock{Text: "kept result"}}, | |||
| 1225 | + | }}, | |||
| 1226 | + | }}); err != nil { | |||
| 1227 | + | t.Fatal(err) | |||
| 1228 | + | } | |||
| 1229 | + | if err := journal.RecordTurnFinished(spec, "turn-1", nil); err != nil { | |||
| 1230 | + | t.Fatal(err) | |||
| 1231 | + | } | |||
| 1232 | + | if err := journal.Close(); err != nil { | |||
| 1233 | + | t.Fatal(err) | |||
| 1234 | + | } | |||
| 1235 | + | first, err := ReconcileResidentJournal(filepath.Join(root, spec.ID)) | |||
| 1236 | + | if err != nil { | |||
| 1237 | + | t.Fatal(err) | |||
| 1238 | + | } | |||
| 1239 | + | second, err := ReconcileResidentJournal(filepath.Join(root, spec.ID)) | |||
| 1240 | + | if err != nil { | |||
| 1241 | + | t.Fatal(err) | |||
| 1242 | + | } | |||
| 1243 | + | if !first.UpdatedAt.Equal(second.UpdatedAt) { | |||
| 1244 | + | t.Fatalf("reconciliation rewrote the lifecycle time: %v then %v", first.UpdatedAt, second.UpdatedAt) | |||
| 1245 | + | } | |||
| 1246 | + | if second.State != ResidentIdle { | |||
| 1247 | + | t.Fatalf("reconciled state = %q, want idle", second.State) | |||
| 1248 | + | } | |||
| 1249 | + | } | |||
| 1250 | + | ||||
| 1251 | + | func TestAppendSyncEnforcesTheReaderRecordBound(t *testing.T) { | |||
| 1252 | + | root := t.TempDir() | |||
| 1253 | + | journal, err := OpenResidentJournal(root, "record-bound") | |||
| 1254 | + | if err != nil { | |||
| 1255 | + | t.Fatal(err) | |||
| 1256 | + | } | |||
| 1257 | + | spec := ResidentChildSpec{ID: "record-bound", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |||
| 1258 | + | if err := journal.Accept(spec, "task"); err != nil { | |||
| 1259 | + | t.Fatal(err) | |||
| 1260 | + | } | |||
| 1261 | + | _, base := residentAssistantRecordSize(t, "") | |||
| 1262 | + | if base >= residentMaxRecordBytes-1 { | |||
| 1263 | + | t.Fatalf("empty assistant record = %d bytes, too large for a boundary check", base) | |||
| 1264 | + | } | |||
| 1265 | + | // A JSON record of max-1 bytes produces a max-byte line with its newline, | |||
| 1266 | + | // which is exactly what the readers accept. | |||
| 1267 | + | padding := residentMaxRecordBytes - 1 - base | |||
| 1268 | + | borderline, borderlineSize := residentAssistantRecordSize(t, strings.Repeat("x", padding)) | |||
| 1269 | + | if borderlineSize != residentMaxRecordBytes-1 { | |||
| 1270 | + | t.Fatalf("borderline record = %d bytes, want %d", borderlineSize, residentMaxRecordBytes-1) | |||
| 1271 | + | } | |||
| 1272 | + | if err := journal.appendSync(borderline); err != nil { | |||
| 1273 | + | t.Fatalf("appendSync rejected a reader-legal record: %v", err) | |||
| 1274 | + | } | |||
| 1275 | + | oversized, oversizedSize := residentAssistantRecordSize(t, strings.Repeat("x", padding+1)) | |||
| 1276 | + | if oversizedSize != residentMaxRecordBytes { | |||
| 1277 | + | t.Fatalf("oversized record = %d bytes, want %d", oversizedSize, residentMaxRecordBytes) | |||
| 1278 | + | } | |||
| 1279 | + | if err := journal.appendSync(oversized); err == nil || !strings.Contains(err.Error(), "too large") { | |||
| 1280 | + | t.Fatalf("appendSync error = %v, want the reader record bound", err) | |||
| 1281 | + | } | |||
| 1282 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |||
| 1283 | + | if err := journal.Close(); err != nil { | |||
| 1284 | + | t.Fatal(err) | |||
| 1285 | + | } | |||
| 1286 | + | records, err := ReadResidentJournal(transcript) | |||
| 1287 | + | if err != nil { | |||
| 1288 | + | t.Fatalf("journal unreadable after a boundary record: %v", err) | |||
| 1289 | + | } | |||
| 1290 | + | if len(records) != 2 { | |||
| 1291 | + | t.Fatalf("records = %d, want the accepted child and the borderline message", len(records)) | |||
| 1292 | + | } | |||
| 1293 | + | } | |||
| 1294 | + | ||||
| 1295 | + | func residentAssistantRecordSize(t *testing.T, text string) (residentRecord, int) { | |||
| 1296 | + | t.Helper() | |||
| 1297 | + | message, err := json.Marshal(provider.Message{ | |||
| 1298 | + | Role: provider.RoleAssistant, | |||
| 1299 | + | Content: []provider.Content{provider.TextBlock{Text: text}}, | |||
| 1300 | + | }) | |||
| 1301 | + | if err != nil { | |||
| 1302 | + | t.Fatal(err) | |||
| 1303 | + | } | |||
| 1304 | + | record := residentRecord{Version: residentJournalVersion, Type: residentRecordAssistant, Message: message} | |||
| 1305 | + | data, err := json.Marshal(record) | |||
| 1306 | + | if err != nil { | |||
| 1307 | + | t.Fatal(err) | |||
| 1308 | + | } | |||
| 1309 | + | return record, len(data) | |||
| 1310 | + | } | |||
| 1311 | + | ||||
| 1312 | + | func TestReconcileResidentJournalRepairsDanglingCheckpointCall(t *testing.T) { | |||
| 1313 | + | root := t.TempDir() | |||
| 1314 | + | journal, err := OpenResidentJournal(root, "checkpoint-dangling") | |||
| 1315 | + | if err != nil { | |||
| 1316 | + | t.Fatal(err) | |||
| 1317 | + | } | |||
| 1318 | + | spec := ResidentChildSpec{ID: "checkpoint-dangling", SessionID: "session", Provider: "openai", Model: "gpt-5"} | |||
| 1319 | + | if err := journal.Accept(spec, "task"); err != nil { | |||
| 1320 | + | t.Fatal(err) | |||
| 1321 | + | } | |||
| 1322 | + | if err := journal.AcceptFollowUp(spec, "turn-1", "follow up"); err != nil { | |||
| 1323 | + | t.Fatal(err) | |||
| 1324 | + | } | |||
| 1325 | + | if err := journal.RecordTurnStarted(spec, "turn-1"); err != nil { | |||
| 1326 | + | t.Fatal(err) | |||
| 1327 | + | } | |||
| 1328 | + | // The checkpoint kept a call whose result never arrived. | |||
| 1329 | + | if err := journal.RecordCompacted([]provider.Message{{ | |||
| 1330 | + | Role: provider.RoleAssistant, | |||
| 1331 | + | Content: []provider.Content{provider.ToolCallBlock{ | |||
| 1332 | + | ID: "compacted-call", | |||
| 1333 | + | Name: "bash", | |||
| 1334 | + | Arguments: json.RawMessage(`{"command":"pwd"}`), | |||
| 1335 | + | }}, | |||
| 1336 | + | }}); err != nil { | |||
| 1337 | + | t.Fatal(err) | |||
| 1338 | + | } | |||
| 1339 | + | if err := journal.RecordTurnFinished(spec, "turn-1", nil); err != nil { | |||
| 1340 | + | t.Fatal(err) | |||
| 1341 | + | } | |||
| 1342 | + | transcript := filepath.Join(journal.Dir(), residentTranscriptName) | |||
| 1343 | + | if err := journal.Close(); err != nil { | |||
| 1344 | + | t.Fatal(err) | |||
| 1345 | + | } | |||
| 1346 | + | dir := filepath.Join(root, spec.ID) | |||
| 1347 | + | if _, err := ReconcileResidentJournal(dir); err != nil { | |||
| 1348 | + | t.Fatal(err) | |||
| 1349 | + | } | |||
| 1350 | + | records, err := ReadResidentJournal(transcript) | |||
| 1351 | + | if err != nil { | |||
| 1352 | + | t.Fatal(err) | |||
| 1353 | + | } | |||
| 1354 | + | repaired := 0 | |||
| 1355 | + | for _, record := range records { | |||
| 1356 | + | if record.Type != residentRecordToolResult || record.ToolID != "compacted-call" { | |||
| 1357 | + | continue | |||
| 1358 | + | } | |||
| 1359 | + | if !isSyntheticInterruption(record.ToolResult) { | |||
| 1360 | + | t.Fatalf("repaired result = %s, want the interruption result", record.ToolResult) | |||
| 1361 | + | } | |||
| 1362 | + | repaired++ | |||
| 1363 | + | } | |||
| 1364 | + | if repaired != 1 { | |||
| 1365 | + | t.Fatalf("repaired results = %d, want exactly one synthetic result", repaired) | |||
| 1366 | + | } | |||
| 1367 | + | // Once the repair is durable, later reconciliations must be stable no-ops. | |||
| 1368 | + | second, err := ReconcileResidentJournal(dir) | |||
| 1369 | + | if err != nil { | |||
| 1370 | + | t.Fatal(err) | |||
| 1371 | + | } | |||
| 1372 | + | third, err := ReconcileResidentJournal(dir) | |||
| 1373 | + | if err != nil { | |||
| 1374 | + | t.Fatal(err) | |||
| 1375 | + | } | |||
| 1376 | + | if !second.UpdatedAt.Equal(third.UpdatedAt) { | |||
| 1377 | + | t.Fatalf("reconciliation kept rewriting the lifecycle time: %v then %v", second.UpdatedAt, third.UpdatedAt) | |||
| 1378 | + | } | |||
| 1379 | + | if records, err = ReadResidentJournal(transcript); err != nil { | |||
| 1380 | + | t.Fatal(err) | |||
| 1381 | + | } | |||
| 1382 | + | results := 0 | |||
| 1383 | + | for _, record := range records { | |||
| 1384 | + | if record.Type == residentRecordToolResult && record.ToolID == "compacted-call" { | |||
| 1385 | + | results++ | |||
| 1386 | + | } | |||
| 1387 | + | } | |||
| 1388 | + | if results != 1 { | |||
| 1389 | + | t.Fatalf("tool results = %d, want the single repaired result", results) | |||
| 1390 | + | } | |||
| 1391 | + | } | |||
M packages/core/dynamic_context.go +9 -0
| @@ -80,6 +80,15 @@ func isInternalContextMessage(message provider.Message) bool { | |||
| 80 | 80 | return message.Role == provider.RoleDeveloper && message.Meta[internalContextMarker] == "true" | |
| 81 | 81 | } | |
| 82 | 82 | ||
| 83 | + | // IsInternalContextMessage reports whether message is host-authored context | |
| 84 | + | // injected by the agent loop, such as a time snapshot or an extension | |
| 85 | + | // reminder, rather than finalized conversation history. Hosts that persist a | |
| 86 | + | // replacement transcript must exclude these messages: their value belongs to | |
| 87 | + | // the live request and replaying a stale copy would contradict it. | |
| 88 | + | func IsInternalContextMessage(message provider.Message) bool { | |
| 89 | + | return isInternalContextMessage(message) | |
| 90 | + | } | |
| 91 | + | ||
| 83 | 92 | func latestInternalContext(messages []provider.Message) (*provider.Message, []provider.Message) { | |
| 84 | 93 | filtered := make([]provider.Message, 0, len(messages)) | |
| 85 | 94 | var latest *provider.Message | |
| @@ -80,6 +80,15 @@ func isInternalContextMessage(message provider.Message) bool { | |||||
| 80 | return message.Role == provider.RoleDeveloper && message.Meta[internalContextMarker] == "true" | 80 | return message.Role == provider.RoleDeveloper && message.Meta[internalContextMarker] == "true" | ||
| 81 | } | 81 | } | ||
| 82 | 82 | ||||
| 83 | + | // IsInternalContextMessage reports whether message is host-authored context | |||
| 84 | + | // injected by the agent loop, such as a time snapshot or an extension | |||
| 85 | + | // reminder, rather than finalized conversation history. Hosts that persist a | |||
| 86 | + | // replacement transcript must exclude these messages: their value belongs to | |||
| 87 | + | // the live request and replaying a stale copy would contradict it. | |||
| 88 | + | func IsInternalContextMessage(message provider.Message) bool { | |||
| 89 | + | return isInternalContextMessage(message) | |||
| 90 | + | } | |||
| 91 | + | ||||
| 83 | func latestInternalContext(messages []provider.Message) (*provider.Message, []provider.Message) { | 92 | func latestInternalContext(messages []provider.Message) (*provider.Message, []provider.Message) { | ||
| 84 | filtered := make([]provider.Message, 0, len(messages)) | 93 | filtered := make([]provider.Message, 0, len(messages)) | ||
| 85 | var latest *provider.Message | 94 | var latest *provider.Message | ||