Merge pull request #79 from bnema/feat/queue-active-subagent-instructions
feat(subagents): queue follow-ups and reset run budgets
22 files · +1687 -195
M docs/subagents.md +3 -3
| @@ -131,10 +131,10 @@ When a subagent deadline expires, the supervisor requests worker shutdown and al | |||
| 131 | 131 | ||
| 132 | 132 | Use `/subagents resume-session <id>` to continue the existing session without replaying its original task. Use `/subagents restart-task <id>` only when intentionally starting the stored task again. Cancellation requests graceful shutdown first, then forcefully cancels after the configured grace period. | |
| 133 | 133 | ||
| 134 | - | When auto-subagents is enabled and permitted by launch-time policy, the model can manage workers with `subagent_status`, `subagent_stop`, and `subagent_resume`. Call `subagent_status` with `{}` to list workers visible to the active supervisor session, or pass `{"agent_id":"<id-or-unique-prefix>"}` to query one worker. Queries use in-process snapshots and never wait for a worker turn or process to finish; repeated calls or state inspection are not completion signals. The JSON response contains the worker id, a normalized lifecycle state (`starting`, `running`, `completed`, `failed`, `cancelled`, or `detached`), the underlying `process_state` and `turn_state`, start/update/finish timestamps, a bounded first-line task summary, and terminal-result metadata/reference when available. It intentionally omits prompts, transcripts, result output, credentials, provider settings, and filesystem paths. | |
| 134 | + | When auto-subagents is enabled and permitted by launch-time policy, the model can manage workers with `subagent_status`, `subagent_stop`, and `subagent_resume`. Call `subagent_status` with `{}` to list workers visible to the active supervisor session, or pass `{"agent_id":"<id-or-unique-prefix>"}` to query one worker. Queries use in-process snapshots and never wait for a worker turn or process to finish; repeated calls or state inspection are not completion signals. The JSON response contains the worker id, a normalized lifecycle state (`starting`, `running`, `completed`, `failed`, `cancelled`, or `detached`), the underlying `process_state` and `turn_state`, `lifetime_turns` and `current_run_turns` counters, start/update/finish timestamps, a bounded first-line task summary, and terminal-result metadata/reference when available. It intentionally omits prompts, transcripts, result output, credentials, provider settings, and filesystem paths. | |
| 135 | 135 | ||
| 136 | - | Use `subagent_stop` with an `agent_id` to request termination of a stuck worker through the normal graceful-then-forceful supervisor lifecycle. Its `stop_requested` result confirms that shutdown has started; wait for the host completion update before attempting a restart. Use `subagent_resume` with an `agent_id` and `prompt` to send a new follow-up to an idle worker or to restart a terminal worker from its retained session with that follow-up as the initial turn. It never replays the original task, and its completion is delivered to the manager through the normal auto-subagent update. An unknown or ambiguous id is returned as a model-visible tool error; the existing `/subagents` dashboard and final-result notifications remain unchanged. An explicit `--tools` allowlist must include each manager tool that the model needs, and `subagent_status` must not be polled solely to wait for completion. | |
| 136 | + | Use `subagent_stop` with an `agent_id` to request termination of a stuck worker through the normal graceful-then-forceful supervisor lifecycle. Its `stop_requested` result confirms that shutdown has started; wait for the host completion update before attempting a restart. Use `subagent_resume` with an `agent_id` and `prompt` to start a fresh max-turn budget while retaining the transcript/session/context, send the follow-up to an idle worker, queue it for the next available message turn when the worker is active, or restart a terminal worker from its retained session with that follow-up as the initial turn. A follow-up remains durably pending until its matching user message is flushed to the session and the delegated `turn.started` boundary is recorded; a worker-ready event alone does not acknowledge it. Automatic retries, model loops, and compaction remain within the current run budget; multiple active follow-ups are delivered in order. Commands have stable identities so cancellation or an ambiguous socket-write outcome can be retried without replaying an accepted command. It never replays the original task, and its completion is delivered to the manager through the normal auto-subagent update. An unknown or ambiguous id is returned as a model-visible tool error; the existing `/subagents` dashboard and final-result notifications remain unchanged. An explicit `--tools` allowlist must include each manager tool that the model needs, and `subagent_status` must not be polled solely to wait for completion. | |
| 137 | 137 | ||
| 138 | 138 | ## Resource policy | |
| 139 | 139 | ||
| 140 | - | The persisted `subagents` config object supports `max_concurrent`, `max_concurrent_per_parent`, `queue_timeout`, `default_timeout`, `max_turns`, output caps, allowed tools/roots, heartbeat and idle timeouts. Previously saved `max_total_spawned` values are accepted but ignored. A missing or non-positive configured `max_turns` uses the default ceiling of 3. The `max_turns` field in `subagent_spawn` is optional: omit it to use the policy ceiling, or provide a value from 1 through the configured maximum. Concurrency limits apply to slash commands, `subagent_spawn`, and batch operations. A child cannot create descendants in v1. Per-agent timeouts are retained across reload/resume. Packaged `zut run` agents keep their declared capability ceiling by disabling subagent delegation; profiles are not a substitute for that permission boundary. A primary whose launch-time policy disables delegation still receives the strict orchestrator contract, but does not receive actionable profile metadata or a replacement implementation path. | |
| 140 | + | The persisted `subagents` config object supports `max_concurrent`, `max_concurrent_per_parent`, `queue_timeout`, `default_timeout`, `max_turns`, output caps, allowed tools/roots, heartbeat and idle timeouts. `max_turns` bounds one active worker run; lifetime and current-run turn counters remain persisted and visible in worker status. An explicit `subagent_resume` starts a new run with a fresh budget without replaying the original task or losing session context. Previously saved `max_total_spawned` values are accepted but ignored. A missing or non-positive configured `max_turns` uses the default ceiling of 3. The `max_turns` field in `subagent_spawn` is optional: omit it to use the policy ceiling, or provide a value from 1 through the configured maximum. Concurrency limits apply to slash commands, `subagent_spawn`, and batch operations. A child cannot create descendants in v1. Per-agent timeouts are retained across reload/resume. Packaged `zut run` agents keep their declared capability ceiling by disabling subagent delegation; profiles are not a substitute for that permission boundary. A primary whose launch-time policy disables delegation still receives the strict orchestrator contract, but does not receive actionable profile metadata or a replacement implementation path. | |
| @@ -131,10 +131,10 @@ When a subagent deadline expires, the supervisor requests worker shutdown and al | |||||
| 131 | 131 | ||||
| 132 | Use `/subagents resume-session <id>` to continue the existing session without replaying its original task. Use `/subagents restart-task <id>` only when intentionally starting the stored task again. Cancellation requests graceful shutdown first, then forcefully cancels after the configured grace period. | 132 | Use `/subagents resume-session <id>` to continue the existing session without replaying its original task. Use `/subagents restart-task <id>` only when intentionally starting the stored task again. Cancellation requests graceful shutdown first, then forcefully cancels after the configured grace period. | ||
| 133 | 133 | ||||
| 134 | - | When auto-subagents is enabled and permitted by launch-time policy, the model can manage workers with `subagent_status`, `subagent_stop`, and `subagent_resume`. Call `subagent_status` with `{}` to list workers visible to the active supervisor session, or pass `{"agent_id":"<id-or-unique-prefix>"}` to query one worker. Queries use in-process snapshots and never wait for a worker turn or process to finish; repeated calls or state inspection are not completion signals. The JSON response contains the worker id, a normalized lifecycle state (`starting`, `running`, `completed`, `failed`, `cancelled`, or `detached`), the underlying `process_state` and `turn_state`, start/update/finish timestamps, a bounded first-line task summary, and terminal-result metadata/reference when available. It intentionally omits prompts, transcripts, result output, credentials, provider settings, and filesystem paths. | 134 | + | When auto-subagents is enabled and permitted by launch-time policy, the model can manage workers with `subagent_status`, `subagent_stop`, and `subagent_resume`. Call `subagent_status` with `{}` to list workers visible to the active supervisor session, or pass `{"agent_id":"<id-or-unique-prefix>"}` to query one worker. Queries use in-process snapshots and never wait for a worker turn or process to finish; repeated calls or state inspection are not completion signals. The JSON response contains the worker id, a normalized lifecycle state (`starting`, `running`, `completed`, `failed`, `cancelled`, or `detached`), the underlying `process_state` and `turn_state`, `lifetime_turns` and `current_run_turns` counters, start/update/finish timestamps, a bounded first-line task summary, and terminal-result metadata/reference when available. It intentionally omits prompts, transcripts, result output, credentials, provider settings, and filesystem paths. |
| 135 | 135 | ||||
| 136 | - | Use `subagent_stop` with an `agent_id` to request termination of a stuck worker through the normal graceful-then-forceful supervisor lifecycle. Its `stop_requested` result confirms that shutdown has started; wait for the host completion update before attempting a restart. Use `subagent_resume` with an `agent_id` and `prompt` to send a new follow-up to an idle worker or to restart a terminal worker from its retained session with that follow-up as the initial turn. It never replays the original task, and its completion is delivered to the manager through the normal auto-subagent update. An unknown or ambiguous id is returned as a model-visible tool error; the existing `/subagents` dashboard and final-result notifications remain unchanged. An explicit `--tools` allowlist must include each manager tool that the model needs, and `subagent_status` must not be polled solely to wait for completion. | 136 | + | Use `subagent_stop` with an `agent_id` to request termination of a stuck worker through the normal graceful-then-forceful supervisor lifecycle. Its `stop_requested` result confirms that shutdown has started; wait for the host completion update before attempting a restart. Use `subagent_resume` with an `agent_id` and `prompt` to start a fresh max-turn budget while retaining the transcript/session/context, send the follow-up to an idle worker, queue it for the next available message turn when the worker is active, or restart a terminal worker from its retained session with that follow-up as the initial turn. A follow-up remains durably pending until its matching user message is flushed to the session and the delegated `turn.started` boundary is recorded; a worker-ready event alone does not acknowledge it. Automatic retries, model loops, and compaction remain within the current run budget; multiple active follow-ups are delivered in order. Commands have stable identities so cancellation or an ambiguous socket-write outcome can be retried without replaying an accepted command. It never replays the original task, and its completion is delivered to the manager through the normal auto-subagent update. An unknown or ambiguous id is returned as a model-visible tool error; the existing `/subagents` dashboard and final-result notifications remain unchanged. An explicit `--tools` allowlist must include each manager tool that the model needs, and `subagent_status` must not be polled solely to wait for completion. |
| 137 | 137 | ||||
| 138 | ## Resource policy | 138 | ## Resource policy | ||
| 139 | 139 | ||||
| 140 | - | The persisted `subagents` config object supports `max_concurrent`, `max_concurrent_per_parent`, `queue_timeout`, `default_timeout`, `max_turns`, output caps, allowed tools/roots, heartbeat and idle timeouts. Previously saved `max_total_spawned` values are accepted but ignored. A missing or non-positive configured `max_turns` uses the default ceiling of 3. The `max_turns` field in `subagent_spawn` is optional: omit it to use the policy ceiling, or provide a value from 1 through the configured maximum. Concurrency limits apply to slash commands, `subagent_spawn`, and batch operations. A child cannot create descendants in v1. Per-agent timeouts are retained across reload/resume. Packaged `zut run` agents keep their declared capability ceiling by disabling subagent delegation; profiles are not a substitute for that permission boundary. A primary whose launch-time policy disables delegation still receives the strict orchestrator contract, but does not receive actionable profile metadata or a replacement implementation path. | 140 | + | The persisted `subagents` config object supports `max_concurrent`, `max_concurrent_per_parent`, `queue_timeout`, `default_timeout`, `max_turns`, output caps, allowed tools/roots, heartbeat and idle timeouts. `max_turns` bounds one active worker run; lifetime and current-run turn counters remain persisted and visible in worker status. An explicit `subagent_resume` starts a new run with a fresh budget without replaying the original task or losing session context. Previously saved `max_total_spawned` values are accepted but ignored. A missing or non-positive configured `max_turns` uses the default ceiling of 3. The `max_turns` field in `subagent_spawn` is optional: omit it to use the policy ceiling, or provide a value from 1 through the configured maximum. Concurrency limits apply to slash commands, `subagent_spawn`, and batch operations. A child cannot create descendants in v1. Per-agent timeouts are retained across reload/resume. Packaged `zut run` agents keep their declared capability ceiling by disabling subagent delegation; profiles are not a substitute for that permission boundary. A primary whose launch-time policy disables delegation still receives the strict orchestrator contract, but does not receive actionable profile metadata or a replacement implementation path. |
M packages/agent/args.go +26 -1
| @@ -139,8 +139,13 @@ type Args struct { | |||
| 139 | 139 | // --subagent-worker. | |
| 140 | 140 | SubagentWorker string | |
| 141 | 141 | ||
| 142 | - | // SubagentMaxTurns limits prompt-level turns in worker mode. | |
| 142 | + | // SubagentMaxTurns limits message turns in one worker run. | |
| 143 | 143 | SubagentMaxTurns int | |
| 144 | + | // SubagentLifetimeTurns and SubagentRunTurns are supervisor-provided | |
| 145 | + | // counters used to resume a worker without losing observability or its | |
| 146 | + | // current-run budget. They are internal worker flags, not user policy. | |
| 147 | + | SubagentLifetimeTurns int | |
| 148 | + | SubagentRunTurns int | |
| 144 | 149 | ||
| 145 | 150 | // Subagent selects a named markdown profile for a subagent child. | |
| 146 | 151 | // It is intentionally an internal child flag; the parent subagent tool | |
| @@ -371,6 +376,26 @@ func ParseArgs(in []string) (Args, error) { | |||
| 371 | 376 | return a, fmt.Errorf("--max-turns must be a positive integer") | |
| 372 | 377 | } | |
| 373 | 378 | a.SubagentMaxTurns = n | |
| 379 | + | case "--subagent-lifetime-turns": | |
| 380 | + | v, err := want(&i, arg) | |
| 381 | + | if err != nil { | |
| 382 | + | return a, err | |
| 383 | + | } | |
| 384 | + | var n int | |
| 385 | + | if _, err := fmt.Sscanf(v, "%d", &n); err != nil || n < 0 { | |
| 386 | + | return a, fmt.Errorf("--subagent-lifetime-turns must be a non-negative integer") | |
| 387 | + | } | |
| 388 | + | a.SubagentLifetimeTurns = n | |
| 389 | + | case "--subagent-run-turns": | |
| 390 | + | v, err := want(&i, arg) | |
| 391 | + | if err != nil { | |
| 392 | + | return a, err | |
| 393 | + | } | |
| 394 | + | var n int | |
| 395 | + | if _, err := fmt.Sscanf(v, "%d", &n); err != nil || n < 0 { | |
| 396 | + | return a, fmt.Errorf("--subagent-run-turns must be a non-negative integer") | |
| 397 | + | } | |
| 398 | + | a.SubagentRunTurns = n | |
| 374 | 399 | default: | |
| 375 | 400 | if strings.HasPrefix(arg, "-") && arg != "-" { | |
| 376 | 401 | return a, fmt.Errorf("unknown flag %q", arg) | |
| @@ -139,8 +139,13 @@ type Args struct { | |||||
| 139 | // --subagent-worker. | 139 | // --subagent-worker. | ||
| 140 | SubagentWorker string | 140 | SubagentWorker string | ||
| 141 | 141 | ||||
| 142 | - | // SubagentMaxTurns limits prompt-level turns in worker mode. | 142 | + | // SubagentMaxTurns limits message turns in one worker run. |
| 143 | SubagentMaxTurns int | 143 | SubagentMaxTurns int | ||
| 144 | + | // SubagentLifetimeTurns and SubagentRunTurns are supervisor-provided | |||
| 145 | + | // counters used to resume a worker without losing observability or its | |||
| 146 | + | // current-run budget. They are internal worker flags, not user policy. | |||
| 147 | + | SubagentLifetimeTurns int | |||
| 148 | + | SubagentRunTurns int | |||
| 144 | 149 | ||||
| 145 | // Subagent selects a named markdown profile for a subagent child. | 150 | // Subagent selects a named markdown profile for a subagent child. | ||
| 146 | // It is intentionally an internal child flag; the parent subagent tool | 151 | // It is intentionally an internal child flag; the parent subagent tool | ||
| @@ -371,6 +376,26 @@ func ParseArgs(in []string) (Args, error) { | |||||
| 371 | return a, fmt.Errorf("--max-turns must be a positive integer") | 376 | return a, fmt.Errorf("--max-turns must be a positive integer") | ||
| 372 | } | 377 | } | ||
| 373 | a.SubagentMaxTurns = n | 378 | a.SubagentMaxTurns = n | ||
| 379 | + | case "--subagent-lifetime-turns": | |||
| 380 | + | v, err := want(&i, arg) | |||
| 381 | + | if err != nil { | |||
| 382 | + | return a, err | |||
| 383 | + | } | |||
| 384 | + | var n int | |||
| 385 | + | if _, err := fmt.Sscanf(v, "%d", &n); err != nil || n < 0 { | |||
| 386 | + | return a, fmt.Errorf("--subagent-lifetime-turns must be a non-negative integer") | |||
| 387 | + | } | |||
| 388 | + | a.SubagentLifetimeTurns = n | |||
| 389 | + | case "--subagent-run-turns": | |||
| 390 | + | v, err := want(&i, arg) | |||
| 391 | + | if err != nil { | |||
| 392 | + | return a, err | |||
| 393 | + | } | |||
| 394 | + | var n int | |||
| 395 | + | if _, err := fmt.Sscanf(v, "%d", &n); err != nil || n < 0 { | |||
| 396 | + | return a, fmt.Errorf("--subagent-run-turns must be a non-negative integer") | |||
| 397 | + | } | |||
| 398 | + | a.SubagentRunTurns = n | |||
| 374 | default: | 399 | default: | ||
| 375 | if strings.HasPrefix(arg, "-") && arg != "-" { | 400 | if strings.HasPrefix(arg, "-") && arg != "-" { | ||
| 376 | return a, fmt.Errorf("unknown flag %q", arg) | 401 | return a, fmt.Errorf("unknown flag %q", arg) | ||
M packages/agent/subagent_worker.go +149 -44
| @@ -36,6 +36,56 @@ import ( | |||
| 36 | 36 | // production; tests use the stubchild binary under | |
| 37 | 37 | // packages/agent/subagents/testdata/cmd/stubchild instead of the real model | |
| 38 | 38 | // loop. | |
| 39 | + | // workerTurnBudget is deliberately independent of the model loop: one | |
| 40 | + | // admitted message turn consumes one current-run and lifetime turn, while | |
| 41 | + | // retries, compaction, and provider/tool loops remain inside that turn. | |
| 42 | + | type workerTurnBudget struct { | |
| 43 | + | sequence int | |
| 44 | + | lifetime int | |
| 45 | + | current int | |
| 46 | + | } | |
| 47 | + | ||
| 48 | + | type sessionPersistenceState struct { | |
| 49 | + | mu sync.Mutex | |
| 50 | + | errValue error | |
| 51 | + | } | |
| 52 | + | ||
| 53 | + | func (s *sessionPersistenceState) record(err error) { | |
| 54 | + | if err == nil { | |
| 55 | + | return | |
| 56 | + | } | |
| 57 | + | s.mu.Lock() | |
| 58 | + | if s.errValue == nil { | |
| 59 | + | s.errValue = err | |
| 60 | + | } | |
| 61 | + | s.mu.Unlock() | |
| 62 | + | } | |
| 63 | + | ||
| 64 | + | func (s *sessionPersistenceState) failed() bool { | |
| 65 | + | s.mu.Lock() | |
| 66 | + | defer s.mu.Unlock() | |
| 67 | + | return s.errValue != nil | |
| 68 | + | } | |
| 69 | + | ||
| 70 | + | func (s *sessionPersistenceState) err() error { | |
| 71 | + | s.mu.Lock() | |
| 72 | + | defer s.mu.Unlock() | |
| 73 | + | return s.errValue | |
| 74 | + | } | |
| 75 | + | ||
| 76 | + | func (b *workerTurnBudget) start(maxTurns int, newRun bool) (step, lifetime, current int, admitted bool) { | |
| 77 | + | if newRun { | |
| 78 | + | b.current = 0 | |
| 79 | + | } | |
| 80 | + | b.sequence++ | |
| 81 | + | if maxTurns > 0 && b.current >= maxTurns { | |
| 82 | + | return b.sequence, b.lifetime, b.current, false | |
| 83 | + | } | |
| 84 | + | b.lifetime++ | |
| 85 | + | b.current++ | |
| 86 | + | return b.sequence, b.lifetime, b.current, true | |
| 87 | + | } | |
| 88 | + | ||
| 39 | 89 | func runSubagentWorkerMode(ctx context.Context, args Args, version string) error { | |
| 40 | 90 | if args.SubagentWorker == "" { | |
| 41 | 91 | return fmt.Errorf("--subagent-worker requires a socket path") | |
| @@ -74,6 +124,7 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||
| 74 | 124 | if err != nil { | |
| 75 | 125 | return err | |
| 76 | 126 | } | |
| 127 | + | var sessionPersistence sessionPersistenceState | |
| 77 | 128 | if sess != nil { | |
| 78 | 129 | var providerName, model string | |
| 79 | 130 | sess, ag, providerName, model, err = applyInitialSessionResume(ctx, args, r, extMgr, sess, ag) | |
| @@ -82,6 +133,18 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||
| 82 | 133 | } | |
| 83 | 134 | r.Provider, r.Model = providerName, model | |
| 84 | 135 | ag.OnToolResult = func(_ string, result core.ToolResult) { persistExtensionToolResult(extMgr, sess, result) } | |
| 136 | + | // A prompt is acknowledged by the supervisor only after this callback | |
| 137 | + | // has appended and synced the matching user message. This closes the | |
| 138 | + | // crash window between accepting a follow-up and writing session.json. | |
| 139 | + | ag.OnMessageAppended = func(message provider.Message) { | |
| 140 | + | if err := sess.AppendMessage(message); err != nil { | |
| 141 | + | sessionPersistence.record(err) | |
| 142 | + | return | |
| 143 | + | } | |
| 144 | + | if err := sess.Sync(); err != nil { | |
| 145 | + | sessionPersistence.record(err) | |
| 146 | + | } | |
| 147 | + | } | |
| 85 | 148 | defer sess.Close() | |
| 86 | 149 | } | |
| 87 | 150 | announceSession(extMgr, sess) | |
| @@ -113,9 +176,12 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||
| 113 | 176 | em := newSubagentEmitter(os.Stdout, logMirror) | |
| 114 | 177 | em.setProtocolIdentity(os.Getenv("ZUT_SUBAGENT_AGENT_ID")) | |
| 115 | 178 | em.emit("agent.ready", map[string]any{ | |
| 116 | - | "version": version, | |
| 117 | - | "cwd": r.CWD, | |
| 118 | - | "model": r.Model, | |
| 179 | + | "version": version, | |
| 180 | + | "cwd": r.CWD, | |
| 181 | + | "model": r.Model, | |
| 182 | + | "lifetime_turns": args.SubagentLifetimeTurns, | |
| 183 | + | "current_run_turns": args.SubagentRunTurns, | |
| 184 | + | "max_turns": args.SubagentMaxTurns, | |
| 119 | 185 | }) | |
| 120 | 186 | ||
| 121 | 187 | // Heartbeats make a live-but-detached child distinguishable from a | |
| @@ -145,63 +211,82 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||
| 145 | 211 | // Keep a per-turn cancel so the "cancel" inbox message can interrupt | |
| 146 | 212 | // an in-flight turn without tearing down the whole daemon. | |
| 147 | 213 | var ( | |
| 148 | - | mu sync.Mutex | |
| 149 | - | cancelFn context.CancelFunc | |
| 150 | - | busyTurn bool | |
| 151 | - | turnNo = initialTurnNumber(os.Getenv("ZUT_SUBAGENT_EVENT_LOG")) | |
| 214 | + | mu sync.Mutex | |
| 215 | + | cancelFn context.CancelFunc | |
| 216 | + | busyTurn bool | |
| 217 | + | budget = workerTurnBudget{ | |
| 218 | + | sequence: initialTurnNumber(os.Getenv("ZUT_SUBAGENT_EVENT_LOG")), | |
| 219 | + | lifetime: args.SubagentLifetimeTurns, | |
| 220 | + | current: args.SubagentRunTurns, | |
| 221 | + | } | |
| 152 | 222 | turnWG sync.WaitGroup | |
| 153 | 223 | closing bool | |
| 224 | + | turnPending bool | |
| 225 | + | seenCommands = make(map[string]struct{}) | |
| 154 | 226 | shutdown = make(chan struct{}) | |
| 155 | 227 | shutdownOnce sync.Once | |
| 156 | 228 | ) | |
| 157 | 229 | maxOutputBytes, maxOutputLines := workerOutputLimits() | |
| 158 | 230 | ||
| 159 | - | runOne := func(prompt string) { | |
| 231 | + | runOne := func(prompt string, newRun bool, commandID string) { | |
| 160 | 232 | mu.Lock() | |
| 161 | 233 | // launchTurn admits the goroutine to turnWG before it starts so | |
| 162 | 234 | // shutdown can join it. A shutdown can win the scheduling race | |
| 163 | 235 | // after that admission but before this goroutine acquires mu; in | |
| 164 | 236 | // that case do not start a new Prompt after waitForActiveTurn has | |
| 165 | 237 | // declared the worker closing. | |
| 166 | - | if closing { | |
| 238 | + | if closing || busyTurn { | |
| 167 | 239 | mu.Unlock() | |
| 240 | + | em.emit("error", map[string]any{"code": "turn_rejected", "reason": "busy", "command_id": commandID, "message": "worker is busy"}) | |
| 168 | 241 | return | |
| 169 | 242 | } | |
| 170 | - | if args.SubagentMaxTurns > 0 && turnNo >= args.SubagentMaxTurns { | |
| 171 | - | // Consume a unique rejected attempt so repeated prompts after | |
| 172 | - | // the limit cannot reuse the same turn id in the event log. | |
| 173 | - | turnNo++ | |
| 174 | - | step := turnNo | |
| 243 | + | busyTurn = true | |
| 244 | + | turnPending = false | |
| 245 | + | step, lifetime, currentRun, admitted := budget.start(args.SubagentMaxTurns, newRun) | |
| 246 | + | if !admitted { | |
| 247 | + | busyTurn = false | |
| 175 | 248 | mu.Unlock() | |
| 176 | 249 | turnID := fmt.Sprintf("turn-%d", step) | |
| 177 | - | errPayload := map[string]any{"code": "max_turns", "message": "maximum subagent turns reached"} | |
| 250 | + | errPayload := map[string]any{"code": "turn_rejected", "reason": "max_turns", "command_id": commandID, "message": "maximum subagent turns reached"} | |
| 251 | + | em.emit("error", errPayload) | |
| 178 | 252 | em.emit("turn.result", map[string]any{"status": "failed", "turn_id": turnID, "error": errPayload}) | |
| 179 | 253 | em.emit("turn.failed", map[string]any{"turn_id": turnID, "error": errPayload}) | |
| 180 | 254 | em.emit("turn_end", map[string]any{"step": step, "turn_id": turnID, "error": errPayload["message"]}) | |
| 181 | - | em.emit("agent.idle", map[string]any{"turn_id": turnID}) | |
| 182 | - | return | |
| 183 | - | } | |
| 184 | - | if busyTurn { | |
| 185 | - | // Drop concurrent turns rather than queuing. The | |
| 186 | - | // supervisor protocol assumes one outstanding turn per | |
| 187 | - | // agent; if a user really wants to interrupt and start | |
| 188 | - | // another, they should send "cancel" first. | |
| 189 | - | mu.Unlock() | |
| 190 | - | em.emit("error", map[string]any{"message": "agent busy; send 'cancel' first"}) | |
| 255 | + | // Max-turn rejection ends the current run, but the worker remains | |
| 256 | + | // alive and can accept a queued follow-up as a fresh run. | |
| 257 | + | em.emit("agent.idle", map[string]any{ | |
| 258 | + | "turn_id": turnID, | |
| 259 | + | "lifetime_turns": lifetime, | |
| 260 | + | "current_run_turns": currentRun, | |
| 261 | + | }) | |
| 191 | 262 | return | |
| 192 | 263 | } | |
| 193 | - | busyTurn = true | |
| 194 | - | turnNo++ | |
| 195 | - | step := turnNo | |
| 264 | + | turnID := fmt.Sprintf("turn-%d", step) | |
| 196 | 265 | c, cancel := context.WithCancel(ctx) | |
| 197 | 266 | cancelFn = cancel | |
| 198 | 267 | mu.Unlock() | |
| 199 | 268 | ||
| 200 | - | turnID := fmt.Sprintf("turn-%d", step) | |
| 201 | - | em.emit("turn.started", map[string]any{"step": step, "turn_id": turnID}) | |
| 202 | - | ||
| 269 | + | turnStarted := false | |
| 203 | 270 | sink := func(ev core.AgentEvent) { | |
| 204 | - | em.emit(ev.Type(), subagentEventData(ev)) | |
| 271 | + | data := subagentEventData(ev) | |
| 272 | + | if ev.Type() == "user_message" && !turnStarted { | |
| 273 | + | // Prompt invokes OnMessageAppended before delivering this sink | |
| 274 | + | // event, so the session row is durable before the lifecycle | |
| 275 | + | // acknowledgement is written. | |
| 276 | + | em.emit(ev.Type(), data) | |
| 277 | + | if !sessionPersistence.failed() { | |
| 278 | + | em.emit("turn.started", map[string]any{"step": step, "turn_id": turnID, "command_id": commandID, "lifetime_turns": lifetime, "current_run_turns": currentRun}) | |
| 279 | + | turnStarted = true | |
| 280 | + | } | |
| 281 | + | return | |
| 282 | + | } | |
| 283 | + | if ev.Type() == "turn_start" { | |
| 284 | + | // A provider/model-loop turn is nested inside the one delegated | |
| 285 | + | // message turn admitted above. Mark it on the wire so the | |
| 286 | + | // supervisor cannot mistake it for a new worker turn. | |
| 287 | + | data["nested_turn"] = true | |
| 288 | + | } | |
| 289 | + | em.emit(ev.Type(), data) | |
| 205 | 290 | } | |
| 206 | 291 | ||
| 207 | 292 | var persistCompaction func([]provider.Message) error | |
| @@ -209,7 +294,12 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||
| 209 | 294 | persistCompaction = sess.AppendCompaction | |
| 210 | 295 | } | |
| 211 | 296 | start, err := promptWithContextRecovery(c, ag, prompt, sink, persistCompaction) | |
| 212 | - | WriteNewTranscript(ag, sess, start) | |
| 297 | + | if sess != nil && ag.OnMessageAppended == nil { | |
| 298 | + | WriteNewTranscript(ag, sess, start) | |
| 299 | + | } | |
| 300 | + | if sessionErr := sessionPersistence.err(); err == nil && sessionErr != nil { | |
| 301 | + | err = sessionErr | |
| 302 | + | } | |
| 213 | 303 | ||
| 214 | 304 | output := finalAssistantText(ag.Messages()[start:]) | |
| 215 | 305 | resultStatus := "succeeded" | |
| @@ -245,24 +335,26 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||
| 245 | 335 | // The supervisor treats agent.idle as permission to start a follow-up. | |
| 246 | 336 | // Publish it only after the busy gate has opened, otherwise a manager | |
| 247 | 337 | // can send a turn that the inbox loop drops as still busy. | |
| 248 | - | em.emit("agent.idle", map[string]any{"turn_id": turnID}) | |
| 338 | + | em.emit("agent.idle", map[string]any{"turn_id": turnID, "lifetime_turns": lifetime, "current_run_turns": currentRun}) | |
| 249 | 339 | } | |
| 250 | 340 | ||
| 251 | - | launchTurn := func(prompt string) { | |
| 341 | + | launchTurn := func(prompt string, newRun bool, commandID string) bool { | |
| 252 | 342 | mu.Lock() | |
| 253 | - | if closing { | |
| 343 | + | if closing || busyTurn || turnPending { | |
| 254 | 344 | mu.Unlock() | |
| 255 | - | return | |
| 345 | + | em.emit("error", map[string]any{"code": "turn_rejected", "reason": "busy", "command_id": commandID, "message": "worker is busy"}) | |
| 346 | + | return false | |
| 256 | 347 | } | |
| 257 | - | // Register the goroutine before launching it. Shutdown can then | |
| 258 | - | // close the gate and wait for every turn that was admitted, even | |
| 259 | - | // if runOne has not acquired mu and set busyTurn yet. | |
| 348 | + | // Reserve the worker before launching the goroutine. A second command | |
| 349 | + | // cannot be accepted in the gap before runOne acquires mu. | |
| 350 | + | turnPending = true | |
| 260 | 351 | turnWG.Add(1) | |
| 261 | 352 | mu.Unlock() | |
| 262 | 353 | go func() { | |
| 263 | 354 | defer turnWG.Done() | |
| 264 | - | runOne(prompt) | |
| 355 | + | runOne(prompt, newRun, commandID) | |
| 265 | 356 | }() | |
| 357 | + | return true | |
| 266 | 358 | } | |
| 267 | 359 | ||
| 268 | 360 | waitForActiveTurn := func(cancel bool) { | |
| @@ -287,7 +379,7 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||
| 287 | 379 | // "starts working" the moment it boots, matching what users | |
| 288 | 380 | // expect from `/subagents new <task>`. | |
| 289 | 381 | if args.Prompt != "" { | |
| 290 | - | launchTurn(args.Prompt) | |
| 382 | + | launchTurn(args.Prompt, false, "") | |
| 291 | 383 | } | |
| 292 | 384 | ||
| 293 | 385 | // Inbox loop: one supervisor message at a time. We don't spawn | |
| @@ -333,10 +425,20 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||
| 333 | 425 | case subagents.CommandTurnStart: | |
| 334 | 426 | var payload subagents.TurnStartPayload | |
| 335 | 427 | if err := command.DecodePayload(&payload); err != nil { | |
| 336 | - | em.emit("error", map[string]any{"message": "invalid turn.start payload"}) | |
| 428 | + | em.emit("error", map[string]any{"code": "turn_rejected", "reason": "invalid_payload", "command_id": command.MessageID, "message": "invalid turn.start payload"}) | |
| 337 | 429 | continue | |
| 338 | 430 | } | |
| 339 | - | launchTurn(payload.Prompt) | |
| 431 | + | mu.Lock() | |
| 432 | + | _, duplicate := seenCommands[command.MessageID] | |
| 433 | + | mu.Unlock() | |
| 434 | + | if duplicate && command.MessageID != "" { | |
| 435 | + | continue | |
| 436 | + | } | |
| 437 | + | if launchTurn(payload.Prompt, payload.NewRun, command.MessageID) && command.MessageID != "" { | |
| 438 | + | mu.Lock() | |
| 439 | + | seenCommands[command.MessageID] = struct{}{} | |
| 440 | + | mu.Unlock() | |
| 441 | + | } | |
| 340 | 442 | default: | |
| 341 | 443 | // Unknown commands are rejected rather than interpreted as | |
| 342 | 444 | // child-originated control requests. | |
| @@ -555,6 +657,9 @@ func initialTurnNumber(path string) int { | |||
| 555 | 657 | } | |
| 556 | 658 | maxStep := 0 | |
| 557 | 659 | for _, ev := range events { | |
| 660 | + | if !subagents.IsDelegatedTurnStart(ev) { | |
| 661 | + | continue | |
| 662 | + | } | |
| 558 | 663 | if step, ok := ev.Data["step"].(float64); ok && int(step) > maxStep { | |
| 559 | 664 | maxStep = int(step) | |
| 560 | 665 | } | |
| @@ -36,6 +36,56 @@ import ( | |||||
| 36 | // production; tests use the stubchild binary under | 36 | // production; tests use the stubchild binary under | ||
| 37 | // packages/agent/subagents/testdata/cmd/stubchild instead of the real model | 37 | // packages/agent/subagents/testdata/cmd/stubchild instead of the real model | ||
| 38 | // loop. | 38 | // loop. | ||
| 39 | + | // workerTurnBudget is deliberately independent of the model loop: one | |||
| 40 | + | // admitted message turn consumes one current-run and lifetime turn, while | |||
| 41 | + | // retries, compaction, and provider/tool loops remain inside that turn. | |||
| 42 | + | type workerTurnBudget struct { | |||
| 43 | + | sequence int | |||
| 44 | + | lifetime int | |||
| 45 | + | current int | |||
| 46 | + | } | |||
| 47 | + | ||||
| 48 | + | type sessionPersistenceState struct { | |||
| 49 | + | mu sync.Mutex | |||
| 50 | + | errValue error | |||
| 51 | + | } | |||
| 52 | + | ||||
| 53 | + | func (s *sessionPersistenceState) record(err error) { | |||
| 54 | + | if err == nil { | |||
| 55 | + | return | |||
| 56 | + | } | |||
| 57 | + | s.mu.Lock() | |||
| 58 | + | if s.errValue == nil { | |||
| 59 | + | s.errValue = err | |||
| 60 | + | } | |||
| 61 | + | s.mu.Unlock() | |||
| 62 | + | } | |||
| 63 | + | ||||
| 64 | + | func (s *sessionPersistenceState) failed() bool { | |||
| 65 | + | s.mu.Lock() | |||
| 66 | + | defer s.mu.Unlock() | |||
| 67 | + | return s.errValue != nil | |||
| 68 | + | } | |||
| 69 | + | ||||
| 70 | + | func (s *sessionPersistenceState) err() error { | |||
| 71 | + | s.mu.Lock() | |||
| 72 | + | defer s.mu.Unlock() | |||
| 73 | + | return s.errValue | |||
| 74 | + | } | |||
| 75 | + | ||||
| 76 | + | func (b *workerTurnBudget) start(maxTurns int, newRun bool) (step, lifetime, current int, admitted bool) { | |||
| 77 | + | if newRun { | |||
| 78 | + | b.current = 0 | |||
| 79 | + | } | |||
| 80 | + | b.sequence++ | |||
| 81 | + | if maxTurns > 0 && b.current >= maxTurns { | |||
| 82 | + | return b.sequence, b.lifetime, b.current, false | |||
| 83 | + | } | |||
| 84 | + | b.lifetime++ | |||
| 85 | + | b.current++ | |||
| 86 | + | return b.sequence, b.lifetime, b.current, true | |||
| 87 | + | } | |||
| 88 | + | ||||
| 39 | func runSubagentWorkerMode(ctx context.Context, args Args, version string) error { | 89 | func runSubagentWorkerMode(ctx context.Context, args Args, version string) error { | ||
| 40 | if args.SubagentWorker == "" { | 90 | if args.SubagentWorker == "" { | ||
| 41 | return fmt.Errorf("--subagent-worker requires a socket path") | 91 | return fmt.Errorf("--subagent-worker requires a socket path") | ||
| @@ -74,6 +124,7 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||||
| 74 | if err != nil { | 124 | if err != nil { | ||
| 75 | return err | 125 | return err | ||
| 76 | } | 126 | } | ||
| 127 | + | var sessionPersistence sessionPersistenceState | |||
| 77 | if sess != nil { | 128 | if sess != nil { | ||
| 78 | var providerName, model string | 129 | var providerName, model string | ||
| 79 | sess, ag, providerName, model, err = applyInitialSessionResume(ctx, args, r, extMgr, sess, ag) | 130 | sess, ag, providerName, model, err = applyInitialSessionResume(ctx, args, r, extMgr, sess, ag) | ||
| @@ -82,6 +133,18 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||||
| 82 | } | 133 | } | ||
| 83 | r.Provider, r.Model = providerName, model | 134 | r.Provider, r.Model = providerName, model | ||
| 84 | ag.OnToolResult = func(_ string, result core.ToolResult) { persistExtensionToolResult(extMgr, sess, result) } | 135 | ag.OnToolResult = func(_ string, result core.ToolResult) { persistExtensionToolResult(extMgr, sess, result) } | ||
| 136 | + | // A prompt is acknowledged by the supervisor only after this callback | |||
| 137 | + | // has appended and synced the matching user message. This closes the | |||
| 138 | + | // crash window between accepting a follow-up and writing session.json. | |||
| 139 | + | ag.OnMessageAppended = func(message provider.Message) { | |||
| 140 | + | if err := sess.AppendMessage(message); err != nil { | |||
| 141 | + | sessionPersistence.record(err) | |||
| 142 | + | return | |||
| 143 | + | } | |||
| 144 | + | if err := sess.Sync(); err != nil { | |||
| 145 | + | sessionPersistence.record(err) | |||
| 146 | + | } | |||
| 147 | + | } | |||
| 85 | defer sess.Close() | 148 | defer sess.Close() | ||
| 86 | } | 149 | } | ||
| 87 | announceSession(extMgr, sess) | 150 | announceSession(extMgr, sess) | ||
| @@ -113,9 +176,12 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||||
| 113 | em := newSubagentEmitter(os.Stdout, logMirror) | 176 | em := newSubagentEmitter(os.Stdout, logMirror) | ||
| 114 | em.setProtocolIdentity(os.Getenv("ZUT_SUBAGENT_AGENT_ID")) | 177 | em.setProtocolIdentity(os.Getenv("ZUT_SUBAGENT_AGENT_ID")) | ||
| 115 | em.emit("agent.ready", map[string]any{ | 178 | em.emit("agent.ready", map[string]any{ | ||
| 116 | - | "version": version, | 179 | + | "version": version, |
| 117 | - | "cwd": r.CWD, | 180 | + | "cwd": r.CWD, |
| 118 | - | "model": r.Model, | 181 | + | "model": r.Model, |
| 182 | + | "lifetime_turns": args.SubagentLifetimeTurns, | |||
| 183 | + | "current_run_turns": args.SubagentRunTurns, | |||
| 184 | + | "max_turns": args.SubagentMaxTurns, | |||
| 119 | }) | 185 | }) | ||
| 120 | 186 | ||||
| 121 | // Heartbeats make a live-but-detached child distinguishable from a | 187 | // Heartbeats make a live-but-detached child distinguishable from a | ||
| @@ -145,63 +211,82 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||||
| 145 | // Keep a per-turn cancel so the "cancel" inbox message can interrupt | 211 | // Keep a per-turn cancel so the "cancel" inbox message can interrupt | ||
| 146 | // an in-flight turn without tearing down the whole daemon. | 212 | // an in-flight turn without tearing down the whole daemon. | ||
| 147 | var ( | 213 | var ( | ||
| 148 | - | mu sync.Mutex | 214 | + | mu sync.Mutex |
| 149 | - | cancelFn context.CancelFunc | 215 | + | cancelFn context.CancelFunc |
| 150 | - | busyTurn bool | 216 | + | busyTurn bool |
| 151 | - | turnNo = initialTurnNumber(os.Getenv("ZUT_SUBAGENT_EVENT_LOG")) | 217 | + | budget = workerTurnBudget{ |
| 218 | + | sequence: initialTurnNumber(os.Getenv("ZUT_SUBAGENT_EVENT_LOG")), | |||
| 219 | + | lifetime: args.SubagentLifetimeTurns, | |||
| 220 | + | current: args.SubagentRunTurns, | |||
| 221 | + | } | |||
| 152 | turnWG sync.WaitGroup | 222 | turnWG sync.WaitGroup | ||
| 153 | closing bool | 223 | closing bool | ||
| 224 | + | turnPending bool | |||
| 225 | + | seenCommands = make(map[string]struct{}) | |||
| 154 | shutdown = make(chan struct{}) | 226 | shutdown = make(chan struct{}) | ||
| 155 | shutdownOnce sync.Once | 227 | shutdownOnce sync.Once | ||
| 156 | ) | 228 | ) | ||
| 157 | maxOutputBytes, maxOutputLines := workerOutputLimits() | 229 | maxOutputBytes, maxOutputLines := workerOutputLimits() | ||
| 158 | 230 | ||||
| 159 | - | runOne := func(prompt string) { | 231 | + | runOne := func(prompt string, newRun bool, commandID string) { |
| 160 | mu.Lock() | 232 | mu.Lock() | ||
| 161 | // launchTurn admits the goroutine to turnWG before it starts so | 233 | // launchTurn admits the goroutine to turnWG before it starts so | ||
| 162 | // shutdown can join it. A shutdown can win the scheduling race | 234 | // shutdown can join it. A shutdown can win the scheduling race | ||
| 163 | // after that admission but before this goroutine acquires mu; in | 235 | // after that admission but before this goroutine acquires mu; in | ||
| 164 | // that case do not start a new Prompt after waitForActiveTurn has | 236 | // that case do not start a new Prompt after waitForActiveTurn has | ||
| 165 | // declared the worker closing. | 237 | // declared the worker closing. | ||
| 166 | - | if closing { | 238 | + | if closing || busyTurn { |
| 167 | mu.Unlock() | 239 | mu.Unlock() | ||
| 240 | + | em.emit("error", map[string]any{"code": "turn_rejected", "reason": "busy", "command_id": commandID, "message": "worker is busy"}) | |||
| 168 | return | 241 | return | ||
| 169 | } | 242 | } | ||
| 170 | - | if args.SubagentMaxTurns > 0 && turnNo >= args.SubagentMaxTurns { | 243 | + | busyTurn = true |
| 171 | - | // Consume a unique rejected attempt so repeated prompts after | 244 | + | turnPending = false |
| 172 | - | // the limit cannot reuse the same turn id in the event log. | 245 | + | step, lifetime, currentRun, admitted := budget.start(args.SubagentMaxTurns, newRun) |
| 173 | - | turnNo++ | 246 | + | if !admitted { |
| 174 | - | step := turnNo | 247 | + | busyTurn = false |
| 175 | mu.Unlock() | 248 | mu.Unlock() | ||
| 176 | turnID := fmt.Sprintf("turn-%d", step) | 249 | turnID := fmt.Sprintf("turn-%d", step) | ||
| 177 | - | errPayload := map[string]any{"code": "max_turns", "message": "maximum subagent turns reached"} | 250 | + | errPayload := map[string]any{"code": "turn_rejected", "reason": "max_turns", "command_id": commandID, "message": "maximum subagent turns reached"} |
| 251 | + | em.emit("error", errPayload) | |||
| 178 | em.emit("turn.result", map[string]any{"status": "failed", "turn_id": turnID, "error": errPayload}) | 252 | em.emit("turn.result", map[string]any{"status": "failed", "turn_id": turnID, "error": errPayload}) | ||
| 179 | em.emit("turn.failed", map[string]any{"turn_id": turnID, "error": errPayload}) | 253 | em.emit("turn.failed", map[string]any{"turn_id": turnID, "error": errPayload}) | ||
| 180 | em.emit("turn_end", map[string]any{"step": step, "turn_id": turnID, "error": errPayload["message"]}) | 254 | em.emit("turn_end", map[string]any{"step": step, "turn_id": turnID, "error": errPayload["message"]}) | ||
| 181 | - | em.emit("agent.idle", map[string]any{"turn_id": turnID}) | 255 | + | // Max-turn rejection ends the current run, but the worker remains |
| 182 | - | return | 256 | + | // alive and can accept a queued follow-up as a fresh run. |
| 183 | - | } | 257 | + | em.emit("agent.idle", map[string]any{ |
| 184 | - | if busyTurn { | 258 | + | "turn_id": turnID, |
| 185 | - | // Drop concurrent turns rather than queuing. The | 259 | + | "lifetime_turns": lifetime, |
| 186 | - | // supervisor protocol assumes one outstanding turn per | 260 | + | "current_run_turns": currentRun, |
| 187 | - | // agent; if a user really wants to interrupt and start | 261 | + | }) |
| 188 | - | // another, they should send "cancel" first. | |||
| 189 | - | mu.Unlock() | |||
| 190 | - | em.emit("error", map[string]any{"message": "agent busy; send 'cancel' first"}) | |||
| 191 | return | 262 | return | ||
| 192 | } | 263 | } | ||
| 193 | - | busyTurn = true | 264 | + | turnID := fmt.Sprintf("turn-%d", step) |
| 194 | - | turnNo++ | |||
| 195 | - | step := turnNo | |||
| 196 | c, cancel := context.WithCancel(ctx) | 265 | c, cancel := context.WithCancel(ctx) | ||
| 197 | cancelFn = cancel | 266 | cancelFn = cancel | ||
| 198 | mu.Unlock() | 267 | mu.Unlock() | ||
| 199 | 268 | ||||
| 200 | - | turnID := fmt.Sprintf("turn-%d", step) | 269 | + | turnStarted := false |
| 201 | - | em.emit("turn.started", map[string]any{"step": step, "turn_id": turnID}) | |||
| 202 | - | ||||
| 203 | sink := func(ev core.AgentEvent) { | 270 | sink := func(ev core.AgentEvent) { | ||
| 204 | - | em.emit(ev.Type(), subagentEventData(ev)) | 271 | + | data := subagentEventData(ev) |
| 272 | + | if ev.Type() == "user_message" && !turnStarted { | |||
| 273 | + | // Prompt invokes OnMessageAppended before delivering this sink | |||
| 274 | + | // event, so the session row is durable before the lifecycle | |||
| 275 | + | // acknowledgement is written. | |||
| 276 | + | em.emit(ev.Type(), data) | |||
| 277 | + | if !sessionPersistence.failed() { | |||
| 278 | + | em.emit("turn.started", map[string]any{"step": step, "turn_id": turnID, "command_id": commandID, "lifetime_turns": lifetime, "current_run_turns": currentRun}) | |||
| 279 | + | turnStarted = true | |||
| 280 | + | } | |||
| 281 | + | return | |||
| 282 | + | } | |||
| 283 | + | if ev.Type() == "turn_start" { | |||
| 284 | + | // A provider/model-loop turn is nested inside the one delegated | |||
| 285 | + | // message turn admitted above. Mark it on the wire so the | |||
| 286 | + | // supervisor cannot mistake it for a new worker turn. | |||
| 287 | + | data["nested_turn"] = true | |||
| 288 | + | } | |||
| 289 | + | em.emit(ev.Type(), data) | |||
| 205 | } | 290 | } | ||
| 206 | 291 | ||||
| 207 | var persistCompaction func([]provider.Message) error | 292 | var persistCompaction func([]provider.Message) error | ||
| @@ -209,7 +294,12 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||||
| 209 | persistCompaction = sess.AppendCompaction | 294 | persistCompaction = sess.AppendCompaction | ||
| 210 | } | 295 | } | ||
| 211 | start, err := promptWithContextRecovery(c, ag, prompt, sink, persistCompaction) | 296 | start, err := promptWithContextRecovery(c, ag, prompt, sink, persistCompaction) | ||
| 212 | - | WriteNewTranscript(ag, sess, start) | 297 | + | if sess != nil && ag.OnMessageAppended == nil { |
| 298 | + | WriteNewTranscript(ag, sess, start) | |||
| 299 | + | } | |||
| 300 | + | if sessionErr := sessionPersistence.err(); err == nil && sessionErr != nil { | |||
| 301 | + | err = sessionErr | |||
| 302 | + | } | |||
| 213 | 303 | ||||
| 214 | output := finalAssistantText(ag.Messages()[start:]) | 304 | output := finalAssistantText(ag.Messages()[start:]) | ||
| 215 | resultStatus := "succeeded" | 305 | resultStatus := "succeeded" | ||
| @@ -245,24 +335,26 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||||
| 245 | // The supervisor treats agent.idle as permission to start a follow-up. | 335 | // The supervisor treats agent.idle as permission to start a follow-up. | ||
| 246 | // Publish it only after the busy gate has opened, otherwise a manager | 336 | // Publish it only after the busy gate has opened, otherwise a manager | ||
| 247 | // can send a turn that the inbox loop drops as still busy. | 337 | // can send a turn that the inbox loop drops as still busy. | ||
| 248 | - | em.emit("agent.idle", map[string]any{"turn_id": turnID}) | 338 | + | em.emit("agent.idle", map[string]any{"turn_id": turnID, "lifetime_turns": lifetime, "current_run_turns": currentRun}) |
| 249 | } | 339 | } | ||
| 250 | 340 | ||||
| 251 | - | launchTurn := func(prompt string) { | 341 | + | launchTurn := func(prompt string, newRun bool, commandID string) bool { |
| 252 | mu.Lock() | 342 | mu.Lock() | ||
| 253 | - | if closing { | 343 | + | if closing || busyTurn || turnPending { |
| 254 | mu.Unlock() | 344 | mu.Unlock() | ||
| 255 | - | return | 345 | + | em.emit("error", map[string]any{"code": "turn_rejected", "reason": "busy", "command_id": commandID, "message": "worker is busy"}) |
| 346 | + | return false | |||
| 256 | } | 347 | } | ||
| 257 | - | // Register the goroutine before launching it. Shutdown can then | 348 | + | // Reserve the worker before launching the goroutine. A second command |
| 258 | - | // close the gate and wait for every turn that was admitted, even | 349 | + | // cannot be accepted in the gap before runOne acquires mu. |
| 259 | - | // if runOne has not acquired mu and set busyTurn yet. | 350 | + | turnPending = true |
| 260 | turnWG.Add(1) | 351 | turnWG.Add(1) | ||
| 261 | mu.Unlock() | 352 | mu.Unlock() | ||
| 262 | go func() { | 353 | go func() { | ||
| 263 | defer turnWG.Done() | 354 | defer turnWG.Done() | ||
| 264 | - | runOne(prompt) | 355 | + | runOne(prompt, newRun, commandID) |
| 265 | }() | 356 | }() | ||
| 357 | + | return true | |||
| 266 | } | 358 | } | ||
| 267 | 359 | ||||
| 268 | waitForActiveTurn := func(cancel bool) { | 360 | waitForActiveTurn := func(cancel bool) { | ||
| @@ -287,7 +379,7 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||||
| 287 | // "starts working" the moment it boots, matching what users | 379 | // "starts working" the moment it boots, matching what users | ||
| 288 | // expect from `/subagents new <task>`. | 380 | // expect from `/subagents new <task>`. | ||
| 289 | if args.Prompt != "" { | 381 | if args.Prompt != "" { | ||
| 290 | - | launchTurn(args.Prompt) | 382 | + | launchTurn(args.Prompt, false, "") |
| 291 | } | 383 | } | ||
| 292 | 384 | ||||
| 293 | // Inbox loop: one supervisor message at a time. We don't spawn | 385 | // Inbox loop: one supervisor message at a time. We don't spawn | ||
| @@ -333,10 +425,20 @@ func runSubagentWorkerMode(ctx context.Context, args Args, version string) error | |||||
| 333 | case subagents.CommandTurnStart: | 425 | case subagents.CommandTurnStart: | ||
| 334 | var payload subagents.TurnStartPayload | 426 | var payload subagents.TurnStartPayload | ||
| 335 | if err := command.DecodePayload(&payload); err != nil { | 427 | if err := command.DecodePayload(&payload); err != nil { | ||
| 336 | - | em.emit("error", map[string]any{"message": "invalid turn.start payload"}) | 428 | + | em.emit("error", map[string]any{"code": "turn_rejected", "reason": "invalid_payload", "command_id": command.MessageID, "message": "invalid turn.start payload"}) |
| 337 | continue | 429 | continue | ||
| 338 | } | 430 | } | ||
| 339 | - | launchTurn(payload.Prompt) | 431 | + | mu.Lock() |
| 432 | + | _, duplicate := seenCommands[command.MessageID] | |||
| 433 | + | mu.Unlock() | |||
| 434 | + | if duplicate && command.MessageID != "" { | |||
| 435 | + | continue | |||
| 436 | + | } | |||
| 437 | + | if launchTurn(payload.Prompt, payload.NewRun, command.MessageID) && command.MessageID != "" { | |||
| 438 | + | mu.Lock() | |||
| 439 | + | seenCommands[command.MessageID] = struct{}{} | |||
| 440 | + | mu.Unlock() | |||
| 441 | + | } | |||
| 340 | default: | 442 | default: | ||
| 341 | // Unknown commands are rejected rather than interpreted as | 443 | // Unknown commands are rejected rather than interpreted as | ||
| 342 | // child-originated control requests. | 444 | // child-originated control requests. | ||
| @@ -555,6 +657,9 @@ func initialTurnNumber(path string) int { | |||||
| 555 | } | 657 | } | ||
| 556 | maxStep := 0 | 658 | maxStep := 0 | ||
| 557 | for _, ev := range events { | 659 | for _, ev := range events { | ||
| 660 | + | if !subagents.IsDelegatedTurnStart(ev) { | |||
| 661 | + | continue | |||
| 662 | + | } | |||
| 558 | if step, ok := ev.Data["step"].(float64); ok && int(step) > maxStep { | 663 | if step, ok := ev.Data["step"].(float64); ok && int(step) > maxStep { | ||
| 559 | maxStep = int(step) | 664 | maxStep = int(step) | ||
| 560 | } | 665 | } | ||
M packages/agent/subagent_worker_test.go +46 -0
| @@ -526,3 +526,49 @@ func TestSupervisorEmitterStdoutShapeMatchesSupervisorParser(t *testing.T) { | |||
| 526 | 526 | t.Errorf("payload step field missing or wrong: %v", object["payload"]) | |
| 527 | 527 | } | |
| 528 | 528 | } | |
| 529 | + | ||
| 530 | + | func TestInitialTurnNumberIgnoresNestedProviderTurns(t *testing.T) { | |
| 531 | + | path := filepath.Join(t.TempDir(), "events.jsonl") | |
| 532 | + | log, err := subagents.OpenEventLog(path) | |
| 533 | + | if err != nil { | |
| 534 | + | t.Fatal(err) | |
| 535 | + | } | |
| 536 | + | if err := log.Append(subagents.NewEvent(subagents.EventTurnStarted, map[string]any{ | |
| 537 | + | "step": float64(1), "lifetime_turns": 1, "current_run_turns": 1, | |
| 538 | + | })); err != nil { | |
| 539 | + | t.Fatal(err) | |
| 540 | + | } | |
| 541 | + | if err := log.Append(subagents.NewEvent(subagents.EventTurnStarted, map[string]any{ | |
| 542 | + | "step": float64(99), "nested_turn": true, | |
| 543 | + | })); err != nil { | |
| 544 | + | t.Fatal(err) | |
| 545 | + | } | |
| 546 | + | if err := log.Close(); err != nil { | |
| 547 | + | t.Fatal(err) | |
| 548 | + | } | |
| 549 | + | if got := initialTurnNumber(path); got != 1 { | |
| 550 | + | t.Fatalf("initial turn number = %d, want 1", got) | |
| 551 | + | } | |
| 552 | + | } | |
| 553 | + | ||
| 554 | + | func TestWorkerTurnBudgetFreshAllowanceOnlyForExplicitRun(t *testing.T) { | |
| 555 | + | var budget workerTurnBudget | |
| 556 | + | ||
| 557 | + | if _, lifetime, current, admitted := budget.start(2, false); !admitted || lifetime != 1 || current != 1 { | |
| 558 | + | t.Fatalf("first turn = (%d, %d, %v), want (1, 1, true)", lifetime, current, admitted) | |
| 559 | + | } | |
| 560 | + | // Provider retries, model loops, and compaction stay inside the admitted | |
| 561 | + | // turn and therefore do not call start or consume another allowance. | |
| 562 | + | if budget.lifetime != 1 || budget.current != 1 { | |
| 563 | + | t.Fatalf("nested work changed budget to (lifetime=%d, current=%d)", budget.lifetime, budget.current) | |
| 564 | + | } | |
| 565 | + | if _, lifetime, current, admitted := budget.start(2, false); !admitted || lifetime != 2 || current != 2 { | |
| 566 | + | t.Fatalf("second turn = (%d, %d, %v), want (2, 2, true)", lifetime, current, admitted) | |
| 567 | + | } | |
| 568 | + | if _, lifetime, current, admitted := budget.start(2, false); admitted || lifetime != 2 || current != 2 { | |
| 569 | + | t.Fatalf("exhausted turn = (%d, %d, %v), want (2, 2, false)", lifetime, current, admitted) | |
| 570 | + | } | |
| 571 | + | if _, lifetime, current, admitted := budget.start(2, true); !admitted || lifetime != 3 || current != 1 { | |
| 572 | + | t.Fatalf("explicit resumed turn = (%d, %d, %v), want (3, 1, true)", lifetime, current, admitted) | |
| 573 | + | } | |
| 574 | + | } | |
| @@ -526,3 +526,49 @@ func TestSupervisorEmitterStdoutShapeMatchesSupervisorParser(t *testing.T) { | |||||
| 526 | t.Errorf("payload step field missing or wrong: %v", object["payload"]) | 526 | t.Errorf("payload step field missing or wrong: %v", object["payload"]) | ||
| 527 | } | 527 | } | ||
| 528 | } | 528 | } | ||
| 529 | + | ||||
| 530 | + | func TestInitialTurnNumberIgnoresNestedProviderTurns(t *testing.T) { | |||
| 531 | + | path := filepath.Join(t.TempDir(), "events.jsonl") | |||
| 532 | + | log, err := subagents.OpenEventLog(path) | |||
| 533 | + | if err != nil { | |||
| 534 | + | t.Fatal(err) | |||
| 535 | + | } | |||
| 536 | + | if err := log.Append(subagents.NewEvent(subagents.EventTurnStarted, map[string]any{ | |||
| 537 | + | "step": float64(1), "lifetime_turns": 1, "current_run_turns": 1, | |||
| 538 | + | })); err != nil { | |||
| 539 | + | t.Fatal(err) | |||
| 540 | + | } | |||
| 541 | + | if err := log.Append(subagents.NewEvent(subagents.EventTurnStarted, map[string]any{ | |||
| 542 | + | "step": float64(99), "nested_turn": true, | |||
| 543 | + | })); err != nil { | |||
| 544 | + | t.Fatal(err) | |||
| 545 | + | } | |||
| 546 | + | if err := log.Close(); err != nil { | |||
| 547 | + | t.Fatal(err) | |||
| 548 | + | } | |||
| 549 | + | if got := initialTurnNumber(path); got != 1 { | |||
| 550 | + | t.Fatalf("initial turn number = %d, want 1", got) | |||
| 551 | + | } | |||
| 552 | + | } | |||
| 553 | + | ||||
| 554 | + | func TestWorkerTurnBudgetFreshAllowanceOnlyForExplicitRun(t *testing.T) { | |||
| 555 | + | var budget workerTurnBudget | |||
| 556 | + | ||||
| 557 | + | if _, lifetime, current, admitted := budget.start(2, false); !admitted || lifetime != 1 || current != 1 { | |||
| 558 | + | t.Fatalf("first turn = (%d, %d, %v), want (1, 1, true)", lifetime, current, admitted) | |||
| 559 | + | } | |||
| 560 | + | // Provider retries, model loops, and compaction stay inside the admitted | |||
| 561 | + | // turn and therefore do not call start or consume another allowance. | |||
| 562 | + | if budget.lifetime != 1 || budget.current != 1 { | |||
| 563 | + | t.Fatalf("nested work changed budget to (lifetime=%d, current=%d)", budget.lifetime, budget.current) | |||
| 564 | + | } | |||
| 565 | + | if _, lifetime, current, admitted := budget.start(2, false); !admitted || lifetime != 2 || current != 2 { | |||
| 566 | + | t.Fatalf("second turn = (%d, %d, %v), want (2, 2, true)", lifetime, current, admitted) | |||
| 567 | + | } | |||
| 568 | + | if _, lifetime, current, admitted := budget.start(2, false); admitted || lifetime != 2 || current != 2 { | |||
| 569 | + | t.Fatalf("exhausted turn = (%d, %d, %v), want (2, 2, false)", lifetime, current, admitted) | |||
| 570 | + | } | |||
| 571 | + | if _, lifetime, current, admitted := budget.start(2, true); !admitted || lifetime != 3 || current != 1 { | |||
| 572 | + | t.Fatalf("explicit resumed turn = (%d, %d, %v), want (3, 1, true)", lifetime, current, admitted) | |||
| 573 | + | } | |||
| 574 | + | } | |||
A packages/agent/subagents/ack_test.go +65 -0
| @@ -0,0 +1,65 @@ | |||
| 1 | + | package subagents | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "testing" | |
| 5 | + | "time" | |
| 6 | + | ) | |
| 7 | + | ||
| 8 | + | func TestResumePromptAcknowledgementRequiresDurableUserMessageAndTurnStart(t *testing.T) { | |
| 9 | + | acceptedAt := time.Now().Add(-time.Second) | |
| 10 | + | prompt := "follow-up prompt" | |
| 11 | + | user := NewEvent("user_message", map[string]any{ | |
| 12 | + | "content": []any{map[string]any{"type": "text", "text": prompt}}, | |
| 13 | + | }) | |
| 14 | + | started := NewEvent(EventTurnStarted, map[string]any{ | |
| 15 | + | "lifetime_turns": 2, | |
| 16 | + | "current_run_turns": 1, | |
| 17 | + | }) | |
| 18 | + | ||
| 19 | + | if resumePromptAcknowledged([]Event{started}, prompt, acceptedAt) { | |
| 20 | + | t.Fatal("turn.started alone acknowledged a pending prompt") | |
| 21 | + | } | |
| 22 | + | if resumePromptAcknowledged([]Event{user}, prompt, acceptedAt) { | |
| 23 | + | t.Fatal("crash point after durable user message acknowledged before turn.started") | |
| 24 | + | } | |
| 25 | + | if !resumePromptAcknowledged([]Event{user, started}, prompt, acceptedAt) { | |
| 26 | + | t.Fatal("matching durable user message followed by delegated turn.started was not acknowledged") | |
| 27 | + | } | |
| 28 | + | } | |
| 29 | + | ||
| 30 | + | func TestResumePromptAcknowledgementIgnoresNestedTurnStart(t *testing.T) { | |
| 31 | + | acceptedAt := time.Now().Add(-time.Second) | |
| 32 | + | prompt := "follow-up prompt" | |
| 33 | + | user := NewEvent("user_message", map[string]any{ | |
| 34 | + | "content": []any{map[string]any{"type": "text", "text": prompt}}, | |
| 35 | + | }) | |
| 36 | + | nested := NewEvent(EventTurnStarted, map[string]any{"nested_turn": true}) | |
| 37 | + | if resumePromptAcknowledged([]Event{user, nested}, prompt, acceptedAt) { | |
| 38 | + | t.Fatal("nested turn.started acknowledged a pending delegated prompt") | |
| 39 | + | } | |
| 40 | + | } | |
| 41 | + | ||
| 42 | + | func TestResumePromptAcknowledgementIgnoresStaleMatchingUserMessage(t *testing.T) { | |
| 43 | + | acceptedAt := time.Now() | |
| 44 | + | prompt := "follow-up prompt" | |
| 45 | + | user := NewEvent("user_message", map[string]any{ | |
| 46 | + | "content": []any{map[string]any{"type": "text", "text": prompt}}, | |
| 47 | + | }) | |
| 48 | + | user.Time = acceptedAt.Add(-time.Millisecond) | |
| 49 | + | started := NewEvent(EventTurnStarted, map[string]any{"lifetime_turns": 2, "current_run_turns": 1}) | |
| 50 | + | if resumePromptAcknowledged([]Event{user, started}, prompt, acceptedAt) { | |
| 51 | + | t.Fatal("stale matching user message acknowledged a pending prompt") | |
| 52 | + | } | |
| 53 | + | } | |
| 54 | + | ||
| 55 | + | func TestResumePromptAcknowledgementIgnoresAmbiguousUserMessage(t *testing.T) { | |
| 56 | + | acceptedAt := time.Now().Add(-time.Second) | |
| 57 | + | prompt := "follow-up prompt" | |
| 58 | + | user := NewEvent("user_message", map[string]any{ | |
| 59 | + | "content": []any{map[string]any{"type": "text", "text": "older prompt"}}, | |
| 60 | + | }) | |
| 61 | + | started := NewEvent(EventTurnStarted, map[string]any{"lifetime_turns": 2, "current_run_turns": 1}) | |
| 62 | + | if resumePromptAcknowledged([]Event{user, started}, prompt, acceptedAt) { | |
| 63 | + | t.Fatal("ambiguous user message acknowledged a pending prompt") | |
| 64 | + | } | |
| 65 | + | } | |
| @@ -0,0 +1,65 @@ | |||||
| 1 | + | package subagents | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "testing" | |||
| 5 | + | "time" | |||
| 6 | + | ) | |||
| 7 | + | ||||
| 8 | + | func TestResumePromptAcknowledgementRequiresDurableUserMessageAndTurnStart(t *testing.T) { | |||
| 9 | + | acceptedAt := time.Now().Add(-time.Second) | |||
| 10 | + | prompt := "follow-up prompt" | |||
| 11 | + | user := NewEvent("user_message", map[string]any{ | |||
| 12 | + | "content": []any{map[string]any{"type": "text", "text": prompt}}, | |||
| 13 | + | }) | |||
| 14 | + | started := NewEvent(EventTurnStarted, map[string]any{ | |||
| 15 | + | "lifetime_turns": 2, | |||
| 16 | + | "current_run_turns": 1, | |||
| 17 | + | }) | |||
| 18 | + | ||||
| 19 | + | if resumePromptAcknowledged([]Event{started}, prompt, acceptedAt) { | |||
| 20 | + | t.Fatal("turn.started alone acknowledged a pending prompt") | |||
| 21 | + | } | |||
| 22 | + | if resumePromptAcknowledged([]Event{user}, prompt, acceptedAt) { | |||
| 23 | + | t.Fatal("crash point after durable user message acknowledged before turn.started") | |||
| 24 | + | } | |||
| 25 | + | if !resumePromptAcknowledged([]Event{user, started}, prompt, acceptedAt) { | |||
| 26 | + | t.Fatal("matching durable user message followed by delegated turn.started was not acknowledged") | |||
| 27 | + | } | |||
| 28 | + | } | |||
| 29 | + | ||||
| 30 | + | func TestResumePromptAcknowledgementIgnoresNestedTurnStart(t *testing.T) { | |||
| 31 | + | acceptedAt := time.Now().Add(-time.Second) | |||
| 32 | + | prompt := "follow-up prompt" | |||
| 33 | + | user := NewEvent("user_message", map[string]any{ | |||
| 34 | + | "content": []any{map[string]any{"type": "text", "text": prompt}}, | |||
| 35 | + | }) | |||
| 36 | + | nested := NewEvent(EventTurnStarted, map[string]any{"nested_turn": true}) | |||
| 37 | + | if resumePromptAcknowledged([]Event{user, nested}, prompt, acceptedAt) { | |||
| 38 | + | t.Fatal("nested turn.started acknowledged a pending delegated prompt") | |||
| 39 | + | } | |||
| 40 | + | } | |||
| 41 | + | ||||
| 42 | + | func TestResumePromptAcknowledgementIgnoresStaleMatchingUserMessage(t *testing.T) { | |||
| 43 | + | acceptedAt := time.Now() | |||
| 44 | + | prompt := "follow-up prompt" | |||
| 45 | + | user := NewEvent("user_message", map[string]any{ | |||
| 46 | + | "content": []any{map[string]any{"type": "text", "text": prompt}}, | |||
| 47 | + | }) | |||
| 48 | + | user.Time = acceptedAt.Add(-time.Millisecond) | |||
| 49 | + | started := NewEvent(EventTurnStarted, map[string]any{"lifetime_turns": 2, "current_run_turns": 1}) | |||
| 50 | + | if resumePromptAcknowledged([]Event{user, started}, prompt, acceptedAt) { | |||
| 51 | + | t.Fatal("stale matching user message acknowledged a pending prompt") | |||
| 52 | + | } | |||
| 53 | + | } | |||
| 54 | + | ||||
| 55 | + | func TestResumePromptAcknowledgementIgnoresAmbiguousUserMessage(t *testing.T) { | |||
| 56 | + | acceptedAt := time.Now().Add(-time.Second) | |||
| 57 | + | prompt := "follow-up prompt" | |||
| 58 | + | user := NewEvent("user_message", map[string]any{ | |||
| 59 | + | "content": []any{map[string]any{"type": "text", "text": "older prompt"}}, | |||
| 60 | + | }) | |||
| 61 | + | started := NewEvent(EventTurnStarted, map[string]any{"lifetime_turns": 2, "current_run_turns": 1}) | |||
| 62 | + | if resumePromptAcknowledged([]Event{user, started}, prompt, acceptedAt) { | |||
| 63 | + | t.Fatal("ambiguous user message acknowledged a pending prompt") | |||
| 64 | + | } | |||
| 65 | + | } | |||
M packages/agent/subagents/agent.go +244 -10
| @@ -7,6 +7,8 @@ import ( | |||
| 7 | 7 | "strings" | |
| 8 | 8 | "sync" | |
| 9 | 9 | "time" | |
| 10 | + | ||
| 11 | + | "github.com/google/uuid" | |
| 10 | 12 | ) | |
| 11 | 13 | ||
| 12 | 14 | // Agent is one supervised task. Public fields are immutable after | |
| @@ -17,6 +19,18 @@ type turnEndNotice struct { | |||
| 17 | 19 | errMsg string | |
| 18 | 20 | } | |
| 19 | 21 | ||
| 22 | + | type queuedResumePrompt struct { | |
| 23 | + | Prompt string `json:"prompt"` | |
| 24 | + | AcceptedAt time.Time `json:"accepted_at"` | |
| 25 | + | CommandID string `json:"command_id,omitempty"` | |
| 26 | + | } | |
| 27 | + | ||
| 28 | + | type resumePromptSnapshot struct { | |
| 29 | + | Prompt string | |
| 30 | + | AcceptedAt time.Time | |
| 31 | + | CommandID string | |
| 32 | + | } | |
| 33 | + | ||
| 20 | 34 | type Agent struct { | |
| 21 | 35 | ID string | |
| 22 | 36 | Task string | |
| @@ -36,6 +50,14 @@ type Agent struct { | |||
| 36 | 50 | WorkspaceBase string | |
| 37 | 51 | WorkspaceCapture CaptureMode | |
| 38 | 52 | MaxTurns int | |
| 53 | + | ||
| 54 | + | // LifetimeTurns counts every accepted message turn for this worker across | |
| 55 | + | // all explicit runs. CurrentRunTurns resets only when an explicit | |
| 56 | + | // subagent_resume follow-up starts a new run; retries, model loops, | |
| 57 | + | // compaction, and cancellation do not reset it. | |
| 58 | + | LifetimeTurns int | |
| 59 | + | CurrentRunTurns int | |
| 60 | + | ||
| 39 | 61 | // Timeout is the effective per-agent lifetime, persisted so a resumed | |
| 40 | 62 | // worker keeps the timeout selected at its spawn boundary. | |
| 41 | 63 | Timeout time.Duration | |
| @@ -80,12 +102,15 @@ type Agent struct { | |||
| 80 | 102 | // Not persisted — every Resume sets it explicitly. | |
| 81 | 103 | Resuming bool | |
| 82 | 104 | ||
| 83 | - | // resumePromptText is an optional manager follow-up passed to a resumed | |
| 84 | - | // child as its initial prompt. It remains in durable metadata until the | |
| 85 | - | // child reports turn.started, so a host exit while the worker is queued | |
| 86 | - | // cannot lose it. Access it through ResumePromptInfo. | |
| 87 | - | resumePromptText string | |
| 88 | - | resumePromptAt time.Time | |
| 105 | + | // resumePromptText is the manager follow-up currently handed to the | |
| 106 | + | // child. It remains in durable metadata until the child reports | |
| 107 | + | // turn.started, so a host exit while the worker is queued cannot lose it. | |
| 108 | + | // Additional follow-ups accepted while the child is busy stay in | |
| 109 | + | // resumePromptQueue until the next agent idle turn. | |
| 110 | + | resumePromptText string | |
| 111 | + | resumePromptAt time.Time | |
| 112 | + | resumePromptID string | |
| 113 | + | resumePromptQueue []queuedResumePrompt | |
| 89 | 114 | ||
| 90 | 115 | // InboxPath is the unix socket the child agent listens on for | |
| 91 | 116 | // follow-up prompts and control messages. The supervisor opens | |
| @@ -146,10 +171,12 @@ type Agent struct { | |||
| 146 | 171 | outputTruncated bool | |
| 147 | 172 | maxOutputBytes int | |
| 148 | 173 | maxOutputLines int | |
| 149 | - | persistFn func(*Agent) | |
| 174 | + | persistFn func(*Agent) error | |
| 175 | + | persistMu sync.Mutex | |
| 150 | 176 | stateDir string | |
| 151 | 177 | workspaceCleanup func() error | |
| 152 | 178 | workspaceCapture func() (WorkspaceCapture, error) | |
| 179 | + | onTurnIdle func() | |
| 153 | 180 | ||
| 154 | 181 | // OnTurnEnd, if set, fires once per prompt-level turn_end event | |
| 155 | 182 | // emitted by the subagent daemon wrapper. Provider/tool-loop | |
| @@ -234,6 +261,66 @@ func (a *Agent) AttemptValue() int { | |||
| 234 | 261 | return a.Attempt | |
| 235 | 262 | } | |
| 236 | 263 | ||
| 264 | + | // LifetimeTurnsValue returns the number of accepted message turns across all | |
| 265 | + | // explicit runs of this worker. | |
| 266 | + | func (a *Agent) LifetimeTurnsValue() int { | |
| 267 | + | a.lifecycleMu.Lock() | |
| 268 | + | defer a.lifecycleMu.Unlock() | |
| 269 | + | return a.LifetimeTurns | |
| 270 | + | } | |
| 271 | + | ||
| 272 | + | // CurrentRunTurnsValue returns the number of accepted message turns in the | |
| 273 | + | // current explicit run. | |
| 274 | + | func (a *Agent) CurrentRunTurnsValue() int { | |
| 275 | + | a.lifecycleMu.Lock() | |
| 276 | + | defer a.lifecycleMu.Unlock() | |
| 277 | + | return a.CurrentRunTurns | |
| 278 | + | } | |
| 279 | + | ||
| 280 | + | func (a *Agent) setTurnCounts(lifetime, currentRun int) { | |
| 281 | + | if lifetime < 0 { | |
| 282 | + | lifetime = 0 | |
| 283 | + | } | |
| 284 | + | if currentRun < 0 { | |
| 285 | + | currentRun = 0 | |
| 286 | + | } | |
| 287 | + | a.lifecycleMu.Lock() | |
| 288 | + | a.LifetimeTurns = lifetime | |
| 289 | + | a.CurrentRunTurns = currentRun | |
| 290 | + | a.updatedAt = time.Now() | |
| 291 | + | a.lifecycleMu.Unlock() | |
| 292 | + | } | |
| 293 | + | ||
| 294 | + | func (a *Agent) incrementTurnCounts() (lifetime, currentRun int) { | |
| 295 | + | a.lifecycleMu.Lock() | |
| 296 | + | a.LifetimeTurns++ | |
| 297 | + | a.CurrentRunTurns++ | |
| 298 | + | a.updatedAt = time.Now() | |
| 299 | + | lifetime = a.LifetimeTurns | |
| 300 | + | currentRun = a.CurrentRunTurns | |
| 301 | + | a.lifecycleMu.Unlock() | |
| 302 | + | return lifetime, currentRun | |
| 303 | + | } | |
| 304 | + | ||
| 305 | + | func (a *Agent) resetCurrentRunTurns() (previous int) { | |
| 306 | + | a.lifecycleMu.Lock() | |
| 307 | + | previous = a.CurrentRunTurns | |
| 308 | + | a.CurrentRunTurns = 0 | |
| 309 | + | a.updatedAt = time.Now() | |
| 310 | + | a.lifecycleMu.Unlock() | |
| 311 | + | return previous | |
| 312 | + | } | |
| 313 | + | ||
| 314 | + | func (a *Agent) setCurrentRunTurns(currentRun int) { | |
| 315 | + | if currentRun < 0 { | |
| 316 | + | currentRun = 0 | |
| 317 | + | } | |
| 318 | + | a.lifecycleMu.Lock() | |
| 319 | + | a.CurrentRunTurns = currentRun | |
| 320 | + | a.updatedAt = time.Now() | |
| 321 | + | a.lifecycleMu.Unlock() | |
| 322 | + | } | |
| 323 | + | ||
| 237 | 324 | // ProcessPIDValue returns the current worker pid, when known. | |
| 238 | 325 | func (a *Agent) ProcessPIDValue() int { | |
| 239 | 326 | a.lifecycleMu.Lock() | |
| @@ -404,13 +491,45 @@ func (a *Agent) ResumePromptInfo() (string, time.Time) { | |||
| 404 | 491 | return a.resumePromptText, a.resumePromptAt | |
| 405 | 492 | } | |
| 406 | 493 | ||
| 407 | - | func (a *Agent) setResumePrompt(prompt string, acceptedAt time.Time) (string, time.Time) { | |
| 494 | + | func (a *Agent) setResumePrompt(prompt string, acceptedAt time.Time) resumePromptSnapshot { | |
| 408 | 495 | a.mu.Lock() | |
| 409 | 496 | defer a.mu.Unlock() | |
| 410 | - | previousPrompt, previousAcceptedAt := a.resumePromptText, a.resumePromptAt | |
| 497 | + | previous := resumePromptSnapshot{ | |
| 498 | + | Prompt: a.resumePromptText, | |
| 499 | + | AcceptedAt: a.resumePromptAt, | |
| 500 | + | CommandID: a.resumePromptID, | |
| 501 | + | } | |
| 411 | 502 | a.resumePromptText = prompt | |
| 412 | 503 | a.resumePromptAt = acceptedAt | |
| 413 | - | return previousPrompt, previousAcceptedAt | |
| 504 | + | a.resumePromptID = uuid.NewString() | |
| 505 | + | return previous | |
| 506 | + | } | |
| 507 | + | ||
| 508 | + | func (a *Agent) restoreResumePrompt(previous resumePromptSnapshot) { | |
| 509 | + | a.mu.Lock() | |
| 510 | + | defer a.mu.Unlock() | |
| 511 | + | a.resumePromptText = previous.Prompt | |
| 512 | + | a.resumePromptAt = previous.AcceptedAt | |
| 513 | + | a.resumePromptID = previous.CommandID | |
| 514 | + | if a.resumePromptText != "" && a.resumePromptID == "" { | |
| 515 | + | a.resumePromptID = uuid.NewString() | |
| 516 | + | } | |
| 517 | + | } | |
| 518 | + | ||
| 519 | + | func (a *Agent) resumePromptState() resumePromptSnapshot { | |
| 520 | + | a.mu.Lock() | |
| 521 | + | defer a.mu.Unlock() | |
| 522 | + | return resumePromptSnapshot{ | |
| 523 | + | Prompt: a.resumePromptText, | |
| 524 | + | AcceptedAt: a.resumePromptAt, | |
| 525 | + | CommandID: a.resumePromptID, | |
| 526 | + | } | |
| 527 | + | } | |
| 528 | + | ||
| 529 | + | func (a *Agent) resumePromptCommandID() string { | |
| 530 | + | a.mu.Lock() | |
| 531 | + | defer a.mu.Unlock() | |
| 532 | + | return a.resumePromptID | |
| 414 | 533 | } | |
| 415 | 534 | ||
| 416 | 535 | func (a *Agent) clearResumePrompt() bool { | |
| @@ -421,9 +540,110 @@ func (a *Agent) clearResumePrompt() bool { | |||
| 421 | 540 | } | |
| 422 | 541 | a.resumePromptText = "" | |
| 423 | 542 | a.resumePromptAt = time.Time{} | |
| 543 | + | a.resumePromptID = "" | |
| 424 | 544 | return true | |
| 425 | 545 | } | |
| 426 | 546 | ||
| 547 | + | func (a *Agent) queueResumePrompt(prompt string, acceptedAt time.Time) string { | |
| 548 | + | commandID := uuid.NewString() | |
| 549 | + | a.mu.Lock() | |
| 550 | + | a.resumePromptQueue = append(a.resumePromptQueue, queuedResumePrompt{Prompt: prompt, AcceptedAt: acceptedAt, CommandID: commandID}) | |
| 551 | + | a.mu.Unlock() | |
| 552 | + | return commandID | |
| 553 | + | } | |
| 554 | + | ||
| 555 | + | func (a *Agent) removeQueuedResumePrompt(commandID string) bool { | |
| 556 | + | if commandID == "" { | |
| 557 | + | return false | |
| 558 | + | } | |
| 559 | + | a.mu.Lock() | |
| 560 | + | defer a.mu.Unlock() | |
| 561 | + | for i := len(a.resumePromptQueue) - 1; i >= 0; i-- { | |
| 562 | + | if a.resumePromptQueue[i].CommandID == commandID { | |
| 563 | + | a.resumePromptQueue = append(a.resumePromptQueue[:i], a.resumePromptQueue[i+1:]...) | |
| 564 | + | return true | |
| 565 | + | } | |
| 566 | + | } | |
| 567 | + | return false | |
| 568 | + | } | |
| 569 | + | ||
| 570 | + | func (a *Agent) resumePromptQueueSnapshot() []queuedResumePrompt { | |
| 571 | + | a.mu.Lock() | |
| 572 | + | defer a.mu.Unlock() | |
| 573 | + | return append([]queuedResumePrompt(nil), a.resumePromptQueue...) | |
| 574 | + | } | |
| 575 | + | ||
| 576 | + | func (a *Agent) hasQueuedResumePrompt() bool { | |
| 577 | + | a.mu.Lock() | |
| 578 | + | defer a.mu.Unlock() | |
| 579 | + | return len(a.resumePromptQueue) != 0 | |
| 580 | + | } | |
| 581 | + | ||
| 582 | + | // startQueuedResumePrompt moves the oldest active follow-up into the | |
| 583 | + | // in-flight prompt slot. The caller must persist the returned state before | |
| 584 | + | // delivering the command so a host exit cannot lose the prompt. | |
| 585 | + | func (a *Agent) startQueuedResumePrompt() (string, time.Time, string, bool) { | |
| 586 | + | a.mu.Lock() | |
| 587 | + | defer a.mu.Unlock() | |
| 588 | + | if a.resumePromptText != "" || len(a.resumePromptQueue) == 0 { | |
| 589 | + | return "", time.Time{}, "", false | |
| 590 | + | } | |
| 591 | + | queued := a.resumePromptQueue[0] | |
| 592 | + | a.resumePromptQueue = a.resumePromptQueue[1:] | |
| 593 | + | a.resumePromptText = queued.Prompt | |
| 594 | + | a.resumePromptAt = queued.AcceptedAt | |
| 595 | + | a.resumePromptID = queued.CommandID | |
| 596 | + | if a.resumePromptID == "" { | |
| 597 | + | a.resumePromptID = uuid.NewString() | |
| 598 | + | } | |
| 599 | + | return queued.Prompt, queued.AcceptedAt, a.resumePromptID, true | |
| 600 | + | } | |
| 601 | + | ||
| 602 | + | func (a *Agent) restoreResumePromptToQueue(prompt string, acceptedAt time.Time) bool { | |
| 603 | + | a.mu.Lock() | |
| 604 | + | defer a.mu.Unlock() | |
| 605 | + | if a.resumePromptText != prompt || !a.resumePromptAt.Equal(acceptedAt) { | |
| 606 | + | return false | |
| 607 | + | } | |
| 608 | + | commandID := a.resumePromptID | |
| 609 | + | if commandID == "" { | |
| 610 | + | commandID = uuid.NewString() | |
| 611 | + | } | |
| 612 | + | a.resumePromptText = "" | |
| 613 | + | a.resumePromptAt = time.Time{} | |
| 614 | + | a.resumePromptID = "" | |
| 615 | + | a.resumePromptQueue = append([]queuedResumePrompt{{Prompt: prompt, AcceptedAt: acceptedAt, CommandID: commandID}}, a.resumePromptQueue...) | |
| 616 | + | return true | |
| 617 | + | } | |
| 618 | + | ||
| 619 | + | func (a *Agent) rejectActiveResumePrompt() bool { | |
| 620 | + | a.mu.Lock() | |
| 621 | + | defer a.mu.Unlock() | |
| 622 | + | if a.resumePromptText == "" { | |
| 623 | + | return false | |
| 624 | + | } | |
| 625 | + | a.resumePromptQueue = append([]queuedResumePrompt{{Prompt: a.resumePromptText, AcceptedAt: a.resumePromptAt, CommandID: a.resumePromptID}}, a.resumePromptQueue...) | |
| 626 | + | a.resumePromptText = "" | |
| 627 | + | a.resumePromptAt = time.Time{} | |
| 628 | + | a.resumePromptID = "" | |
| 629 | + | return true | |
| 630 | + | } | |
| 631 | + | ||
| 632 | + | func (a *Agent) setOnTurnIdle(fn func()) { | |
| 633 | + | a.mu.Lock() | |
| 634 | + | a.onTurnIdle = fn | |
| 635 | + | a.mu.Unlock() | |
| 636 | + | } | |
| 637 | + | ||
| 638 | + | func (a *Agent) notifyTurnIdle() { | |
| 639 | + | a.mu.Lock() | |
| 640 | + | fn := a.onTurnIdle | |
| 641 | + | a.mu.Unlock() | |
| 642 | + | if fn != nil { | |
| 643 | + | fn() | |
| 644 | + | } | |
| 645 | + | } | |
| 646 | + | ||
| 427 | 647 | func (a *Agent) SetOnTurnEnd(fn func(step int, errMsg string)) { | |
| 428 | 648 | a.mu.Lock() | |
| 429 | 649 | a.OnTurnEnd = fn | |
| @@ -437,6 +657,20 @@ func (a *Agent) SetOnTurnEnd(fn func(step int, errMsg string)) { | |||
| 437 | 657 | } | |
| 438 | 658 | } | |
| 439 | 659 | ||
| 660 | + | func (a *Agent) recordPersistenceError(err error) { | |
| 661 | + | if a == nil || err == nil { | |
| 662 | + | return | |
| 663 | + | } | |
| 664 | + | a.mu.Lock() | |
| 665 | + | if a.lastErr == nil { | |
| 666 | + | a.lastErr = fmt.Errorf("subagent metadata persistence: %w", err) | |
| 667 | + | } | |
| 668 | + | if a.status != StatusDone && a.status != StatusFailed && a.status != StatusKilled { | |
| 669 | + | a.activity = "metadata error: " + truncate(err.Error(), 120) | |
| 670 | + | } | |
| 671 | + | a.mu.Unlock() | |
| 672 | + | } | |
| 673 | + | ||
| 440 | 674 | func (a *Agent) setActivity(msg string) { | |
| 441 | 675 | a.mu.Lock() | |
| 442 | 676 | a.activity = strings.TrimSpace(msg) | |
| @@ -7,6 +7,8 @@ import ( | |||||
| 7 | "strings" | 7 | "strings" | ||
| 8 | "sync" | 8 | "sync" | ||
| 9 | "time" | 9 | "time" | ||
| 10 | + | ||||
| 11 | + | "github.com/google/uuid" | |||
| 10 | ) | 12 | ) | ||
| 11 | 13 | ||||
| 12 | // Agent is one supervised task. Public fields are immutable after | 14 | // Agent is one supervised task. Public fields are immutable after | ||
| @@ -17,6 +19,18 @@ type turnEndNotice struct { | |||||
| 17 | errMsg string | 19 | errMsg string | ||
| 18 | } | 20 | } | ||
| 19 | 21 | ||||
| 22 | + | type queuedResumePrompt struct { | |||
| 23 | + | Prompt string `json:"prompt"` | |||
| 24 | + | AcceptedAt time.Time `json:"accepted_at"` | |||
| 25 | + | CommandID string `json:"command_id,omitempty"` | |||
| 26 | + | } | |||
| 27 | + | ||||
| 28 | + | type resumePromptSnapshot struct { | |||
| 29 | + | Prompt string | |||
| 30 | + | AcceptedAt time.Time | |||
| 31 | + | CommandID string | |||
| 32 | + | } | |||
| 33 | + | ||||
| 20 | type Agent struct { | 34 | type Agent struct { | ||
| 21 | ID string | 35 | ID string | ||
| 22 | Task string | 36 | Task string | ||
| @@ -36,6 +50,14 @@ type Agent struct { | |||||
| 36 | WorkspaceBase string | 50 | WorkspaceBase string | ||
| 37 | WorkspaceCapture CaptureMode | 51 | WorkspaceCapture CaptureMode | ||
| 38 | MaxTurns int | 52 | MaxTurns int | ||
| 53 | + | ||||
| 54 | + | // LifetimeTurns counts every accepted message turn for this worker across | |||
| 55 | + | // all explicit runs. CurrentRunTurns resets only when an explicit | |||
| 56 | + | // subagent_resume follow-up starts a new run; retries, model loops, | |||
| 57 | + | // compaction, and cancellation do not reset it. | |||
| 58 | + | LifetimeTurns int | |||
| 59 | + | CurrentRunTurns int | |||
| 60 | + | ||||
| 39 | // Timeout is the effective per-agent lifetime, persisted so a resumed | 61 | // Timeout is the effective per-agent lifetime, persisted so a resumed | ||
| 40 | // worker keeps the timeout selected at its spawn boundary. | 62 | // worker keeps the timeout selected at its spawn boundary. | ||
| 41 | Timeout time.Duration | 63 | Timeout time.Duration | ||
| @@ -80,12 +102,15 @@ type Agent struct { | |||||
| 80 | // Not persisted — every Resume sets it explicitly. | 102 | // Not persisted — every Resume sets it explicitly. | ||
| 81 | Resuming bool | 103 | Resuming bool | ||
| 82 | 104 | ||||
| 83 | - | // resumePromptText is an optional manager follow-up passed to a resumed | 105 | + | // resumePromptText is the manager follow-up currently handed to the |
| 84 | - | // child as its initial prompt. It remains in durable metadata until the | 106 | + | // child. It remains in durable metadata until the child reports |
| 85 | - | // child reports turn.started, so a host exit while the worker is queued | 107 | + | // turn.started, so a host exit while the worker is queued cannot lose it. |
| 86 | - | // cannot lose it. Access it through ResumePromptInfo. | 108 | + | // Additional follow-ups accepted while the child is busy stay in |
| 87 | - | resumePromptText string | 109 | + | // resumePromptQueue until the next agent idle turn. |
| 88 | - | resumePromptAt time.Time | 110 | + | resumePromptText string |
| 111 | + | resumePromptAt time.Time | |||
| 112 | + | resumePromptID string | |||
| 113 | + | resumePromptQueue []queuedResumePrompt | |||
| 89 | 114 | ||||
| 90 | // InboxPath is the unix socket the child agent listens on for | 115 | // InboxPath is the unix socket the child agent listens on for | ||
| 91 | // follow-up prompts and control messages. The supervisor opens | 116 | // follow-up prompts and control messages. The supervisor opens | ||
| @@ -146,10 +171,12 @@ type Agent struct { | |||||
| 146 | outputTruncated bool | 171 | outputTruncated bool | ||
| 147 | maxOutputBytes int | 172 | maxOutputBytes int | ||
| 148 | maxOutputLines int | 173 | maxOutputLines int | ||
| 149 | - | persistFn func(*Agent) | 174 | + | persistFn func(*Agent) error |
| 175 | + | persistMu sync.Mutex | |||
| 150 | stateDir string | 176 | stateDir string | ||
| 151 | workspaceCleanup func() error | 177 | workspaceCleanup func() error | ||
| 152 | workspaceCapture func() (WorkspaceCapture, error) | 178 | workspaceCapture func() (WorkspaceCapture, error) | ||
| 179 | + | onTurnIdle func() | |||
| 153 | 180 | ||||
| 154 | // OnTurnEnd, if set, fires once per prompt-level turn_end event | 181 | // OnTurnEnd, if set, fires once per prompt-level turn_end event | ||
| 155 | // emitted by the subagent daemon wrapper. Provider/tool-loop | 182 | // emitted by the subagent daemon wrapper. Provider/tool-loop | ||
| @@ -234,6 +261,66 @@ func (a *Agent) AttemptValue() int { | |||||
| 234 | return a.Attempt | 261 | return a.Attempt | ||
| 235 | } | 262 | } | ||
| 236 | 263 | ||||
| 264 | + | // LifetimeTurnsValue returns the number of accepted message turns across all | |||
| 265 | + | // explicit runs of this worker. | |||
| 266 | + | func (a *Agent) LifetimeTurnsValue() int { | |||
| 267 | + | a.lifecycleMu.Lock() | |||
| 268 | + | defer a.lifecycleMu.Unlock() | |||
| 269 | + | return a.LifetimeTurns | |||
| 270 | + | } | |||
| 271 | + | ||||
| 272 | + | // CurrentRunTurnsValue returns the number of accepted message turns in the | |||
| 273 | + | // current explicit run. | |||
| 274 | + | func (a *Agent) CurrentRunTurnsValue() int { | |||
| 275 | + | a.lifecycleMu.Lock() | |||
| 276 | + | defer a.lifecycleMu.Unlock() | |||
| 277 | + | return a.CurrentRunTurns | |||
| 278 | + | } | |||
| 279 | + | ||||
| 280 | + | func (a *Agent) setTurnCounts(lifetime, currentRun int) { | |||
| 281 | + | if lifetime < 0 { | |||
| 282 | + | lifetime = 0 | |||
| 283 | + | } | |||
| 284 | + | if currentRun < 0 { | |||
| 285 | + | currentRun = 0 | |||
| 286 | + | } | |||
| 287 | + | a.lifecycleMu.Lock() | |||
| 288 | + | a.LifetimeTurns = lifetime | |||
| 289 | + | a.CurrentRunTurns = currentRun | |||
| 290 | + | a.updatedAt = time.Now() | |||
| 291 | + | a.lifecycleMu.Unlock() | |||
| 292 | + | } | |||
| 293 | + | ||||
| 294 | + | func (a *Agent) incrementTurnCounts() (lifetime, currentRun int) { | |||
| 295 | + | a.lifecycleMu.Lock() | |||
| 296 | + | a.LifetimeTurns++ | |||
| 297 | + | a.CurrentRunTurns++ | |||
| 298 | + | a.updatedAt = time.Now() | |||
| 299 | + | lifetime = a.LifetimeTurns | |||
| 300 | + | currentRun = a.CurrentRunTurns | |||
| 301 | + | a.lifecycleMu.Unlock() | |||
| 302 | + | return lifetime, currentRun | |||
| 303 | + | } | |||
| 304 | + | ||||
| 305 | + | func (a *Agent) resetCurrentRunTurns() (previous int) { | |||
| 306 | + | a.lifecycleMu.Lock() | |||
| 307 | + | previous = a.CurrentRunTurns | |||
| 308 | + | a.CurrentRunTurns = 0 | |||
| 309 | + | a.updatedAt = time.Now() | |||
| 310 | + | a.lifecycleMu.Unlock() | |||
| 311 | + | return previous | |||
| 312 | + | } | |||
| 313 | + | ||||
| 314 | + | func (a *Agent) setCurrentRunTurns(currentRun int) { | |||
| 315 | + | if currentRun < 0 { | |||
| 316 | + | currentRun = 0 | |||
| 317 | + | } | |||
| 318 | + | a.lifecycleMu.Lock() | |||
| 319 | + | a.CurrentRunTurns = currentRun | |||
| 320 | + | a.updatedAt = time.Now() | |||
| 321 | + | a.lifecycleMu.Unlock() | |||
| 322 | + | } | |||
| 323 | + | ||||
| 237 | // ProcessPIDValue returns the current worker pid, when known. | 324 | // ProcessPIDValue returns the current worker pid, when known. | ||
| 238 | func (a *Agent) ProcessPIDValue() int { | 325 | func (a *Agent) ProcessPIDValue() int { | ||
| 239 | a.lifecycleMu.Lock() | 326 | a.lifecycleMu.Lock() | ||
| @@ -404,13 +491,45 @@ func (a *Agent) ResumePromptInfo() (string, time.Time) { | |||||
| 404 | return a.resumePromptText, a.resumePromptAt | 491 | return a.resumePromptText, a.resumePromptAt | ||
| 405 | } | 492 | } | ||
| 406 | 493 | ||||
| 407 | - | func (a *Agent) setResumePrompt(prompt string, acceptedAt time.Time) (string, time.Time) { | 494 | + | func (a *Agent) setResumePrompt(prompt string, acceptedAt time.Time) resumePromptSnapshot { |
| 408 | a.mu.Lock() | 495 | a.mu.Lock() | ||
| 409 | defer a.mu.Unlock() | 496 | defer a.mu.Unlock() | ||
| 410 | - | previousPrompt, previousAcceptedAt := a.resumePromptText, a.resumePromptAt | 497 | + | previous := resumePromptSnapshot{ |
| 498 | + | Prompt: a.resumePromptText, | |||
| 499 | + | AcceptedAt: a.resumePromptAt, | |||
| 500 | + | CommandID: a.resumePromptID, | |||
| 501 | + | } | |||
| 411 | a.resumePromptText = prompt | 502 | a.resumePromptText = prompt | ||
| 412 | a.resumePromptAt = acceptedAt | 503 | a.resumePromptAt = acceptedAt | ||
| 413 | - | return previousPrompt, previousAcceptedAt | 504 | + | a.resumePromptID = uuid.NewString() |
| 505 | + | return previous | |||
| 506 | + | } | |||
| 507 | + | ||||
| 508 | + | func (a *Agent) restoreResumePrompt(previous resumePromptSnapshot) { | |||
| 509 | + | a.mu.Lock() | |||
| 510 | + | defer a.mu.Unlock() | |||
| 511 | + | a.resumePromptText = previous.Prompt | |||
| 512 | + | a.resumePromptAt = previous.AcceptedAt | |||
| 513 | + | a.resumePromptID = previous.CommandID | |||
| 514 | + | if a.resumePromptText != "" && a.resumePromptID == "" { | |||
| 515 | + | a.resumePromptID = uuid.NewString() | |||
| 516 | + | } | |||
| 517 | + | } | |||
| 518 | + | ||||
| 519 | + | func (a *Agent) resumePromptState() resumePromptSnapshot { | |||
| 520 | + | a.mu.Lock() | |||
| 521 | + | defer a.mu.Unlock() | |||
| 522 | + | return resumePromptSnapshot{ | |||
| 523 | + | Prompt: a.resumePromptText, | |||
| 524 | + | AcceptedAt: a.resumePromptAt, | |||
| 525 | + | CommandID: a.resumePromptID, | |||
| 526 | + | } | |||
| 527 | + | } | |||
| 528 | + | ||||
| 529 | + | func (a *Agent) resumePromptCommandID() string { | |||
| 530 | + | a.mu.Lock() | |||
| 531 | + | defer a.mu.Unlock() | |||
| 532 | + | return a.resumePromptID | |||
| 414 | } | 533 | } | ||
| 415 | 534 | ||||
| 416 | func (a *Agent) clearResumePrompt() bool { | 535 | func (a *Agent) clearResumePrompt() bool { | ||
| @@ -421,9 +540,110 @@ func (a *Agent) clearResumePrompt() bool { | |||||
| 421 | } | 540 | } | ||
| 422 | a.resumePromptText = "" | 541 | a.resumePromptText = "" | ||
| 423 | a.resumePromptAt = time.Time{} | 542 | a.resumePromptAt = time.Time{} | ||
| 543 | + | a.resumePromptID = "" | |||
| 424 | return true | 544 | return true | ||
| 425 | } | 545 | } | ||
| 426 | 546 | ||||
| 547 | + | func (a *Agent) queueResumePrompt(prompt string, acceptedAt time.Time) string { | |||
| 548 | + | commandID := uuid.NewString() | |||
| 549 | + | a.mu.Lock() | |||
| 550 | + | a.resumePromptQueue = append(a.resumePromptQueue, queuedResumePrompt{Prompt: prompt, AcceptedAt: acceptedAt, CommandID: commandID}) | |||
| 551 | + | a.mu.Unlock() | |||
| 552 | + | return commandID | |||
| 553 | + | } | |||
| 554 | + | ||||
| 555 | + | func (a *Agent) removeQueuedResumePrompt(commandID string) bool { | |||
| 556 | + | if commandID == "" { | |||
| 557 | + | return false | |||
| 558 | + | } | |||
| 559 | + | a.mu.Lock() | |||
| 560 | + | defer a.mu.Unlock() | |||
| 561 | + | for i := len(a.resumePromptQueue) - 1; i >= 0; i-- { | |||
| 562 | + | if a.resumePromptQueue[i].CommandID == commandID { | |||
| 563 | + | a.resumePromptQueue = append(a.resumePromptQueue[:i], a.resumePromptQueue[i+1:]...) | |||
| 564 | + | return true | |||
| 565 | + | } | |||
| 566 | + | } | |||
| 567 | + | return false | |||
| 568 | + | } | |||
| 569 | + | ||||
| 570 | + | func (a *Agent) resumePromptQueueSnapshot() []queuedResumePrompt { | |||
| 571 | + | a.mu.Lock() | |||
| 572 | + | defer a.mu.Unlock() | |||
| 573 | + | return append([]queuedResumePrompt(nil), a.resumePromptQueue...) | |||
| 574 | + | } | |||
| 575 | + | ||||
| 576 | + | func (a *Agent) hasQueuedResumePrompt() bool { | |||
| 577 | + | a.mu.Lock() | |||
| 578 | + | defer a.mu.Unlock() | |||
| 579 | + | return len(a.resumePromptQueue) != 0 | |||
| 580 | + | } | |||
| 581 | + | ||||
| 582 | + | // startQueuedResumePrompt moves the oldest active follow-up into the | |||
| 583 | + | // in-flight prompt slot. The caller must persist the returned state before | |||
| 584 | + | // delivering the command so a host exit cannot lose the prompt. | |||
| 585 | + | func (a *Agent) startQueuedResumePrompt() (string, time.Time, string, bool) { | |||
| 586 | + | a.mu.Lock() | |||
| 587 | + | defer a.mu.Unlock() | |||
| 588 | + | if a.resumePromptText != "" || len(a.resumePromptQueue) == 0 { | |||
| 589 | + | return "", time.Time{}, "", false | |||
| 590 | + | } | |||
| 591 | + | queued := a.resumePromptQueue[0] | |||
| 592 | + | a.resumePromptQueue = a.resumePromptQueue[1:] | |||
| 593 | + | a.resumePromptText = queued.Prompt | |||
| 594 | + | a.resumePromptAt = queued.AcceptedAt | |||
| 595 | + | a.resumePromptID = queued.CommandID | |||
| 596 | + | if a.resumePromptID == "" { | |||
| 597 | + | a.resumePromptID = uuid.NewString() | |||
| 598 | + | } | |||
| 599 | + | return queued.Prompt, queued.AcceptedAt, a.resumePromptID, true | |||
| 600 | + | } | |||
| 601 | + | ||||
| 602 | + | func (a *Agent) restoreResumePromptToQueue(prompt string, acceptedAt time.Time) bool { | |||
| 603 | + | a.mu.Lock() | |||
| 604 | + | defer a.mu.Unlock() | |||
| 605 | + | if a.resumePromptText != prompt || !a.resumePromptAt.Equal(acceptedAt) { | |||
| 606 | + | return false | |||
| 607 | + | } | |||
| 608 | + | commandID := a.resumePromptID | |||
| 609 | + | if commandID == "" { | |||
| 610 | + | commandID = uuid.NewString() | |||
| 611 | + | } | |||
| 612 | + | a.resumePromptText = "" | |||
| 613 | + | a.resumePromptAt = time.Time{} | |||
| 614 | + | a.resumePromptID = "" | |||
| 615 | + | a.resumePromptQueue = append([]queuedResumePrompt{{Prompt: prompt, AcceptedAt: acceptedAt, CommandID: commandID}}, a.resumePromptQueue...) | |||
| 616 | + | return true | |||
| 617 | + | } | |||
| 618 | + | ||||
| 619 | + | func (a *Agent) rejectActiveResumePrompt() bool { | |||
| 620 | + | a.mu.Lock() | |||
| 621 | + | defer a.mu.Unlock() | |||
| 622 | + | if a.resumePromptText == "" { | |||
| 623 | + | return false | |||
| 624 | + | } | |||
| 625 | + | a.resumePromptQueue = append([]queuedResumePrompt{{Prompt: a.resumePromptText, AcceptedAt: a.resumePromptAt, CommandID: a.resumePromptID}}, a.resumePromptQueue...) | |||
| 626 | + | a.resumePromptText = "" | |||
| 627 | + | a.resumePromptAt = time.Time{} | |||
| 628 | + | a.resumePromptID = "" | |||
| 629 | + | return true | |||
| 630 | + | } | |||
| 631 | + | ||||
| 632 | + | func (a *Agent) setOnTurnIdle(fn func()) { | |||
| 633 | + | a.mu.Lock() | |||
| 634 | + | a.onTurnIdle = fn | |||
| 635 | + | a.mu.Unlock() | |||
| 636 | + | } | |||
| 637 | + | ||||
| 638 | + | func (a *Agent) notifyTurnIdle() { | |||
| 639 | + | a.mu.Lock() | |||
| 640 | + | fn := a.onTurnIdle | |||
| 641 | + | a.mu.Unlock() | |||
| 642 | + | if fn != nil { | |||
| 643 | + | fn() | |||
| 644 | + | } | |||
| 645 | + | } | |||
| 646 | + | ||||
| 427 | func (a *Agent) SetOnTurnEnd(fn func(step int, errMsg string)) { | 647 | func (a *Agent) SetOnTurnEnd(fn func(step int, errMsg string)) { | ||
| 428 | a.mu.Lock() | 648 | a.mu.Lock() | ||
| 429 | a.OnTurnEnd = fn | 649 | a.OnTurnEnd = fn | ||
| @@ -437,6 +657,20 @@ func (a *Agent) SetOnTurnEnd(fn func(step int, errMsg string)) { | |||||
| 437 | } | 657 | } | ||
| 438 | } | 658 | } | ||
| 439 | 659 | ||||
| 660 | + | func (a *Agent) recordPersistenceError(err error) { | |||
| 661 | + | if a == nil || err == nil { | |||
| 662 | + | return | |||
| 663 | + | } | |||
| 664 | + | a.mu.Lock() | |||
| 665 | + | if a.lastErr == nil { | |||
| 666 | + | a.lastErr = fmt.Errorf("subagent metadata persistence: %w", err) | |||
| 667 | + | } | |||
| 668 | + | if a.status != StatusDone && a.status != StatusFailed && a.status != StatusKilled { | |||
| 669 | + | a.activity = "metadata error: " + truncate(err.Error(), 120) | |||
| 670 | + | } | |||
| 671 | + | a.mu.Unlock() | |||
| 672 | + | } | |||
| 673 | + | ||||
| 440 | func (a *Agent) setActivity(msg string) { | 674 | func (a *Agent) setActivity(msg string) { | ||
| 441 | a.mu.Lock() | 675 | a.mu.Lock() | ||
| 442 | a.activity = strings.TrimSpace(msg) | 676 | a.activity = strings.TrimSpace(msg) | ||
M packages/agent/subagents/inbox.go +33 -12
| @@ -112,34 +112,49 @@ func (b *Inbox) sendBytesContext(ctx context.Context, data []byte) error { | |||
| 112 | 112 | } | |
| 113 | 113 | defer conn.SetWriteDeadline(time.Time{}) | |
| 114 | 114 | } | |
| 115 | - | if _, err := conn.Write(data); err != nil { | |
| 116 | - | // Drop the connection so the next call redials. The previous error is | |
| 117 | - | // more informative than the redial's would be, so surface this one. | |
| 115 | + | n, writeErr := conn.Write(data) | |
| 116 | + | if writeErr != nil { | |
| 117 | + | // Drop the connection so the next call redials. Once any bytes have | |
| 118 | + | // crossed the socket boundary, the outcome is ambiguous; returning the | |
| 119 | + | // caller's context error would invite a duplicate command with a new | |
| 120 | + | // identity. | |
| 118 | 121 | stopCancel() | |
| 119 | - | b.mu.Lock() | |
| 120 | - | if b.conn == conn { | |
| 121 | - | _ = b.conn.Close() | |
| 122 | - | b.conn = nil | |
| 122 | + | b.dropConnection(conn) | |
| 123 | + | if n > 0 { | |
| 124 | + | return fmt.Errorf("%w: wrote %d of %d bytes: %v", ErrDeliveryUnknown, n, len(data), writeErr) | |
| 123 | 125 | } | |
| 124 | - | b.mu.Unlock() | |
| 125 | 126 | if ctxErr := ctx.Err(); ctxErr != nil { | |
| 126 | 127 | return ctxErr | |
| 127 | 128 | } | |
| 128 | - | return err | |
| 129 | + | return writeErr | |
| 130 | + | } | |
| 131 | + | if n != len(data) { | |
| 132 | + | stopCancel() | |
| 133 | + | b.dropConnection(conn) | |
| 134 | + | return fmt.Errorf("%w: wrote %d of %d bytes", ErrDeliveryUnknown, n, len(data)) | |
| 129 | 135 | } | |
| 130 | 136 | if !stopCancel() { | |
| 131 | - | // Cancellation closed this connection after the write completed. Drop it | |
| 132 | - | // so the next command redials instead of reusing a closed stream. | |
| 137 | + | // Cancellation closed this connection after the complete command was | |
| 138 | + | // written. The worker can parse the newline-delimited command, so the | |
| 139 | + | // delivery succeeded even though the context is now canceled. | |
| 133 | 140 | b.mu.Lock() | |
| 134 | 141 | if b.conn == conn { | |
| 135 | 142 | b.conn = nil | |
| 136 | 143 | } | |
| 137 | 144 | b.mu.Unlock() | |
| 138 | - | return ctx.Err() | |
| 139 | 145 | } | |
| 140 | 146 | return nil | |
| 141 | 147 | } | |
| 142 | 148 | ||
| 149 | + | func (b *Inbox) dropConnection(conn net.Conn) { | |
| 150 | + | b.mu.Lock() | |
| 151 | + | defer b.mu.Unlock() | |
| 152 | + | if b.conn == conn { | |
| 153 | + | _ = b.conn.Close() | |
| 154 | + | b.conn = nil | |
| 155 | + | } | |
| 156 | + | } | |
| 157 | + | ||
| 143 | 158 | func (b *Inbox) acquireSend(ctx context.Context) (chan struct{}, error) { | |
| 144 | 159 | b.mu.Lock() | |
| 145 | 160 | if b.send == nil { | |
| @@ -179,6 +194,12 @@ func (b *Inbox) Close() error { | |||
| 179 | 194 | // the TUI surfaces it as "agent <id> not listening yet". | |
| 180 | 195 | var ErrNotReady = errors.New("subagent: agent inbox not ready") | |
| 181 | 196 | ||
| 197 | + | // ErrDeliveryUnknown means a command write made progress but did not finish. | |
| 198 | + | // Callers must not replace the command with a fresh identity: the worker may | |
| 199 | + | // have received enough bytes to parse it, so retries must reuse the same | |
| 200 | + | // message id and be handled idempotently. | |
| 201 | + | var ErrDeliveryUnknown = errors.New("subagent: command delivery unknown") | |
| 202 | + | ||
| 182 | 203 | // dialUnixContext retries a Unix-socket connection briefly so the happy path | |
| 183 | 204 | // "child just started, supervisor sends right away" works without forcing the | |
| 184 | 205 | // caller to poll. Errors collapse onto ErrNotReady whenever the failure means | |
| @@ -112,34 +112,49 @@ func (b *Inbox) sendBytesContext(ctx context.Context, data []byte) error { | |||||
| 112 | } | 112 | } | ||
| 113 | defer conn.SetWriteDeadline(time.Time{}) | 113 | defer conn.SetWriteDeadline(time.Time{}) | ||
| 114 | } | 114 | } | ||
| 115 | - | if _, err := conn.Write(data); err != nil { | 115 | + | n, writeErr := conn.Write(data) |
| 116 | - | // Drop the connection so the next call redials. The previous error is | 116 | + | if writeErr != nil { |
| 117 | - | // more informative than the redial's would be, so surface this one. | 117 | + | // Drop the connection so the next call redials. Once any bytes have |
| 118 | + | // crossed the socket boundary, the outcome is ambiguous; returning the | |||
| 119 | + | // caller's context error would invite a duplicate command with a new | |||
| 120 | + | // identity. | |||
| 118 | stopCancel() | 121 | stopCancel() | ||
| 119 | - | b.mu.Lock() | 122 | + | b.dropConnection(conn) |
| 120 | - | if b.conn == conn { | 123 | + | if n > 0 { |
| 121 | - | _ = b.conn.Close() | 124 | + | return fmt.Errorf("%w: wrote %d of %d bytes: %v", ErrDeliveryUnknown, n, len(data), writeErr) |
| 122 | - | b.conn = nil | |||
| 123 | } | 125 | } | ||
| 124 | - | b.mu.Unlock() | |||
| 125 | if ctxErr := ctx.Err(); ctxErr != nil { | 126 | if ctxErr := ctx.Err(); ctxErr != nil { | ||
| 126 | return ctxErr | 127 | return ctxErr | ||
| 127 | } | 128 | } | ||
| 128 | - | return err | 129 | + | return writeErr |
| 130 | + | } | |||
| 131 | + | if n != len(data) { | |||
| 132 | + | stopCancel() | |||
| 133 | + | b.dropConnection(conn) | |||
| 134 | + | return fmt.Errorf("%w: wrote %d of %d bytes", ErrDeliveryUnknown, n, len(data)) | |||
| 129 | } | 135 | } | ||
| 130 | if !stopCancel() { | 136 | if !stopCancel() { | ||
| 131 | - | // Cancellation closed this connection after the write completed. Drop it | 137 | + | // Cancellation closed this connection after the complete command was |
| 132 | - | // so the next command redials instead of reusing a closed stream. | 138 | + | // written. The worker can parse the newline-delimited command, so the |
| 139 | + | // delivery succeeded even though the context is now canceled. | |||
| 133 | b.mu.Lock() | 140 | b.mu.Lock() | ||
| 134 | if b.conn == conn { | 141 | if b.conn == conn { | ||
| 135 | b.conn = nil | 142 | b.conn = nil | ||
| 136 | } | 143 | } | ||
| 137 | b.mu.Unlock() | 144 | b.mu.Unlock() | ||
| 138 | - | return ctx.Err() | |||
| 139 | } | 145 | } | ||
| 140 | return nil | 146 | return nil | ||
| 141 | } | 147 | } | ||
| 142 | 148 | ||||
| 149 | + | func (b *Inbox) dropConnection(conn net.Conn) { | |||
| 150 | + | b.mu.Lock() | |||
| 151 | + | defer b.mu.Unlock() | |||
| 152 | + | if b.conn == conn { | |||
| 153 | + | _ = b.conn.Close() | |||
| 154 | + | b.conn = nil | |||
| 155 | + | } | |||
| 156 | + | } | |||
| 157 | + | ||||
| 143 | func (b *Inbox) acquireSend(ctx context.Context) (chan struct{}, error) { | 158 | func (b *Inbox) acquireSend(ctx context.Context) (chan struct{}, error) { | ||
| 144 | b.mu.Lock() | 159 | b.mu.Lock() | ||
| 145 | if b.send == nil { | 160 | if b.send == nil { | ||
| @@ -179,6 +194,12 @@ func (b *Inbox) Close() error { | |||||
| 179 | // the TUI surfaces it as "agent <id> not listening yet". | 194 | // the TUI surfaces it as "agent <id> not listening yet". | ||
| 180 | var ErrNotReady = errors.New("subagent: agent inbox not ready") | 195 | var ErrNotReady = errors.New("subagent: agent inbox not ready") | ||
| 181 | 196 | ||||
| 197 | + | // ErrDeliveryUnknown means a command write made progress but did not finish. | |||
| 198 | + | // Callers must not replace the command with a fresh identity: the worker may | |||
| 199 | + | // have received enough bytes to parse it, so retries must reuse the same | |||
| 200 | + | // message id and be handled idempotently. | |||
| 201 | + | var ErrDeliveryUnknown = errors.New("subagent: command delivery unknown") | |||
| 202 | + | ||||
| 182 | // dialUnixContext retries a Unix-socket connection briefly so the happy path | 203 | // dialUnixContext retries a Unix-socket connection briefly so the happy path | ||
| 183 | // "child just started, supervisor sends right away" works without forcing the | 204 | // "child just started, supervisor sends right away" works without forcing the | ||
| 184 | // caller to poll. Errors collapse onto ErrNotReady whenever the failure means | 205 | // caller to poll. Errors collapse onto ErrNotReady whenever the failure means | ||
M packages/agent/subagents/inbox_test.go +60 -0
| @@ -127,6 +127,32 @@ func (c *blockingWriteConn) Close() error { | |||
| 127 | 127 | return c.Conn.Close() | |
| 128 | 128 | } | |
| 129 | 129 | ||
| 130 | + | type cancelAfterWriteConn struct { | |
| 131 | + | net.Conn | |
| 132 | + | cancel context.CancelFunc | |
| 133 | + | } | |
| 134 | + | ||
| 135 | + | func (c *cancelAfterWriteConn) Write(data []byte) (int, error) { | |
| 136 | + | c.cancel() | |
| 137 | + | return len(data), nil | |
| 138 | + | } | |
| 139 | + | ||
| 140 | + | type partialWriteConn struct { | |
| 141 | + | net.Conn | |
| 142 | + | withError bool | |
| 143 | + | } | |
| 144 | + | ||
| 145 | + | func (c *partialWriteConn) Write(data []byte) (int, error) { | |
| 146 | + | written := len(data) / 2 | |
| 147 | + | if written == 0 { | |
| 148 | + | written = 1 | |
| 149 | + | } | |
| 150 | + | if c.withError { | |
| 151 | + | return written, errors.New("partial write") | |
| 152 | + | } | |
| 153 | + | return written, nil | |
| 154 | + | } | |
| 155 | + | ||
| 130 | 156 | type notifyingContext struct { | |
| 131 | 157 | context.Context | |
| 132 | 158 | doneCalled chan struct{} | |
| @@ -138,6 +164,40 @@ func (c *notifyingContext) Done() <-chan struct{} { | |||
| 138 | 164 | return c.Context.Done() | |
| 139 | 165 | } | |
| 140 | 166 | ||
| 167 | + | func TestInboxSendContextTreatsCompleteWriteBeforeCancellationAsDelivered(t *testing.T) { | |
| 168 | + | local, remote := net.Pipe() | |
| 169 | + | defer remote.Close() | |
| 170 | + | ctx, cancel := context.WithCancel(context.Background()) | |
| 171 | + | inbox := NewInbox("unused") | |
| 172 | + | inbox.conn = &cancelAfterWriteConn{Conn: local, cancel: cancel} | |
| 173 | + | defer inbox.Close() | |
| 174 | + | ||
| 175 | + | if err := inbox.sendBytesContext(ctx, []byte("follow-up\n")); err != nil { | |
| 176 | + | t.Fatalf("complete write after cancellation = %v, want success", err) | |
| 177 | + | } | |
| 178 | + | } | |
| 179 | + | ||
| 180 | + | func TestInboxSendContextTreatsPartialWritesAsUnknownDelivery(t *testing.T) { | |
| 181 | + | for _, withError := range []bool{false, true} { | |
| 182 | + | name := "short write" | |
| 183 | + | if withError { | |
| 184 | + | name = "partial write error" | |
| 185 | + | } | |
| 186 | + | t.Run(name, func(t *testing.T) { | |
| 187 | + | local, remote := net.Pipe() | |
| 188 | + | defer remote.Close() | |
| 189 | + | inbox := NewInbox("unused") | |
| 190 | + | inbox.conn = &partialWriteConn{Conn: local, withError: withError} | |
| 191 | + | defer inbox.Close() | |
| 192 | + | ||
| 193 | + | err := inbox.sendBytesContext(context.Background(), []byte("follow-up\n")) | |
| 194 | + | if !errors.Is(err, ErrDeliveryUnknown) { | |
| 195 | + | t.Fatalf("partial write error = %v, want ErrDeliveryUnknown", err) | |
| 196 | + | } | |
| 197 | + | }) | |
| 198 | + | } | |
| 199 | + | } | |
| 200 | + | ||
| 141 | 201 | func TestInboxSendContextCancelsBlockedWrite(t *testing.T) { | |
| 142 | 202 | local, remote := net.Pipe() | |
| 143 | 203 | defer remote.Close() | |
| @@ -127,6 +127,32 @@ func (c *blockingWriteConn) Close() error { | |||||
| 127 | return c.Conn.Close() | 127 | return c.Conn.Close() | ||
| 128 | } | 128 | } | ||
| 129 | 129 | ||||
| 130 | + | type cancelAfterWriteConn struct { | |||
| 131 | + | net.Conn | |||
| 132 | + | cancel context.CancelFunc | |||
| 133 | + | } | |||
| 134 | + | ||||
| 135 | + | func (c *cancelAfterWriteConn) Write(data []byte) (int, error) { | |||
| 136 | + | c.cancel() | |||
| 137 | + | return len(data), nil | |||
| 138 | + | } | |||
| 139 | + | ||||
| 140 | + | type partialWriteConn struct { | |||
| 141 | + | net.Conn | |||
| 142 | + | withError bool | |||
| 143 | + | } | |||
| 144 | + | ||||
| 145 | + | func (c *partialWriteConn) Write(data []byte) (int, error) { | |||
| 146 | + | written := len(data) / 2 | |||
| 147 | + | if written == 0 { | |||
| 148 | + | written = 1 | |||
| 149 | + | } | |||
| 150 | + | if c.withError { | |||
| 151 | + | return written, errors.New("partial write") | |||
| 152 | + | } | |||
| 153 | + | return written, nil | |||
| 154 | + | } | |||
| 155 | + | ||||
| 130 | type notifyingContext struct { | 156 | type notifyingContext struct { | ||
| 131 | context.Context | 157 | context.Context | ||
| 132 | doneCalled chan struct{} | 158 | doneCalled chan struct{} | ||
| @@ -138,6 +164,40 @@ func (c *notifyingContext) Done() <-chan struct{} { | |||||
| 138 | return c.Context.Done() | 164 | return c.Context.Done() | ||
| 139 | } | 165 | } | ||
| 140 | 166 | ||||
| 167 | + | func TestInboxSendContextTreatsCompleteWriteBeforeCancellationAsDelivered(t *testing.T) { | |||
| 168 | + | local, remote := net.Pipe() | |||
| 169 | + | defer remote.Close() | |||
| 170 | + | ctx, cancel := context.WithCancel(context.Background()) | |||
| 171 | + | inbox := NewInbox("unused") | |||
| 172 | + | inbox.conn = &cancelAfterWriteConn{Conn: local, cancel: cancel} | |||
| 173 | + | defer inbox.Close() | |||
| 174 | + | ||||
| 175 | + | if err := inbox.sendBytesContext(ctx, []byte("follow-up\n")); err != nil { | |||
| 176 | + | t.Fatalf("complete write after cancellation = %v, want success", err) | |||
| 177 | + | } | |||
| 178 | + | } | |||
| 179 | + | ||||
| 180 | + | func TestInboxSendContextTreatsPartialWritesAsUnknownDelivery(t *testing.T) { | |||
| 181 | + | for _, withError := range []bool{false, true} { | |||
| 182 | + | name := "short write" | |||
| 183 | + | if withError { | |||
| 184 | + | name = "partial write error" | |||
| 185 | + | } | |||
| 186 | + | t.Run(name, func(t *testing.T) { | |||
| 187 | + | local, remote := net.Pipe() | |||
| 188 | + | defer remote.Close() | |||
| 189 | + | inbox := NewInbox("unused") | |||
| 190 | + | inbox.conn = &partialWriteConn{Conn: local, withError: withError} | |||
| 191 | + | defer inbox.Close() | |||
| 192 | + | ||||
| 193 | + | err := inbox.sendBytesContext(context.Background(), []byte("follow-up\n")) | |||
| 194 | + | if !errors.Is(err, ErrDeliveryUnknown) { | |||
| 195 | + | t.Fatalf("partial write error = %v, want ErrDeliveryUnknown", err) | |||
| 196 | + | } | |||
| 197 | + | }) | |||
| 198 | + | } | |||
| 199 | + | } | |||
| 200 | + | ||||
| 141 | func TestInboxSendContextCancelsBlockedWrite(t *testing.T) { | 201 | func TestInboxSendContextCancelsBlockedWrite(t *testing.T) { | ||
| 142 | local, remote := net.Pipe() | 202 | local, remote := net.Pipe() | ||
| 143 | defer remote.Close() | 203 | defer remote.Close() | ||
M packages/agent/subagents/persist.go +316 -80
| @@ -22,6 +22,8 @@ import ( | |||
| 22 | 22 | "sort" | |
| 23 | 23 | "strings" | |
| 24 | 24 | "time" | |
| 25 | + | ||
| 26 | + | "github.com/google/uuid" | |
| 25 | 27 | ) | |
| 26 | 28 | ||
| 27 | 29 | // agentMeta is the durable identity record for one agent. Only fields | |
| @@ -54,6 +56,8 @@ type agentMeta struct { | |||
| 54 | 56 | WorkspaceBase string `json:"workspace_base,omitempty"` | |
| 55 | 57 | WorkspaceCapture CaptureMode `json:"workspace_capture,omitempty"` | |
| 56 | 58 | MaxTurns int `json:"max_turns,omitempty"` | |
| 59 | + | LifetimeTurns int `json:"lifetime_turns,omitempty"` | |
| 60 | + | CurrentRunTurns int `json:"current_run_turns,omitempty"` | |
| 57 | 61 | Timeout time.Duration `json:"timeout,omitempty"` | |
| 58 | 62 | Tools []string `json:"tools,omitempty"` | |
| 59 | 63 | WebSearchPolicy WebSearchPolicy `json:"web_search_policy,omitempty"` | |
| @@ -77,8 +81,10 @@ type agentMeta struct { | |||
| 77 | 81 | // ResumePrompt is a follow-up accepted by subagent_resume but not yet | |
| 78 | 82 | // acknowledged by the child with a turn.started event. It is durable so a | |
| 79 | 83 | // host exit while the worker is queued does not lose the manager signal. | |
| 80 | - | ResumePrompt string `json:"resume_prompt,omitempty"` | |
| 81 | - | ResumePromptAt time.Time `json:"resume_prompt_at,omitempty"` | |
| 84 | + | ResumePrompt string `json:"resume_prompt,omitempty"` | |
| 85 | + | ResumePromptAt time.Time `json:"resume_prompt_at,omitempty"` | |
| 86 | + | ResumePromptID string `json:"resume_prompt_id,omitempty"` | |
| 87 | + | ResumeQueue []queuedResumePrompt `json:"resume_queue,omitempty"` | |
| 82 | 88 | ||
| 83 | 89 | // SessionID, when non-empty, scopes the agent to a particular | |
| 84 | 90 | // host zut session: the dashboard only shows agents whose | |
| @@ -96,6 +102,12 @@ func metaPath(stateDir string) string { return filepath.Join(stateDir, "meta.jso | |||
| 96 | 102 | // write is atomic (tmp + rename) so a crash mid-write can't leave a | |
| 97 | 103 | // half-parsable file that fails Reload. | |
| 98 | 104 | func writeAgentMeta(stateDir string, a *Agent) error { | |
| 105 | + | if a == nil { | |
| 106 | + | return errors.New("subagent meta: nil agent") | |
| 107 | + | } | |
| 108 | + | a.persistMu.Lock() | |
| 109 | + | defer a.persistMu.Unlock() | |
| 110 | + | ||
| 99 | 111 | fastMode := a.FastMode | |
| 100 | 112 | webSearchPolicy := childWebSearchPolicy(a.WebSearchPolicy, a.Subagent, a.Tools) | |
| 101 | 113 | a.mu.Lock() | |
| @@ -104,6 +116,8 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||
| 104 | 116 | finished := a.finished | |
| 105 | 117 | resumePrompt := a.resumePromptText | |
| 106 | 118 | resumePromptAt := a.resumePromptAt | |
| 119 | + | resumePromptID := a.resumePromptID | |
| 120 | + | resumeQueue := append([]queuedResumePrompt(nil), a.resumePromptQueue...) | |
| 107 | 121 | errText := "" | |
| 108 | 122 | if a.lastErr != nil { | |
| 109 | 123 | errText = a.lastErr.Error() | |
| @@ -113,6 +127,8 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||
| 113 | 127 | processState := a.processState | |
| 114 | 128 | turnState := a.turnState | |
| 115 | 129 | currentTurnID := a.currentTurnID | |
| 130 | + | lifetimeTurns := a.LifetimeTurns | |
| 131 | + | currentRunTurns := a.CurrentRunTurns | |
| 116 | 132 | attempt := a.Attempt | |
| 117 | 133 | processPID := a.ProcessPID | |
| 118 | 134 | updatedAt := a.updatedAt | |
| @@ -143,6 +159,8 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||
| 143 | 159 | WorkspaceBase: a.WorkspaceBase, | |
| 144 | 160 | WorkspaceCapture: a.WorkspaceCapture, | |
| 145 | 161 | MaxTurns: a.MaxTurns, | |
| 162 | + | LifetimeTurns: lifetimeTurns, | |
| 163 | + | CurrentRunTurns: currentRunTurns, | |
| 146 | 164 | Timeout: a.Timeout, | |
| 147 | 165 | Tools: append([]string(nil), a.Tools...), | |
| 148 | 166 | WebSearchPolicy: webSearchPolicy, | |
| @@ -165,6 +183,8 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||
| 165 | 183 | SessionPath: a.SessionPath, | |
| 166 | 184 | ResumePrompt: resumePrompt, | |
| 167 | 185 | ResumePromptAt: resumePromptAt, | |
| 186 | + | ResumePromptID: resumePromptID, | |
| 187 | + | ResumeQueue: resumeQueue, | |
| 168 | 188 | SessionID: a.SessionID, | |
| 169 | 189 | } | |
| 170 | 190 | b, err := json.MarshalIndent(m, "", " ") | |
| @@ -175,14 +195,50 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||
| 175 | 195 | return fmt.Errorf("subagents meta dir: %w", err) | |
| 176 | 196 | } | |
| 177 | 197 | final := metaPath(stateDir) | |
| 178 | - | tmp := final + ".tmp" | |
| 179 | - | if err := os.WriteFile(tmp, append(b, '\n'), 0o600); err != nil { | |
| 198 | + | // Use a unique temporary file while holding the per-agent persistence | |
| 199 | + | // lock. A fixed meta.json.tmp is vulnerable to a concurrent writer (or a | |
| 200 | + | // stale file after a crash) replacing another writer's snapshot. | |
| 201 | + | tmpFile, err := os.CreateTemp(stateDir, ".meta.json.tmp-*") | |
| 202 | + | if err != nil { | |
| 203 | + | return fmt.Errorf("subagents meta temp: %w", err) | |
| 204 | + | } | |
| 205 | + | tmp := tmpFile.Name() | |
| 206 | + | removeTemp := true | |
| 207 | + | defer func() { | |
| 208 | + | _ = tmpFile.Close() | |
| 209 | + | if removeTemp { | |
| 210 | + | _ = os.Remove(tmp) | |
| 211 | + | } | |
| 212 | + | }() | |
| 213 | + | if err := tmpFile.Chmod(0o600); err != nil { | |
| 214 | + | return fmt.Errorf("subagents meta permissions: %w", err) | |
| 215 | + | } | |
| 216 | + | if _, err := tmpFile.Write(append(b, '\n')); err != nil { | |
| 180 | 217 | return fmt.Errorf("subagents meta write: %w", err) | |
| 181 | 218 | } | |
| 219 | + | if err := tmpFile.Sync(); err != nil { | |
| 220 | + | return fmt.Errorf("subagents meta sync: %w", err) | |
| 221 | + | } | |
| 222 | + | if err := tmpFile.Close(); err != nil { | |
| 223 | + | return fmt.Errorf("subagents meta close: %w", err) | |
| 224 | + | } | |
| 182 | 225 | if err := os.Rename(tmp, final); err != nil { | |
| 183 | - | _ = os.Remove(tmp) | |
| 184 | 226 | return fmt.Errorf("subagent meta rename: %w", err) | |
| 185 | 227 | } | |
| 228 | + | removeTemp = false | |
| 229 | + | // Sync the containing directory so the rename itself survives a host | |
| 230 | + | // crash, not just the bytes in the temporary file. | |
| 231 | + | dir, err := os.Open(stateDir) | |
| 232 | + | if err != nil { | |
| 233 | + | return fmt.Errorf("subagents meta directory open: %w", err) | |
| 234 | + | } | |
| 235 | + | if err := dir.Sync(); err != nil { | |
| 236 | + | _ = dir.Close() | |
| 237 | + | return fmt.Errorf("subagents meta directory sync: %w", err) | |
| 238 | + | } | |
| 239 | + | if err := dir.Close(); err != nil { | |
| 240 | + | return fmt.Errorf("subagents meta directory close: %w", err) | |
| 241 | + | } | |
| 186 | 242 | return nil | |
| 187 | 243 | } | |
| 188 | 244 | ||
| @@ -420,57 +476,61 @@ func (f *Supervisor) buildDetachedAgent(m agentMeta) *Agent { | |||
| 420 | 476 | agentStateDir = filepath.Dir(m.EventLogPath) | |
| 421 | 477 | } | |
| 422 | 478 | a := &Agent{ | |
| 423 | - | ID: m.ID, | |
| 424 | - | Task: m.Task, | |
| 425 | - | OriginalTask: firstNonEmpty(m.OriginalTask, m.Task), | |
| 426 | - | Dir: dir, | |
| 427 | - | RepositoryRoot: firstNonEmpty(m.RepositoryRoot, f.cfg.RepoRoot), | |
| 428 | - | Started: m.Started, | |
| 429 | - | ParentID: m.ParentID, | |
| 430 | - | BatchID: m.BatchID, | |
| 431 | - | RootSessionID: m.RootSessionID, | |
| 432 | - | Model: m.Model, | |
| 433 | - | Provider: m.Provider, | |
| 434 | - | BaseURL: m.BaseURL, | |
| 435 | - | InsecureTLS: m.InsecureTLS, | |
| 436 | - | Reasoning: m.Reasoning, | |
| 437 | - | FastMode: fastMode, | |
| 438 | - | Subagent: m.Subagent, | |
| 439 | - | WorkspaceMode: m.WorkspaceMode, | |
| 440 | - | WorkspacePath: m.WorkspacePath, | |
| 441 | - | WorkspaceBase: m.WorkspaceBase, | |
| 442 | - | WorkspaceCapture: m.WorkspaceCapture, | |
| 443 | - | MaxTurns: m.MaxTurns, | |
| 444 | - | Timeout: m.Timeout, | |
| 445 | - | Tools: append([]string(nil), m.Tools...), | |
| 446 | - | WebSearchPolicy: childWebSearchPolicy(m.WebSearchPolicy, m.Subagent, m.Tools), | |
| 447 | - | Attempt: m.Attempt, | |
| 448 | - | ProcessPID: m.ProcessPID, | |
| 449 | - | InboxPath: m.InboxPath, | |
| 450 | - | EventLogPath: m.EventLogPath, | |
| 451 | - | SessionPath: m.SessionPath, | |
| 452 | - | resumePromptText: m.ResumePrompt, | |
| 453 | - | resumePromptAt: m.ResumePromptAt, | |
| 454 | - | SessionID: m.SessionID, | |
| 455 | - | inbox: NewInbox(m.InboxPath), | |
| 456 | - | status: status, | |
| 457 | - | activity: activity, | |
| 458 | - | finished: m.Finished, | |
| 459 | - | lastErr: lastErr, | |
| 460 | - | legacyEventState: legacyEventState, | |
| 461 | - | processState: processState, | |
| 462 | - | turnState: turnState, | |
| 463 | - | currentTurnID: m.CurrentTurnID, | |
| 464 | - | updatedAt: m.UpdatedAt, | |
| 465 | - | lastActivity: lastActivity, | |
| 466 | - | stateDir: agentStateDir, | |
| 467 | - | resultRef: m.ResultRef, | |
| 468 | - | patchRef: m.PatchRef, | |
| 469 | - | changedFiles: append([]string(nil), m.ChangedFiles...), | |
| 470 | - | maxOutputBytes: f.cfg.Policy.MaxOutputBytes, | |
| 471 | - | maxOutputLines: f.cfg.Policy.MaxOutputLines, | |
| 472 | - | done: make(chan struct{}), | |
| 473 | - | turnResults: make(chan *TurnResult, 16), | |
| 479 | + | ID: m.ID, | |
| 480 | + | Task: m.Task, | |
| 481 | + | OriginalTask: firstNonEmpty(m.OriginalTask, m.Task), | |
| 482 | + | Dir: dir, | |
| 483 | + | RepositoryRoot: firstNonEmpty(m.RepositoryRoot, f.cfg.RepoRoot), | |
| 484 | + | Started: m.Started, | |
| 485 | + | ParentID: m.ParentID, | |
| 486 | + | BatchID: m.BatchID, | |
| 487 | + | RootSessionID: m.RootSessionID, | |
| 488 | + | Model: m.Model, | |
| 489 | + | Provider: m.Provider, | |
| 490 | + | BaseURL: m.BaseURL, | |
| 491 | + | InsecureTLS: m.InsecureTLS, | |
| 492 | + | Reasoning: m.Reasoning, | |
| 493 | + | FastMode: fastMode, | |
| 494 | + | Subagent: m.Subagent, | |
| 495 | + | WorkspaceMode: m.WorkspaceMode, | |
| 496 | + | WorkspacePath: m.WorkspacePath, | |
| 497 | + | WorkspaceBase: m.WorkspaceBase, | |
| 498 | + | WorkspaceCapture: m.WorkspaceCapture, | |
| 499 | + | MaxTurns: m.MaxTurns, | |
| 500 | + | LifetimeTurns: m.LifetimeTurns, | |
| 501 | + | CurrentRunTurns: m.CurrentRunTurns, | |
| 502 | + | Timeout: m.Timeout, | |
| 503 | + | Tools: append([]string(nil), m.Tools...), | |
| 504 | + | WebSearchPolicy: childWebSearchPolicy(m.WebSearchPolicy, m.Subagent, m.Tools), | |
| 505 | + | Attempt: m.Attempt, | |
| 506 | + | ProcessPID: m.ProcessPID, | |
| 507 | + | InboxPath: m.InboxPath, | |
| 508 | + | EventLogPath: m.EventLogPath, | |
| 509 | + | SessionPath: m.SessionPath, | |
| 510 | + | resumePromptText: m.ResumePrompt, | |
| 511 | + | resumePromptAt: m.ResumePromptAt, | |
| 512 | + | resumePromptID: m.ResumePromptID, | |
| 513 | + | resumePromptQueue: append([]queuedResumePrompt(nil), m.ResumeQueue...), | |
| 514 | + | SessionID: m.SessionID, | |
| 515 | + | inbox: NewInbox(m.InboxPath), | |
| 516 | + | status: status, | |
| 517 | + | activity: activity, | |
| 518 | + | finished: m.Finished, | |
| 519 | + | lastErr: lastErr, | |
| 520 | + | legacyEventState: legacyEventState, | |
| 521 | + | processState: processState, | |
| 522 | + | turnState: turnState, | |
| 523 | + | currentTurnID: m.CurrentTurnID, | |
| 524 | + | updatedAt: m.UpdatedAt, | |
| 525 | + | lastActivity: lastActivity, | |
| 526 | + | stateDir: agentStateDir, | |
| 527 | + | resultRef: m.ResultRef, | |
| 528 | + | patchRef: m.PatchRef, | |
| 529 | + | changedFiles: append([]string(nil), m.ChangedFiles...), | |
| 530 | + | maxOutputBytes: f.cfg.Policy.MaxOutputBytes, | |
| 531 | + | maxOutputLines: f.cfg.Policy.MaxOutputLines, | |
| 532 | + | done: make(chan struct{}), | |
| 533 | + | turnResults: make(chan *TurnResult, 16), | |
| 474 | 534 | } | |
| 475 | 535 | if a.updatedAt.IsZero() { | |
| 476 | 536 | a.updatedAt = lastActivity | |
| @@ -542,7 +602,7 @@ func (a *Agent) loadTranscript() error { | |||
| 542 | 602 | if err != nil { | |
| 543 | 603 | return fmt.Errorf("read subagent transcript: %w", err) | |
| 544 | 604 | } | |
| 545 | - | if prompt, acceptedAt := a.ResumePromptInfo(); prompt != "" && resumePromptAcknowledged(events, acceptedAt) { | |
| 605 | + | if prompt, acceptedAt := a.ResumePromptInfo(); prompt != "" && resumePromptAcknowledged(events, prompt, acceptedAt) { | |
| 546 | 606 | a.clearResumePrompt() | |
| 547 | 607 | } | |
| 548 | 608 | a.mu.Lock() | |
| @@ -583,12 +643,23 @@ func (a *Agent) loadTranscript() error { | |||
| 583 | 643 | // Status precedence: explicit lifecycle events (agent_stopped) win | |
| 584 | 644 | // over inferred ones (assistant_message → idle). If the log contains | |
| 585 | 645 | // no terminator we keep status=StatusDetached so the user can resume. | |
| 586 | - | func resumePromptAcknowledged(events []Event, acceptedAt time.Time) bool { | |
| 587 | - | if acceptedAt.IsZero() { | |
| 646 | + | func resumePromptAcknowledged(events []Event, prompt string, acceptedAt time.Time) bool { | |
| 647 | + | if prompt == "" || acceptedAt.IsZero() { | |
| 588 | 648 | return false | |
| 589 | 649 | } | |
| 650 | + | userDurable := false | |
| 590 | 651 | for _, ev := range events { | |
| 591 | - | if (ev.Type == EventTurnStarted || ev.Type == "turn_start") && !ev.Time.Before(acceptedAt) { | |
| 652 | + | if ev.Time.Before(acceptedAt) { | |
| 653 | + | continue | |
| 654 | + | } | |
| 655 | + | if ev.Type == "user_message" && userMessageText(ev) == prompt { | |
| 656 | + | // The worker emits this event only after the matching session | |
| 657 | + | // message has been appended and flushed. A crash here leaves the | |
| 658 | + | // prompt pending, which is intentionally safe to replay. | |
| 659 | + | userDurable = true | |
| 660 | + | continue | |
| 661 | + | } | |
| 662 | + | if userDurable && isDelegatedTurnStart(ev) { | |
| 592 | 663 | return true | |
| 593 | 664 | } | |
| 594 | 665 | } | |
| @@ -600,14 +671,11 @@ func replayEventsIntoAgent(a *Agent, evs []Event) { | |||
| 600 | 671 | for _, ev := range evs { | |
| 601 | 672 | replayEventTranscript(a, ev) | |
| 602 | 673 | switch ev.Type { | |
| 603 | - | case EventTurnStarted, "turn_start": | |
| 604 | - | turnID := ev.TurnID | |
| 605 | - | if turnID == "" { | |
| 606 | - | if step, ok := ev.Data["step"].(float64); ok { | |
| 607 | - | turnID = fmt.Sprintf("turn-%d", int(step)) | |
| 608 | - | } | |
| 674 | + | case EventTurnStarted: | |
| 675 | + | if !isDelegatedTurnStart(ev) { | |
| 676 | + | continue | |
| 609 | 677 | } | |
| 610 | - | a.setTurnState(TurnRunning, turnID) | |
| 678 | + | a.setTurnState(TurnRunning, eventTurnID(ev)) | |
| 611 | 679 | case EventTurnResult, "turn_result": | |
| 612 | 680 | if result, err := decodeTurnResultEvent(ev, a.ID, a.maxOutputBytes, a.maxOutputLines); err == nil { | |
| 613 | 681 | a.setResult(result) | |
| @@ -741,9 +809,16 @@ func (f *Supervisor) ResumeSession(ctx context.Context, id string) (*Agent, erro | |||
| 741 | 809 | ||
| 742 | 810 | // ResumeWithPrompt continues an agent with a manager follow-up while retaining | |
| 743 | 811 | // its existing session. An idle live worker receives the turn through its | |
| 744 | - | // inbox; a terminal worker is restarted with the follow-up as its initial | |
| 745 | - | // prompt, avoiding a race with inbox listener startup. | |
| 812 | + | // inbox; an active live worker queues the follow-up for the next idle turn; a | |
| 813 | + | // terminal worker is restarted with the follow-up as its initial prompt, | |
| 814 | + | // avoiding a race with inbox listener startup. | |
| 746 | 815 | func (f *Supervisor) ResumeWithPrompt(ctx context.Context, id, prompt string) (*Agent, error) { | |
| 816 | + | if ctx == nil { | |
| 817 | + | ctx = context.Background() | |
| 818 | + | } | |
| 819 | + | if err := ctx.Err(); err != nil { | |
| 820 | + | return nil, err | |
| 821 | + | } | |
| 747 | 822 | if strings.TrimSpace(prompt) == "" { | |
| 748 | 823 | return nil, fmt.Errorf("subagent: follow-up prompt is required") | |
| 749 | 824 | } | |
| @@ -756,34 +831,82 @@ func (f *Supervisor) ResumeWithPrompt(ctx context.Context, id, prompt string) (* | |||
| 756 | 831 | } | |
| 757 | 832 | status := existing.Status() | |
| 758 | 833 | if status == StatusRunning { | |
| 759 | - | if existing.ProcessState() != ProcessAlive || existing.TurnState() != TurnIdle { | |
| 834 | + | if existing.ProcessState() != ProcessAlive { | |
| 760 | 835 | f.operationMu.Unlock() | |
| 761 | - | return nil, fmt.Errorf("subagent: agent %s is still working; wait for it to become idle before sending a follow-up", existing.ID) | |
| 836 | + | return nil, fmt.Errorf("subagent: agent %s is still working; wait for its worker to become available before sending a follow-up", existing.ID) | |
| 762 | 837 | } | |
| 763 | 838 | if existing.inbox == nil { | |
| 764 | 839 | f.operationMu.Unlock() | |
| 765 | 840 | return nil, fmt.Errorf("subagent: agent %s has no inbox", existing.ID) | |
| 766 | 841 | } | |
| 842 | + | if err := ctx.Err(); err != nil { | |
| 843 | + | f.operationMu.Unlock() | |
| 844 | + | return nil, err | |
| 845 | + | } | |
| 846 | + | if existing.TurnState() != TurnRunning && existing.hasQueuedResumePrompt() { | |
| 847 | + | f.operationMu.Unlock() | |
| 848 | + | return nil, fmt.Errorf("subagent: agent %s already has a follow-up pending delivery", existing.ID) | |
| 849 | + | } | |
| 850 | + | if existing.TurnState() == TurnRunning { | |
| 851 | + | // Active turns cannot accept a command through the worker inbox yet. | |
| 852 | + | // Keep the prompt durable and let the idle transition deliver it in | |
| 853 | + | // order after the current message turn finishes. | |
| 854 | + | acceptedAt := f.cfg.Now() | |
| 855 | + | commandID := existing.queueResumePrompt(prompt, acceptedAt) | |
| 856 | + | if err := f.persistAgent(existing); err != nil { | |
| 857 | + | existing.removeQueuedResumePrompt(commandID) | |
| 858 | + | f.operationMu.Unlock() | |
| 859 | + | return nil, fmt.Errorf("subagent: persist queued follow-up for %s: %w", existing.ID, err) | |
| 860 | + | } | |
| 861 | + | f.operationMu.Unlock() | |
| 862 | + | return existing, nil | |
| 863 | + | } | |
| 864 | + | if existing.TurnState() != TurnIdle { | |
| 865 | + | f.operationMu.Unlock() | |
| 866 | + | return nil, fmt.Errorf("subagent: agent %s is still working; wait for it to become idle before sending a follow-up", existing.ID) | |
| 867 | + | } | |
| 767 | 868 | turnID := existing.CurrentTurnID() | |
| 869 | + | previousRunTurns := existing.resetCurrentRunTurns() | |
| 768 | 870 | // Reserve the worker before writing to its inbox. A second manager call | |
| 769 | 871 | // must not enqueue another turn in the small interval before the worker | |
| 770 | 872 | // reports its turn.start event. Persist the prompt first so a host exit | |
| 771 | 873 | // before that event can resume and redeliver it. | |
| 772 | - | previousPrompt, previousPromptAt := existing.setResumePrompt(prompt, f.cfg.Now()) | |
| 874 | + | previousPrompt := existing.setResumePrompt(prompt, f.cfg.Now()) | |
| 773 | 875 | existing.setTurnState(TurnQueued, turnID) | |
| 774 | 876 | inbox := existing.inbox | |
| 775 | - | command := NewCommand(CommandTurnStart, existing.ID, turnID, TurnStartPayload{Prompt: prompt}) | |
| 776 | - | f.persistAgent(existing) | |
| 877 | + | command := NewCommand(CommandTurnStart, existing.ID, turnID, TurnStartPayload{Prompt: prompt, NewRun: true}) | |
| 878 | + | command.MessageID = existing.resumePromptCommandID() | |
| 879 | + | if err := f.persistAgent(existing); err != nil { | |
| 880 | + | existing.setTurnState(TurnIdle, turnID) | |
| 881 | + | existing.setCurrentRunTurns(previousRunTurns) | |
| 882 | + | existing.restoreResumePrompt(previousPrompt) | |
| 883 | + | f.operationMu.Unlock() | |
| 884 | + | return nil, fmt.Errorf("subagent: persist follow-up for %s: %w", existing.ID, err) | |
| 885 | + | } | |
| 777 | 886 | f.operationMu.Unlock() | |
| 778 | 887 | ||
| 779 | 888 | if err := inbox.SendCommandContext(ctx, command); err != nil { | |
| 889 | + | retryUnknown := false | |
| 780 | 890 | f.operationMu.Lock() | |
| 781 | 891 | if existing.Status() == StatusRunning && existing.TurnState() == TurnQueued && existing.CurrentTurnID() == turnID { | |
| 782 | 892 | existing.setTurnState(TurnIdle, turnID) | |
| 783 | - | existing.setResumePrompt(previousPrompt, previousPromptAt) | |
| 784 | - | f.persistAgent(existing) | |
| 893 | + | existing.setCurrentRunTurns(previousRunTurns) | |
| 894 | + | if errors.Is(err, ErrDeliveryUnknown) { | |
| 895 | + | existing.rejectActiveResumePrompt() | |
| 896 | + | retryUnknown = true | |
| 897 | + | } else { | |
| 898 | + | existing.restoreResumePrompt(previousPrompt) | |
| 899 | + | } | |
| 900 | + | if persistErr := f.persistAgent(existing); persistErr != nil { | |
| 901 | + | err = fmt.Errorf("%w; persist recovery: %v", err, persistErr) | |
| 902 | + | } | |
| 785 | 903 | } | |
| 786 | 904 | f.operationMu.Unlock() | |
| 905 | + | if retryUnknown { | |
| 906 | + | // The command keeps its durable identity. A retry is safe whether | |
| 907 | + | // the worker parsed the first write or only received a prefix. | |
| 908 | + | f.dispatchQueuedResume(ctx, existing) | |
| 909 | + | } | |
| 787 | 910 | return nil, fmt.Errorf("subagent: send follow-up to %s: %w", existing.ID, err) | |
| 788 | 911 | } | |
| 789 | 912 | return existing, nil | |
| @@ -793,6 +916,77 @@ func (f *Supervisor) ResumeWithPrompt(ctx context.Context, id, prompt string) (* | |||
| 793 | 916 | return f.resume(ctx, id, true, prompt) | |
| 794 | 917 | } | |
| 795 | 918 | ||
| 919 | + | // dispatchQueuedResumeWithTimeout keeps the idle callback bounded. Unlike a | |
| 920 | + | // manager request, an idle transition has no caller context to propagate. | |
| 921 | + | func (f *Supervisor) dispatchQueuedResumeWithTimeout(a *Agent) { | |
| 922 | + | timeout := f.cfg.Policy.ReconnectTimeout | |
| 923 | + | if timeout <= 0 { | |
| 924 | + | timeout = 5 * time.Second | |
| 925 | + | } | |
| 926 | + | ctx, cancel := context.WithTimeout(context.Background(), timeout) | |
| 927 | + | defer cancel() | |
| 928 | + | f.dispatchQueuedResume(ctx, a) | |
| 929 | + | } | |
| 930 | + | ||
| 931 | + | // dispatchQueuedResume delivers the oldest follow-up accepted during an | |
| 932 | + | // active turn after the worker reports that its inbox can accept a new turn. | |
| 933 | + | // The operation lock serializes this transition with manager resume calls and | |
| 934 | + | // the agent's durable prompt state prevents a host exit from losing it. | |
| 935 | + | func (f *Supervisor) dispatchQueuedResume(ctx context.Context, a *Agent) { | |
| 936 | + | if ctx == nil { | |
| 937 | + | ctx = context.Background() | |
| 938 | + | } | |
| 939 | + | if a == nil { | |
| 940 | + | return | |
| 941 | + | } | |
| 942 | + | f.operationMu.Lock() | |
| 943 | + | if a.Status() != StatusRunning || a.ProcessState() != ProcessAlive || a.TurnState() != TurnIdle || a.inbox == nil { | |
| 944 | + | f.operationMu.Unlock() | |
| 945 | + | return | |
| 946 | + | } | |
| 947 | + | prompt, acceptedAt, commandID, ok := a.startQueuedResumePrompt() | |
| 948 | + | if !ok { | |
| 949 | + | f.operationMu.Unlock() | |
| 950 | + | return | |
| 951 | + | } | |
| 952 | + | turnID := a.CurrentTurnID() | |
| 953 | + | previousRunTurns := a.resetCurrentRunTurns() | |
| 954 | + | a.setTurnState(TurnQueued, turnID) | |
| 955 | + | inbox := a.inbox | |
| 956 | + | command := NewCommand(CommandTurnStart, a.ID, turnID, TurnStartPayload{Prompt: prompt, NewRun: true}) | |
| 957 | + | command.MessageID = commandID | |
| 958 | + | if err := f.persistAgent(a); err != nil { | |
| 959 | + | a.restoreResumePromptToQueue(prompt, acceptedAt) | |
| 960 | + | a.setCurrentRunTurns(previousRunTurns) | |
| 961 | + | a.setTurnState(TurnIdle, turnID) | |
| 962 | + | f.operationMu.Unlock() | |
| 963 | + | return | |
| 964 | + | } | |
| 965 | + | f.operationMu.Unlock() | |
| 966 | + | ||
| 967 | + | if err := inbox.SendCommandContext(ctx, command); err == nil { | |
| 968 | + | return | |
| 969 | + | } | |
| 970 | + | ||
| 971 | + | // A failed delivery must put the prompt back at the head of the queue. | |
| 972 | + | // Keep the worker idle so a later retry, or an explicit resume, can send | |
| 973 | + | // it without skipping any subsequent follow-ups. | |
| 974 | + | f.operationMu.Lock() | |
| 975 | + | if a.Status() == StatusRunning && a.ProcessState() == ProcessAlive && a.TurnState() == TurnQueued && a.CurrentTurnID() == turnID { | |
| 976 | + | if a.restoreResumePromptToQueue(prompt, acceptedAt) { | |
| 977 | + | a.setCurrentRunTurns(previousRunTurns) | |
| 978 | + | a.setTurnState(TurnIdle, turnID) | |
| 979 | + | if persistErr := f.persistAgent(a); persistErr != nil { | |
| 980 | + | // persistAgent records the failure on the agent. Keep the explicit | |
| 981 | + | // check here because this recovery write is the only durable copy | |
| 982 | + | // of the restored queue head. | |
| 983 | + | a.recordPersistenceError(persistErr) | |
| 984 | + | } | |
| 985 | + | } | |
| 986 | + | } | |
| 987 | + | f.operationMu.Unlock() | |
| 988 | + | } | |
| 989 | + | ||
| 796 | 990 | // RestartTask intentionally runs the stored original task again in a fresh | |
| 797 | 991 | // worker attempt. Callers should use ResumeSession for normal recovery. | |
| 798 | 992 | func (f *Supervisor) RestartTask(ctx context.Context, id string) (*Agent, error) { | |
| @@ -858,10 +1052,45 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||
| 858 | 1052 | } | |
| 859 | 1053 | now := f.cfg.Now() | |
| 860 | 1054 | fastMode := existing.FastMode | |
| 861 | - | pendingResumePrompt, pendingResumePromptAt := existing.ResumePromptInfo() | |
| 1055 | + | lifetimeTurns := existing.LifetimeTurnsValue() | |
| 1056 | + | currentRunTurns := existing.CurrentRunTurnsValue() | |
| 1057 | + | pendingResume := existing.resumePromptState() | |
| 1058 | + | pendingResumePrompt, pendingResumePromptAt, pendingResumePromptID := pendingResume.Prompt, pendingResume.AcceptedAt, pendingResume.CommandID | |
| 1059 | + | pendingResumeQueue := existing.resumePromptQueueSnapshot() | |
| 1060 | + | if pendingResumePrompt != "" && pendingResumePromptID == "" { | |
| 1061 | + | pendingResumePromptID = uuid.NewString() | |
| 1062 | + | } | |
| 862 | 1063 | if resumePrompt != "" { | |
| 863 | - | pendingResumePrompt = resumePrompt | |
| 864 | - | pendingResumePromptAt = now | |
| 1064 | + | currentRunTurns = 0 | |
| 1065 | + | if pendingResumePrompt == "" { | |
| 1066 | + | pendingResumePrompt = resumePrompt | |
| 1067 | + | pendingResumePromptAt = now | |
| 1068 | + | pendingResumePromptID = uuid.NewString() | |
| 1069 | + | } else { | |
| 1070 | + | pendingResumeQueue = append(pendingResumeQueue, queuedResumePrompt{Prompt: resumePrompt, AcceptedAt: now, CommandID: uuid.NewString()}) | |
| 1071 | + | } | |
| 1072 | + | } | |
| 1073 | + | if resuming && resumePrompt == "" && pendingResumePrompt == "" && len(pendingResumeQueue) > 0 { | |
| 1074 | + | // A host can exit after accepting an active-worker follow-up but before | |
| 1075 | + | // dispatching it. Make the first durable queued prompt the resumed | |
| 1076 | + | // worker's initial turn rather than leaving the queue stranded. | |
| 1077 | + | queued := pendingResumeQueue[0] | |
| 1078 | + | pendingResumePrompt = queued.Prompt | |
| 1079 | + | pendingResumePromptAt = queued.AcceptedAt | |
| 1080 | + | pendingResumePromptID = queued.CommandID | |
| 1081 | + | if pendingResumePromptID == "" { | |
| 1082 | + | pendingResumePromptID = uuid.NewString() | |
| 1083 | + | } | |
| 1084 | + | pendingResumeQueue = pendingResumeQueue[1:] | |
| 1085 | + | } | |
| 1086 | + | if resuming && pendingResumePrompt != "" { | |
| 1087 | + | // A retained follow-up is an explicit new run, including a queued | |
| 1088 | + | // follow-up recovered after a host restart. Preserve lifetime turns but | |
| 1089 | + | // never carry the previous run's allowance into this one. | |
| 1090 | + | currentRunTurns = 0 | |
| 1091 | + | } | |
| 1092 | + | if !resuming { | |
| 1093 | + | currentRunTurns = 0 | |
| 865 | 1094 | } | |
| 866 | 1095 | m := agentMeta{ | |
| 867 | 1096 | ID: existing.ID, Task: existing.Task, | |
| @@ -874,7 +1103,7 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||
| 874 | 1103 | Subagent: existing.Subagent, | |
| 875 | 1104 | WorkspaceMode: existing.WorkspaceMode, WorkspacePath: existing.WorkspacePath, | |
| 876 | 1105 | WorkspaceBase: existing.WorkspaceBase, WorkspaceCapture: existing.WorkspaceCapture, | |
| 877 | - | MaxTurns: existing.MaxTurns, Timeout: existing.Timeout, Tools: append([]string(nil), existing.Tools...), | |
| 1106 | + | MaxTurns: existing.MaxTurns, LifetimeTurns: lifetimeTurns, CurrentRunTurns: currentRunTurns, Timeout: existing.Timeout, Tools: append([]string(nil), existing.Tools...), | |
| 878 | 1107 | WebSearchPolicy: childWebSearchPolicy(existing.WebSearchPolicy, existing.Subagent, existing.Tools), | |
| 879 | 1108 | CurrentTurnID: existingSnapshot.CurrentTurnID, Attempt: existingSnapshot.Attempt, | |
| 880 | 1109 | SessionID: existing.SessionID, | |
| @@ -882,6 +1111,8 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||
| 882 | 1111 | SessionPath: existing.SessionPath, | |
| 883 | 1112 | ResumePrompt: pendingResumePrompt, | |
| 884 | 1113 | ResumePromptAt: pendingResumePromptAt, | |
| 1114 | + | ResumePromptID: pendingResumePromptID, | |
| 1115 | + | ResumeQueue: pendingResumeQueue, | |
| 885 | 1116 | } | |
| 886 | 1117 | ||
| 887 | 1118 | if len(m.Tools) == 0 && len(f.cfg.Policy.AllowedTools) > 0 { | |
| @@ -960,6 +1191,8 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||
| 960 | 1191 | WorkspaceBase: m.WorkspaceBase, | |
| 961 | 1192 | WorkspaceCapture: m.WorkspaceCapture, | |
| 962 | 1193 | MaxTurns: maxTurns, | |
| 1194 | + | LifetimeTurns: m.LifetimeTurns, | |
| 1195 | + | CurrentRunTurns: m.CurrentRunTurns, | |
| 963 | 1196 | Timeout: timeout, | |
| 964 | 1197 | Attempt: m.Attempt, | |
| 965 | 1198 | HeartbeatInterval: f.cfg.Policy.HeartbeatInterval, | |
| @@ -971,6 +1204,8 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||
| 971 | 1204 | Resuming: resuming, | |
| 972 | 1205 | resumePromptText: pendingResumePrompt, | |
| 973 | 1206 | resumePromptAt: pendingResumePromptAt, | |
| 1207 | + | resumePromptID: m.ResumePromptID, | |
| 1208 | + | resumePromptQueue: append([]queuedResumePrompt(nil), pendingResumeQueue...), | |
| 974 | 1209 | inbox: NewInbox(m.InboxPath), | |
| 975 | 1210 | status: StatusPending, | |
| 976 | 1211 | activity: "resuming", | |
| @@ -995,6 +1230,7 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||
| 995 | 1230 | } | |
| 996 | 1231 | a.ctx, a.cancel = runCtx, cancel | |
| 997 | 1232 | a.persistFn = f.persistAgent | |
| 1233 | + | a.setOnTurnIdle(func() { f.dispatchQueuedResumeWithTimeout(a) }) | |
| 998 | 1234 | a.workspaceCleanup = func() error { return workspace.Cleanup(context.Background()) } | |
| 999 | 1235 | a.workspaceCapture = func() (WorkspaceCapture, error) { return workspace.Capture(context.Background()) } | |
| 1000 | 1236 | a.runner = f.cfg.NewRunner(a) | |
| @@ -22,6 +22,8 @@ import ( | |||||
| 22 | "sort" | 22 | "sort" | ||
| 23 | "strings" | 23 | "strings" | ||
| 24 | "time" | 24 | "time" | ||
| 25 | + | ||||
| 26 | + | "github.com/google/uuid" | |||
| 25 | ) | 27 | ) | ||
| 26 | 28 | ||||
| 27 | // agentMeta is the durable identity record for one agent. Only fields | 29 | // agentMeta is the durable identity record for one agent. Only fields | ||
| @@ -54,6 +56,8 @@ type agentMeta struct { | |||||
| 54 | WorkspaceBase string `json:"workspace_base,omitempty"` | 56 | WorkspaceBase string `json:"workspace_base,omitempty"` | ||
| 55 | WorkspaceCapture CaptureMode `json:"workspace_capture,omitempty"` | 57 | WorkspaceCapture CaptureMode `json:"workspace_capture,omitempty"` | ||
| 56 | MaxTurns int `json:"max_turns,omitempty"` | 58 | MaxTurns int `json:"max_turns,omitempty"` | ||
| 59 | + | LifetimeTurns int `json:"lifetime_turns,omitempty"` | |||
| 60 | + | CurrentRunTurns int `json:"current_run_turns,omitempty"` | |||
| 57 | Timeout time.Duration `json:"timeout,omitempty"` | 61 | Timeout time.Duration `json:"timeout,omitempty"` | ||
| 58 | Tools []string `json:"tools,omitempty"` | 62 | Tools []string `json:"tools,omitempty"` | ||
| 59 | WebSearchPolicy WebSearchPolicy `json:"web_search_policy,omitempty"` | 63 | WebSearchPolicy WebSearchPolicy `json:"web_search_policy,omitempty"` | ||
| @@ -77,8 +81,10 @@ type agentMeta struct { | |||||
| 77 | // ResumePrompt is a follow-up accepted by subagent_resume but not yet | 81 | // ResumePrompt is a follow-up accepted by subagent_resume but not yet | ||
| 78 | // acknowledged by the child with a turn.started event. It is durable so a | 82 | // acknowledged by the child with a turn.started event. It is durable so a | ||
| 79 | // host exit while the worker is queued does not lose the manager signal. | 83 | // host exit while the worker is queued does not lose the manager signal. | ||
| 80 | - | ResumePrompt string `json:"resume_prompt,omitempty"` | 84 | + | ResumePrompt string `json:"resume_prompt,omitempty"` |
| 81 | - | ResumePromptAt time.Time `json:"resume_prompt_at,omitempty"` | 85 | + | ResumePromptAt time.Time `json:"resume_prompt_at,omitempty"` |
| 86 | + | ResumePromptID string `json:"resume_prompt_id,omitempty"` | |||
| 87 | + | ResumeQueue []queuedResumePrompt `json:"resume_queue,omitempty"` | |||
| 82 | 88 | ||||
| 83 | // SessionID, when non-empty, scopes the agent to a particular | 89 | // SessionID, when non-empty, scopes the agent to a particular | ||
| 84 | // host zut session: the dashboard only shows agents whose | 90 | // host zut session: the dashboard only shows agents whose | ||
| @@ -96,6 +102,12 @@ func metaPath(stateDir string) string { return filepath.Join(stateDir, "meta.jso | |||||
| 96 | // write is atomic (tmp + rename) so a crash mid-write can't leave a | 102 | // write is atomic (tmp + rename) so a crash mid-write can't leave a | ||
| 97 | // half-parsable file that fails Reload. | 103 | // half-parsable file that fails Reload. | ||
| 98 | func writeAgentMeta(stateDir string, a *Agent) error { | 104 | func writeAgentMeta(stateDir string, a *Agent) error { | ||
| 105 | + | if a == nil { | |||
| 106 | + | return errors.New("subagent meta: nil agent") | |||
| 107 | + | } | |||
| 108 | + | a.persistMu.Lock() | |||
| 109 | + | defer a.persistMu.Unlock() | |||
| 110 | + | ||||
| 99 | fastMode := a.FastMode | 111 | fastMode := a.FastMode | ||
| 100 | webSearchPolicy := childWebSearchPolicy(a.WebSearchPolicy, a.Subagent, a.Tools) | 112 | webSearchPolicy := childWebSearchPolicy(a.WebSearchPolicy, a.Subagent, a.Tools) | ||
| 101 | a.mu.Lock() | 113 | a.mu.Lock() | ||
| @@ -104,6 +116,8 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||||
| 104 | finished := a.finished | 116 | finished := a.finished | ||
| 105 | resumePrompt := a.resumePromptText | 117 | resumePrompt := a.resumePromptText | ||
| 106 | resumePromptAt := a.resumePromptAt | 118 | resumePromptAt := a.resumePromptAt | ||
| 119 | + | resumePromptID := a.resumePromptID | |||
| 120 | + | resumeQueue := append([]queuedResumePrompt(nil), a.resumePromptQueue...) | |||
| 107 | errText := "" | 121 | errText := "" | ||
| 108 | if a.lastErr != nil { | 122 | if a.lastErr != nil { | ||
| 109 | errText = a.lastErr.Error() | 123 | errText = a.lastErr.Error() | ||
| @@ -113,6 +127,8 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||||
| 113 | processState := a.processState | 127 | processState := a.processState | ||
| 114 | turnState := a.turnState | 128 | turnState := a.turnState | ||
| 115 | currentTurnID := a.currentTurnID | 129 | currentTurnID := a.currentTurnID | ||
| 130 | + | lifetimeTurns := a.LifetimeTurns | |||
| 131 | + | currentRunTurns := a.CurrentRunTurns | |||
| 116 | attempt := a.Attempt | 132 | attempt := a.Attempt | ||
| 117 | processPID := a.ProcessPID | 133 | processPID := a.ProcessPID | ||
| 118 | updatedAt := a.updatedAt | 134 | updatedAt := a.updatedAt | ||
| @@ -143,6 +159,8 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||||
| 143 | WorkspaceBase: a.WorkspaceBase, | 159 | WorkspaceBase: a.WorkspaceBase, | ||
| 144 | WorkspaceCapture: a.WorkspaceCapture, | 160 | WorkspaceCapture: a.WorkspaceCapture, | ||
| 145 | MaxTurns: a.MaxTurns, | 161 | MaxTurns: a.MaxTurns, | ||
| 162 | + | LifetimeTurns: lifetimeTurns, | |||
| 163 | + | CurrentRunTurns: currentRunTurns, | |||
| 146 | Timeout: a.Timeout, | 164 | Timeout: a.Timeout, | ||
| 147 | Tools: append([]string(nil), a.Tools...), | 165 | Tools: append([]string(nil), a.Tools...), | ||
| 148 | WebSearchPolicy: webSearchPolicy, | 166 | WebSearchPolicy: webSearchPolicy, | ||
| @@ -165,6 +183,8 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||||
| 165 | SessionPath: a.SessionPath, | 183 | SessionPath: a.SessionPath, | ||
| 166 | ResumePrompt: resumePrompt, | 184 | ResumePrompt: resumePrompt, | ||
| 167 | ResumePromptAt: resumePromptAt, | 185 | ResumePromptAt: resumePromptAt, | ||
| 186 | + | ResumePromptID: resumePromptID, | |||
| 187 | + | ResumeQueue: resumeQueue, | |||
| 168 | SessionID: a.SessionID, | 188 | SessionID: a.SessionID, | ||
| 169 | } | 189 | } | ||
| 170 | b, err := json.MarshalIndent(m, "", " ") | 190 | b, err := json.MarshalIndent(m, "", " ") | ||
| @@ -175,14 +195,50 @@ func writeAgentMeta(stateDir string, a *Agent) error { | |||||
| 175 | return fmt.Errorf("subagents meta dir: %w", err) | 195 | return fmt.Errorf("subagents meta dir: %w", err) | ||
| 176 | } | 196 | } | ||
| 177 | final := metaPath(stateDir) | 197 | final := metaPath(stateDir) | ||
| 178 | - | tmp := final + ".tmp" | 198 | + | // Use a unique temporary file while holding the per-agent persistence |
| 179 | - | if err := os.WriteFile(tmp, append(b, '\n'), 0o600); err != nil { | 199 | + | // lock. A fixed meta.json.tmp is vulnerable to a concurrent writer (or a |
| 200 | + | // stale file after a crash) replacing another writer's snapshot. | |||
| 201 | + | tmpFile, err := os.CreateTemp(stateDir, ".meta.json.tmp-*") | |||
| 202 | + | if err != nil { | |||
| 203 | + | return fmt.Errorf("subagents meta temp: %w", err) | |||
| 204 | + | } | |||
| 205 | + | tmp := tmpFile.Name() | |||
| 206 | + | removeTemp := true | |||
| 207 | + | defer func() { | |||
| 208 | + | _ = tmpFile.Close() | |||
| 209 | + | if removeTemp { | |||
| 210 | + | _ = os.Remove(tmp) | |||
| 211 | + | } | |||
| 212 | + | }() | |||
| 213 | + | if err := tmpFile.Chmod(0o600); err != nil { | |||
| 214 | + | return fmt.Errorf("subagents meta permissions: %w", err) | |||
| 215 | + | } | |||
| 216 | + | if _, err := tmpFile.Write(append(b, '\n')); err != nil { | |||
| 180 | return fmt.Errorf("subagents meta write: %w", err) | 217 | return fmt.Errorf("subagents meta write: %w", err) | ||
| 181 | } | 218 | } | ||
| 219 | + | if err := tmpFile.Sync(); err != nil { | |||
| 220 | + | return fmt.Errorf("subagents meta sync: %w", err) | |||
| 221 | + | } | |||
| 222 | + | if err := tmpFile.Close(); err != nil { | |||
| 223 | + | return fmt.Errorf("subagents meta close: %w", err) | |||
| 224 | + | } | |||
| 182 | if err := os.Rename(tmp, final); err != nil { | 225 | if err := os.Rename(tmp, final); err != nil { | ||
| 183 | - | _ = os.Remove(tmp) | |||
| 184 | return fmt.Errorf("subagent meta rename: %w", err) | 226 | return fmt.Errorf("subagent meta rename: %w", err) | ||
| 185 | } | 227 | } | ||
| 228 | + | removeTemp = false | |||
| 229 | + | // Sync the containing directory so the rename itself survives a host | |||
| 230 | + | // crash, not just the bytes in the temporary file. | |||
| 231 | + | dir, err := os.Open(stateDir) | |||
| 232 | + | if err != nil { | |||
| 233 | + | return fmt.Errorf("subagents meta directory open: %w", err) | |||
| 234 | + | } | |||
| 235 | + | if err := dir.Sync(); err != nil { | |||
| 236 | + | _ = dir.Close() | |||
| 237 | + | return fmt.Errorf("subagents meta directory sync: %w", err) | |||
| 238 | + | } | |||
| 239 | + | if err := dir.Close(); err != nil { | |||
| 240 | + | return fmt.Errorf("subagents meta directory close: %w", err) | |||
| 241 | + | } | |||
| 186 | return nil | 242 | return nil | ||
| 187 | } | 243 | } | ||
| 188 | 244 | ||||
| @@ -420,57 +476,61 @@ func (f *Supervisor) buildDetachedAgent(m agentMeta) *Agent { | |||||
| 420 | agentStateDir = filepath.Dir(m.EventLogPath) | 476 | agentStateDir = filepath.Dir(m.EventLogPath) | ||
| 421 | } | 477 | } | ||
| 422 | a := &Agent{ | 478 | a := &Agent{ | ||
| 423 | - | ID: m.ID, | 479 | + | ID: m.ID, |
| 424 | - | Task: m.Task, | 480 | + | Task: m.Task, |
| 425 | - | OriginalTask: firstNonEmpty(m.OriginalTask, m.Task), | 481 | + | OriginalTask: firstNonEmpty(m.OriginalTask, m.Task), |
| 426 | - | Dir: dir, | 482 | + | Dir: dir, |
| 427 | - | RepositoryRoot: firstNonEmpty(m.RepositoryRoot, f.cfg.RepoRoot), | 483 | + | RepositoryRoot: firstNonEmpty(m.RepositoryRoot, f.cfg.RepoRoot), |
| 428 | - | Started: m.Started, | 484 | + | Started: m.Started, |
| 429 | - | ParentID: m.ParentID, | 485 | + | ParentID: m.ParentID, |
| 430 | - | BatchID: m.BatchID, | 486 | + | BatchID: m.BatchID, |
| 431 | - | RootSessionID: m.RootSessionID, | 487 | + | RootSessionID: m.RootSessionID, |
| 432 | - | Model: m.Model, | 488 | + | Model: m.Model, |
| 433 | - | Provider: m.Provider, | 489 | + | Provider: m.Provider, |
| 434 | - | BaseURL: m.BaseURL, | 490 | + | BaseURL: m.BaseURL, |
| 435 | - | InsecureTLS: m.InsecureTLS, | 491 | + | InsecureTLS: m.InsecureTLS, |
| 436 | - | Reasoning: m.Reasoning, | 492 | + | Reasoning: m.Reasoning, |
| 437 | - | FastMode: fastMode, | 493 | + | FastMode: fastMode, |
| 438 | - | Subagent: m.Subagent, | 494 | + | Subagent: m.Subagent, |
| 439 | - | WorkspaceMode: m.WorkspaceMode, | 495 | + | WorkspaceMode: m.WorkspaceMode, |
| 440 | - | WorkspacePath: m.WorkspacePath, | 496 | + | WorkspacePath: m.WorkspacePath, |
| 441 | - | WorkspaceBase: m.WorkspaceBase, | 497 | + | WorkspaceBase: m.WorkspaceBase, |
| 442 | - | WorkspaceCapture: m.WorkspaceCapture, | 498 | + | WorkspaceCapture: m.WorkspaceCapture, |
| 443 | - | MaxTurns: m.MaxTurns, | 499 | + | MaxTurns: m.MaxTurns, |
| 444 | - | Timeout: m.Timeout, | 500 | + | LifetimeTurns: m.LifetimeTurns, |
| 445 | - | Tools: append([]string(nil), m.Tools...), | 501 | + | CurrentRunTurns: m.CurrentRunTurns, |
| 446 | - | WebSearchPolicy: childWebSearchPolicy(m.WebSearchPolicy, m.Subagent, m.Tools), | 502 | + | Timeout: m.Timeout, |
| 447 | - | Attempt: m.Attempt, | 503 | + | Tools: append([]string(nil), m.Tools...), |
| 448 | - | ProcessPID: m.ProcessPID, | 504 | + | WebSearchPolicy: childWebSearchPolicy(m.WebSearchPolicy, m.Subagent, m.Tools), |
| 449 | - | InboxPath: m.InboxPath, | 505 | + | Attempt: m.Attempt, |
| 450 | - | EventLogPath: m.EventLogPath, | 506 | + | ProcessPID: m.ProcessPID, |
| 451 | - | SessionPath: m.SessionPath, | 507 | + | InboxPath: m.InboxPath, |
| 452 | - | resumePromptText: m.ResumePrompt, | 508 | + | EventLogPath: m.EventLogPath, |
| 453 | - | resumePromptAt: m.ResumePromptAt, | 509 | + | SessionPath: m.SessionPath, |
| 454 | - | SessionID: m.SessionID, | 510 | + | resumePromptText: m.ResumePrompt, |
| 455 | - | inbox: NewInbox(m.InboxPath), | 511 | + | resumePromptAt: m.ResumePromptAt, |
| 456 | - | status: status, | 512 | + | resumePromptID: m.ResumePromptID, |
| 457 | - | activity: activity, | 513 | + | resumePromptQueue: append([]queuedResumePrompt(nil), m.ResumeQueue...), |
| 458 | - | finished: m.Finished, | 514 | + | SessionID: m.SessionID, |
| 459 | - | lastErr: lastErr, | 515 | + | inbox: NewInbox(m.InboxPath), |
| 460 | - | legacyEventState: legacyEventState, | 516 | + | status: status, |
| 461 | - | processState: processState, | 517 | + | activity: activity, |
| 462 | - | turnState: turnState, | 518 | + | finished: m.Finished, |
| 463 | - | currentTurnID: m.CurrentTurnID, | 519 | + | lastErr: lastErr, |
| 464 | - | updatedAt: m.UpdatedAt, | 520 | + | legacyEventState: legacyEventState, |
| 465 | - | lastActivity: lastActivity, | 521 | + | processState: processState, |
| 466 | - | stateDir: agentStateDir, | 522 | + | turnState: turnState, |
| 467 | - | resultRef: m.ResultRef, | 523 | + | currentTurnID: m.CurrentTurnID, |
| 468 | - | patchRef: m.PatchRef, | 524 | + | updatedAt: m.UpdatedAt, |
| 469 | - | changedFiles: append([]string(nil), m.ChangedFiles...), | 525 | + | lastActivity: lastActivity, |
| 470 | - | maxOutputBytes: f.cfg.Policy.MaxOutputBytes, | 526 | + | stateDir: agentStateDir, |
| 471 | - | maxOutputLines: f.cfg.Policy.MaxOutputLines, | 527 | + | resultRef: m.ResultRef, |
| 472 | - | done: make(chan struct{}), | 528 | + | patchRef: m.PatchRef, |
| 473 | - | turnResults: make(chan *TurnResult, 16), | 529 | + | changedFiles: append([]string(nil), m.ChangedFiles...), |
| 530 | + | maxOutputBytes: f.cfg.Policy.MaxOutputBytes, | |||
| 531 | + | maxOutputLines: f.cfg.Policy.MaxOutputLines, | |||
| 532 | + | done: make(chan struct{}), | |||
| 533 | + | turnResults: make(chan *TurnResult, 16), | |||
| 474 | } | 534 | } | ||
| 475 | if a.updatedAt.IsZero() { | 535 | if a.updatedAt.IsZero() { | ||
| 476 | a.updatedAt = lastActivity | 536 | a.updatedAt = lastActivity | ||
| @@ -542,7 +602,7 @@ func (a *Agent) loadTranscript() error { | |||||
| 542 | if err != nil { | 602 | if err != nil { | ||
| 543 | return fmt.Errorf("read subagent transcript: %w", err) | 603 | return fmt.Errorf("read subagent transcript: %w", err) | ||
| 544 | } | 604 | } | ||
| 545 | - | if prompt, acceptedAt := a.ResumePromptInfo(); prompt != "" && resumePromptAcknowledged(events, acceptedAt) { | 605 | + | if prompt, acceptedAt := a.ResumePromptInfo(); prompt != "" && resumePromptAcknowledged(events, prompt, acceptedAt) { |
| 546 | a.clearResumePrompt() | 606 | a.clearResumePrompt() | ||
| 547 | } | 607 | } | ||
| 548 | a.mu.Lock() | 608 | a.mu.Lock() | ||
| @@ -583,12 +643,23 @@ func (a *Agent) loadTranscript() error { | |||||
| 583 | // Status precedence: explicit lifecycle events (agent_stopped) win | 643 | // Status precedence: explicit lifecycle events (agent_stopped) win | ||
| 584 | // over inferred ones (assistant_message → idle). If the log contains | 644 | // over inferred ones (assistant_message → idle). If the log contains | ||
| 585 | // no terminator we keep status=StatusDetached so the user can resume. | 645 | // no terminator we keep status=StatusDetached so the user can resume. | ||
| 586 | - | func resumePromptAcknowledged(events []Event, acceptedAt time.Time) bool { | 646 | + | func resumePromptAcknowledged(events []Event, prompt string, acceptedAt time.Time) bool { |
| 587 | - | if acceptedAt.IsZero() { | 647 | + | if prompt == "" || acceptedAt.IsZero() { |
| 588 | return false | 648 | return false | ||
| 589 | } | 649 | } | ||
| 650 | + | userDurable := false | |||
| 590 | for _, ev := range events { | 651 | for _, ev := range events { | ||
| 591 | - | if (ev.Type == EventTurnStarted || ev.Type == "turn_start") && !ev.Time.Before(acceptedAt) { | 652 | + | if ev.Time.Before(acceptedAt) { |
| 653 | + | continue | |||
| 654 | + | } | |||
| 655 | + | if ev.Type == "user_message" && userMessageText(ev) == prompt { | |||
| 656 | + | // The worker emits this event only after the matching session | |||
| 657 | + | // message has been appended and flushed. A crash here leaves the | |||
| 658 | + | // prompt pending, which is intentionally safe to replay. | |||
| 659 | + | userDurable = true | |||
| 660 | + | continue | |||
| 661 | + | } | |||
| 662 | + | if userDurable && isDelegatedTurnStart(ev) { | |||
| 592 | return true | 663 | return true | ||
| 593 | } | 664 | } | ||
| 594 | } | 665 | } | ||
| @@ -600,14 +671,11 @@ func replayEventsIntoAgent(a *Agent, evs []Event) { | |||||
| 600 | for _, ev := range evs { | 671 | for _, ev := range evs { | ||
| 601 | replayEventTranscript(a, ev) | 672 | replayEventTranscript(a, ev) | ||
| 602 | switch ev.Type { | 673 | switch ev.Type { | ||
| 603 | - | case EventTurnStarted, "turn_start": | 674 | + | case EventTurnStarted: |
| 604 | - | turnID := ev.TurnID | 675 | + | if !isDelegatedTurnStart(ev) { |
| 605 | - | if turnID == "" { | 676 | + | continue |
| 606 | - | if step, ok := ev.Data["step"].(float64); ok { | |||
| 607 | - | turnID = fmt.Sprintf("turn-%d", int(step)) | |||
| 608 | - | } | |||
| 609 | } | 677 | } | ||
| 610 | - | a.setTurnState(TurnRunning, turnID) | 678 | + | a.setTurnState(TurnRunning, eventTurnID(ev)) |
| 611 | case EventTurnResult, "turn_result": | 679 | case EventTurnResult, "turn_result": | ||
| 612 | if result, err := decodeTurnResultEvent(ev, a.ID, a.maxOutputBytes, a.maxOutputLines); err == nil { | 680 | if result, err := decodeTurnResultEvent(ev, a.ID, a.maxOutputBytes, a.maxOutputLines); err == nil { | ||
| 613 | a.setResult(result) | 681 | a.setResult(result) | ||
| @@ -741,9 +809,16 @@ func (f *Supervisor) ResumeSession(ctx context.Context, id string) (*Agent, erro | |||||
| 741 | 809 | ||||
| 742 | // ResumeWithPrompt continues an agent with a manager follow-up while retaining | 810 | // ResumeWithPrompt continues an agent with a manager follow-up while retaining | ||
| 743 | // its existing session. An idle live worker receives the turn through its | 811 | // its existing session. An idle live worker receives the turn through its | ||
| 744 | - | // inbox; a terminal worker is restarted with the follow-up as its initial | 812 | + | // inbox; an active live worker queues the follow-up for the next idle turn; a |
| 745 | - | // prompt, avoiding a race with inbox listener startup. | 813 | + | // terminal worker is restarted with the follow-up as its initial prompt, |
| 814 | + | // avoiding a race with inbox listener startup. | |||
| 746 | func (f *Supervisor) ResumeWithPrompt(ctx context.Context, id, prompt string) (*Agent, error) { | 815 | func (f *Supervisor) ResumeWithPrompt(ctx context.Context, id, prompt string) (*Agent, error) { | ||
| 816 | + | if ctx == nil { | |||
| 817 | + | ctx = context.Background() | |||
| 818 | + | } | |||
| 819 | + | if err := ctx.Err(); err != nil { | |||
| 820 | + | return nil, err | |||
| 821 | + | } | |||
| 747 | if strings.TrimSpace(prompt) == "" { | 822 | if strings.TrimSpace(prompt) == "" { | ||
| 748 | return nil, fmt.Errorf("subagent: follow-up prompt is required") | 823 | return nil, fmt.Errorf("subagent: follow-up prompt is required") | ||
| 749 | } | 824 | } | ||
| @@ -756,34 +831,82 @@ func (f *Supervisor) ResumeWithPrompt(ctx context.Context, id, prompt string) (* | |||||
| 756 | } | 831 | } | ||
| 757 | status := existing.Status() | 832 | status := existing.Status() | ||
| 758 | if status == StatusRunning { | 833 | if status == StatusRunning { | ||
| 759 | - | if existing.ProcessState() != ProcessAlive || existing.TurnState() != TurnIdle { | 834 | + | if existing.ProcessState() != ProcessAlive { |
| 760 | f.operationMu.Unlock() | 835 | f.operationMu.Unlock() | ||
| 761 | - | return nil, fmt.Errorf("subagent: agent %s is still working; wait for it to become idle before sending a follow-up", existing.ID) | 836 | + | return nil, fmt.Errorf("subagent: agent %s is still working; wait for its worker to become available before sending a follow-up", existing.ID) |
| 762 | } | 837 | } | ||
| 763 | if existing.inbox == nil { | 838 | if existing.inbox == nil { | ||
| 764 | f.operationMu.Unlock() | 839 | f.operationMu.Unlock() | ||
| 765 | return nil, fmt.Errorf("subagent: agent %s has no inbox", existing.ID) | 840 | return nil, fmt.Errorf("subagent: agent %s has no inbox", existing.ID) | ||
| 766 | } | 841 | } | ||
| 842 | + | if err := ctx.Err(); err != nil { | |||
| 843 | + | f.operationMu.Unlock() | |||
| 844 | + | return nil, err | |||
| 845 | + | } | |||
| 846 | + | if existing.TurnState() != TurnRunning && existing.hasQueuedResumePrompt() { | |||
| 847 | + | f.operationMu.Unlock() | |||
| 848 | + | return nil, fmt.Errorf("subagent: agent %s already has a follow-up pending delivery", existing.ID) | |||
| 849 | + | } | |||
| 850 | + | if existing.TurnState() == TurnRunning { | |||
| 851 | + | // Active turns cannot accept a command through the worker inbox yet. | |||
| 852 | + | // Keep the prompt durable and let the idle transition deliver it in | |||
| 853 | + | // order after the current message turn finishes. | |||
| 854 | + | acceptedAt := f.cfg.Now() | |||
| 855 | + | commandID := existing.queueResumePrompt(prompt, acceptedAt) | |||
| 856 | + | if err := f.persistAgent(existing); err != nil { | |||
| 857 | + | existing.removeQueuedResumePrompt(commandID) | |||
| 858 | + | f.operationMu.Unlock() | |||
| 859 | + | return nil, fmt.Errorf("subagent: persist queued follow-up for %s: %w", existing.ID, err) | |||
| 860 | + | } | |||
| 861 | + | f.operationMu.Unlock() | |||
| 862 | + | return existing, nil | |||
| 863 | + | } | |||
| 864 | + | if existing.TurnState() != TurnIdle { | |||
| 865 | + | f.operationMu.Unlock() | |||
| 866 | + | return nil, fmt.Errorf("subagent: agent %s is still working; wait for it to become idle before sending a follow-up", existing.ID) | |||
| 867 | + | } | |||
| 767 | turnID := existing.CurrentTurnID() | 868 | turnID := existing.CurrentTurnID() | ||
| 869 | + | previousRunTurns := existing.resetCurrentRunTurns() | |||
| 768 | // Reserve the worker before writing to its inbox. A second manager call | 870 | // Reserve the worker before writing to its inbox. A second manager call | ||
| 769 | // must not enqueue another turn in the small interval before the worker | 871 | // must not enqueue another turn in the small interval before the worker | ||
| 770 | // reports its turn.start event. Persist the prompt first so a host exit | 872 | // reports its turn.start event. Persist the prompt first so a host exit | ||
| 771 | // before that event can resume and redeliver it. | 873 | // before that event can resume and redeliver it. | ||
| 772 | - | previousPrompt, previousPromptAt := existing.setResumePrompt(prompt, f.cfg.Now()) | 874 | + | previousPrompt := existing.setResumePrompt(prompt, f.cfg.Now()) |
| 773 | existing.setTurnState(TurnQueued, turnID) | 875 | existing.setTurnState(TurnQueued, turnID) | ||
| 774 | inbox := existing.inbox | 876 | inbox := existing.inbox | ||
| 775 | - | command := NewCommand(CommandTurnStart, existing.ID, turnID, TurnStartPayload{Prompt: prompt}) | 877 | + | command := NewCommand(CommandTurnStart, existing.ID, turnID, TurnStartPayload{Prompt: prompt, NewRun: true}) |
| 776 | - | f.persistAgent(existing) | 878 | + | command.MessageID = existing.resumePromptCommandID() |
| 879 | + | if err := f.persistAgent(existing); err != nil { | |||
| 880 | + | existing.setTurnState(TurnIdle, turnID) | |||
| 881 | + | existing.setCurrentRunTurns(previousRunTurns) | |||
| 882 | + | existing.restoreResumePrompt(previousPrompt) | |||
| 883 | + | f.operationMu.Unlock() | |||
| 884 | + | return nil, fmt.Errorf("subagent: persist follow-up for %s: %w", existing.ID, err) | |||
| 885 | + | } | |||
| 777 | f.operationMu.Unlock() | 886 | f.operationMu.Unlock() | ||
| 778 | 887 | ||||
| 779 | if err := inbox.SendCommandContext(ctx, command); err != nil { | 888 | if err := inbox.SendCommandContext(ctx, command); err != nil { | ||
| 889 | + | retryUnknown := false | |||
| 780 | f.operationMu.Lock() | 890 | f.operationMu.Lock() | ||
| 781 | if existing.Status() == StatusRunning && existing.TurnState() == TurnQueued && existing.CurrentTurnID() == turnID { | 891 | if existing.Status() == StatusRunning && existing.TurnState() == TurnQueued && existing.CurrentTurnID() == turnID { | ||
| 782 | existing.setTurnState(TurnIdle, turnID) | 892 | existing.setTurnState(TurnIdle, turnID) | ||
| 783 | - | existing.setResumePrompt(previousPrompt, previousPromptAt) | 893 | + | existing.setCurrentRunTurns(previousRunTurns) |
| 784 | - | f.persistAgent(existing) | 894 | + | if errors.Is(err, ErrDeliveryUnknown) { |
| 895 | + | existing.rejectActiveResumePrompt() | |||
| 896 | + | retryUnknown = true | |||
| 897 | + | } else { | |||
| 898 | + | existing.restoreResumePrompt(previousPrompt) | |||
| 899 | + | } | |||
| 900 | + | if persistErr := f.persistAgent(existing); persistErr != nil { | |||
| 901 | + | err = fmt.Errorf("%w; persist recovery: %v", err, persistErr) | |||
| 902 | + | } | |||
| 785 | } | 903 | } | ||
| 786 | f.operationMu.Unlock() | 904 | f.operationMu.Unlock() | ||
| 905 | + | if retryUnknown { | |||
| 906 | + | // The command keeps its durable identity. A retry is safe whether | |||
| 907 | + | // the worker parsed the first write or only received a prefix. | |||
| 908 | + | f.dispatchQueuedResume(ctx, existing) | |||
| 909 | + | } | |||
| 787 | return nil, fmt.Errorf("subagent: send follow-up to %s: %w", existing.ID, err) | 910 | return nil, fmt.Errorf("subagent: send follow-up to %s: %w", existing.ID, err) | ||
| 788 | } | 911 | } | ||
| 789 | return existing, nil | 912 | return existing, nil | ||
| @@ -793,6 +916,77 @@ func (f *Supervisor) ResumeWithPrompt(ctx context.Context, id, prompt string) (* | |||||
| 793 | return f.resume(ctx, id, true, prompt) | 916 | return f.resume(ctx, id, true, prompt) | ||
| 794 | } | 917 | } | ||
| 795 | 918 | ||||
| 919 | + | // dispatchQueuedResumeWithTimeout keeps the idle callback bounded. Unlike a | |||
| 920 | + | // manager request, an idle transition has no caller context to propagate. | |||
| 921 | + | func (f *Supervisor) dispatchQueuedResumeWithTimeout(a *Agent) { | |||
| 922 | + | timeout := f.cfg.Policy.ReconnectTimeout | |||
| 923 | + | if timeout <= 0 { | |||
| 924 | + | timeout = 5 * time.Second | |||
| 925 | + | } | |||
| 926 | + | ctx, cancel := context.WithTimeout(context.Background(), timeout) | |||
| 927 | + | defer cancel() | |||
| 928 | + | f.dispatchQueuedResume(ctx, a) | |||
| 929 | + | } | |||
| 930 | + | ||||
| 931 | + | // dispatchQueuedResume delivers the oldest follow-up accepted during an | |||
| 932 | + | // active turn after the worker reports that its inbox can accept a new turn. | |||
| 933 | + | // The operation lock serializes this transition with manager resume calls and | |||
| 934 | + | // the agent's durable prompt state prevents a host exit from losing it. | |||
| 935 | + | func (f *Supervisor) dispatchQueuedResume(ctx context.Context, a *Agent) { | |||
| 936 | + | if ctx == nil { | |||
| 937 | + | ctx = context.Background() | |||
| 938 | + | } | |||
| 939 | + | if a == nil { | |||
| 940 | + | return | |||
| 941 | + | } | |||
| 942 | + | f.operationMu.Lock() | |||
| 943 | + | if a.Status() != StatusRunning || a.ProcessState() != ProcessAlive || a.TurnState() != TurnIdle || a.inbox == nil { | |||
| 944 | + | f.operationMu.Unlock() | |||
| 945 | + | return | |||
| 946 | + | } | |||
| 947 | + | prompt, acceptedAt, commandID, ok := a.startQueuedResumePrompt() | |||
| 948 | + | if !ok { | |||
| 949 | + | f.operationMu.Unlock() | |||
| 950 | + | return | |||
| 951 | + | } | |||
| 952 | + | turnID := a.CurrentTurnID() | |||
| 953 | + | previousRunTurns := a.resetCurrentRunTurns() | |||
| 954 | + | a.setTurnState(TurnQueued, turnID) | |||
| 955 | + | inbox := a.inbox | |||
| 956 | + | command := NewCommand(CommandTurnStart, a.ID, turnID, TurnStartPayload{Prompt: prompt, NewRun: true}) | |||
| 957 | + | command.MessageID = commandID | |||
| 958 | + | if err := f.persistAgent(a); err != nil { | |||
| 959 | + | a.restoreResumePromptToQueue(prompt, acceptedAt) | |||
| 960 | + | a.setCurrentRunTurns(previousRunTurns) | |||
| 961 | + | a.setTurnState(TurnIdle, turnID) | |||
| 962 | + | f.operationMu.Unlock() | |||
| 963 | + | return | |||
| 964 | + | } | |||
| 965 | + | f.operationMu.Unlock() | |||
| 966 | + | ||||
| 967 | + | if err := inbox.SendCommandContext(ctx, command); err == nil { | |||
| 968 | + | return | |||
| 969 | + | } | |||
| 970 | + | ||||
| 971 | + | // A failed delivery must put the prompt back at the head of the queue. | |||
| 972 | + | // Keep the worker idle so a later retry, or an explicit resume, can send | |||
| 973 | + | // it without skipping any subsequent follow-ups. | |||
| 974 | + | f.operationMu.Lock() | |||
| 975 | + | if a.Status() == StatusRunning && a.ProcessState() == ProcessAlive && a.TurnState() == TurnQueued && a.CurrentTurnID() == turnID { | |||
| 976 | + | if a.restoreResumePromptToQueue(prompt, acceptedAt) { | |||
| 977 | + | a.setCurrentRunTurns(previousRunTurns) | |||
| 978 | + | a.setTurnState(TurnIdle, turnID) | |||
| 979 | + | if persistErr := f.persistAgent(a); persistErr != nil { | |||
| 980 | + | // persistAgent records the failure on the agent. Keep the explicit | |||
| 981 | + | // check here because this recovery write is the only durable copy | |||
| 982 | + | // of the restored queue head. | |||
| 983 | + | a.recordPersistenceError(persistErr) | |||
| 984 | + | } | |||
| 985 | + | } | |||
| 986 | + | } | |||
| 987 | + | f.operationMu.Unlock() | |||
| 988 | + | } | |||
| 989 | + | ||||
| 796 | // RestartTask intentionally runs the stored original task again in a fresh | 990 | // RestartTask intentionally runs the stored original task again in a fresh | ||
| 797 | // worker attempt. Callers should use ResumeSession for normal recovery. | 991 | // worker attempt. Callers should use ResumeSession for normal recovery. | ||
| 798 | func (f *Supervisor) RestartTask(ctx context.Context, id string) (*Agent, error) { | 992 | func (f *Supervisor) RestartTask(ctx context.Context, id string) (*Agent, error) { | ||
| @@ -858,10 +1052,45 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||||
| 858 | } | 1052 | } | ||
| 859 | now := f.cfg.Now() | 1053 | now := f.cfg.Now() | ||
| 860 | fastMode := existing.FastMode | 1054 | fastMode := existing.FastMode | ||
| 861 | - | pendingResumePrompt, pendingResumePromptAt := existing.ResumePromptInfo() | 1055 | + | lifetimeTurns := existing.LifetimeTurnsValue() |
| 1056 | + | currentRunTurns := existing.CurrentRunTurnsValue() | |||
| 1057 | + | pendingResume := existing.resumePromptState() | |||
| 1058 | + | pendingResumePrompt, pendingResumePromptAt, pendingResumePromptID := pendingResume.Prompt, pendingResume.AcceptedAt, pendingResume.CommandID | |||
| 1059 | + | pendingResumeQueue := existing.resumePromptQueueSnapshot() | |||
| 1060 | + | if pendingResumePrompt != "" && pendingResumePromptID == "" { | |||
| 1061 | + | pendingResumePromptID = uuid.NewString() | |||
| 1062 | + | } | |||
| 862 | if resumePrompt != "" { | 1063 | if resumePrompt != "" { | ||
| 863 | - | pendingResumePrompt = resumePrompt | 1064 | + | currentRunTurns = 0 |
| 864 | - | pendingResumePromptAt = now | 1065 | + | if pendingResumePrompt == "" { |
| 1066 | + | pendingResumePrompt = resumePrompt | |||
| 1067 | + | pendingResumePromptAt = now | |||
| 1068 | + | pendingResumePromptID = uuid.NewString() | |||
| 1069 | + | } else { | |||
| 1070 | + | pendingResumeQueue = append(pendingResumeQueue, queuedResumePrompt{Prompt: resumePrompt, AcceptedAt: now, CommandID: uuid.NewString()}) | |||
| 1071 | + | } | |||
| 1072 | + | } | |||
| 1073 | + | if resuming && resumePrompt == "" && pendingResumePrompt == "" && len(pendingResumeQueue) > 0 { | |||
| 1074 | + | // A host can exit after accepting an active-worker follow-up but before | |||
| 1075 | + | // dispatching it. Make the first durable queued prompt the resumed | |||
| 1076 | + | // worker's initial turn rather than leaving the queue stranded. | |||
| 1077 | + | queued := pendingResumeQueue[0] | |||
| 1078 | + | pendingResumePrompt = queued.Prompt | |||
| 1079 | + | pendingResumePromptAt = queued.AcceptedAt | |||
| 1080 | + | pendingResumePromptID = queued.CommandID | |||
| 1081 | + | if pendingResumePromptID == "" { | |||
| 1082 | + | pendingResumePromptID = uuid.NewString() | |||
| 1083 | + | } | |||
| 1084 | + | pendingResumeQueue = pendingResumeQueue[1:] | |||
| 1085 | + | } | |||
| 1086 | + | if resuming && pendingResumePrompt != "" { | |||
| 1087 | + | // A retained follow-up is an explicit new run, including a queued | |||
| 1088 | + | // follow-up recovered after a host restart. Preserve lifetime turns but | |||
| 1089 | + | // never carry the previous run's allowance into this one. | |||
| 1090 | + | currentRunTurns = 0 | |||
| 1091 | + | } | |||
| 1092 | + | if !resuming { | |||
| 1093 | + | currentRunTurns = 0 | |||
| 865 | } | 1094 | } | ||
| 866 | m := agentMeta{ | 1095 | m := agentMeta{ | ||
| 867 | ID: existing.ID, Task: existing.Task, | 1096 | ID: existing.ID, Task: existing.Task, | ||
| @@ -874,7 +1103,7 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||||
| 874 | Subagent: existing.Subagent, | 1103 | Subagent: existing.Subagent, | ||
| 875 | WorkspaceMode: existing.WorkspaceMode, WorkspacePath: existing.WorkspacePath, | 1104 | WorkspaceMode: existing.WorkspaceMode, WorkspacePath: existing.WorkspacePath, | ||
| 876 | WorkspaceBase: existing.WorkspaceBase, WorkspaceCapture: existing.WorkspaceCapture, | 1105 | WorkspaceBase: existing.WorkspaceBase, WorkspaceCapture: existing.WorkspaceCapture, | ||
| 877 | - | MaxTurns: existing.MaxTurns, Timeout: existing.Timeout, Tools: append([]string(nil), existing.Tools...), | 1106 | + | MaxTurns: existing.MaxTurns, LifetimeTurns: lifetimeTurns, CurrentRunTurns: currentRunTurns, Timeout: existing.Timeout, Tools: append([]string(nil), existing.Tools...), |
| 878 | WebSearchPolicy: childWebSearchPolicy(existing.WebSearchPolicy, existing.Subagent, existing.Tools), | 1107 | WebSearchPolicy: childWebSearchPolicy(existing.WebSearchPolicy, existing.Subagent, existing.Tools), | ||
| 879 | CurrentTurnID: existingSnapshot.CurrentTurnID, Attempt: existingSnapshot.Attempt, | 1108 | CurrentTurnID: existingSnapshot.CurrentTurnID, Attempt: existingSnapshot.Attempt, | ||
| 880 | SessionID: existing.SessionID, | 1109 | SessionID: existing.SessionID, | ||
| @@ -882,6 +1111,8 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||||
| 882 | SessionPath: existing.SessionPath, | 1111 | SessionPath: existing.SessionPath, | ||
| 883 | ResumePrompt: pendingResumePrompt, | 1112 | ResumePrompt: pendingResumePrompt, | ||
| 884 | ResumePromptAt: pendingResumePromptAt, | 1113 | ResumePromptAt: pendingResumePromptAt, | ||
| 1114 | + | ResumePromptID: pendingResumePromptID, | |||
| 1115 | + | ResumeQueue: pendingResumeQueue, | |||
| 885 | } | 1116 | } | ||
| 886 | 1117 | ||||
| 887 | if len(m.Tools) == 0 && len(f.cfg.Policy.AllowedTools) > 0 { | 1118 | if len(m.Tools) == 0 && len(f.cfg.Policy.AllowedTools) > 0 { | ||
| @@ -960,6 +1191,8 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||||
| 960 | WorkspaceBase: m.WorkspaceBase, | 1191 | WorkspaceBase: m.WorkspaceBase, | ||
| 961 | WorkspaceCapture: m.WorkspaceCapture, | 1192 | WorkspaceCapture: m.WorkspaceCapture, | ||
| 962 | MaxTurns: maxTurns, | 1193 | MaxTurns: maxTurns, | ||
| 1194 | + | LifetimeTurns: m.LifetimeTurns, | |||
| 1195 | + | CurrentRunTurns: m.CurrentRunTurns, | |||
| 963 | Timeout: timeout, | 1196 | Timeout: timeout, | ||
| 964 | Attempt: m.Attempt, | 1197 | Attempt: m.Attempt, | ||
| 965 | HeartbeatInterval: f.cfg.Policy.HeartbeatInterval, | 1198 | HeartbeatInterval: f.cfg.Policy.HeartbeatInterval, | ||
| @@ -971,6 +1204,8 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||||
| 971 | Resuming: resuming, | 1204 | Resuming: resuming, | ||
| 972 | resumePromptText: pendingResumePrompt, | 1205 | resumePromptText: pendingResumePrompt, | ||
| 973 | resumePromptAt: pendingResumePromptAt, | 1206 | resumePromptAt: pendingResumePromptAt, | ||
| 1207 | + | resumePromptID: m.ResumePromptID, | |||
| 1208 | + | resumePromptQueue: append([]queuedResumePrompt(nil), pendingResumeQueue...), | |||
| 974 | inbox: NewInbox(m.InboxPath), | 1209 | inbox: NewInbox(m.InboxPath), | ||
| 975 | status: StatusPending, | 1210 | status: StatusPending, | ||
| 976 | activity: "resuming", | 1211 | activity: "resuming", | ||
| @@ -995,6 +1230,7 @@ func (f *Supervisor) resume(ctx context.Context, id string, resuming bool, resum | |||||
| 995 | } | 1230 | } | ||
| 996 | a.ctx, a.cancel = runCtx, cancel | 1231 | a.ctx, a.cancel = runCtx, cancel | ||
| 997 | a.persistFn = f.persistAgent | 1232 | a.persistFn = f.persistAgent | ||
| 1233 | + | a.setOnTurnIdle(func() { f.dispatchQueuedResumeWithTimeout(a) }) | |||
| 998 | a.workspaceCleanup = func() error { return workspace.Cleanup(context.Background()) } | 1234 | a.workspaceCleanup = func() error { return workspace.Cleanup(context.Background()) } | ||
| 999 | a.workspaceCapture = func() (WorkspaceCapture, error) { return workspace.Capture(context.Background()) } | 1235 | a.workspaceCapture = func() (WorkspaceCapture, error) { return workspace.Capture(context.Background()) } | ||
| 1000 | a.runner = f.cfg.NewRunner(a) | 1236 | a.runner = f.cfg.NewRunner(a) | ||
M packages/agent/subagents/persist_test.go +32 -1
| @@ -14,6 +14,34 @@ import ( | |||
| 14 | 14 | "time" | |
| 15 | 15 | ) | |
| 16 | 16 | ||
| 17 | + | func TestWriteAgentMetaSerializesConcurrentSnapshots(t *testing.T) { | |
| 18 | + | stateDir := t.TempDir() | |
| 19 | + | a := &Agent{ID: "concurrent-meta", Task: "retain queued prompts", stateDir: stateDir} | |
| 20 | + | var wg sync.WaitGroup | |
| 21 | + | for i := 0; i < 32; i++ { | |
| 22 | + | wg.Add(1) | |
| 23 | + | go func(i int) { | |
| 24 | + | defer wg.Done() | |
| 25 | + | a.setTurnCounts(i, i/2) | |
| 26 | + | a.queueResumePrompt("prompt", time.Now()) | |
| 27 | + | if err := writeAgentMeta(stateDir, a); err != nil { | |
| 28 | + | t.Errorf("write metadata: %v", err) | |
| 29 | + | } | |
| 30 | + | }(i) | |
| 31 | + | } | |
| 32 | + | wg.Wait() | |
| 33 | + | if _, err := readAgentMeta(stateDir); err != nil { | |
| 34 | + | t.Fatalf("final metadata is not a complete JSON snapshot: %v", err) | |
| 35 | + | } | |
| 36 | + | matches, err := filepath.Glob(filepath.Join(stateDir, ".meta.json.tmp-*")) | |
| 37 | + | if err != nil { | |
| 38 | + | t.Fatal(err) | |
| 39 | + | } | |
| 40 | + | if len(matches) != 0 { | |
| 41 | + | t.Fatalf("temporary metadata files remain: %v", matches) | |
| 42 | + | } | |
| 43 | + | } | |
| 44 | + | ||
| 17 | 45 | // TestSpawnWritesMetaJSON asserts the durability contract: every | |
| 18 | 46 | // successful Spawn leaves a meta.json on disk with the agent's | |
| 19 | 47 | // identity bits. Without this, Reload on the next zut launch can't | |
| @@ -662,7 +690,7 @@ func TestReloadClearsResumePromptAcknowledgedInEventLog(t *testing.T) { | |||
| 662 | 690 | EventLogPath: filepath.Join(stateDir, "events.jsonl"), | |
| 663 | 691 | SessionPath: filepath.Join(stateDir, "session.json"), | |
| 664 | 692 | ResumePrompt: "I applied your review. What do you think now?", | |
| 665 | - | ResumePromptAt: time.Now(), | |
| 693 | + | ResumePromptAt: time.Now().Add(-time.Second), | |
| 666 | 694 | } | |
| 667 | 695 | data, err := json.Marshal(m) | |
| 668 | 696 | if err != nil { | |
| @@ -675,6 +703,9 @@ func TestReloadClearsResumePromptAcknowledgedInEventLog(t *testing.T) { | |||
| 675 | 703 | if err != nil { | |
| 676 | 704 | t.Fatal(err) | |
| 677 | 705 | } | |
| 706 | + | if err := log.Append(NewEvent("user_message", map[string]any{"content": []any{map[string]any{"type": "text", "text": m.ResumePrompt}}})); err != nil { | |
| 707 | + | t.Fatal(err) | |
| 708 | + | } | |
| 678 | 709 | if err := log.Append(NewEvent(EventTurnStarted, map[string]any{"step": float64(1)})); err != nil { | |
| 679 | 710 | t.Fatal(err) | |
| 680 | 711 | } | |
| @@ -14,6 +14,34 @@ import ( | |||||
| 14 | "time" | 14 | "time" | ||
| 15 | ) | 15 | ) | ||
| 16 | 16 | ||||
| 17 | + | func TestWriteAgentMetaSerializesConcurrentSnapshots(t *testing.T) { | |||
| 18 | + | stateDir := t.TempDir() | |||
| 19 | + | a := &Agent{ID: "concurrent-meta", Task: "retain queued prompts", stateDir: stateDir} | |||
| 20 | + | var wg sync.WaitGroup | |||
| 21 | + | for i := 0; i < 32; i++ { | |||
| 22 | + | wg.Add(1) | |||
| 23 | + | go func(i int) { | |||
| 24 | + | defer wg.Done() | |||
| 25 | + | a.setTurnCounts(i, i/2) | |||
| 26 | + | a.queueResumePrompt("prompt", time.Now()) | |||
| 27 | + | if err := writeAgentMeta(stateDir, a); err != nil { | |||
| 28 | + | t.Errorf("write metadata: %v", err) | |||
| 29 | + | } | |||
| 30 | + | }(i) | |||
| 31 | + | } | |||
| 32 | + | wg.Wait() | |||
| 33 | + | if _, err := readAgentMeta(stateDir); err != nil { | |||
| 34 | + | t.Fatalf("final metadata is not a complete JSON snapshot: %v", err) | |||
| 35 | + | } | |||
| 36 | + | matches, err := filepath.Glob(filepath.Join(stateDir, ".meta.json.tmp-*")) | |||
| 37 | + | if err != nil { | |||
| 38 | + | t.Fatal(err) | |||
| 39 | + | } | |||
| 40 | + | if len(matches) != 0 { | |||
| 41 | + | t.Fatalf("temporary metadata files remain: %v", matches) | |||
| 42 | + | } | |||
| 43 | + | } | |||
| 44 | + | ||||
| 17 | // TestSpawnWritesMetaJSON asserts the durability contract: every | 45 | // TestSpawnWritesMetaJSON asserts the durability contract: every | ||
| 18 | // successful Spawn leaves a meta.json on disk with the agent's | 46 | // successful Spawn leaves a meta.json on disk with the agent's | ||
| 19 | // identity bits. Without this, Reload on the next zut launch can't | 47 | // identity bits. Without this, Reload on the next zut launch can't | ||
| @@ -662,7 +690,7 @@ func TestReloadClearsResumePromptAcknowledgedInEventLog(t *testing.T) { | |||||
| 662 | EventLogPath: filepath.Join(stateDir, "events.jsonl"), | 690 | EventLogPath: filepath.Join(stateDir, "events.jsonl"), | ||
| 663 | SessionPath: filepath.Join(stateDir, "session.json"), | 691 | SessionPath: filepath.Join(stateDir, "session.json"), | ||
| 664 | ResumePrompt: "I applied your review. What do you think now?", | 692 | ResumePrompt: "I applied your review. What do you think now?", | ||
| 665 | - | ResumePromptAt: time.Now(), | 693 | + | ResumePromptAt: time.Now().Add(-time.Second), |
| 666 | } | 694 | } | ||
| 667 | data, err := json.Marshal(m) | 695 | data, err := json.Marshal(m) | ||
| 668 | if err != nil { | 696 | if err != nil { | ||
| @@ -675,6 +703,9 @@ func TestReloadClearsResumePromptAcknowledgedInEventLog(t *testing.T) { | |||||
| 675 | if err != nil { | 703 | if err != nil { | ||
| 676 | t.Fatal(err) | 704 | t.Fatal(err) | ||
| 677 | } | 705 | } | ||
| 706 | + | if err := log.Append(NewEvent("user_message", map[string]any{"content": []any{map[string]any{"type": "text", "text": m.ResumePrompt}}})); err != nil { | |||
| 707 | + | t.Fatal(err) | |||
| 708 | + | } | |||
| 678 | if err := log.Append(NewEvent(EventTurnStarted, map[string]any{"step": float64(1)})); err != nil { | 709 | if err := log.Append(NewEvent(EventTurnStarted, map[string]any{"step": float64(1)})); err != nil { | ||
| 679 | t.Fatal(err) | 710 | t.Fatal(err) | ||
| 680 | } | 711 | } | ||
M packages/agent/subagents/protocol.go +4 -0
| @@ -579,6 +579,10 @@ type AgentHeartbeatPayload struct { | |||
| 579 | 579 | } | |
| 580 | 580 | type TurnStartPayload struct { | |
| 581 | 581 | Prompt string `json:"prompt"` | |
| 582 | + | // NewRun starts a fresh max_turns budget while retaining the worker's | |
| 583 | + | // existing session and transcript. It is set only for explicit | |
| 584 | + | // subagent_resume follow-ups. | |
| 585 | + | NewRun bool `json:"new_run,omitempty"` | |
| 582 | 586 | } | |
| 583 | 587 | type TurnCancelPayload struct { | |
| 584 | 588 | Reason string `json:"reason,omitempty"` | |
| @@ -579,6 +579,10 @@ type AgentHeartbeatPayload struct { | |||||
| 579 | } | 579 | } | ||
| 580 | type TurnStartPayload struct { | 580 | type TurnStartPayload struct { | ||
| 581 | Prompt string `json:"prompt"` | 581 | Prompt string `json:"prompt"` | ||
| 582 | + | // NewRun starts a fresh max_turns budget while retaining the worker's | |||
| 583 | + | // existing session and transcript. It is set only for explicit | |||
| 584 | + | // subagent_resume follow-ups. | |||
| 585 | + | NewRun bool `json:"new_run,omitempty"` | |||
| 582 | } | 586 | } | ||
| 583 | type TurnCancelPayload struct { | 587 | type TurnCancelPayload struct { | ||
| 584 | Reason string `json:"reason,omitempty"` | 588 | Reason string `json:"reason,omitempty"` | ||
M packages/agent/subagents/resume_prompt_test.go +149 -0
| @@ -2,12 +2,151 @@ package subagents | |||
| 2 | 2 | ||
| 3 | 3 | import ( | |
| 4 | 4 | "context" | |
| 5 | + | "errors" | |
| 5 | 6 | "path/filepath" | |
| 6 | 7 | "runtime" | |
| 7 | 8 | "testing" | |
| 8 | 9 | "time" | |
| 9 | 10 | ) | |
| 10 | 11 | ||
| 12 | + | func TestResumeWithPromptQueuesFollowUpsForActiveWorker(t *testing.T) { | |
| 13 | + | if runtime.GOOS == "windows" { | |
| 14 | + | t.Skip("subagent inbox transport uses Unix-domain sockets") | |
| 15 | + | } | |
| 16 | + | ||
| 17 | + | root := shortSocketDir(t) | |
| 18 | + | ready := make(chan struct{}, 1) | |
| 19 | + | commands := make(chan Envelope, 2) | |
| 20 | + | f := New(Config{ | |
| 21 | + | Root: root, RepoRoot: root, | |
| 22 | + | NewRunner: func(a *Agent) Runner { | |
| 23 | + | return RunnerFunc(func(ctx context.Context, _ Sink) error { | |
| 24 | + | listener, err := Listen(a.InboxPath) | |
| 25 | + | if err != nil { | |
| 26 | + | return err | |
| 27 | + | } | |
| 28 | + | defer listener.Close() | |
| 29 | + | a.setProcessState(ProcessAlive) | |
| 30 | + | a.setTurnState(TurnRunning, "turn-1") | |
| 31 | + | ready <- struct{}{} | |
| 32 | + | for { | |
| 33 | + | select { | |
| 34 | + | case <-ctx.Done(): | |
| 35 | + | return ctx.Err() | |
| 36 | + | case line, ok := <-listener.Lines(): | |
| 37 | + | if !ok { | |
| 38 | + | return nil | |
| 39 | + | } | |
| 40 | + | command, err := ParseCommand(line) | |
| 41 | + | if err != nil { | |
| 42 | + | return err | |
| 43 | + | } | |
| 44 | + | if command.Type == CommandAgentShutdown { | |
| 45 | + | return nil | |
| 46 | + | } | |
| 47 | + | commands <- command | |
| 48 | + | } | |
| 49 | + | } | |
| 50 | + | }) | |
| 51 | + | }, | |
| 52 | + | }) | |
| 53 | + | defer f.StopAll() | |
| 54 | + | ||
| 55 | + | a, err := f.Spawn(context.Background(), "review the implementation") | |
| 56 | + | if err != nil { | |
| 57 | + | t.Fatal(err) | |
| 58 | + | } | |
| 59 | + | select { | |
| 60 | + | case <-ready: | |
| 61 | + | case <-time.After(time.Second): | |
| 62 | + | t.Fatal("active worker did not open its inbox") | |
| 63 | + | } | |
| 64 | + | canceled, cancel := context.WithCancel(context.Background()) | |
| 65 | + | cancel() | |
| 66 | + | if _, err := f.ResumeWithPrompt(canceled, a.ID, "must not be accepted"); !errors.Is(err, context.Canceled) { | |
| 67 | + | t.Fatalf("canceled active follow-up error = %v, want context canceled", err) | |
| 68 | + | } | |
| 69 | + | ||
| 70 | + | const ( | |
| 71 | + | firstFollowUp = "Check the parser behavior." | |
| 72 | + | secondFollowUp = "Then check the error handling." | |
| 73 | + | ) | |
| 74 | + | if _, err := f.ResumeWithPrompt(context.Background(), a.ID, firstFollowUp); err != nil { | |
| 75 | + | t.Fatalf("queue first active follow-up: %v", err) | |
| 76 | + | } | |
| 77 | + | if _, err := f.ResumeWithPrompt(context.Background(), a.ID, secondFollowUp); err != nil { | |
| 78 | + | t.Fatalf("queue second active follow-up: %v", err) | |
| 79 | + | } | |
| 80 | + | select { | |
| 81 | + | case command := <-commands: | |
| 82 | + | t.Fatalf("active follow-up was delivered before idle: %v", command.Type) | |
| 83 | + | default: | |
| 84 | + | } | |
| 85 | + | ||
| 86 | + | persisted, err := readAgentMeta(filepath.Join(root, "agents", a.ID)) | |
| 87 | + | if err != nil { | |
| 88 | + | t.Fatal(err) | |
| 89 | + | } | |
| 90 | + | if persisted.ResumePrompt != "" || len(persisted.ResumeQueue) != 2 { | |
| 91 | + | t.Fatalf("active follow-ups persisted as prompt %q queue %d, want empty prompt and queue 2", persisted.ResumePrompt, len(persisted.ResumeQueue)) | |
| 92 | + | } | |
| 93 | + | if firstID, secondID := persisted.ResumeQueue[0].CommandID, persisted.ResumeQueue[1].CommandID; firstID == "" || secondID == "" || firstID == secondID { | |
| 94 | + | t.Fatalf("queued follow-up command IDs = %q, %q; want distinct non-empty IDs", firstID, secondID) | |
| 95 | + | } | |
| 96 | + | ||
| 97 | + | if err := updateAgentFromEvent(a, NewEvent(EventAgentIdle, map[string]any{"turn_id": "turn-1"})); err != nil { | |
| 98 | + | t.Fatalf("apply first idle event: %v", err) | |
| 99 | + | } | |
| 100 | + | select { | |
| 101 | + | case command := <-commands: | |
| 102 | + | var payload TurnStartPayload | |
| 103 | + | if command.Type != CommandTurnStart || command.DecodePayload(&payload) != nil || payload.Prompt != firstFollowUp { | |
| 104 | + | t.Fatalf("first idle delivery = type %q prompt %q, want turn start %q", command.Type, payload.Prompt, firstFollowUp) | |
| 105 | + | } | |
| 106 | + | if !payload.NewRun { | |
| 107 | + | t.Fatal("first queued follow-up did not request a fresh run budget") | |
| 108 | + | } | |
| 109 | + | case <-time.After(time.Second): | |
| 110 | + | t.Fatal("first active follow-up was not delivered at idle") | |
| 111 | + | } | |
| 112 | + | if got := a.TurnState(); got != TurnQueued { | |
| 113 | + | t.Fatalf("turn state after first queued delivery = %s, want %s", got, TurnQueued) | |
| 114 | + | } | |
| 115 | + | ||
| 116 | + | if err := updateAgentFromEvent(a, NewEvent(EventTurnStarted, map[string]any{ | |
| 117 | + | "turn_id": "turn-2", "step": float64(2), "lifetime_turns": 2, "current_run_turns": 1, | |
| 118 | + | })); err != nil { | |
| 119 | + | t.Fatalf("apply second turn started event: %v", err) | |
| 120 | + | } | |
| 121 | + | if err := updateAgentFromEvent(a, NewEvent(EventAgentIdle, map[string]any{"turn_id": "turn-2"})); err != nil { | |
| 122 | + | t.Fatalf("apply second idle event: %v", err) | |
| 123 | + | } | |
| 124 | + | select { | |
| 125 | + | case command := <-commands: | |
| 126 | + | var payload TurnStartPayload | |
| 127 | + | if command.Type != CommandTurnStart || command.DecodePayload(&payload) != nil || payload.Prompt != secondFollowUp { | |
| 128 | + | t.Fatalf("second idle delivery = type %q prompt %q, want turn start %q", command.Type, payload.Prompt, secondFollowUp) | |
| 129 | + | } | |
| 130 | + | case <-time.After(time.Second): | |
| 131 | + | t.Fatal("second active follow-up was not delivered at the next idle turn") | |
| 132 | + | } | |
| 133 | + | ||
| 134 | + | persisted, err = readAgentMeta(filepath.Join(root, "agents", a.ID)) | |
| 135 | + | if err != nil { | |
| 136 | + | t.Fatal(err) | |
| 137 | + | } | |
| 138 | + | if persisted.ResumePrompt != secondFollowUp || len(persisted.ResumeQueue) != 0 { | |
| 139 | + | t.Fatalf("second follow-up persistence = prompt %q queue %d, want prompt %q and empty queue", persisted.ResumePrompt, len(persisted.ResumeQueue), secondFollowUp) | |
| 140 | + | } | |
| 141 | + | if persisted.ResumePromptID == "" { | |
| 142 | + | t.Fatal("promoted follow-up lost its command ID") | |
| 143 | + | } | |
| 144 | + | if err := f.Stop(a.ID); err != nil { | |
| 145 | + | t.Fatal(err) | |
| 146 | + | } | |
| 147 | + | a.Wait() | |
| 148 | + | } | |
| 149 | + | ||
| 11 | 150 | func TestResumeWithPromptDeliversFollowUpToIdleWorker(t *testing.T) { | |
| 12 | 151 | if runtime.GOOS == "windows" { | |
| 13 | 152 | t.Skip("subagent inbox transport uses Unix-domain sockets") | |
| @@ -60,6 +199,7 @@ func TestResumeWithPromptDeliversFollowUpToIdleWorker(t *testing.T) { | |||
| 60 | 199 | case <-time.After(time.Second): | |
| 61 | 200 | t.Fatal("idle worker did not open its inbox") | |
| 62 | 201 | } | |
| 202 | + | a.setTurnCounts(4, 2) | |
| 63 | 203 | ||
| 64 | 204 | const followUp = "I applied your review. What do you think now?" | |
| 65 | 205 | continued, err := f.ResumeWithPrompt(context.Background(), a.ID, followUp) | |
| @@ -72,6 +212,12 @@ func TestResumeWithPromptDeliversFollowUpToIdleWorker(t *testing.T) { | |||
| 72 | 212 | if got := a.TurnState(); got != TurnQueued { | |
| 73 | 213 | t.Fatalf("turn state after sending follow-up = %s, want %s", got, TurnQueued) | |
| 74 | 214 | } | |
| 215 | + | if got := a.CurrentRunTurnsValue(); got != 0 { | |
| 216 | + | t.Fatalf("current-run turns after explicit resume = %d, want 0 before turn starts", got) | |
| 217 | + | } | |
| 218 | + | if got := a.LifetimeTurnsValue(); got != 4 { | |
| 219 | + | t.Fatalf("lifetime turns after explicit resume = %d, want 4", got) | |
| 220 | + | } | |
| 75 | 221 | if _, err := f.ResumeWithPrompt(context.Background(), a.ID, "duplicate follow-up"); err == nil { | |
| 76 | 222 | t.Fatal("second follow-up succeeded while the first was queued") | |
| 77 | 223 | } | |
| @@ -87,6 +233,9 @@ func TestResumeWithPromptDeliversFollowUpToIdleWorker(t *testing.T) { | |||
| 87 | 233 | if payload.Prompt != followUp { | |
| 88 | 234 | t.Fatalf("follow-up prompt = %q, want %q", payload.Prompt, followUp) | |
| 89 | 235 | } | |
| 236 | + | if !payload.NewRun { | |
| 237 | + | t.Fatal("idle follow-up did not request a fresh run budget") | |
| 238 | + | } | |
| 90 | 239 | case <-time.After(time.Second): | |
| 91 | 240 | t.Fatal("idle worker did not receive the follow-up") | |
| 92 | 241 | } | |
| @@ -2,12 +2,151 @@ package subagents | |||||
| 2 | 2 | ||||
| 3 | import ( | 3 | import ( | ||
| 4 | "context" | 4 | "context" | ||
| 5 | + | "errors" | |||
| 5 | "path/filepath" | 6 | "path/filepath" | ||
| 6 | "runtime" | 7 | "runtime" | ||
| 7 | "testing" | 8 | "testing" | ||
| 8 | "time" | 9 | "time" | ||
| 9 | ) | 10 | ) | ||
| 10 | 11 | ||||
| 12 | + | func TestResumeWithPromptQueuesFollowUpsForActiveWorker(t *testing.T) { | |||
| 13 | + | if runtime.GOOS == "windows" { | |||
| 14 | + | t.Skip("subagent inbox transport uses Unix-domain sockets") | |||
| 15 | + | } | |||
| 16 | + | ||||
| 17 | + | root := shortSocketDir(t) | |||
| 18 | + | ready := make(chan struct{}, 1) | |||
| 19 | + | commands := make(chan Envelope, 2) | |||
| 20 | + | f := New(Config{ | |||
| 21 | + | Root: root, RepoRoot: root, | |||
| 22 | + | NewRunner: func(a *Agent) Runner { | |||
| 23 | + | return RunnerFunc(func(ctx context.Context, _ Sink) error { | |||
| 24 | + | listener, err := Listen(a.InboxPath) | |||
| 25 | + | if err != nil { | |||
| 26 | + | return err | |||
| 27 | + | } | |||
| 28 | + | defer listener.Close() | |||
| 29 | + | a.setProcessState(ProcessAlive) | |||
| 30 | + | a.setTurnState(TurnRunning, "turn-1") | |||
| 31 | + | ready <- struct{}{} | |||
| 32 | + | for { | |||
| 33 | + | select { | |||
| 34 | + | case <-ctx.Done(): | |||
| 35 | + | return ctx.Err() | |||
| 36 | + | case line, ok := <-listener.Lines(): | |||
| 37 | + | if !ok { | |||
| 38 | + | return nil | |||
| 39 | + | } | |||
| 40 | + | command, err := ParseCommand(line) | |||
| 41 | + | if err != nil { | |||
| 42 | + | return err | |||
| 43 | + | } | |||
| 44 | + | if command.Type == CommandAgentShutdown { | |||
| 45 | + | return nil | |||
| 46 | + | } | |||
| 47 | + | commands <- command | |||
| 48 | + | } | |||
| 49 | + | } | |||
| 50 | + | }) | |||
| 51 | + | }, | |||
| 52 | + | }) | |||
| 53 | + | defer f.StopAll() | |||
| 54 | + | ||||
| 55 | + | a, err := f.Spawn(context.Background(), "review the implementation") | |||
| 56 | + | if err != nil { | |||
| 57 | + | t.Fatal(err) | |||
| 58 | + | } | |||
| 59 | + | select { | |||
| 60 | + | case <-ready: | |||
| 61 | + | case <-time.After(time.Second): | |||
| 62 | + | t.Fatal("active worker did not open its inbox") | |||
| 63 | + | } | |||
| 64 | + | canceled, cancel := context.WithCancel(context.Background()) | |||
| 65 | + | cancel() | |||
| 66 | + | if _, err := f.ResumeWithPrompt(canceled, a.ID, "must not be accepted"); !errors.Is(err, context.Canceled) { | |||
| 67 | + | t.Fatalf("canceled active follow-up error = %v, want context canceled", err) | |||
| 68 | + | } | |||
| 69 | + | ||||
| 70 | + | const ( | |||
| 71 | + | firstFollowUp = "Check the parser behavior." | |||
| 72 | + | secondFollowUp = "Then check the error handling." | |||
| 73 | + | ) | |||
| 74 | + | if _, err := f.ResumeWithPrompt(context.Background(), a.ID, firstFollowUp); err != nil { | |||
| 75 | + | t.Fatalf("queue first active follow-up: %v", err) | |||
| 76 | + | } | |||
| 77 | + | if _, err := f.ResumeWithPrompt(context.Background(), a.ID, secondFollowUp); err != nil { | |||
| 78 | + | t.Fatalf("queue second active follow-up: %v", err) | |||
| 79 | + | } | |||
| 80 | + | select { | |||
| 81 | + | case command := <-commands: | |||
| 82 | + | t.Fatalf("active follow-up was delivered before idle: %v", command.Type) | |||
| 83 | + | default: | |||
| 84 | + | } | |||
| 85 | + | ||||
| 86 | + | persisted, err := readAgentMeta(filepath.Join(root, "agents", a.ID)) | |||
| 87 | + | if err != nil { | |||
| 88 | + | t.Fatal(err) | |||
| 89 | + | } | |||
| 90 | + | if persisted.ResumePrompt != "" || len(persisted.ResumeQueue) != 2 { | |||
| 91 | + | t.Fatalf("active follow-ups persisted as prompt %q queue %d, want empty prompt and queue 2", persisted.ResumePrompt, len(persisted.ResumeQueue)) | |||
| 92 | + | } | |||
| 93 | + | if firstID, secondID := persisted.ResumeQueue[0].CommandID, persisted.ResumeQueue[1].CommandID; firstID == "" || secondID == "" || firstID == secondID { | |||
| 94 | + | t.Fatalf("queued follow-up command IDs = %q, %q; want distinct non-empty IDs", firstID, secondID) | |||
| 95 | + | } | |||
| 96 | + | ||||
| 97 | + | if err := updateAgentFromEvent(a, NewEvent(EventAgentIdle, map[string]any{"turn_id": "turn-1"})); err != nil { | |||
| 98 | + | t.Fatalf("apply first idle event: %v", err) | |||
| 99 | + | } | |||
| 100 | + | select { | |||
| 101 | + | case command := <-commands: | |||
| 102 | + | var payload TurnStartPayload | |||
| 103 | + | if command.Type != CommandTurnStart || command.DecodePayload(&payload) != nil || payload.Prompt != firstFollowUp { | |||
| 104 | + | t.Fatalf("first idle delivery = type %q prompt %q, want turn start %q", command.Type, payload.Prompt, firstFollowUp) | |||
| 105 | + | } | |||
| 106 | + | if !payload.NewRun { | |||
| 107 | + | t.Fatal("first queued follow-up did not request a fresh run budget") | |||
| 108 | + | } | |||
| 109 | + | case <-time.After(time.Second): | |||
| 110 | + | t.Fatal("first active follow-up was not delivered at idle") | |||
| 111 | + | } | |||
| 112 | + | if got := a.TurnState(); got != TurnQueued { | |||
| 113 | + | t.Fatalf("turn state after first queued delivery = %s, want %s", got, TurnQueued) | |||
| 114 | + | } | |||
| 115 | + | ||||
| 116 | + | if err := updateAgentFromEvent(a, NewEvent(EventTurnStarted, map[string]any{ | |||
| 117 | + | "turn_id": "turn-2", "step": float64(2), "lifetime_turns": 2, "current_run_turns": 1, | |||
| 118 | + | })); err != nil { | |||
| 119 | + | t.Fatalf("apply second turn started event: %v", err) | |||
| 120 | + | } | |||
| 121 | + | if err := updateAgentFromEvent(a, NewEvent(EventAgentIdle, map[string]any{"turn_id": "turn-2"})); err != nil { | |||
| 122 | + | t.Fatalf("apply second idle event: %v", err) | |||
| 123 | + | } | |||
| 124 | + | select { | |||
| 125 | + | case command := <-commands: | |||
| 126 | + | var payload TurnStartPayload | |||
| 127 | + | if command.Type != CommandTurnStart || command.DecodePayload(&payload) != nil || payload.Prompt != secondFollowUp { | |||
| 128 | + | t.Fatalf("second idle delivery = type %q prompt %q, want turn start %q", command.Type, payload.Prompt, secondFollowUp) | |||
| 129 | + | } | |||
| 130 | + | case <-time.After(time.Second): | |||
| 131 | + | t.Fatal("second active follow-up was not delivered at the next idle turn") | |||
| 132 | + | } | |||
| 133 | + | ||||
| 134 | + | persisted, err = readAgentMeta(filepath.Join(root, "agents", a.ID)) | |||
| 135 | + | if err != nil { | |||
| 136 | + | t.Fatal(err) | |||
| 137 | + | } | |||
| 138 | + | if persisted.ResumePrompt != secondFollowUp || len(persisted.ResumeQueue) != 0 { | |||
| 139 | + | t.Fatalf("second follow-up persistence = prompt %q queue %d, want prompt %q and empty queue", persisted.ResumePrompt, len(persisted.ResumeQueue), secondFollowUp) | |||
| 140 | + | } | |||
| 141 | + | if persisted.ResumePromptID == "" { | |||
| 142 | + | t.Fatal("promoted follow-up lost its command ID") | |||
| 143 | + | } | |||
| 144 | + | if err := f.Stop(a.ID); err != nil { | |||
| 145 | + | t.Fatal(err) | |||
| 146 | + | } | |||
| 147 | + | a.Wait() | |||
| 148 | + | } | |||
| 149 | + | ||||
| 11 | func TestResumeWithPromptDeliversFollowUpToIdleWorker(t *testing.T) { | 150 | func TestResumeWithPromptDeliversFollowUpToIdleWorker(t *testing.T) { | ||
| 12 | if runtime.GOOS == "windows" { | 151 | if runtime.GOOS == "windows" { | ||
| 13 | t.Skip("subagent inbox transport uses Unix-domain sockets") | 152 | t.Skip("subagent inbox transport uses Unix-domain sockets") | ||
| @@ -60,6 +199,7 @@ func TestResumeWithPromptDeliversFollowUpToIdleWorker(t *testing.T) { | |||||
| 60 | case <-time.After(time.Second): | 199 | case <-time.After(time.Second): | ||
| 61 | t.Fatal("idle worker did not open its inbox") | 200 | t.Fatal("idle worker did not open its inbox") | ||
| 62 | } | 201 | } | ||
| 202 | + | a.setTurnCounts(4, 2) | |||
| 63 | 203 | ||||
| 64 | const followUp = "I applied your review. What do you think now?" | 204 | const followUp = "I applied your review. What do you think now?" | ||
| 65 | continued, err := f.ResumeWithPrompt(context.Background(), a.ID, followUp) | 205 | continued, err := f.ResumeWithPrompt(context.Background(), a.ID, followUp) | ||
| @@ -72,6 +212,12 @@ func TestResumeWithPromptDeliversFollowUpToIdleWorker(t *testing.T) { | |||||
| 72 | if got := a.TurnState(); got != TurnQueued { | 212 | if got := a.TurnState(); got != TurnQueued { | ||
| 73 | t.Fatalf("turn state after sending follow-up = %s, want %s", got, TurnQueued) | 213 | t.Fatalf("turn state after sending follow-up = %s, want %s", got, TurnQueued) | ||
| 74 | } | 214 | } | ||
| 215 | + | if got := a.CurrentRunTurnsValue(); got != 0 { | |||
| 216 | + | t.Fatalf("current-run turns after explicit resume = %d, want 0 before turn starts", got) | |||
| 217 | + | } | |||
| 218 | + | if got := a.LifetimeTurnsValue(); got != 4 { | |||
| 219 | + | t.Fatalf("lifetime turns after explicit resume = %d, want 4", got) | |||
| 220 | + | } | |||
| 75 | if _, err := f.ResumeWithPrompt(context.Background(), a.ID, "duplicate follow-up"); err == nil { | 221 | if _, err := f.ResumeWithPrompt(context.Background(), a.ID, "duplicate follow-up"); err == nil { | ||
| 76 | t.Fatal("second follow-up succeeded while the first was queued") | 222 | t.Fatal("second follow-up succeeded while the first was queued") | ||
| 77 | } | 223 | } | ||
| @@ -87,6 +233,9 @@ func TestResumeWithPromptDeliversFollowUpToIdleWorker(t *testing.T) { | |||||
| 87 | if payload.Prompt != followUp { | 233 | if payload.Prompt != followUp { | ||
| 88 | t.Fatalf("follow-up prompt = %q, want %q", payload.Prompt, followUp) | 234 | t.Fatalf("follow-up prompt = %q, want %q", payload.Prompt, followUp) | ||
| 89 | } | 235 | } | ||
| 236 | + | if !payload.NewRun { | |||
| 237 | + | t.Fatal("idle follow-up did not request a fresh run budget") | |||
| 238 | + | } | |||
| 90 | case <-time.After(time.Second): | 239 | case <-time.After(time.Second): | ||
| 91 | t.Fatal("idle worker did not receive the follow-up") | 240 | t.Fatal("idle worker did not receive the follow-up") | ||
| 92 | } | 241 | } | ||
M packages/agent/subagents/runner.go +151 -14
| @@ -7,6 +7,7 @@ import ( | |||
| 7 | 7 | "encoding/json" | |
| 8 | 8 | "errors" | |
| 9 | 9 | "fmt" | |
| 10 | + | "math" | |
| 10 | 11 | "os" | |
| 11 | 12 | "os/exec" | |
| 12 | 13 | "path/filepath" | |
| @@ -79,6 +80,9 @@ type subagentWorkerArgsOpts struct { | |||
| 79 | 80 | FastModeSet bool | |
| 80 | 81 | Subagent string | |
| 81 | 82 | MaxTurns int | |
| 83 | + | LifetimeTurns int | |
| 84 | + | RunTurns int | |
| 85 | + | CountersSet bool | |
| 82 | 86 | Tools []string | |
| 83 | 87 | WebSearchPolicy WebSearchPolicy | |
| 84 | 88 | } | |
| @@ -114,6 +118,9 @@ func defaultChildArgs(exe string, a *Agent, sessionPath, inboxPath string) []str | |||
| 114 | 118 | FastModeSet: true, | |
| 115 | 119 | Subagent: a.Subagent, | |
| 116 | 120 | MaxTurns: a.MaxTurns, | |
| 121 | + | LifetimeTurns: a.LifetimeTurnsValue(), | |
| 122 | + | RunTurns: a.CurrentRunTurnsValue(), | |
| 123 | + | CountersSet: true, | |
| 117 | 124 | Tools: a.Tools, | |
| 118 | 125 | WebSearchPolicy: a.WebSearchPolicy, | |
| 119 | 126 | }) | |
| @@ -157,6 +164,12 @@ func subagentWorkerArgs(opts subagentWorkerArgsOpts) []string { | |||
| 157 | 164 | if opts.MaxTurns > 0 { | |
| 158 | 165 | args = append(args, "--max-turns", fmt.Sprint(opts.MaxTurns)) | |
| 159 | 166 | } | |
| 167 | + | if opts.CountersSet { | |
| 168 | + | args = append(args, | |
| 169 | + | "--subagent-lifetime-turns", fmt.Sprint(opts.LifetimeTurns), | |
| 170 | + | "--subagent-run-turns", fmt.Sprint(opts.RunTurns), | |
| 171 | + | ) | |
| 172 | + | } | |
| 160 | 173 | webSearchPolicy := childWebSearchPolicy(opts.WebSearchPolicy, opts.Subagent, opts.Tools) | |
| 161 | 174 | // Always propagate the final capability decision, including deny, so a | |
| 162 | 175 | // child with no ordinary --tools list cannot re-enable web search from its | |
| @@ -414,7 +427,9 @@ func (r *execRunner) Run(ctx context.Context, sink Sink) error { | |||
| 414 | 427 | if trimmed != "" && !truncated { | |
| 415 | 428 | if ev, ok := parseEventLine(trimmed); ok { | |
| 416 | 429 | appendLog(ev) | |
| 417 | - | updateAgentFromEvent(r.agent, ev) | |
| 430 | + | if persistErr := updateAgentFromEvent(r.agent, ev); persistErr != nil { | |
| 431 | + | sink.Transcript("error: metadata persistence failed: " + persistErr.Error()) | |
| 432 | + | } | |
| 418 | 433 | applyEventToSink(ev, sink) | |
| 419 | 434 | // Fan prompt-level task completions up to any | |
| 420 | 435 | // subscriber on the supervised Agent. The child | |
| @@ -657,9 +672,71 @@ func parseEventLine(line string) (Event, bool) { | |||
| 657 | 672 | }, true | |
| 658 | 673 | } | |
| 659 | 674 | ||
| 660 | - | func updateAgentFromEvent(a *Agent, ev Event) { | |
| 675 | + | func eventMatchesPendingResume(a *Agent, ev Event) bool { | |
| 661 | 676 | if a == nil { | |
| 662 | - | return | |
| 677 | + | return false | |
| 678 | + | } | |
| 679 | + | commandID, _ := ev.Data["command_id"].(string) | |
| 680 | + | if commandID == "" { | |
| 681 | + | // Protocol-v1 turn.started events predate command identities, but a | |
| 682 | + | // commandless rejection cannot be safely associated with the current | |
| 683 | + | // pending prompt and must not consume it. | |
| 684 | + | code, _ := ev.Data["code"].(string) | |
| 685 | + | return code != "turn_rejected" | |
| 686 | + | } | |
| 687 | + | expected := a.resumePromptCommandID() | |
| 688 | + | return expected == "" || expected == commandID | |
| 689 | + | } | |
| 690 | + | ||
| 691 | + | func eventCounter(data map[string]any, key string) (int, bool) { | |
| 692 | + | value, ok := data[key] | |
| 693 | + | if !ok { | |
| 694 | + | return 0, false | |
| 695 | + | } | |
| 696 | + | maxInt := int(^uint(0) >> 1) | |
| 697 | + | valid := func(value int64) (int, bool) { | |
| 698 | + | if value < 0 || value > int64(maxInt) { | |
| 699 | + | return 0, false | |
| 700 | + | } | |
| 701 | + | return int(value), true | |
| 702 | + | } | |
| 703 | + | switch value := value.(type) { | |
| 704 | + | case int: | |
| 705 | + | if value < 0 { | |
| 706 | + | return 0, false | |
| 707 | + | } | |
| 708 | + | return value, true | |
| 709 | + | case int64: | |
| 710 | + | return valid(value) | |
| 711 | + | case uint: | |
| 712 | + | if uint64(value) > uint64(maxInt) { | |
| 713 | + | return 0, false | |
| 714 | + | } | |
| 715 | + | return int(value), true | |
| 716 | + | case uint64: | |
| 717 | + | if value > uint64(maxInt) { | |
| 718 | + | return 0, false | |
| 719 | + | } | |
| 720 | + | return int(value), true | |
| 721 | + | case float64: | |
| 722 | + | if value < 0 || math.Trunc(value) != value || value >= float64(maxInt)+1 { | |
| 723 | + | return 0, false | |
| 724 | + | } | |
| 725 | + | return int(value), true | |
| 726 | + | case json.Number: | |
| 727 | + | parsed, err := value.Int64() | |
| 728 | + | if err != nil { | |
| 729 | + | return 0, false | |
| 730 | + | } | |
| 731 | + | return valid(parsed) | |
| 732 | + | default: | |
| 733 | + | return 0, false | |
| 734 | + | } | |
| 735 | + | } | |
| 736 | + | ||
| 737 | + | func updateAgentFromEvent(a *Agent, ev Event) error { | |
| 738 | + | if a == nil { | |
| 739 | + | return nil | |
| 663 | 740 | } | |
| 664 | 741 | now := ev.Time | |
| 665 | 742 | if now.IsZero() { | |
| @@ -667,25 +744,56 @@ func updateAgentFromEvent(a *Agent, ev Event) { | |||
| 667 | 744 | } | |
| 668 | 745 | a.markActivity(now) | |
| 669 | 746 | persist := false | |
| 747 | + | notifyIdle := false | |
| 670 | 748 | switch ev.Type { | |
| 671 | 749 | case EventAgentReady, "agent_ready": | |
| 672 | 750 | a.setProcessState(ProcessAlive) | |
| 673 | - | a.setTurnState(TurnIdle, ev.TurnID) | |
| 751 | + | if lifetime, ok := eventCounter(ev.Data, "lifetime_turns"); ok { | |
| 752 | + | if currentRun, currentOK := eventCounter(ev.Data, "current_run_turns"); currentOK { | |
| 753 | + | a.setTurnCounts(lifetime, currentRun) | |
| 754 | + | } | |
| 755 | + | } | |
| 756 | + | // A resumed worker may have a durable initial follow-up in its argv. | |
| 757 | + | // Keep it queued until the worker emits its own turn.started event; | |
| 758 | + | // treating readiness as idle here could dispatch another queued prompt | |
| 759 | + | // concurrently with that initial turn. | |
| 760 | + | if prompt, _ := a.ResumePromptInfo(); prompt == "" && a.TurnState() != TurnQueued { | |
| 761 | + | a.setTurnState(TurnIdle, ev.TurnID) | |
| 762 | + | notifyIdle = true | |
| 763 | + | } | |
| 674 | 764 | persist = true | |
| 675 | 765 | case EventAgentHeartbeat, "agent_heartbeat": | |
| 676 | 766 | a.setProcessState(ProcessAlive) | |
| 677 | 767 | persist = true | |
| 678 | - | case EventTurnStarted, "turn_start": | |
| 679 | - | turnID := ev.TurnID | |
| 680 | - | if turnID == "" { | |
| 681 | - | if step, ok := ev.Data["step"].(float64); ok { | |
| 682 | - | turnID = fmt.Sprintf("turn-%d", int(step)) | |
| 683 | - | } | |
| 768 | + | case EventTurnStarted: | |
| 769 | + | if !isDelegatedTurnStart(ev) { | |
| 770 | + | // Provider/model-loop turn starts are nested inside one worker | |
| 771 | + | // message turn and are activity only. | |
| 772 | + | a.setProcessState(ProcessAlive) | |
| 773 | + | break | |
| 774 | + | } | |
| 775 | + | if !eventMatchesPendingResume(a, ev) { | |
| 776 | + | break | |
| 684 | 777 | } | |
| 778 | + | turnID := eventTurnID(ev) | |
| 685 | 779 | a.setProcessState(ProcessAlive) | |
| 780 | + | if lifetime, ok := eventCounter(ev.Data, "lifetime_turns"); ok { | |
| 781 | + | if currentRun, currentOK := eventCounter(ev.Data, "current_run_turns"); currentOK { | |
| 782 | + | a.setTurnCounts(lifetime, currentRun) | |
| 783 | + | } else { | |
| 784 | + | a.incrementTurnCounts() | |
| 785 | + | } | |
| 786 | + | } else { | |
| 787 | + | a.incrementTurnCounts() | |
| 788 | + | } | |
| 686 | 789 | a.setTurnState(TurnRunning, turnID) | |
| 687 | 790 | a.clearResumePrompt() | |
| 688 | 791 | persist = true | |
| 792 | + | case "turn_start": | |
| 793 | + | // Provider/model-loop turn starts are nested inside one worker message | |
| 794 | + | // turn. They are activity only and must not consume or reset the | |
| 795 | + | // worker's lifetime or current-run budget. | |
| 796 | + | a.setProcessState(ProcessAlive) | |
| 689 | 797 | case EventTurnResult, "turn_result": | |
| 690 | 798 | if result, err := decodeTurnResultEvent(ev, a.ID, a.maxOutputBytes, a.maxOutputLines); err == nil { | |
| 691 | 799 | a.setResult(result) | |
| @@ -716,17 +824,36 @@ func updateAgentFromEvent(a *Agent, ev Event) { | |||
| 716 | 824 | a.setTurnState(TurnFailed, ev.TurnID) | |
| 717 | 825 | persist = true | |
| 718 | 826 | case EventAgentIdle, "agent_idle": | |
| 827 | + | if lifetime, ok := eventCounter(ev.Data, "lifetime_turns"); ok { | |
| 828 | + | if currentRun, currentOK := eventCounter(ev.Data, "current_run_turns"); currentOK { | |
| 829 | + | a.setTurnCounts(lifetime, currentRun) | |
| 830 | + | } | |
| 831 | + | } | |
| 719 | 832 | a.setTurnState(TurnIdle, ev.TurnID) | |
| 833 | + | notifyIdle = true | |
| 720 | 834 | persist = true | |
| 721 | 835 | case EventAgentExited, "agent_exited", "agent_stopped": | |
| 722 | 836 | a.setProcessState(ProcessExited) | |
| 723 | 837 | persist = true | |
| 838 | + | case "error": | |
| 839 | + | if code, _ := ev.Data["code"].(string); code == "turn_rejected" && eventMatchesPendingResume(a, ev) { | |
| 840 | + | if a.rejectActiveResumePrompt() { | |
| 841 | + | reason, _ := ev.Data["reason"].(string) | |
| 842 | + | if reason == "max_turns" { | |
| 843 | + | a.setTurnState(TurnFailed, ev.TurnID) | |
| 844 | + | } else { | |
| 845 | + | a.setTurnState(TurnIdle, ev.TurnID) | |
| 846 | + | notifyIdle = true | |
| 847 | + | } | |
| 848 | + | persist = true | |
| 849 | + | } | |
| 850 | + | } | |
| 724 | 851 | case "turn_end": | |
| 725 | 852 | // Provider/tool-loop turn_end events (for example stop=tool_use) | |
| 726 | 853 | // do not carry the daemon's prompt step and are not terminal for | |
| 727 | 854 | // the delegated task. They remain in the event log, but must not | |
| 728 | 855 | // overwrite the delegated turn state or trigger persistence. | |
| 729 | - | if _, ok := ev.Data["step"].(float64); !ok { | |
| 856 | + | if !isDelegatedTurnEnd(ev) { | |
| 730 | 857 | break | |
| 731 | 858 | } | |
| 732 | 859 | if message, _ := ev.Data["error"].(string); message != "" { | |
| @@ -737,8 +864,15 @@ func updateAgentFromEvent(a *Agent, ev Event) { | |||
| 737 | 864 | persist = true | |
| 738 | 865 | } | |
| 739 | 866 | if persist && a.persistFn != nil { | |
| 740 | - | a.persistFn(a) | |
| 867 | + | if err := a.persistFn(a); err != nil { | |
| 868 | + | a.recordPersistenceError(err) | |
| 869 | + | return err | |
| 870 | + | } | |
| 871 | + | } | |
| 872 | + | if notifyIdle { | |
| 873 | + | a.notifyTurnIdle() | |
| 741 | 874 | } | |
| 875 | + | return nil | |
| 742 | 876 | } | |
| 743 | 877 | ||
| 744 | 878 | // notifyPromptTurnEnd calls Agent.OnTurnEnd only for the subagent | |
| @@ -749,7 +883,10 @@ func notifyPromptTurnEnd(a *Agent, ev Event) { | |||
| 749 | 883 | if a == nil || ev.Type != "turn_end" { | |
| 750 | 884 | return | |
| 751 | 885 | } | |
| 752 | - | step, ok := ev.Data["step"].(float64) | |
| 886 | + | if !isDelegatedTurnEnd(ev) { | |
| 887 | + | return | |
| 888 | + | } | |
| 889 | + | step, ok := eventCounter(ev.Data, "step") | |
| 753 | 890 | if !ok { | |
| 754 | 891 | return | |
| 755 | 892 | } | |
| @@ -763,7 +900,7 @@ func notifyPromptTurnEnd(a *Agent, ev Event) { | |||
| 763 | 900 | return | |
| 764 | 901 | } | |
| 765 | 902 | a.mu.Unlock() | |
| 766 | - | go fn(int(step), errMsg) | |
| 903 | + | go fn(step, errMsg) | |
| 767 | 904 | } | |
| 768 | 905 | ||
| 769 | 906 | func isAssistantStreamBoundary(eventType string) bool { | |
| @@ -7,6 +7,7 @@ import ( | |||||
| 7 | "encoding/json" | 7 | "encoding/json" | ||
| 8 | "errors" | 8 | "errors" | ||
| 9 | "fmt" | 9 | "fmt" | ||
| 10 | + | "math" | |||
| 10 | "os" | 11 | "os" | ||
| 11 | "os/exec" | 12 | "os/exec" | ||
| 12 | "path/filepath" | 13 | "path/filepath" | ||
| @@ -79,6 +80,9 @@ type subagentWorkerArgsOpts struct { | |||||
| 79 | FastModeSet bool | 80 | FastModeSet bool | ||
| 80 | Subagent string | 81 | Subagent string | ||
| 81 | MaxTurns int | 82 | MaxTurns int | ||
| 83 | + | LifetimeTurns int | |||
| 84 | + | RunTurns int | |||
| 85 | + | CountersSet bool | |||
| 82 | Tools []string | 86 | Tools []string | ||
| 83 | WebSearchPolicy WebSearchPolicy | 87 | WebSearchPolicy WebSearchPolicy | ||
| 84 | } | 88 | } | ||
| @@ -114,6 +118,9 @@ func defaultChildArgs(exe string, a *Agent, sessionPath, inboxPath string) []str | |||||
| 114 | FastModeSet: true, | 118 | FastModeSet: true, | ||
| 115 | Subagent: a.Subagent, | 119 | Subagent: a.Subagent, | ||
| 116 | MaxTurns: a.MaxTurns, | 120 | MaxTurns: a.MaxTurns, | ||
| 121 | + | LifetimeTurns: a.LifetimeTurnsValue(), | |||
| 122 | + | RunTurns: a.CurrentRunTurnsValue(), | |||
| 123 | + | CountersSet: true, | |||
| 117 | Tools: a.Tools, | 124 | Tools: a.Tools, | ||
| 118 | WebSearchPolicy: a.WebSearchPolicy, | 125 | WebSearchPolicy: a.WebSearchPolicy, | ||
| 119 | }) | 126 | }) | ||
| @@ -157,6 +164,12 @@ func subagentWorkerArgs(opts subagentWorkerArgsOpts) []string { | |||||
| 157 | if opts.MaxTurns > 0 { | 164 | if opts.MaxTurns > 0 { | ||
| 158 | args = append(args, "--max-turns", fmt.Sprint(opts.MaxTurns)) | 165 | args = append(args, "--max-turns", fmt.Sprint(opts.MaxTurns)) | ||
| 159 | } | 166 | } | ||
| 167 | + | if opts.CountersSet { | |||
| 168 | + | args = append(args, | |||
| 169 | + | "--subagent-lifetime-turns", fmt.Sprint(opts.LifetimeTurns), | |||
| 170 | + | "--subagent-run-turns", fmt.Sprint(opts.RunTurns), | |||
| 171 | + | ) | |||
| 172 | + | } | |||
| 160 | webSearchPolicy := childWebSearchPolicy(opts.WebSearchPolicy, opts.Subagent, opts.Tools) | 173 | webSearchPolicy := childWebSearchPolicy(opts.WebSearchPolicy, opts.Subagent, opts.Tools) | ||
| 161 | // Always propagate the final capability decision, including deny, so a | 174 | // Always propagate the final capability decision, including deny, so a | ||
| 162 | // child with no ordinary --tools list cannot re-enable web search from its | 175 | // child with no ordinary --tools list cannot re-enable web search from its | ||
| @@ -414,7 +427,9 @@ func (r *execRunner) Run(ctx context.Context, sink Sink) error { | |||||
| 414 | if trimmed != "" && !truncated { | 427 | if trimmed != "" && !truncated { | ||
| 415 | if ev, ok := parseEventLine(trimmed); ok { | 428 | if ev, ok := parseEventLine(trimmed); ok { | ||
| 416 | appendLog(ev) | 429 | appendLog(ev) | ||
| 417 | - | updateAgentFromEvent(r.agent, ev) | 430 | + | if persistErr := updateAgentFromEvent(r.agent, ev); persistErr != nil { |
| 431 | + | sink.Transcript("error: metadata persistence failed: " + persistErr.Error()) | |||
| 432 | + | } | |||
| 418 | applyEventToSink(ev, sink) | 433 | applyEventToSink(ev, sink) | ||
| 419 | // Fan prompt-level task completions up to any | 434 | // Fan prompt-level task completions up to any | ||
| 420 | // subscriber on the supervised Agent. The child | 435 | // subscriber on the supervised Agent. The child | ||
| @@ -657,9 +672,71 @@ func parseEventLine(line string) (Event, bool) { | |||||
| 657 | }, true | 672 | }, true | ||
| 658 | } | 673 | } | ||
| 659 | 674 | ||||
| 660 | - | func updateAgentFromEvent(a *Agent, ev Event) { | 675 | + | func eventMatchesPendingResume(a *Agent, ev Event) bool { |
| 661 | if a == nil { | 676 | if a == nil { | ||
| 662 | - | return | 677 | + | return false |
| 678 | + | } | |||
| 679 | + | commandID, _ := ev.Data["command_id"].(string) | |||
| 680 | + | if commandID == "" { | |||
| 681 | + | // Protocol-v1 turn.started events predate command identities, but a | |||
| 682 | + | // commandless rejection cannot be safely associated with the current | |||
| 683 | + | // pending prompt and must not consume it. | |||
| 684 | + | code, _ := ev.Data["code"].(string) | |||
| 685 | + | return code != "turn_rejected" | |||
| 686 | + | } | |||
| 687 | + | expected := a.resumePromptCommandID() | |||
| 688 | + | return expected == "" || expected == commandID | |||
| 689 | + | } | |||
| 690 | + | ||||
| 691 | + | func eventCounter(data map[string]any, key string) (int, bool) { | |||
| 692 | + | value, ok := data[key] | |||
| 693 | + | if !ok { | |||
| 694 | + | return 0, false | |||
| 695 | + | } | |||
| 696 | + | maxInt := int(^uint(0) >> 1) | |||
| 697 | + | valid := func(value int64) (int, bool) { | |||
| 698 | + | if value < 0 || value > int64(maxInt) { | |||
| 699 | + | return 0, false | |||
| 700 | + | } | |||
| 701 | + | return int(value), true | |||
| 702 | + | } | |||
| 703 | + | switch value := value.(type) { | |||
| 704 | + | case int: | |||
| 705 | + | if value < 0 { | |||
| 706 | + | return 0, false | |||
| 707 | + | } | |||
| 708 | + | return value, true | |||
| 709 | + | case int64: | |||
| 710 | + | return valid(value) | |||
| 711 | + | case uint: | |||
| 712 | + | if uint64(value) > uint64(maxInt) { | |||
| 713 | + | return 0, false | |||
| 714 | + | } | |||
| 715 | + | return int(value), true | |||
| 716 | + | case uint64: | |||
| 717 | + | if value > uint64(maxInt) { | |||
| 718 | + | return 0, false | |||
| 719 | + | } | |||
| 720 | + | return int(value), true | |||
| 721 | + | case float64: | |||
| 722 | + | if value < 0 || math.Trunc(value) != value || value >= float64(maxInt)+1 { | |||
| 723 | + | return 0, false | |||
| 724 | + | } | |||
| 725 | + | return int(value), true | |||
| 726 | + | case json.Number: | |||
| 727 | + | parsed, err := value.Int64() | |||
| 728 | + | if err != nil { | |||
| 729 | + | return 0, false | |||
| 730 | + | } | |||
| 731 | + | return valid(parsed) | |||
| 732 | + | default: | |||
| 733 | + | return 0, false | |||
| 734 | + | } | |||
| 735 | + | } | |||
| 736 | + | ||||
| 737 | + | func updateAgentFromEvent(a *Agent, ev Event) error { | |||
| 738 | + | if a == nil { | |||
| 739 | + | return nil | |||
| 663 | } | 740 | } | ||
| 664 | now := ev.Time | 741 | now := ev.Time | ||
| 665 | if now.IsZero() { | 742 | if now.IsZero() { | ||
| @@ -667,25 +744,56 @@ func updateAgentFromEvent(a *Agent, ev Event) { | |||||
| 667 | } | 744 | } | ||
| 668 | a.markActivity(now) | 745 | a.markActivity(now) | ||
| 669 | persist := false | 746 | persist := false | ||
| 747 | + | notifyIdle := false | |||
| 670 | switch ev.Type { | 748 | switch ev.Type { | ||
| 671 | case EventAgentReady, "agent_ready": | 749 | case EventAgentReady, "agent_ready": | ||
| 672 | a.setProcessState(ProcessAlive) | 750 | a.setProcessState(ProcessAlive) | ||
| 673 | - | a.setTurnState(TurnIdle, ev.TurnID) | 751 | + | if lifetime, ok := eventCounter(ev.Data, "lifetime_turns"); ok { |
| 752 | + | if currentRun, currentOK := eventCounter(ev.Data, "current_run_turns"); currentOK { | |||
| 753 | + | a.setTurnCounts(lifetime, currentRun) | |||
| 754 | + | } | |||
| 755 | + | } | |||
| 756 | + | // A resumed worker may have a durable initial follow-up in its argv. | |||
| 757 | + | // Keep it queued until the worker emits its own turn.started event; | |||
| 758 | + | // treating readiness as idle here could dispatch another queued prompt | |||
| 759 | + | // concurrently with that initial turn. | |||
| 760 | + | if prompt, _ := a.ResumePromptInfo(); prompt == "" && a.TurnState() != TurnQueued { | |||
| 761 | + | a.setTurnState(TurnIdle, ev.TurnID) | |||
| 762 | + | notifyIdle = true | |||
| 763 | + | } | |||
| 674 | persist = true | 764 | persist = true | ||
| 675 | case EventAgentHeartbeat, "agent_heartbeat": | 765 | case EventAgentHeartbeat, "agent_heartbeat": | ||
| 676 | a.setProcessState(ProcessAlive) | 766 | a.setProcessState(ProcessAlive) | ||
| 677 | persist = true | 767 | persist = true | ||
| 678 | - | case EventTurnStarted, "turn_start": | 768 | + | case EventTurnStarted: |
| 679 | - | turnID := ev.TurnID | 769 | + | if !isDelegatedTurnStart(ev) { |
| 680 | - | if turnID == "" { | 770 | + | // Provider/model-loop turn starts are nested inside one worker |
| 681 | - | if step, ok := ev.Data["step"].(float64); ok { | 771 | + | // message turn and are activity only. |
| 682 | - | turnID = fmt.Sprintf("turn-%d", int(step)) | 772 | + | a.setProcessState(ProcessAlive) |
| 683 | - | } | 773 | + | break |
| 774 | + | } | |||
| 775 | + | if !eventMatchesPendingResume(a, ev) { | |||
| 776 | + | break | |||
| 684 | } | 777 | } | ||
| 778 | + | turnID := eventTurnID(ev) | |||
| 685 | a.setProcessState(ProcessAlive) | 779 | a.setProcessState(ProcessAlive) | ||
| 780 | + | if lifetime, ok := eventCounter(ev.Data, "lifetime_turns"); ok { | |||
| 781 | + | if currentRun, currentOK := eventCounter(ev.Data, "current_run_turns"); currentOK { | |||
| 782 | + | a.setTurnCounts(lifetime, currentRun) | |||
| 783 | + | } else { | |||
| 784 | + | a.incrementTurnCounts() | |||
| 785 | + | } | |||
| 786 | + | } else { | |||
| 787 | + | a.incrementTurnCounts() | |||
| 788 | + | } | |||
| 686 | a.setTurnState(TurnRunning, turnID) | 789 | a.setTurnState(TurnRunning, turnID) | ||
| 687 | a.clearResumePrompt() | 790 | a.clearResumePrompt() | ||
| 688 | persist = true | 791 | persist = true | ||
| 792 | + | case "turn_start": | |||
| 793 | + | // Provider/model-loop turn starts are nested inside one worker message | |||
| 794 | + | // turn. They are activity only and must not consume or reset the | |||
| 795 | + | // worker's lifetime or current-run budget. | |||
| 796 | + | a.setProcessState(ProcessAlive) | |||
| 689 | case EventTurnResult, "turn_result": | 797 | case EventTurnResult, "turn_result": | ||
| 690 | if result, err := decodeTurnResultEvent(ev, a.ID, a.maxOutputBytes, a.maxOutputLines); err == nil { | 798 | if result, err := decodeTurnResultEvent(ev, a.ID, a.maxOutputBytes, a.maxOutputLines); err == nil { | ||
| 691 | a.setResult(result) | 799 | a.setResult(result) | ||
| @@ -716,17 +824,36 @@ func updateAgentFromEvent(a *Agent, ev Event) { | |||||
| 716 | a.setTurnState(TurnFailed, ev.TurnID) | 824 | a.setTurnState(TurnFailed, ev.TurnID) | ||
| 717 | persist = true | 825 | persist = true | ||
| 718 | case EventAgentIdle, "agent_idle": | 826 | case EventAgentIdle, "agent_idle": | ||
| 827 | + | if lifetime, ok := eventCounter(ev.Data, "lifetime_turns"); ok { | |||
| 828 | + | if currentRun, currentOK := eventCounter(ev.Data, "current_run_turns"); currentOK { | |||
| 829 | + | a.setTurnCounts(lifetime, currentRun) | |||
| 830 | + | } | |||
| 831 | + | } | |||
| 719 | a.setTurnState(TurnIdle, ev.TurnID) | 832 | a.setTurnState(TurnIdle, ev.TurnID) | ||
| 833 | + | notifyIdle = true | |||
| 720 | persist = true | 834 | persist = true | ||
| 721 | case EventAgentExited, "agent_exited", "agent_stopped": | 835 | case EventAgentExited, "agent_exited", "agent_stopped": | ||
| 722 | a.setProcessState(ProcessExited) | 836 | a.setProcessState(ProcessExited) | ||
| 723 | persist = true | 837 | persist = true | ||
| 838 | + | case "error": | |||
| 839 | + | if code, _ := ev.Data["code"].(string); code == "turn_rejected" && eventMatchesPendingResume(a, ev) { | |||
| 840 | + | if a.rejectActiveResumePrompt() { | |||
| 841 | + | reason, _ := ev.Data["reason"].(string) | |||
| 842 | + | if reason == "max_turns" { | |||
| 843 | + | a.setTurnState(TurnFailed, ev.TurnID) | |||
| 844 | + | } else { | |||
| 845 | + | a.setTurnState(TurnIdle, ev.TurnID) | |||
| 846 | + | notifyIdle = true | |||
| 847 | + | } | |||
| 848 | + | persist = true | |||
| 849 | + | } | |||
| 850 | + | } | |||
| 724 | case "turn_end": | 851 | case "turn_end": | ||
| 725 | // Provider/tool-loop turn_end events (for example stop=tool_use) | 852 | // Provider/tool-loop turn_end events (for example stop=tool_use) | ||
| 726 | // do not carry the daemon's prompt step and are not terminal for | 853 | // do not carry the daemon's prompt step and are not terminal for | ||
| 727 | // the delegated task. They remain in the event log, but must not | 854 | // the delegated task. They remain in the event log, but must not | ||
| 728 | // overwrite the delegated turn state or trigger persistence. | 855 | // overwrite the delegated turn state or trigger persistence. | ||
| 729 | - | if _, ok := ev.Data["step"].(float64); !ok { | 856 | + | if !isDelegatedTurnEnd(ev) { |
| 730 | break | 857 | break | ||
| 731 | } | 858 | } | ||
| 732 | if message, _ := ev.Data["error"].(string); message != "" { | 859 | if message, _ := ev.Data["error"].(string); message != "" { | ||
| @@ -737,8 +864,15 @@ func updateAgentFromEvent(a *Agent, ev Event) { | |||||
| 737 | persist = true | 864 | persist = true | ||
| 738 | } | 865 | } | ||
| 739 | if persist && a.persistFn != nil { | 866 | if persist && a.persistFn != nil { | ||
| 740 | - | a.persistFn(a) | 867 | + | if err := a.persistFn(a); err != nil { |
| 868 | + | a.recordPersistenceError(err) | |||
| 869 | + | return err | |||
| 870 | + | } | |||
| 871 | + | } | |||
| 872 | + | if notifyIdle { | |||
| 873 | + | a.notifyTurnIdle() | |||
| 741 | } | 874 | } | ||
| 875 | + | return nil | |||
| 742 | } | 876 | } | ||
| 743 | 877 | ||||
| 744 | // notifyPromptTurnEnd calls Agent.OnTurnEnd only for the subagent | 878 | // notifyPromptTurnEnd calls Agent.OnTurnEnd only for the subagent | ||
| @@ -749,7 +883,10 @@ func notifyPromptTurnEnd(a *Agent, ev Event) { | |||||
| 749 | if a == nil || ev.Type != "turn_end" { | 883 | if a == nil || ev.Type != "turn_end" { | ||
| 750 | return | 884 | return | ||
| 751 | } | 885 | } | ||
| 752 | - | step, ok := ev.Data["step"].(float64) | 886 | + | if !isDelegatedTurnEnd(ev) { |
| 887 | + | return | |||
| 888 | + | } | |||
| 889 | + | step, ok := eventCounter(ev.Data, "step") | |||
| 753 | if !ok { | 890 | if !ok { | ||
| 754 | return | 891 | return | ||
| 755 | } | 892 | } | ||
| @@ -763,7 +900,7 @@ func notifyPromptTurnEnd(a *Agent, ev Event) { | |||||
| 763 | return | 900 | return | ||
| 764 | } | 901 | } | ||
| 765 | a.mu.Unlock() | 902 | a.mu.Unlock() | ||
| 766 | - | go fn(int(step), errMsg) | 903 | + | go fn(step, errMsg) |
| 767 | } | 904 | } | ||
| 768 | 905 | ||||
| 769 | func isAssistantStreamBoundary(eventType string) bool { | 906 | func isAssistantStreamBoundary(eventType string) bool { | ||
M packages/agent/subagents/runner_test.go +168 -2
| @@ -483,11 +483,69 @@ func TestDefaultChildArgsResumeUsesFollowUpPrompt(t *testing.T) { | |||
| 483 | 483 | } | |
| 484 | 484 | } | |
| 485 | 485 | ||
| 486 | + | func TestReadyDoesNotMarkInitialDelegatedTurnIdle(t *testing.T) { | |
| 487 | + | a := &Agent{Task: "initial delegated task"} | |
| 488 | + | a.setTurnState(TurnQueued, "") | |
| 489 | + | ||
| 490 | + | if err := updateAgentFromEvent(a, NewEvent(EventAgentReady, map[string]any{"lifetime_turns": 0, "current_run_turns": 0})); err != nil { | |
| 491 | + | t.Fatal(err) | |
| 492 | + | } | |
| 493 | + | if got := a.TurnState(); got != TurnQueued { | |
| 494 | + | t.Fatalf("ready changed turn state to %s, want %s", got, TurnQueued) | |
| 495 | + | } | |
| 496 | + | if err := updateAgentFromEvent(a, NewEvent(EventTurnStarted, map[string]any{ | |
| 497 | + | "turn_id": "turn-1", | |
| 498 | + | "lifetime_turns": 1, | |
| 499 | + | "current_run_turns": 1, | |
| 500 | + | })); err != nil { | |
| 501 | + | t.Fatal(err) | |
| 502 | + | } | |
| 503 | + | if got := a.TurnState(); got != TurnRunning { | |
| 504 | + | t.Fatalf("turn.started changed turn state to %s, want %s", got, TurnRunning) | |
| 505 | + | } | |
| 506 | + | } | |
| 507 | + | ||
| 508 | + | func TestUpdateAgentFromEventIgnoresStaleRejectedFollowUp(t *testing.T) { | |
| 509 | + | a := &Agent{} | |
| 510 | + | a.setResumePrompt("current prompt", time.Now()) | |
| 511 | + | if err := updateAgentFromEvent(a, NewEvent("error", map[string]any{ | |
| 512 | + | "code": "turn_rejected", "reason": "busy", "command_id": "stale-command", | |
| 513 | + | })); err != nil { | |
| 514 | + | t.Fatal(err) | |
| 515 | + | } | |
| 516 | + | if prompt, _ := a.ResumePromptInfo(); prompt != "current prompt" { | |
| 517 | + | t.Fatalf("stale rejection changed active prompt to %q", prompt) | |
| 518 | + | } | |
| 519 | + | } | |
| 520 | + | ||
| 521 | + | func TestUpdateAgentFromEventRecoversRejectedFollowUp(t *testing.T) { | |
| 522 | + | a := &Agent{} | |
| 523 | + | a.setResumePrompt("retry me", time.Now()) | |
| 524 | + | a.setTurnState(TurnQueued, "turn-1") | |
| 525 | + | if err := updateAgentFromEvent(a, NewEvent("error", map[string]any{ | |
| 526 | + | "code": "turn_rejected", | |
| 527 | + | "reason": "busy", | |
| 528 | + | "command_id": a.resumePromptCommandID(), | |
| 529 | + | })); err != nil { | |
| 530 | + | t.Fatal(err) | |
| 531 | + | } | |
| 532 | + | if got := a.TurnState(); got != TurnIdle { | |
| 533 | + | t.Fatalf("rejected turn state = %s, want %s", got, TurnIdle) | |
| 534 | + | } | |
| 535 | + | if prompt, _ := a.ResumePromptInfo(); prompt != "" { | |
| 536 | + | t.Fatalf("rejected prompt remained active: %q", prompt) | |
| 537 | + | } | |
| 538 | + | queued := a.resumePromptQueueSnapshot() | |
| 539 | + | if len(queued) != 1 || queued[0].Prompt != "retry me" { | |
| 540 | + | t.Fatalf("rejected prompt queue = %+v, want retry me", queued) | |
| 541 | + | } | |
| 542 | + | } | |
| 543 | + | ||
| 486 | 544 | func TestUpdateAgentFromEventIgnoresProviderToolLoopTurnEnd(t *testing.T) { | |
| 487 | 545 | a := &Agent{} | |
| 488 | 546 | a.setTurnState(TurnRunning, "turn-1") | |
| 489 | 547 | persisted := 0 | |
| 490 | - | a.persistFn = func(*Agent) { persisted++ } | |
| 548 | + | a.persistFn = func(*Agent) error { persisted++; return nil } | |
| 491 | 549 | ||
| 492 | 550 | updateAgentFromEvent(a, NewEvent("turn_end", map[string]any{"stop": "tool_use"})) | |
| 493 | 551 | ||
| @@ -505,7 +563,7 @@ func TestUpdateAgentFromEventIgnoresProviderToolLoopTurnEnd(t *testing.T) { | |||
| 505 | 563 | func TestUpdateAgentFromEventPersistsTerminalTurnEndWithStep(t *testing.T) { | |
| 506 | 564 | a := &Agent{} | |
| 507 | 565 | persisted := 0 | |
| 508 | - | a.persistFn = func(*Agent) { persisted++ } | |
| 566 | + | a.persistFn = func(*Agent) error { persisted++; return nil } | |
| 509 | 567 | ||
| 510 | 568 | updateAgentFromEvent(a, NewEvent("turn_end", map[string]any{"step": float64(2), "error": "boom"})) | |
| 511 | 569 | ||
| @@ -581,6 +639,114 @@ func TestSetOnTurnEndReplaysPendingNoticesInArrivalOrder(t *testing.T) { | |||
| 581 | 639 | } | |
| 582 | 640 | } | |
| 583 | 641 | ||
| 642 | + | func TestUpdateAgentFromEventDoesNotCountProviderTurnStarts(t *testing.T) { | |
| 643 | + | a := &Agent{} | |
| 644 | + | a.setTurnCounts(4, 2) | |
| 645 | + | a.setTurnState(TurnRunning, "turn-1") | |
| 646 | + | persisted := 0 | |
| 647 | + | a.persistFn = func(*Agent) error { persisted++; return nil } | |
| 648 | + | ||
| 649 | + | updateAgentFromEvent(a, NewEvent(EventTurnStarted, map[string]any{"step": float64(7), "nested_turn": true})) | |
| 650 | + | ||
| 651 | + | if got := a.LifetimeTurnsValue(); got != 4 { | |
| 652 | + | t.Fatalf("lifetime turns = %d, want 4", got) | |
| 653 | + | } | |
| 654 | + | if got := a.CurrentRunTurnsValue(); got != 2 { | |
| 655 | + | t.Fatalf("current-run turns = %d, want 2", got) | |
| 656 | + | } | |
| 657 | + | if got := a.TurnState(); got != TurnRunning { | |
| 658 | + | t.Fatalf("turn state = %s, want %s", got, TurnRunning) | |
| 659 | + | } | |
| 660 | + | if persisted != 0 { | |
| 661 | + | t.Fatalf("provider turn start persisted %d times, want 0", persisted) | |
| 662 | + | } | |
| 663 | + | } | |
| 664 | + | ||
| 665 | + | func TestUpdateAgentFromEventPersistsCanonicalTurnCounters(t *testing.T) { | |
| 666 | + | stateDir := t.TempDir() | |
| 667 | + | a := &Agent{ID: "counter-agent", stateDir: stateDir} | |
| 668 | + | a.persistFn = func(agent *Agent) error { | |
| 669 | + | if err := writeAgentMeta(stateDir, agent); err != nil { | |
| 670 | + | t.Errorf("write agent metadata: %v", err) | |
| 671 | + | return err | |
| 672 | + | } | |
| 673 | + | return nil | |
| 674 | + | } | |
| 675 | + | ||
| 676 | + | updateAgentFromEvent(a, NewEvent(EventTurnStarted, map[string]any{ | |
| 677 | + | "turn_id": "turn-3", | |
| 678 | + | "lifetime_turns": 3, | |
| 679 | + | "current_run_turns": 1, | |
| 680 | + | })) | |
| 681 | + | ||
| 682 | + | meta, err := readAgentMeta(stateDir) | |
| 683 | + | if err != nil { | |
| 684 | + | t.Fatal(err) | |
| 685 | + | } | |
| 686 | + | if meta.LifetimeTurns != 3 || meta.CurrentRunTurns != 1 { | |
| 687 | + | t.Fatalf("persisted counters = (%d, %d), want (3, 1)", meta.LifetimeTurns, meta.CurrentRunTurns) | |
| 688 | + | } | |
| 689 | + | if got := a.Snapshot(); got.LifetimeTurns != 3 || got.CurrentRunTurns != 1 { | |
| 690 | + | t.Fatalf("snapshot counters = (%d, %d), want (3, 1)", got.LifetimeTurns, got.CurrentRunTurns) | |
| 691 | + | } | |
| 692 | + | } | |
| 693 | + | ||
| 694 | + | func TestRecordPersistenceErrorPreservesFirstFailure(t *testing.T) { | |
| 695 | + | a := &Agent{} | |
| 696 | + | a.recordPersistenceError(fmt.Errorf("first failure")) | |
| 697 | + | a.recordPersistenceError(fmt.Errorf("later failure")) | |
| 698 | + | if got := a.Snapshot().Err; !strings.Contains(got, "first failure") || strings.Contains(got, "later failure") { | |
| 699 | + | t.Fatalf("recorded persistence error = %q, want only first failure", got) | |
| 700 | + | } | |
| 701 | + | } | |
| 702 | + | ||
| 703 | + | func TestRestoreResumePromptPreservesCommandIdentity(t *testing.T) { | |
| 704 | + | a := &Agent{} | |
| 705 | + | a.setResumePrompt("old prompt", time.Now()) | |
| 706 | + | oldID := a.resumePromptCommandID() | |
| 707 | + | previous := a.setResumePrompt("new prompt", time.Now()) | |
| 708 | + | a.restoreResumePrompt(previous) | |
| 709 | + | if got := a.resumePromptCommandID(); got != oldID { | |
| 710 | + | t.Fatalf("restored command ID = %q, want %q", got, oldID) | |
| 711 | + | } | |
| 712 | + | } | |
| 713 | + | ||
| 714 | + | func TestEventMatchesPendingResumeRejectsIdentityLessRejection(t *testing.T) { | |
| 715 | + | a := &Agent{resumePromptID: "command-1"} | |
| 716 | + | if eventMatchesPendingResume(a, Event{Type: "error", Data: map[string]any{"code": "turn_rejected"}}) { | |
| 717 | + | t.Fatal("identity-less turn rejection matched a pending prompt") | |
| 718 | + | } | |
| 719 | + | if !eventMatchesPendingResume(a, Event{Type: EventTurnStarted, Data: map[string]any{}}) { | |
| 720 | + | t.Fatal("identity-less legacy turn.started did not match a pending prompt") | |
| 721 | + | } | |
| 722 | + | if !eventMatchesPendingResume(a, Event{Type: "error", Data: map[string]any{"code": "turn_rejected", "command_id": "command-1"}}) { | |
| 723 | + | t.Fatal("matching turn rejection did not match a pending prompt") | |
| 724 | + | } | |
| 725 | + | } | |
| 726 | + | ||
| 727 | + | func TestEventCounterRejectsNegativeFractionalAndOverflowValues(t *testing.T) { | |
| 728 | + | cases := []struct { | |
| 729 | + | name string | |
| 730 | + | value any | |
| 731 | + | ok bool | |
| 732 | + | }{ | |
| 733 | + | {name: "valid integer", value: float64(3), ok: true}, | |
| 734 | + | {name: "negative integer", value: int64(-1)}, | |
| 735 | + | {name: "fractional float", value: 1.5}, | |
| 736 | + | {name: "negative float", value: -1.0}, | |
| 737 | + | {name: "fractional json number", value: json.Number("1.5")}, | |
| 738 | + | {name: "overflow float", value: float64(^uint(0) >> 1)}, | |
| 739 | + | } | |
| 740 | + | for _, tc := range cases { | |
| 741 | + | t.Run(tc.name, func(t *testing.T) { | |
| 742 | + | got, ok := eventCounter(map[string]any{"counter": tc.value}, "counter") | |
| 743 | + | if ok != tc.ok { | |
| 744 | + | t.Fatalf("eventCounter ok = %t, want %t (value %v, got %d)", ok, tc.ok, tc.value, got) | |
| 745 | + | } | |
| 746 | + | }) | |
| 747 | + | } | |
| 748 | + | } | |
| 749 | + | ||
| 584 | 750 | func indexOf(xs []string, want string) int { | |
| 585 | 751 | for i, x := range xs { | |
| 586 | 752 | if x == want { | |
| @@ -483,11 +483,69 @@ func TestDefaultChildArgsResumeUsesFollowUpPrompt(t *testing.T) { | |||||
| 483 | } | 483 | } | ||
| 484 | } | 484 | } | ||
| 485 | 485 | ||||
| 486 | + | func TestReadyDoesNotMarkInitialDelegatedTurnIdle(t *testing.T) { | |||
| 487 | + | a := &Agent{Task: "initial delegated task"} | |||
| 488 | + | a.setTurnState(TurnQueued, "") | |||
| 489 | + | ||||
| 490 | + | if err := updateAgentFromEvent(a, NewEvent(EventAgentReady, map[string]any{"lifetime_turns": 0, "current_run_turns": 0})); err != nil { | |||
| 491 | + | t.Fatal(err) | |||
| 492 | + | } | |||
| 493 | + | if got := a.TurnState(); got != TurnQueued { | |||
| 494 | + | t.Fatalf("ready changed turn state to %s, want %s", got, TurnQueued) | |||
| 495 | + | } | |||
| 496 | + | if err := updateAgentFromEvent(a, NewEvent(EventTurnStarted, map[string]any{ | |||
| 497 | + | "turn_id": "turn-1", | |||
| 498 | + | "lifetime_turns": 1, | |||
| 499 | + | "current_run_turns": 1, | |||
| 500 | + | })); err != nil { | |||
| 501 | + | t.Fatal(err) | |||
| 502 | + | } | |||
| 503 | + | if got := a.TurnState(); got != TurnRunning { | |||
| 504 | + | t.Fatalf("turn.started changed turn state to %s, want %s", got, TurnRunning) | |||
| 505 | + | } | |||
| 506 | + | } | |||
| 507 | + | ||||
| 508 | + | func TestUpdateAgentFromEventIgnoresStaleRejectedFollowUp(t *testing.T) { | |||
| 509 | + | a := &Agent{} | |||
| 510 | + | a.setResumePrompt("current prompt", time.Now()) | |||
| 511 | + | if err := updateAgentFromEvent(a, NewEvent("error", map[string]any{ | |||
| 512 | + | "code": "turn_rejected", "reason": "busy", "command_id": "stale-command", | |||
| 513 | + | })); err != nil { | |||
| 514 | + | t.Fatal(err) | |||
| 515 | + | } | |||
| 516 | + | if prompt, _ := a.ResumePromptInfo(); prompt != "current prompt" { | |||
| 517 | + | t.Fatalf("stale rejection changed active prompt to %q", prompt) | |||
| 518 | + | } | |||
| 519 | + | } | |||
| 520 | + | ||||
| 521 | + | func TestUpdateAgentFromEventRecoversRejectedFollowUp(t *testing.T) { | |||
| 522 | + | a := &Agent{} | |||
| 523 | + | a.setResumePrompt("retry me", time.Now()) | |||
| 524 | + | a.setTurnState(TurnQueued, "turn-1") | |||
| 525 | + | if err := updateAgentFromEvent(a, NewEvent("error", map[string]any{ | |||
| 526 | + | "code": "turn_rejected", | |||
| 527 | + | "reason": "busy", | |||
| 528 | + | "command_id": a.resumePromptCommandID(), | |||
| 529 | + | })); err != nil { | |||
| 530 | + | t.Fatal(err) | |||
| 531 | + | } | |||
| 532 | + | if got := a.TurnState(); got != TurnIdle { | |||
| 533 | + | t.Fatalf("rejected turn state = %s, want %s", got, TurnIdle) | |||
| 534 | + | } | |||
| 535 | + | if prompt, _ := a.ResumePromptInfo(); prompt != "" { | |||
| 536 | + | t.Fatalf("rejected prompt remained active: %q", prompt) | |||
| 537 | + | } | |||
| 538 | + | queued := a.resumePromptQueueSnapshot() | |||
| 539 | + | if len(queued) != 1 || queued[0].Prompt != "retry me" { | |||
| 540 | + | t.Fatalf("rejected prompt queue = %+v, want retry me", queued) | |||
| 541 | + | } | |||
| 542 | + | } | |||
| 543 | + | ||||
| 486 | func TestUpdateAgentFromEventIgnoresProviderToolLoopTurnEnd(t *testing.T) { | 544 | func TestUpdateAgentFromEventIgnoresProviderToolLoopTurnEnd(t *testing.T) { | ||
| 487 | a := &Agent{} | 545 | a := &Agent{} | ||
| 488 | a.setTurnState(TurnRunning, "turn-1") | 546 | a.setTurnState(TurnRunning, "turn-1") | ||
| 489 | persisted := 0 | 547 | persisted := 0 | ||
| 490 | - | a.persistFn = func(*Agent) { persisted++ } | 548 | + | a.persistFn = func(*Agent) error { persisted++; return nil } |
| 491 | 549 | ||||
| 492 | updateAgentFromEvent(a, NewEvent("turn_end", map[string]any{"stop": "tool_use"})) | 550 | updateAgentFromEvent(a, NewEvent("turn_end", map[string]any{"stop": "tool_use"})) | ||
| 493 | 551 | ||||
| @@ -505,7 +563,7 @@ func TestUpdateAgentFromEventIgnoresProviderToolLoopTurnEnd(t *testing.T) { | |||||
| 505 | func TestUpdateAgentFromEventPersistsTerminalTurnEndWithStep(t *testing.T) { | 563 | func TestUpdateAgentFromEventPersistsTerminalTurnEndWithStep(t *testing.T) { | ||
| 506 | a := &Agent{} | 564 | a := &Agent{} | ||
| 507 | persisted := 0 | 565 | persisted := 0 | ||
| 508 | - | a.persistFn = func(*Agent) { persisted++ } | 566 | + | a.persistFn = func(*Agent) error { persisted++; return nil } |
| 509 | 567 | ||||
| 510 | updateAgentFromEvent(a, NewEvent("turn_end", map[string]any{"step": float64(2), "error": "boom"})) | 568 | updateAgentFromEvent(a, NewEvent("turn_end", map[string]any{"step": float64(2), "error": "boom"})) | ||
| 511 | 569 | ||||
| @@ -581,6 +639,114 @@ func TestSetOnTurnEndReplaysPendingNoticesInArrivalOrder(t *testing.T) { | |||||
| 581 | } | 639 | } | ||
| 582 | } | 640 | } | ||
| 583 | 641 | ||||
| 642 | + | func TestUpdateAgentFromEventDoesNotCountProviderTurnStarts(t *testing.T) { | |||
| 643 | + | a := &Agent{} | |||
| 644 | + | a.setTurnCounts(4, 2) | |||
| 645 | + | a.setTurnState(TurnRunning, "turn-1") | |||
| 646 | + | persisted := 0 | |||
| 647 | + | a.persistFn = func(*Agent) error { persisted++; return nil } | |||
| 648 | + | ||||
| 649 | + | updateAgentFromEvent(a, NewEvent(EventTurnStarted, map[string]any{"step": float64(7), "nested_turn": true})) | |||
| 650 | + | ||||
| 651 | + | if got := a.LifetimeTurnsValue(); got != 4 { | |||
| 652 | + | t.Fatalf("lifetime turns = %d, want 4", got) | |||
| 653 | + | } | |||
| 654 | + | if got := a.CurrentRunTurnsValue(); got != 2 { | |||
| 655 | + | t.Fatalf("current-run turns = %d, want 2", got) | |||
| 656 | + | } | |||
| 657 | + | if got := a.TurnState(); got != TurnRunning { | |||
| 658 | + | t.Fatalf("turn state = %s, want %s", got, TurnRunning) | |||
| 659 | + | } | |||
| 660 | + | if persisted != 0 { | |||
| 661 | + | t.Fatalf("provider turn start persisted %d times, want 0", persisted) | |||
| 662 | + | } | |||
| 663 | + | } | |||
| 664 | + | ||||
| 665 | + | func TestUpdateAgentFromEventPersistsCanonicalTurnCounters(t *testing.T) { | |||
| 666 | + | stateDir := t.TempDir() | |||
| 667 | + | a := &Agent{ID: "counter-agent", stateDir: stateDir} | |||
| 668 | + | a.persistFn = func(agent *Agent) error { | |||
| 669 | + | if err := writeAgentMeta(stateDir, agent); err != nil { | |||
| 670 | + | t.Errorf("write agent metadata: %v", err) | |||
| 671 | + | return err | |||
| 672 | + | } | |||
| 673 | + | return nil | |||
| 674 | + | } | |||
| 675 | + | ||||
| 676 | + | updateAgentFromEvent(a, NewEvent(EventTurnStarted, map[string]any{ | |||
| 677 | + | "turn_id": "turn-3", | |||
| 678 | + | "lifetime_turns": 3, | |||
| 679 | + | "current_run_turns": 1, | |||
| 680 | + | })) | |||
| 681 | + | ||||
| 682 | + | meta, err := readAgentMeta(stateDir) | |||
| 683 | + | if err != nil { | |||
| 684 | + | t.Fatal(err) | |||
| 685 | + | } | |||
| 686 | + | if meta.LifetimeTurns != 3 || meta.CurrentRunTurns != 1 { | |||
| 687 | + | t.Fatalf("persisted counters = (%d, %d), want (3, 1)", meta.LifetimeTurns, meta.CurrentRunTurns) | |||
| 688 | + | } | |||
| 689 | + | if got := a.Snapshot(); got.LifetimeTurns != 3 || got.CurrentRunTurns != 1 { | |||
| 690 | + | t.Fatalf("snapshot counters = (%d, %d), want (3, 1)", got.LifetimeTurns, got.CurrentRunTurns) | |||
| 691 | + | } | |||
| 692 | + | } | |||
| 693 | + | ||||
| 694 | + | func TestRecordPersistenceErrorPreservesFirstFailure(t *testing.T) { | |||
| 695 | + | a := &Agent{} | |||
| 696 | + | a.recordPersistenceError(fmt.Errorf("first failure")) | |||
| 697 | + | a.recordPersistenceError(fmt.Errorf("later failure")) | |||
| 698 | + | if got := a.Snapshot().Err; !strings.Contains(got, "first failure") || strings.Contains(got, "later failure") { | |||
| 699 | + | t.Fatalf("recorded persistence error = %q, want only first failure", got) | |||
| 700 | + | } | |||
| 701 | + | } | |||
| 702 | + | ||||
| 703 | + | func TestRestoreResumePromptPreservesCommandIdentity(t *testing.T) { | |||
| 704 | + | a := &Agent{} | |||
| 705 | + | a.setResumePrompt("old prompt", time.Now()) | |||
| 706 | + | oldID := a.resumePromptCommandID() | |||
| 707 | + | previous := a.setResumePrompt("new prompt", time.Now()) | |||
| 708 | + | a.restoreResumePrompt(previous) | |||
| 709 | + | if got := a.resumePromptCommandID(); got != oldID { | |||
| 710 | + | t.Fatalf("restored command ID = %q, want %q", got, oldID) | |||
| 711 | + | } | |||
| 712 | + | } | |||
| 713 | + | ||||
| 714 | + | func TestEventMatchesPendingResumeRejectsIdentityLessRejection(t *testing.T) { | |||
| 715 | + | a := &Agent{resumePromptID: "command-1"} | |||
| 716 | + | if eventMatchesPendingResume(a, Event{Type: "error", Data: map[string]any{"code": "turn_rejected"}}) { | |||
| 717 | + | t.Fatal("identity-less turn rejection matched a pending prompt") | |||
| 718 | + | } | |||
| 719 | + | if !eventMatchesPendingResume(a, Event{Type: EventTurnStarted, Data: map[string]any{}}) { | |||
| 720 | + | t.Fatal("identity-less legacy turn.started did not match a pending prompt") | |||
| 721 | + | } | |||
| 722 | + | if !eventMatchesPendingResume(a, Event{Type: "error", Data: map[string]any{"code": "turn_rejected", "command_id": "command-1"}}) { | |||
| 723 | + | t.Fatal("matching turn rejection did not match a pending prompt") | |||
| 724 | + | } | |||
| 725 | + | } | |||
| 726 | + | ||||
| 727 | + | func TestEventCounterRejectsNegativeFractionalAndOverflowValues(t *testing.T) { | |||
| 728 | + | cases := []struct { | |||
| 729 | + | name string | |||
| 730 | + | value any | |||
| 731 | + | ok bool | |||
| 732 | + | }{ | |||
| 733 | + | {name: "valid integer", value: float64(3), ok: true}, | |||
| 734 | + | {name: "negative integer", value: int64(-1)}, | |||
| 735 | + | {name: "fractional float", value: 1.5}, | |||
| 736 | + | {name: "negative float", value: -1.0}, | |||
| 737 | + | {name: "fractional json number", value: json.Number("1.5")}, | |||
| 738 | + | {name: "overflow float", value: float64(^uint(0) >> 1)}, | |||
| 739 | + | } | |||
| 740 | + | for _, tc := range cases { | |||
| 741 | + | t.Run(tc.name, func(t *testing.T) { | |||
| 742 | + | got, ok := eventCounter(map[string]any{"counter": tc.value}, "counter") | |||
| 743 | + | if ok != tc.ok { | |||
| 744 | + | t.Fatalf("eventCounter ok = %t, want %t (value %v, got %d)", ok, tc.ok, tc.value, got) | |||
| 745 | + | } | |||
| 746 | + | }) | |||
| 747 | + | } | |||
| 748 | + | } | |||
| 749 | + | ||||
| 584 | func indexOf(xs []string, want string) int { | 750 | func indexOf(xs []string, want string) int { | ||
| 585 | for i, x := range xs { | 751 | for i, x := range xs { | ||
| 586 | if x == want { | 752 | if x == want { | ||
M packages/agent/subagents/scheduler.go +17 -6
| @@ -2,6 +2,7 @@ package subagents | |||
| 2 | 2 | ||
| 3 | 3 | import ( | |
| 4 | 4 | "context" | |
| 5 | + | "errors" | |
| 5 | 6 | "fmt" | |
| 6 | 7 | "os" | |
| 7 | 8 | "path/filepath" | |
| @@ -184,7 +185,13 @@ func (f *Supervisor) schedule() { | |||
| 184 | 185 | a.activity = "starting" | |
| 185 | 186 | a.mu.Unlock() | |
| 186 | 187 | a.setProcessState(ProcessStarting) | |
| 187 | - | a.setTurnState(TurnIdle, "") | |
| 188 | + | prompt, _ := a.ResumePromptInfo() | |
| 189 | + | initialTurn := (!a.Resuming && strings.TrimSpace(a.Task) != "") || prompt != "" | |
| 190 | + | if initialTurn { | |
| 191 | + | a.setTurnState(TurnQueued, "") | |
| 192 | + | } else { | |
| 193 | + | a.setTurnState(TurnIdle, "") | |
| 194 | + | } | |
| 188 | 195 | a.incrementAttempt() | |
| 189 | 196 | ||
| 190 | 197 | f.active++ | |
| @@ -374,13 +381,17 @@ func (f *Supervisor) releaseCapacity(a *Agent) { | |||
| 374 | 381 | f.schedule() | |
| 375 | 382 | } | |
| 376 | 383 | ||
| 377 | - | func (f *Supervisor) persistAgent(a *Agent) { | |
| 384 | + | func (f *Supervisor) persistAgent(a *Agent) error { | |
| 378 | 385 | if a == nil { | |
| 379 | - | return | |
| 386 | + | return errors.New("subagents: nil agent") | |
| 380 | 387 | } | |
| 381 | 388 | stateDir := a.stateDirectory(f.cfg.Root) | |
| 382 | - | if _, err := os.Stat(stateDir); err != nil { | |
| 383 | - | return | |
| 389 | + | if err := writeAgentMeta(stateDir, a); err != nil { | |
| 390 | + | // Lifecycle callbacks cannot all return an error to their caller. Keep | |
| 391 | + | // the failure visible in the agent snapshot instead of silently | |
| 392 | + | // allowing a queued prompt or counter to disappear on restart. | |
| 393 | + | a.recordPersistenceError(err) | |
| 394 | + | return err | |
| 384 | 395 | } | |
| 385 | - | _ = writeAgentMeta(stateDir, a) | |
| 396 | + | return nil | |
| 386 | 397 | } | |
| @@ -2,6 +2,7 @@ package subagents | |||||
| 2 | 2 | ||||
| 3 | import ( | 3 | import ( | ||
| 4 | "context" | 4 | "context" | ||
| 5 | + | "errors" | |||
| 5 | "fmt" | 6 | "fmt" | ||
| 6 | "os" | 7 | "os" | ||
| 7 | "path/filepath" | 8 | "path/filepath" | ||
| @@ -184,7 +185,13 @@ func (f *Supervisor) schedule() { | |||||
| 184 | a.activity = "starting" | 185 | a.activity = "starting" | ||
| 185 | a.mu.Unlock() | 186 | a.mu.Unlock() | ||
| 186 | a.setProcessState(ProcessStarting) | 187 | a.setProcessState(ProcessStarting) | ||
| 187 | - | a.setTurnState(TurnIdle, "") | 188 | + | prompt, _ := a.ResumePromptInfo() |
| 189 | + | initialTurn := (!a.Resuming && strings.TrimSpace(a.Task) != "") || prompt != "" | |||
| 190 | + | if initialTurn { | |||
| 191 | + | a.setTurnState(TurnQueued, "") | |||
| 192 | + | } else { | |||
| 193 | + | a.setTurnState(TurnIdle, "") | |||
| 194 | + | } | |||
| 188 | a.incrementAttempt() | 195 | a.incrementAttempt() | ||
| 189 | 196 | ||||
| 190 | f.active++ | 197 | f.active++ | ||
| @@ -374,13 +381,17 @@ func (f *Supervisor) releaseCapacity(a *Agent) { | |||||
| 374 | f.schedule() | 381 | f.schedule() | ||
| 375 | } | 382 | } | ||
| 376 | 383 | ||||
| 377 | - | func (f *Supervisor) persistAgent(a *Agent) { | 384 | + | func (f *Supervisor) persistAgent(a *Agent) error { |
| 378 | if a == nil { | 385 | if a == nil { | ||
| 379 | - | return | 386 | + | return errors.New("subagents: nil agent") |
| 380 | } | 387 | } | ||
| 381 | stateDir := a.stateDirectory(f.cfg.Root) | 388 | stateDir := a.stateDirectory(f.cfg.Root) | ||
| 382 | - | if _, err := os.Stat(stateDir); err != nil { | 389 | + | if err := writeAgentMeta(stateDir, a); err != nil { |
| 383 | - | return | 390 | + | // Lifecycle callbacks cannot all return an error to their caller. Keep |
| 391 | + | // the failure visible in the agent snapshot instead of silently | |||
| 392 | + | // allowing a queued prompt or counter to disappear on restart. | |||
| 393 | + | a.recordPersistenceError(err) | |||
| 394 | + | return err | |||
| 384 | } | 395 | } | ||
| 385 | - | _ = writeAgentMeta(stateDir, a) | 396 | + | return nil |
| 386 | } | 397 | } | ||
M packages/agent/subagents/supervisor.go +6 -1
| @@ -547,6 +547,7 @@ func (f *Supervisor) SpawnReq(ctx context.Context, req SpawnRequest) (*Agent, er | |||
| 547 | 547 | } | |
| 548 | 548 | a.ctx, a.cancel = runCtx, cancel | |
| 549 | 549 | a.persistFn = f.persistAgent | |
| 550 | + | a.setOnTurnIdle(func() { f.dispatchQueuedResumeWithTimeout(a) }) | |
| 550 | 551 | a.workspaceCleanup = func() error { return workspace.Cleanup(context.Background()) } | |
| 551 | 552 | a.workspaceCapture = func() (WorkspaceCapture, error) { return workspace.Capture(context.Background()) } | |
| 552 | 553 | a.runner = f.cfg.NewRunner(a) | |
| @@ -1079,6 +1080,8 @@ type AgentSnapshot struct { | |||
| 1079 | 1080 | ProcessState ProcessState | |
| 1080 | 1081 | TurnState TurnState | |
| 1081 | 1082 | CurrentTurnID string | |
| 1083 | + | LifetimeTurns int | |
| 1084 | + | CurrentRunTurns int | |
| 1082 | 1085 | Attempt int | |
| 1083 | 1086 | Activity string | |
| 1084 | 1087 | Started time.Time | |
| @@ -1141,6 +1144,8 @@ func (a *Agent) Snapshot() AgentSnapshot { | |||
| 1141 | 1144 | processState := a.processState | |
| 1142 | 1145 | turnState := a.turnState | |
| 1143 | 1146 | currentTurnID := a.currentTurnID | |
| 1147 | + | lifetimeTurns := a.LifetimeTurns | |
| 1148 | + | currentRunTurns := a.CurrentRunTurns | |
| 1144 | 1149 | attempt := a.Attempt | |
| 1145 | 1150 | lastActivity := a.lastActivity | |
| 1146 | 1151 | updatedAt := a.updatedAt | |
| @@ -1152,7 +1157,7 @@ func (a *Agent) Snapshot() AgentSnapshot { | |||
| 1152 | 1157 | return AgentSnapshot{ | |
| 1153 | 1158 | ID: a.ID, Task: a.Task, Dir: a.Dir, | |
| 1154 | 1159 | Status: status, ProcessState: processState, TurnState: turnState, | |
| 1155 | - | CurrentTurnID: currentTurnID, Attempt: attempt, Activity: activity, | |
| 1160 | + | CurrentTurnID: currentTurnID, LifetimeTurns: lifetimeTurns, CurrentRunTurns: currentRunTurns, Attempt: attempt, Activity: activity, | |
| 1156 | 1161 | Started: a.Started, Finished: finished, CreatedAt: a.Started, UpdatedAt: updatedAt, LastActivity: lastActivity, | |
| 1157 | 1162 | Err: errStr, Tail: tail, Lines: lines, | |
| 1158 | 1163 | LastAssistant: lastAssistant, Result: result, ResultRef: resultRef, | |
| @@ -547,6 +547,7 @@ func (f *Supervisor) SpawnReq(ctx context.Context, req SpawnRequest) (*Agent, er | |||||
| 547 | } | 547 | } | ||
| 548 | a.ctx, a.cancel = runCtx, cancel | 548 | a.ctx, a.cancel = runCtx, cancel | ||
| 549 | a.persistFn = f.persistAgent | 549 | a.persistFn = f.persistAgent | ||
| 550 | + | a.setOnTurnIdle(func() { f.dispatchQueuedResumeWithTimeout(a) }) | |||
| 550 | a.workspaceCleanup = func() error { return workspace.Cleanup(context.Background()) } | 551 | a.workspaceCleanup = func() error { return workspace.Cleanup(context.Background()) } | ||
| 551 | a.workspaceCapture = func() (WorkspaceCapture, error) { return workspace.Capture(context.Background()) } | 552 | a.workspaceCapture = func() (WorkspaceCapture, error) { return workspace.Capture(context.Background()) } | ||
| 552 | a.runner = f.cfg.NewRunner(a) | 553 | a.runner = f.cfg.NewRunner(a) | ||
| @@ -1079,6 +1080,8 @@ type AgentSnapshot struct { | |||||
| 1079 | ProcessState ProcessState | 1080 | ProcessState ProcessState | ||
| 1080 | TurnState TurnState | 1081 | TurnState TurnState | ||
| 1081 | CurrentTurnID string | 1082 | CurrentTurnID string | ||
| 1083 | + | LifetimeTurns int | |||
| 1084 | + | CurrentRunTurns int | |||
| 1082 | Attempt int | 1085 | Attempt int | ||
| 1083 | Activity string | 1086 | Activity string | ||
| 1084 | Started time.Time | 1087 | Started time.Time | ||
| @@ -1141,6 +1144,8 @@ func (a *Agent) Snapshot() AgentSnapshot { | |||||
| 1141 | processState := a.processState | 1144 | processState := a.processState | ||
| 1142 | turnState := a.turnState | 1145 | turnState := a.turnState | ||
| 1143 | currentTurnID := a.currentTurnID | 1146 | currentTurnID := a.currentTurnID | ||
| 1147 | + | lifetimeTurns := a.LifetimeTurns | |||
| 1148 | + | currentRunTurns := a.CurrentRunTurns | |||
| 1144 | attempt := a.Attempt | 1149 | attempt := a.Attempt | ||
| 1145 | lastActivity := a.lastActivity | 1150 | lastActivity := a.lastActivity | ||
| 1146 | updatedAt := a.updatedAt | 1151 | updatedAt := a.updatedAt | ||
| @@ -1152,7 +1157,7 @@ func (a *Agent) Snapshot() AgentSnapshot { | |||||
| 1152 | return AgentSnapshot{ | 1157 | return AgentSnapshot{ | ||
| 1153 | ID: a.ID, Task: a.Task, Dir: a.Dir, | 1158 | ID: a.ID, Task: a.Task, Dir: a.Dir, | ||
| 1154 | Status: status, ProcessState: processState, TurnState: turnState, | 1159 | Status: status, ProcessState: processState, TurnState: turnState, | ||
| 1155 | - | CurrentTurnID: currentTurnID, Attempt: attempt, Activity: activity, | 1160 | + | CurrentTurnID: currentTurnID, LifetimeTurns: lifetimeTurns, CurrentRunTurns: currentRunTurns, Attempt: attempt, Activity: activity, |
| 1156 | Started: a.Started, Finished: finished, CreatedAt: a.Started, UpdatedAt: updatedAt, LastActivity: lastActivity, | 1161 | Started: a.Started, Finished: finished, CreatedAt: a.Started, UpdatedAt: updatedAt, LastActivity: lastActivity, | ||
| 1157 | Err: errStr, Tail: tail, Lines: lines, | 1162 | Err: errStr, Tail: tail, Lines: lines, | ||
| 1158 | LastAssistant: lastAssistant, Result: result, ResultRef: resultRef, | 1163 | LastAssistant: lastAssistant, Result: result, ResultRef: resultRef, | ||
A packages/agent/subagents/turn_classification.go +93 -0
| @@ -0,0 +1,93 @@ | |||
| 1 | + | package subagents | |
| 2 | + | ||
| 3 | + | import "fmt" | |
| 4 | + | ||
| 5 | + | // turnEventClass identifies the boundary that owns a turn event. The worker | |
| 6 | + | // daemon's turn.started event is the delegated message boundary; provider | |
| 7 | + | // turn_start events are nested model-loop activity. New workers carry stable | |
| 8 | + | // lifetime/current counters on the delegated boundary and nested_turn on the | |
| 9 | + | // provider boundary. The fallbacks preserve protocol-v1 logs, which used | |
| 10 | + | // turn.started without counters for delegated turns and turn_start for nested | |
| 11 | + | // turns. | |
| 12 | + | type turnEventClass uint8 | |
| 13 | + | ||
| 14 | + | const ( | |
| 15 | + | turnEventUnknown turnEventClass = iota | |
| 16 | + | turnEventDelegated | |
| 17 | + | turnEventNested | |
| 18 | + | ) | |
| 19 | + | ||
| 20 | + | func classifyTurnEvent(ev Event) turnEventClass { | |
| 21 | + | if nested, ok := ev.Data["nested_turn"].(bool); ok && nested { | |
| 22 | + | return turnEventNested | |
| 23 | + | } | |
| 24 | + | switch ev.Type { | |
| 25 | + | case "turn_start": | |
| 26 | + | return turnEventNested | |
| 27 | + | case EventTurnStarted: | |
| 28 | + | // v1 turn.started without counters is the delegated boundary. A | |
| 29 | + | // current worker marks nested canonical events explicitly above. | |
| 30 | + | return turnEventDelegated | |
| 31 | + | case "turn_end": | |
| 32 | + | if _, ok := eventCounter(ev.Data, "step"); ok { | |
| 33 | + | return turnEventDelegated | |
| 34 | + | } | |
| 35 | + | return turnEventNested | |
| 36 | + | } | |
| 37 | + | return turnEventUnknown | |
| 38 | + | } | |
| 39 | + | ||
| 40 | + | func isDelegatedTurnStart(ev Event) bool { | |
| 41 | + | return ev.Type == EventTurnStarted && classifyTurnEvent(ev) == turnEventDelegated | |
| 42 | + | } | |
| 43 | + | ||
| 44 | + | // IsDelegatedTurnStart is shared with the worker's replay bootstrap so live, | |
| 45 | + | // replayed, and protocol-v1 event handling use the same boundary rules. | |
| 46 | + | func IsDelegatedTurnStart(ev Event) bool { | |
| 47 | + | return isDelegatedTurnStart(ev) | |
| 48 | + | } | |
| 49 | + | ||
| 50 | + | func isDelegatedTurnEnd(ev Event) bool { | |
| 51 | + | return ev.Type == "turn_end" && classifyTurnEvent(ev) == turnEventDelegated | |
| 52 | + | } | |
| 53 | + | ||
| 54 | + | func eventTurnID(ev Event) string { | |
| 55 | + | if ev.TurnID != "" { | |
| 56 | + | return ev.TurnID | |
| 57 | + | } | |
| 58 | + | if step, ok := eventCounter(ev.Data, "step"); ok { | |
| 59 | + | return fmt.Sprintf("turn-%d", step) | |
| 60 | + | } | |
| 61 | + | return "" | |
| 62 | + | } | |
| 63 | + | ||
| 64 | + | func userMessageText(ev Event) string { | |
| 65 | + | if ev.Type != "user_message" { | |
| 66 | + | return "" | |
| 67 | + | } | |
| 68 | + | content := ev.Data["content"] | |
| 69 | + | var text string | |
| 70 | + | appendBlock := func(fields map[string]any) { | |
| 71 | + | if typ, _ := fields["type"].(string); typ == "text" { | |
| 72 | + | if value, _ := fields["text"].(string); value != "" { | |
| 73 | + | if text != "" { | |
| 74 | + | text += "\n" | |
| 75 | + | } | |
| 76 | + | text += value | |
| 77 | + | } | |
| 78 | + | } | |
| 79 | + | } | |
| 80 | + | switch blocks := content.(type) { | |
| 81 | + | case []any: | |
| 82 | + | for _, block := range blocks { | |
| 83 | + | if fields, ok := block.(map[string]any); ok { | |
| 84 | + | appendBlock(fields) | |
| 85 | + | } | |
| 86 | + | } | |
| 87 | + | case []map[string]any: | |
| 88 | + | for _, fields := range blocks { | |
| 89 | + | appendBlock(fields) | |
| 90 | + | } | |
| 91 | + | } | |
| 92 | + | return text | |
| 93 | + | } | |
| @@ -0,0 +1,93 @@ | |||||
| 1 | + | package subagents | |||
| 2 | + | ||||
| 3 | + | import "fmt" | |||
| 4 | + | ||||
| 5 | + | // turnEventClass identifies the boundary that owns a turn event. The worker | |||
| 6 | + | // daemon's turn.started event is the delegated message boundary; provider | |||
| 7 | + | // turn_start events are nested model-loop activity. New workers carry stable | |||
| 8 | + | // lifetime/current counters on the delegated boundary and nested_turn on the | |||
| 9 | + | // provider boundary. The fallbacks preserve protocol-v1 logs, which used | |||
| 10 | + | // turn.started without counters for delegated turns and turn_start for nested | |||
| 11 | + | // turns. | |||
| 12 | + | type turnEventClass uint8 | |||
| 13 | + | ||||
| 14 | + | const ( | |||
| 15 | + | turnEventUnknown turnEventClass = iota | |||
| 16 | + | turnEventDelegated | |||
| 17 | + | turnEventNested | |||
| 18 | + | ) | |||
| 19 | + | ||||
| 20 | + | func classifyTurnEvent(ev Event) turnEventClass { | |||
| 21 | + | if nested, ok := ev.Data["nested_turn"].(bool); ok && nested { | |||
| 22 | + | return turnEventNested | |||
| 23 | + | } | |||
| 24 | + | switch ev.Type { | |||
| 25 | + | case "turn_start": | |||
| 26 | + | return turnEventNested | |||
| 27 | + | case EventTurnStarted: | |||
| 28 | + | // v1 turn.started without counters is the delegated boundary. A | |||
| 29 | + | // current worker marks nested canonical events explicitly above. | |||
| 30 | + | return turnEventDelegated | |||
| 31 | + | case "turn_end": | |||
| 32 | + | if _, ok := eventCounter(ev.Data, "step"); ok { | |||
| 33 | + | return turnEventDelegated | |||
| 34 | + | } | |||
| 35 | + | return turnEventNested | |||
| 36 | + | } | |||
| 37 | + | return turnEventUnknown | |||
| 38 | + | } | |||
| 39 | + | ||||
| 40 | + | func isDelegatedTurnStart(ev Event) bool { | |||
| 41 | + | return ev.Type == EventTurnStarted && classifyTurnEvent(ev) == turnEventDelegated | |||
| 42 | + | } | |||
| 43 | + | ||||
| 44 | + | // IsDelegatedTurnStart is shared with the worker's replay bootstrap so live, | |||
| 45 | + | // replayed, and protocol-v1 event handling use the same boundary rules. | |||
| 46 | + | func IsDelegatedTurnStart(ev Event) bool { | |||
| 47 | + | return isDelegatedTurnStart(ev) | |||
| 48 | + | } | |||
| 49 | + | ||||
| 50 | + | func isDelegatedTurnEnd(ev Event) bool { | |||
| 51 | + | return ev.Type == "turn_end" && classifyTurnEvent(ev) == turnEventDelegated | |||
| 52 | + | } | |||
| 53 | + | ||||
| 54 | + | func eventTurnID(ev Event) string { | |||
| 55 | + | if ev.TurnID != "" { | |||
| 56 | + | return ev.TurnID | |||
| 57 | + | } | |||
| 58 | + | if step, ok := eventCounter(ev.Data, "step"); ok { | |||
| 59 | + | return fmt.Sprintf("turn-%d", step) | |||
| 60 | + | } | |||
| 61 | + | return "" | |||
| 62 | + | } | |||
| 63 | + | ||||
| 64 | + | func userMessageText(ev Event) string { | |||
| 65 | + | if ev.Type != "user_message" { | |||
| 66 | + | return "" | |||
| 67 | + | } | |||
| 68 | + | content := ev.Data["content"] | |||
| 69 | + | var text string | |||
| 70 | + | appendBlock := func(fields map[string]any) { | |||
| 71 | + | if typ, _ := fields["type"].(string); typ == "text" { | |||
| 72 | + | if value, _ := fields["text"].(string); value != "" { | |||
| 73 | + | if text != "" { | |||
| 74 | + | text += "\n" | |||
| 75 | + | } | |||
| 76 | + | text += value | |||
| 77 | + | } | |||
| 78 | + | } | |||
| 79 | + | } | |||
| 80 | + | switch blocks := content.(type) { | |||
| 81 | + | case []any: | |||
| 82 | + | for _, block := range blocks { | |||
| 83 | + | if fields, ok := block.(map[string]any); ok { | |||
| 84 | + | appendBlock(fields) | |||
| 85 | + | } | |||
| 86 | + | } | |||
| 87 | + | case []map[string]any: | |||
| 88 | + | for _, fields := range blocks { | |||
| 89 | + | appendBlock(fields) | |||
| 90 | + | } | |||
| 91 | + | } | |||
| 92 | + | return text | |||
| 93 | + | } | |||
A packages/agent/subagents/turn_classification_test.go +63 -0
| @@ -0,0 +1,63 @@ | |||
| 1 | + | package subagents | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "testing" | |
| 5 | + | "time" | |
| 6 | + | ) | |
| 7 | + | ||
| 8 | + | func TestTurnClassificationUsesStableBoundaryFieldsAcrossProtocolVersions(t *testing.T) { | |
| 9 | + | cases := []struct { | |
| 10 | + | name string | |
| 11 | + | event Event | |
| 12 | + | class turnEventClass | |
| 13 | + | delegated bool | |
| 14 | + | }{ | |
| 15 | + | { | |
| 16 | + | name: "current delegated", | |
| 17 | + | event: NewEvent(EventTurnStarted, map[string]any{ | |
| 18 | + | "lifetime_turns": 4, | |
| 19 | + | "current_run_turns": 2, | |
| 20 | + | }), | |
| 21 | + | class: turnEventDelegated, | |
| 22 | + | delegated: true, | |
| 23 | + | }, | |
| 24 | + | { | |
| 25 | + | name: "current nested", | |
| 26 | + | event: NewEvent(EventTurnStarted, map[string]any{ | |
| 27 | + | "nested_turn": true, | |
| 28 | + | "step": 3, | |
| 29 | + | }), | |
| 30 | + | class: turnEventNested, | |
| 31 | + | }, | |
| 32 | + | { | |
| 33 | + | name: "protocol v1 delegated fallback", | |
| 34 | + | event: Event{Type: EventTurnStarted, Version: 1, Time: time.Now()}, | |
| 35 | + | class: turnEventDelegated, | |
| 36 | + | delegated: true, | |
| 37 | + | }, | |
| 38 | + | { | |
| 39 | + | name: "protocol v1 nested turn_start", | |
| 40 | + | event: Event{Type: "turn_start", Version: 1, Time: time.Now()}, | |
| 41 | + | class: turnEventNested, | |
| 42 | + | }, | |
| 43 | + | } | |
| 44 | + | for _, tc := range cases { | |
| 45 | + | t.Run(tc.name, func(t *testing.T) { | |
| 46 | + | if got := classifyTurnEvent(tc.event); got != tc.class { | |
| 47 | + | t.Fatalf("classifyTurnEvent = %d, want %d", got, tc.class) | |
| 48 | + | } | |
| 49 | + | if got := isDelegatedTurnStart(tc.event); got != tc.delegated { | |
| 50 | + | t.Fatalf("isDelegatedTurnStart = %v, want %v", got, tc.delegated) | |
| 51 | + | } | |
| 52 | + | }) | |
| 53 | + | } | |
| 54 | + | } | |
| 55 | + | ||
| 56 | + | func TestTurnClassificationKeepsNestedAndDelegatedEndsSeparate(t *testing.T) { | |
| 57 | + | if isDelegatedTurnEnd(NewEvent("turn_end", map[string]any{"stop": "tool_use"})) { | |
| 58 | + | t.Fatal("provider turn_end was classified as delegated") | |
| 59 | + | } | |
| 60 | + | if !isDelegatedTurnEnd(NewEvent("turn_end", map[string]any{"step": 2})) { | |
| 61 | + | t.Fatal("worker turn_end was not classified as delegated") | |
| 62 | + | } | |
| 63 | + | } | |
| @@ -0,0 +1,63 @@ | |||||
| 1 | + | package subagents | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "testing" | |||
| 5 | + | "time" | |||
| 6 | + | ) | |||
| 7 | + | ||||
| 8 | + | func TestTurnClassificationUsesStableBoundaryFieldsAcrossProtocolVersions(t *testing.T) { | |||
| 9 | + | cases := []struct { | |||
| 10 | + | name string | |||
| 11 | + | event Event | |||
| 12 | + | class turnEventClass | |||
| 13 | + | delegated bool | |||
| 14 | + | }{ | |||
| 15 | + | { | |||
| 16 | + | name: "current delegated", | |||
| 17 | + | event: NewEvent(EventTurnStarted, map[string]any{ | |||
| 18 | + | "lifetime_turns": 4, | |||
| 19 | + | "current_run_turns": 2, | |||
| 20 | + | }), | |||
| 21 | + | class: turnEventDelegated, | |||
| 22 | + | delegated: true, | |||
| 23 | + | }, | |||
| 24 | + | { | |||
| 25 | + | name: "current nested", | |||
| 26 | + | event: NewEvent(EventTurnStarted, map[string]any{ | |||
| 27 | + | "nested_turn": true, | |||
| 28 | + | "step": 3, | |||
| 29 | + | }), | |||
| 30 | + | class: turnEventNested, | |||
| 31 | + | }, | |||
| 32 | + | { | |||
| 33 | + | name: "protocol v1 delegated fallback", | |||
| 34 | + | event: Event{Type: EventTurnStarted, Version: 1, Time: time.Now()}, | |||
| 35 | + | class: turnEventDelegated, | |||
| 36 | + | delegated: true, | |||
| 37 | + | }, | |||
| 38 | + | { | |||
| 39 | + | name: "protocol v1 nested turn_start", | |||
| 40 | + | event: Event{Type: "turn_start", Version: 1, Time: time.Now()}, | |||
| 41 | + | class: turnEventNested, | |||
| 42 | + | }, | |||
| 43 | + | } | |||
| 44 | + | for _, tc := range cases { | |||
| 45 | + | t.Run(tc.name, func(t *testing.T) { | |||
| 46 | + | if got := classifyTurnEvent(tc.event); got != tc.class { | |||
| 47 | + | t.Fatalf("classifyTurnEvent = %d, want %d", got, tc.class) | |||
| 48 | + | } | |||
| 49 | + | if got := isDelegatedTurnStart(tc.event); got != tc.delegated { | |||
| 50 | + | t.Fatalf("isDelegatedTurnStart = %v, want %v", got, tc.delegated) | |||
| 51 | + | } | |||
| 52 | + | }) | |||
| 53 | + | } | |||
| 54 | + | } | |||
| 55 | + | ||||
| 56 | + | func TestTurnClassificationKeepsNestedAndDelegatedEndsSeparate(t *testing.T) { | |||
| 57 | + | if isDelegatedTurnEnd(NewEvent("turn_end", map[string]any{"stop": "tool_use"})) { | |||
| 58 | + | t.Fatal("provider turn_end was classified as delegated") | |||
| 59 | + | } | |||
| 60 | + | if !isDelegatedTurnEnd(NewEvent("turn_end", map[string]any{"step": 2})) { | |||
| 61 | + | t.Fatal("worker turn_end was not classified as delegated") | |||
| 62 | + | } | |||
| 63 | + | } | |||
M packages/agent/tools/subagent_resume.go +7 -5
| @@ -10,10 +10,12 @@ import ( | |||
| 10 | 10 | "github.com/bnema/zut/packages/core" | |
| 11 | 11 | ) | |
| 12 | 12 | ||
| 13 | - | // SubagentResumeTool gives a completed or stopped sub-agent a follow-up turn | |
| 14 | - | // while preserving its existing session context. For an idle live worker, the | |
| 15 | - | // follow-up is delivered directly. For a terminal worker, the supervisor | |
| 16 | - | // restarts the retained session with the follow-up as its initial turn. | |
| 13 | + | // SubagentResumeTool gives a sub-agent a follow-up turn while preserving its | |
| 14 | + | // existing session context. Every explicit follow-up starts a fresh max_turns | |
| 15 | + | // budget. For an idle live worker, the follow-up is delivered directly; for an | |
| 16 | + | // active live worker, it is queued for the next available message turn; for a | |
| 17 | + | // terminal worker, the supervisor restarts the retained session with the | |
| 18 | + | // follow-up as its initial turn. | |
| 17 | 19 | type SubagentResumeTool struct { | |
| 18 | 20 | Supervisor *subagents.Supervisor | |
| 19 | 21 | Enabled func() bool | |
| @@ -48,7 +50,7 @@ const subagentResumeSchema = `{ | |||
| 48 | 50 | func (t *SubagentResumeTool) Name() string { return "subagent_resume" } | |
| 49 | 51 | ||
| 50 | 52 | func (t *SubagentResumeTool) Description() string { | |
| 51 | - | return "Send a follow-up to an idle sub-agent, or restart a stopped one with its prior session context preserved." | |
| 53 | + | return "Continue a sub-agent with a fresh turn budget and its retained session; deliver immediately when idle, queue while active, or restart a stopped worker." | |
| 52 | 54 | } | |
| 53 | 55 | ||
| 54 | 56 | func (t *SubagentResumeTool) Schema() json.RawMessage { | |
| @@ -10,10 +10,12 @@ import ( | |||||
| 10 | "github.com/bnema/zut/packages/core" | 10 | "github.com/bnema/zut/packages/core" | ||
| 11 | ) | 11 | ) | ||
| 12 | 12 | ||||
| 13 | - | // SubagentResumeTool gives a completed or stopped sub-agent a follow-up turn | 13 | + | // SubagentResumeTool gives a sub-agent a follow-up turn while preserving its |
| 14 | - | // while preserving its existing session context. For an idle live worker, the | 14 | + | // existing session context. Every explicit follow-up starts a fresh max_turns |
| 15 | - | // follow-up is delivered directly. For a terminal worker, the supervisor | 15 | + | // budget. For an idle live worker, the follow-up is delivered directly; for an |
| 16 | - | // restarts the retained session with the follow-up as its initial turn. | 16 | + | // active live worker, it is queued for the next available message turn; for a |
| 17 | + | // terminal worker, the supervisor restarts the retained session with the | |||
| 18 | + | // follow-up as its initial turn. | |||
| 17 | type SubagentResumeTool struct { | 19 | type SubagentResumeTool struct { | ||
| 18 | Supervisor *subagents.Supervisor | 20 | Supervisor *subagents.Supervisor | ||
| 19 | Enabled func() bool | 21 | Enabled func() bool | ||
| @@ -48,7 +50,7 @@ const subagentResumeSchema = `{ | |||||
| 48 | func (t *SubagentResumeTool) Name() string { return "subagent_resume" } | 50 | func (t *SubagentResumeTool) Name() string { return "subagent_resume" } | ||
| 49 | 51 | ||||
| 50 | func (t *SubagentResumeTool) Description() string { | 52 | func (t *SubagentResumeTool) Description() string { | ||
| 51 | - | return "Send a follow-up to an idle sub-agent, or restart a stopped one with its prior session context preserved." | 53 | + | return "Continue a sub-agent with a fresh turn budget and its retained session; deliver immediately when idle, queue while active, or restart a stopped worker." |
| 52 | } | 54 | } | ||
| 53 | 55 | ||||
| 54 | func (t *SubagentResumeTool) Schema() json.RawMessage { | 56 | func (t *SubagentResumeTool) Schema() json.RawMessage { | ||
M packages/agent/tools/subagent_status.go +20 -16
| @@ -34,15 +34,17 @@ type subagentStatusResponse struct { | |||
| 34 | 34 | } | |
| 35 | 35 | ||
| 36 | 36 | type subagentStatusEntry struct { | |
| 37 | - | ID string `json:"agent_id"` | |
| 38 | - | State string `json:"state"` | |
| 39 | - | ProcessState string `json:"process_state"` | |
| 40 | - | TurnState string `json:"turn_state"` | |
| 41 | - | StartedAt time.Time `json:"started_at"` | |
| 42 | - | UpdatedAt time.Time `json:"updated_at"` | |
| 43 | - | FinishedAt *time.Time `json:"finished_at,omitempty"` | |
| 44 | - | TaskSummary string `json:"task_summary,omitempty"` | |
| 45 | - | Result *subagentStatusResult `json:"result,omitempty"` | |
| 37 | + | ID string `json:"agent_id"` | |
| 38 | + | State string `json:"state"` | |
| 39 | + | ProcessState string `json:"process_state"` | |
| 40 | + | TurnState string `json:"turn_state"` | |
| 41 | + | LifetimeTurns int `json:"lifetime_turns"` | |
| 42 | + | CurrentRunTurns int `json:"current_run_turns"` | |
| 43 | + | StartedAt time.Time `json:"started_at"` | |
| 44 | + | UpdatedAt time.Time `json:"updated_at"` | |
| 45 | + | FinishedAt *time.Time `json:"finished_at,omitempty"` | |
| 46 | + | TaskSummary string `json:"task_summary,omitempty"` | |
| 47 | + | Result *subagentStatusResult `json:"result,omitempty"` | |
| 46 | 48 | } | |
| 47 | 49 | ||
| 48 | 50 | type subagentStatusResult struct { | |
| @@ -132,13 +134,15 @@ func findSubagentStatusSnapshot(snapshots []subagents.AgentSnapshot, id string) | |||
| 132 | 134 | ||
| 133 | 135 | func publicSubagentStatus(snapshot subagents.AgentSnapshot) subagentStatusEntry { | |
| 134 | 136 | entry := subagentStatusEntry{ | |
| 135 | - | ID: snapshot.ID, | |
| 136 | - | State: publicSubagentState(snapshot), | |
| 137 | - | ProcessState: string(snapshot.ProcessState), | |
| 138 | - | TurnState: string(snapshot.TurnState), | |
| 139 | - | StartedAt: snapshot.Started, | |
| 140 | - | UpdatedAt: snapshot.UpdatedAt, | |
| 141 | - | TaskSummary: summarizeSubagentTask(snapshot.Task), | |
| 137 | + | ID: snapshot.ID, | |
| 138 | + | State: publicSubagentState(snapshot), | |
| 139 | + | ProcessState: string(snapshot.ProcessState), | |
| 140 | + | TurnState: string(snapshot.TurnState), | |
| 141 | + | LifetimeTurns: snapshot.LifetimeTurns, | |
| 142 | + | CurrentRunTurns: snapshot.CurrentRunTurns, | |
| 143 | + | StartedAt: snapshot.Started, | |
| 144 | + | UpdatedAt: snapshot.UpdatedAt, | |
| 145 | + | TaskSummary: summarizeSubagentTask(snapshot.Task), | |
| 142 | 146 | } | |
| 143 | 147 | if !snapshot.Finished.IsZero() { | |
| 144 | 148 | finished := snapshot.Finished | |
| @@ -34,15 +34,17 @@ type subagentStatusResponse struct { | |||||
| 34 | } | 34 | } | ||
| 35 | 35 | ||||
| 36 | type subagentStatusEntry struct { | 36 | type subagentStatusEntry struct { | ||
| 37 | - | ID string `json:"agent_id"` | 37 | + | ID string `json:"agent_id"` |
| 38 | - | State string `json:"state"` | 38 | + | State string `json:"state"` |
| 39 | - | ProcessState string `json:"process_state"` | 39 | + | ProcessState string `json:"process_state"` |
| 40 | - | TurnState string `json:"turn_state"` | 40 | + | TurnState string `json:"turn_state"` |
| 41 | - | StartedAt time.Time `json:"started_at"` | 41 | + | LifetimeTurns int `json:"lifetime_turns"` |
| 42 | - | UpdatedAt time.Time `json:"updated_at"` | 42 | + | CurrentRunTurns int `json:"current_run_turns"` |
| 43 | - | FinishedAt *time.Time `json:"finished_at,omitempty"` | 43 | + | StartedAt time.Time `json:"started_at"` |
| 44 | - | TaskSummary string `json:"task_summary,omitempty"` | 44 | + | UpdatedAt time.Time `json:"updated_at"` |
| 45 | - | Result *subagentStatusResult `json:"result,omitempty"` | 45 | + | FinishedAt *time.Time `json:"finished_at,omitempty"` |
| 46 | + | TaskSummary string `json:"task_summary,omitempty"` | |||
| 47 | + | Result *subagentStatusResult `json:"result,omitempty"` | |||
| 46 | } | 48 | } | ||
| 47 | 49 | ||||
| 48 | type subagentStatusResult struct { | 50 | type subagentStatusResult struct { | ||
| @@ -132,13 +134,15 @@ func findSubagentStatusSnapshot(snapshots []subagents.AgentSnapshot, id string) | |||||
| 132 | 134 | ||||
| 133 | func publicSubagentStatus(snapshot subagents.AgentSnapshot) subagentStatusEntry { | 135 | func publicSubagentStatus(snapshot subagents.AgentSnapshot) subagentStatusEntry { | ||
| 134 | entry := subagentStatusEntry{ | 136 | entry := subagentStatusEntry{ | ||
| 135 | - | ID: snapshot.ID, | 137 | + | ID: snapshot.ID, |
| 136 | - | State: publicSubagentState(snapshot), | 138 | + | State: publicSubagentState(snapshot), |
| 137 | - | ProcessState: string(snapshot.ProcessState), | 139 | + | ProcessState: string(snapshot.ProcessState), |
| 138 | - | TurnState: string(snapshot.TurnState), | 140 | + | TurnState: string(snapshot.TurnState), |
| 139 | - | StartedAt: snapshot.Started, | 141 | + | LifetimeTurns: snapshot.LifetimeTurns, |
| 140 | - | UpdatedAt: snapshot.UpdatedAt, | 142 | + | CurrentRunTurns: snapshot.CurrentRunTurns, |
| 141 | - | TaskSummary: summarizeSubagentTask(snapshot.Task), | 143 | + | StartedAt: snapshot.Started, |
| 144 | + | UpdatedAt: snapshot.UpdatedAt, | |||
| 145 | + | TaskSummary: summarizeSubagentTask(snapshot.Task), | |||
| 142 | } | 146 | } | ||
| 143 | if !snapshot.Finished.IsZero() { | 147 | if !snapshot.Finished.IsZero() { | ||
| 144 | finished := snapshot.Finished | 148 | finished := snapshot.Finished | ||
M packages/agent/tools/subagent_status_test.go +22 -0
| @@ -39,6 +39,28 @@ func (r statusTestRunner) Run(ctx context.Context, sink subagents.Sink) error { | |||
| 39 | 39 | } | |
| 40 | 40 | } | |
| 41 | 41 | ||
| 42 | + | func TestSubagentStatusIncludesTurnCounters(t *testing.T) { | |
| 43 | + | entry := publicSubagentStatus(subagents.AgentSnapshot{ | |
| 44 | + | ID: "counter-agent", | |
| 45 | + | LifetimeTurns: 7, | |
| 46 | + | CurrentRunTurns: 2, | |
| 47 | + | }) | |
| 48 | + | if entry.LifetimeTurns != 7 || entry.CurrentRunTurns != 2 { | |
| 49 | + | t.Fatalf("status counters = (%d, %d), want (7, 2)", entry.LifetimeTurns, entry.CurrentRunTurns) | |
| 50 | + | } | |
| 51 | + | encoded, err := json.Marshal(entry) | |
| 52 | + | if err != nil { | |
| 53 | + | t.Fatal(err) | |
| 54 | + | } | |
| 55 | + | var fields map[string]any | |
| 56 | + | if err := json.Unmarshal(encoded, &fields); err != nil { | |
| 57 | + | t.Fatal(err) | |
| 58 | + | } | |
| 59 | + | if fields["lifetime_turns"] != float64(7) || fields["current_run_turns"] != float64(2) { | |
| 60 | + | t.Fatalf("encoded status counters = %#v", fields) | |
| 61 | + | } | |
| 62 | + | } | |
| 63 | + | ||
| 42 | 64 | func TestSubagentStatusSchemaHasOptionalAgentID(t *testing.T) { | |
| 43 | 65 | var schema struct { | |
| 44 | 66 | Type string `json:"type"` | |
| @@ -39,6 +39,28 @@ func (r statusTestRunner) Run(ctx context.Context, sink subagents.Sink) error { | |||||
| 39 | } | 39 | } | ||
| 40 | } | 40 | } | ||
| 41 | 41 | ||||
| 42 | + | func TestSubagentStatusIncludesTurnCounters(t *testing.T) { | |||
| 43 | + | entry := publicSubagentStatus(subagents.AgentSnapshot{ | |||
| 44 | + | ID: "counter-agent", | |||
| 45 | + | LifetimeTurns: 7, | |||
| 46 | + | CurrentRunTurns: 2, | |||
| 47 | + | }) | |||
| 48 | + | if entry.LifetimeTurns != 7 || entry.CurrentRunTurns != 2 { | |||
| 49 | + | t.Fatalf("status counters = (%d, %d), want (7, 2)", entry.LifetimeTurns, entry.CurrentRunTurns) | |||
| 50 | + | } | |||
| 51 | + | encoded, err := json.Marshal(entry) | |||
| 52 | + | if err != nil { | |||
| 53 | + | t.Fatal(err) | |||
| 54 | + | } | |||
| 55 | + | var fields map[string]any | |||
| 56 | + | if err := json.Unmarshal(encoded, &fields); err != nil { | |||
| 57 | + | t.Fatal(err) | |||
| 58 | + | } | |||
| 59 | + | if fields["lifetime_turns"] != float64(7) || fields["current_run_turns"] != float64(2) { | |||
| 60 | + | t.Fatalf("encoded status counters = %#v", fields) | |||
| 61 | + | } | |||
| 62 | + | } | |||
| 63 | + | ||||
| 42 | func TestSubagentStatusSchemaHasOptionalAgentID(t *testing.T) { | 64 | func TestSubagentStatusSchemaHasOptionalAgentID(t *testing.T) { | ||
| 43 | var schema struct { | 65 | var schema struct { | ||
| 44 | Type string `json:"type"` | 66 | Type string `json:"type"` | ||
M packages/core/session.go +13 -0
| @@ -1166,6 +1166,19 @@ func (s *Session) AppendMessage(m provider.Message) error { | |||
| 1166 | 1166 | return nil | |
| 1167 | 1167 | } | |
| 1168 | 1168 | ||
| 1169 | + | // Sync flushes the session file's bytes through the filesystem. Interactive | |
| 1170 | + | // hosts use this at acknowledgement boundaries where a crash must not make a | |
| 1171 | + | // prompt look accepted before its user message is recoverable. | |
| 1172 | + | func (s *Session) Sync() error { | |
| 1173 | + | if s == nil || s.writer == nil { | |
| 1174 | + | return nil | |
| 1175 | + | } | |
| 1176 | + | if err := s.buf.Flush(); err != nil { | |
| 1177 | + | return err | |
| 1178 | + | } | |
| 1179 | + | return s.writer.Sync() | |
| 1180 | + | } | |
| 1181 | + | ||
| 1169 | 1182 | // AppendCompaction writes a checkpoint that replaces all earlier | |
| 1170 | 1183 | // transcript rows when the session is resumed. The old rows remain in | |
| 1171 | 1184 | // the JSONL file for audit/export, while loaders use the latest | |
| @@ -1166,6 +1166,19 @@ func (s *Session) AppendMessage(m provider.Message) error { | |||||
| 1166 | return nil | 1166 | return nil | ||
| 1167 | } | 1167 | } | ||
| 1168 | 1168 | ||||
| 1169 | + | // Sync flushes the session file's bytes through the filesystem. Interactive | |||
| 1170 | + | // hosts use this at acknowledgement boundaries where a crash must not make a | |||
| 1171 | + | // prompt look accepted before its user message is recoverable. | |||
| 1172 | + | func (s *Session) Sync() error { | |||
| 1173 | + | if s == nil || s.writer == nil { | |||
| 1174 | + | return nil | |||
| 1175 | + | } | |||
| 1176 | + | if err := s.buf.Flush(); err != nil { | |||
| 1177 | + | return err | |||
| 1178 | + | } | |||
| 1179 | + | return s.writer.Sync() | |||
| 1180 | + | } | |||
| 1181 | + | ||||
| 1169 | // AppendCompaction writes a checkpoint that replaces all earlier | 1182 | // AppendCompaction writes a checkpoint that replaces all earlier | ||
| 1170 | // transcript rows when the session is resumed. The old rows remain in | 1183 | // transcript rows when the session is resumed. The old rows remain in | ||
| 1171 | // the JSONL file for audit/export, while loaders use the latest | 1184 | // the JSONL file for audit/export, while loaders use the latest | ||