Merge pull request 'feat: keep remote sessions responsive and resumable on flaky links' (#6) from feat/quic-resilience into main

commit: 50dc2b1af6f69930cc81c8e1fce7f45aeb746e55
author: bnema · 15d ago · 2026-10-07 08:16 UTC
merge of: 93f7e39 + 12ef17b · diff against the first parent
signature: unsigned

39 files · +3160 -98

M Makefile +5 -0


@@ -9,6 +9,7 @@ install:
9	9
10	10		test:
11	11		go test ./... -race
	12	+	python3 -m unittest discover -s scripts/remote-resilience-harness -p '*_test.py'
12	13
13	14		test-installer:
14	15		sh scripts/install_platform_test.sh

@@ -31,6 +32,10 @@ mocks:
31	32		remote-acceptance:
32	33		scripts/remote-picker-harness/run.sh
33	34
	35	+	.PHONY: remote-resilience-visual
	36	+	remote-resilience-visual:
	37	+	python3 scripts/remote-resilience-harness/visual.py
	38	+
34	39		# protocol regenerates the Protobuf wire contract from
35	40		# internal/protocol/wire/schema using the pinned Go tools (buf + protoc-gen-go).
36	41		protocol:


@@ -9,6 +9,7 @@ install:
9		9
10	test:	10		test:
11	go test ./... -race	11		go test ./... -race
		12	+	python3 -m unittest discover -s scripts/remote-resilience-harness -p '*_test.py'
12		13
13	test-installer:	14		test-installer:
14	sh scripts/install_platform_test.sh	15		sh scripts/install_platform_test.sh

@@ -31,6 +32,10 @@ mocks:
31	remote-acceptance:	32		remote-acceptance:
32	scripts/remote-picker-harness/run.sh	33		scripts/remote-picker-harness/run.sh
33		34
		35	+	.PHONY: remote-resilience-visual
		36	+	remote-resilience-visual:
		37	+	python3 scripts/remote-resilience-harness/visual.py
		38	+
34	# protocol regenerates the Protobuf wire contract from	39		# protocol regenerates the Protobuf wire contract from
35	# internal/protocol/wire/schema using the pinned Go tools (buf + protoc-gen-go).	40		# internal/protocol/wire/schema using the pinned Go tools (buf + protoc-gen-go).
36	protocol:	41		protocol:

M docs/configuration.md +1 -1


@@ -187,7 +187,7 @@ Configure this in `~/.config/vev/broker.json`. The file is created on first use
187	187		- `warmTransports`: how many idle connections to keep (0–64, default 8). The oldest one closes first. `0` disables it.
188	188		- `warmIdleTimeout`: close an idle connection after this long, such as `5m` (max `24h`). Default `off`: no time limit.
189	189
190		-	Warm connections never keep the broker running.
	190	+	Warm connections never keep the broker running. Each open QUIC connection, warm ones included, sends a small keepalive every 2 seconds so mobile and NAT paths stay open; on battery, set `warmIdleTimeout` to close idle ones.
191	191
192	192		## Logs and durable state
193	193


@@ -187,7 +187,7 @@ Configure this in `~/.config/vev/broker.json`. The file is created on first use
187		- `warmTransports`: how many idle connections to keep (0–64, default 8). The oldest one closes first. `0` disables it.	187		- `warmTransports`: how many idle connections to keep (0–64, default 8). The oldest one closes first. `0` disables it.
188		- `warmIdleTimeout`: close an idle connection after this long, such as `5m` (max `24h`). Default `off`: no time limit.	188		- `warmIdleTimeout`: close an idle connection after this long, such as `5m` (max `24h`). Default `off`: no time limit.
189			189
190	-	Warm connections never keep the broker running.	190	+	Warm connections never keep the broker running. Each open QUIC connection, warm ones included, sends a small keepalive every 2 seconds so mobile and NAT paths stay open; on battery, set `warmIdleTimeout` to close idle ones.
191			191
192		## Logs and durable state	192		## Logs and durable state
193			193

A docs/plans/quic-resilience.md +74 -0

@@ -0,0 +1,74 @@
	1	+	# Plan: mosh-like remote resilience
	2	+
	3	+	Goal: on a lossy or intermittent link (4G, roaming, sleep), the UI never
	4	+	freezes, typed input is kept and delivered when the link returns, and SSH is
	5	+	used only to authenticate the first QUIC handshake.
	6	+
	7	+	Reference: [mosh](https://github.com/mobile-shell/mosh)
	8	+	(`src/network/transportsender-impl.h`, `src/frontend/terminaloverlay.cc`).
	9	+
	10	+	## Diagnosis (before Phases 1 and 2)
	11	+
	12	+	1. Input path blocks on the network: `sessionAttachmentWorker.send` waits for
	13	+	`SendClient`, which waits for the carriage write. While QUIC stalls, the
	14	+	worker loop stops reading keys, servicing overlays, and painting.
	15	+	2. Every reconnect re-runs SSH: `_broker-mux-quic-proxy` serves one
	16	+	connection with one-time credentials and exits with it.
	17	+	3. No connection migration: one UDP socket per connection, no `AddPath`.
	18	+	4. Slow detection, early give-up: keepalive 15 s / idle 60 s; 5 resume
	19	+	attempts over a few seconds, then back to the picker.
	20	+	5. `LinkState` is only ever `Offline` at teardown; `degraded`/`probing` are
	21	+	documented but never produced.
	22	+	6. Daemon ignores `Input.InputSeq`, so input in flight at a drop is lost or
	23	+	would be duplicated by a replay.
	24	+
	25	+	## Phase 1 — non-blocking client send (done)
	26	+
	27	+	- Per-attachment ordered outbox drained by one sender goroutine; the worker
	28	+	loop enqueues and keeps running. Write failure settles the run as before.
	29	+	- Bounded by count and bytes. An input delivery stays undecided until the
	30	+	outbox wrote it, so the pump reads no newer keys meanwhile (backpressure)
	31	+	and a link failure preserves exactly the unwritten keys for the resume.
	32	+	- Lifecycle exits drain the outbox for at most 2 s.
	33	+
	34	+	## Phase 2 — patient resume (done, except status text)
	35	+
	36	+	- QUIC keepalive 2 s; idle timeout stays 60 s.
	37	+	- A route that stays down keeps resuming for the 15-minute park window; a
	38	+	session that drops right after attaching still gives up after 5 flaps.
	39	+	- During the resume backoff, Ctrl-C or a lone Esc returns to the picker;
	40	+	other keys are kept (bounded) and replayed into the resumed session.
	41	+	- TODO: status bar "last contact Ns ago" while resuming.
	42	+
	43	+	## Phase 3 — exactly-once input across resume (protocol bump)
	44	+
	45	+	- Daemon tracks last applied `InputSeq` per attachment; drops `<=` seqs.
	46	+	- `Welcome` on resume carries the last applied seq; client replays unacked
	47	+	input after it. Bump `protocol.Version`.
	48	+
	49	+	## Phase 4 — SSH only for first handshake (remote helpers, trust)
	50	+
	51	+	- Remote QUIC proxy outlives one connection for the park window and issues a
	52	+	resume ticket (bearer secret, bound to cert fingerprint + session, expiry,
	53	+	single active use, memory only).
	54	+	- Reconnect dials QUIC directly with the ticket; SSH only on rejection/expiry.
	55	+	- Picker host health uses the live link + a ticket-based QUIC probe, never
	56	+	SSH.
	57	+	- Needs a security review (ticket theft, replay, expiry) before merge.
	58	+
	59	+	## Phase 5 — real link states and path migration
	60	+
	61	+	- Derive `degraded`/`probing` from RTT and last-packet age in the adapter.
	62	+	- On stall or local address change, `AddPath` + probe + switch.
	63	+
	64	+	## Phase 6 — client-rendered palette on remote sessions
	65	+
	66	+	- The command palette is rendered by the serving daemon, so each keystroke
	67	+	costs a full round trip on a remote session. Render it client-side like
	68	+	the session picker, fed by the daemon's command catalogue, and send only
	69	+	the chosen command.
	70	+
	71	+	## Phase 7 — optional predictive local echo
	72	+
	73	+	- mosh-style overlay with underline-until-confirmed. Breaks the "thin client
	74	+	interprets nothing" rule; needs an explicit design decision.

@@ -0,0 +1,74 @@
	1	+	# Plan: mosh-like remote resilience
	2	+
	3	+	Goal: on a lossy or intermittent link (4G, roaming, sleep), the UI never
	4	+	freezes, typed input is kept and delivered when the link returns, and SSH is
	5	+	used only to authenticate the first QUIC handshake.
	6	+
	7	+	Reference: [mosh](https://github.com/mobile-shell/mosh)
	8	+	(`src/network/transportsender-impl.h`, `src/frontend/terminaloverlay.cc`).
	9	+
	10	+	## Diagnosis (before Phases 1 and 2)
	11	+
	12	+	1. Input path blocks on the network: `sessionAttachmentWorker.send` waits for
	13	+	`SendClient`, which waits for the carriage write. While QUIC stalls, the
	14	+	worker loop stops reading keys, servicing overlays, and painting.
	15	+	2. Every reconnect re-runs SSH: `_broker-mux-quic-proxy` serves one
	16	+	connection with one-time credentials and exits with it.
	17	+	3. No connection migration: one UDP socket per connection, no `AddPath`.
	18	+	4. Slow detection, early give-up: keepalive 15 s / idle 60 s; 5 resume
	19	+	attempts over a few seconds, then back to the picker.
	20	+	5. `LinkState` is only ever `Offline` at teardown; `degraded`/`probing` are
	21	+	documented but never produced.
	22	+	6. Daemon ignores `Input.InputSeq`, so input in flight at a drop is lost or
	23	+	would be duplicated by a replay.
	24	+
	25	+	## Phase 1 — non-blocking client send (done)
	26	+
	27	+	- Per-attachment ordered outbox drained by one sender goroutine; the worker
	28	+	loop enqueues and keeps running. Write failure settles the run as before.
	29	+	- Bounded by count and bytes. An input delivery stays undecided until the
	30	+	outbox wrote it, so the pump reads no newer keys meanwhile (backpressure)
	31	+	and a link failure preserves exactly the unwritten keys for the resume.
	32	+	- Lifecycle exits drain the outbox for at most 2 s.
	33	+
	34	+	## Phase 2 — patient resume (done, except status text)
	35	+
	36	+	- QUIC keepalive 2 s; idle timeout stays 60 s.
	37	+	- A route that stays down keeps resuming for the 15-minute park window; a
	38	+	session that drops right after attaching still gives up after 5 flaps.
	39	+	- During the resume backoff, Ctrl-C or a lone Esc returns to the picker;
	40	+	other keys are kept (bounded) and replayed into the resumed session.
	41	+	- TODO: status bar "last contact Ns ago" while resuming.
	42	+
	43	+	## Phase 3 — exactly-once input across resume (protocol bump)
	44	+
	45	+	- Daemon tracks last applied `InputSeq` per attachment; drops `<=` seqs.
	46	+	- `Welcome` on resume carries the last applied seq; client replays unacked
	47	+	input after it. Bump `protocol.Version`.
	48	+
	49	+	## Phase 4 — SSH only for first handshake (remote helpers, trust)
	50	+
	51	+	- Remote QUIC proxy outlives one connection for the park window and issues a
	52	+	resume ticket (bearer secret, bound to cert fingerprint + session, expiry,
	53	+	single active use, memory only).
	54	+	- Reconnect dials QUIC directly with the ticket; SSH only on rejection/expiry.
	55	+	- Picker host health uses the live link + a ticket-based QUIC probe, never
	56	+	SSH.
	57	+	- Needs a security review (ticket theft, replay, expiry) before merge.
	58	+
	59	+	## Phase 5 — real link states and path migration
	60	+
	61	+	- Derive `degraded`/`probing` from RTT and last-packet age in the adapter.
	62	+	- On stall or local address change, `AddPath` + probe + switch.
	63	+
	64	+	## Phase 6 — client-rendered palette on remote sessions
	65	+
	66	+	- The command palette is rendered by the serving daemon, so each keystroke
	67	+	costs a full round trip on a remote session. Render it client-side like
	68	+	the session picker, fed by the daemon's command catalogue, and send only
	69	+	the chosen command.
	70	+
	71	+	## Phase 7 — optional predictive local echo
	72	+
	73	+	- mosh-style overlay with underline-until-confirmed. Breaks the "thin client
	74	+	interprets nothing" rule; needs an explicit design decision.

M docs/remote-resilience.md +10 -9


@@ -25,21 +25,22 @@ The transport is fixed when the host is added (`quic` by default). To switch, ru
25	25		## What survives a network problem
26	26
27	27		- The remote session and its shells keep running while you are offline.
28		-	- The client reconnects by itself after a network change, a VPN switch, or a laptop sleep.
	28	+	- The client reconnects by itself after a network change, a VPN switch, or a laptop sleep, and keeps retrying for up to 15 minutes of outage before it returns to the picker. Press Ctrl-C or Esc while it reconnects to give up and return to the picker.
	29	+	- A stalled link does not block the client loop. Keys typed while the link stalls or reconnects are kept and sent in order once it recovers. Keys vev had not yet sent when the link dropped are replayed to the resumed session; keys already in flight at the drop may be lost.
29	30		- If the reconnect window has expired but the session still exists, the client opens a fresh attachment to it.
30	31		- Several clients can attach to the same session. Each keeps its own window, tab, focus, and copy mode.
31	32
	33	+	The command palette is rendered by the remote daemon, so opening it needs a working link. A client-owned session picker that is already open remains locally interactive during an outage.
	34	+
32	35		## Connection states
33	36
34		-	\| State \| Meaning \|
	37	+	\| What you see \| Meaning \|
35	38		\|---\|---\|
36		-	\| `connected` \| Everything works. \|
37		-	\| `degraded` \| Packets arrive, but updates are slow. \|
38		-	\| `probing` \| No contact; vev is looking for a working path. \|
39		-	\| `offline` \| No contact for a while; vev is reconnecting. \|
40		-	\| `dead` \| The connection is gone for good. \|
	39	+	\| The session \| The link works. \|
	40	+	\| `Connecting to session…` \| The link dropped; vev is reconnecting and keeps your keys. \|
	41	+	\| The picker with an error \| The session could not be reached again, or you cancelled. \|
41	42
42		-	States appear in the vev status bar, never inside your shell output.
	43	+	vev does not yet report a slow (`degraded`) link separately. Connection notices never appear inside your shell output.
43	44
44	45		## Hybrid mode: local and remote together
45	46

@@ -78,6 +79,6 @@ VEV_LOG=debug vev attach user@host
78	79		scripts/debug-remote-attach.sh user@host # collects redacted connection health
79	80		```
80	81
81		-	Useful manual checks: block UDP, switch networks or VPN, suspend the laptop, then reattach to the same session.
	82	+	Useful manual checks: block UDP, switch networks or VPN, suspend the laptop, then reattach to the same session. For isolated QUIC/SSH impairment and real Wayland terminal captures, see the [visual resilience harness](../scripts/remote-resilience-harness/README.md).
82	83
83	84		Keep hostnames, usernames, and keys out of shared logs and screenshots.

 ## What survives a network problem
 - The remote session and its shells keep running while you are offline.
-- The client reconnects by itself after a network change, a VPN switch, or a laptop sleep.
+- The client reconnects by itself after a network change, a VPN switch, or a laptop sleep, and keeps retrying for up to 15 minutes of outage before it returns to the picker. Press Ctrl-C or Esc while it reconnects to give up and return to the picker.
+- A stalled link does not block the client loop. Keys typed while the link stalls or reconnects are kept and sent in order once it recovers. Keys vev had not yet sent when the link dropped are replayed to the resumed session; keys already in flight at the drop may be lost.
 - If the reconnect window has expired but the session still exists, the client opens a fresh attachment to it.
 - Several clients can attach to the same session. Each keeps its own window, tab, focus, and copy mode.
+The command palette is rendered by the remote daemon, so opening it needs a working link. A client-owned session picker that is already open remains locally interactive during an outage.
 ## Connection states
-| State | Meaning |
+| What you see | Meaning |
 |---|---|
-| `connected` | Everything works. |
+| The session | The link works. |
-| `degraded` | Packets arrive, but updates are slow. |
+| `Connecting to session…` | The link dropped; vev is reconnecting and keeps your keys. |
-| `probing` | No contact; vev is looking for a working path. |
+| The picker with an error | The session could not be reached again, or you cancelled. |
-| `offline` | No contact for a while; vev is reconnecting. |
-| `dead` | The connection is gone for good. |
-States appear in the vev status bar, never inside your shell output.
+vev does not yet report a slow (`degraded`) link separately. Connection notices never appear inside your shell output.
 ## Hybrid mode: local and remote together
 scripts/debug-remote-attach.sh user@host   # collects redacted connection health
 ```
-Useful manual checks: block UDP, switch networks or VPN, suspend the laptop, then reattach to the same session.
+Useful manual checks: block UDP, switch networks or VPN, suspend the laptop, then reattach to the same session. For isolated QUIC/SSH impairment and real Wayland terminal captures, see the [visual resilience harness](../scripts/remote-resilience-harness/README.md).
 Keep hostnames, usernames, and keys out of shared logs and screenshots.

M internal/adapters/quic/quic.go +9 -3


@@ -61,8 +61,14 @@ var (
61	61		// listener and none on the dialer, no unidirectional streams, keepalive /
62	62		// idle limits, bounded admission, no 0-RTT.
63	63		type Config struct {
64		-	// KeepAlivePeriod defaults to 15s; MaxIdleTimeout defaults to 60s;
65		-	// HandshakeIdleTimeout defaults to 5s.
	64	+	// KeepAlivePeriod defaults to 2s; MaxIdleTimeout defaults to 60s;
	65	+	// HandshakeIdleTimeout defaults to 5s. The short keepalive keeps mobile
	66	+	// CGNAT bindings open and probes the path often enough that loss
	67	+	// recovery restarts promptly once the link returns. It does not shorten
	68	+	// failure detection: a dead path is still declared only after
	69	+	// MaxIdleTimeout, which stays long so one connection rides out short
	70	+	// outages and delivers queued input without reconnecting. Both roles use
	71	+	// it; a 2s PING is a few dozen bytes per connection.
66	72		KeepAlivePeriod time.Duration
67	73		MaxIdleTimeout time.Duration
68	74		HandshakeIdleTimeout time.Duration

@@ -86,7 +92,7 @@ const noIncomingStreams int64 = -1
86	92		func baseQUICConfig(config Config) *quicgo.Config {
87	93		keepAlive := config.KeepAlivePeriod
88	94		if keepAlive == 0 {
89		-	keepAlive = 15 * time.Second
	95	+	keepAlive = 2 * time.Second
90	96		}
91	97		maxIdle := config.MaxIdleTimeout
92	98		if maxIdle == 0 {

 // listener and none on the dialer, no unidirectional streams, keepalive /
 // idle limits, bounded admission, no 0-RTT.
 type Config struct {
-	// KeepAlivePeriod defaults to 15s; MaxIdleTimeout defaults to 60s;
+	// KeepAlivePeriod defaults to 2s; MaxIdleTimeout defaults to 60s;
-	// HandshakeIdleTimeout defaults to 5s.
+	// HandshakeIdleTimeout defaults to 5s. The short keepalive keeps mobile
+	// CGNAT bindings open and probes the path often enough that loss
+	// recovery restarts promptly once the link returns. It does not shorten
+	// failure detection: a dead path is still declared only after
+	// MaxIdleTimeout, which stays long so one connection rides out short
+	// outages and delivers queued input without reconnecting. Both roles use
+	// it; a 2s PING is a few dozen bytes per connection.
 	KeepAlivePeriod      time.Duration
 	MaxIdleTimeout       time.Duration
 	HandshakeIdleTimeout time.Duration
 func baseQUICConfig(config Config) *quicgo.Config {
 	keepAlive := config.KeepAlivePeriod
 	if keepAlive == 0 {
-		keepAlive = 15 * time.Second
+		keepAlive = 2 * time.Second
 	}
 	maxIdle := config.MaxIdleTimeout
 	if maxIdle == 0 {

A internal/adapters/quicnettest/listen_test.go +65 -0

@@ -0,0 +1,65 @@
	1	+	package quicnettest
	2	+
	3	+	import (
	4	+	"net"
	5	+	"testing"
	6	+	"time"
	7	+
	8	+	"github.com/stretchr/testify/require"
	9	+	)
	10	+
	11	+	func TestExplicitListenSurvivesRebind(t *testing.T) {
	12	+	for _, ip := range []net.IP{net.IPv4(127, 0, 0, 1), net.IPv6loopback} {
	13	+	t.Run(ip.String(), func(t *testing.T) {
	14	+	server, err := net.ListenUDP("udp", &net.UDPAddr{IP: ip})
	15	+	if err != nil && ip.To4() == nil {
	16	+	t.Skipf("IPv6 loopback unavailable: %v", err)
	17	+	}
	18	+	require.NoError(t, err)
	19	+	defer server.Close()
	20	+	proxy, err := New(Config{ServerAddr: server.LocalAddr().(*net.UDPAddr), ListenAddr: &net.UDPAddr{IP: ip}})
	21	+	require.NoError(t, err)
	22	+	defer proxy.Close()
	23	+	require.True(t, proxy.Addr().IP.Equal(ip), "requested listener address")
	24	+	client, err := net.ListenUDP("udp", &net.UDPAddr{IP: ip})
	25	+	require.NoError(t, err)
	26	+	defer client.Close()
	27	+	var sources []string
	28	+	for attempt := 0; attempt < 2; attempt++ {
	29	+	if attempt == 1 {
	30	+	require.NoError(t, proxy.Rebind())
	31	+	}
	32	+	_, err = client.WriteToUDP([]byte("fixture"), proxy.Addr())
	33	+	require.NoError(t, err)
	34	+	require.NoError(t, server.SetReadDeadline(time.Now().Add(time.Second)))
	35	+	data := make([]byte, 32)
	36	+	n, from, err := server.ReadFromUDP(data)
	37	+	require.NoError(t, err)
	38	+	require.Equal(t, "fixture", string(data[:n]))
	39	+	sources = append(sources, from.String())
	40	+	_, err = server.WriteToUDP(data[:n], from)
	41	+	require.NoError(t, err)
	42	+	require.NoError(t, client.SetReadDeadline(time.Now().Add(time.Second)))
	43	+	n, _, err = client.ReadFromUDP(data)
	44	+	require.NoError(t, err)
	45	+	require.Equal(t, "fixture", string(data[:n]))
	46	+	}
	47	+	require.NotEqual(t, sources[0], sources[1], "rebind must change the upstream source address")
	48	+	})
	49	+	}
	50	+	}
	51	+
	52	+	func TestUpstreamBindAddrMatchesServerFamily(t *testing.T) {
	53	+	for _, tt := range []struct {
	54	+	server, want net.IP
	55	+	}{
	56	+	{net.IPv4(127, 0, 0, 1), net.IPv4(127, 0, 0, 1)},
	57	+	{net.IPv4(192, 0, 2, 7), net.IPv4zero},
	58	+	{net.IPv6loopback, net.IPv6loopback},
	59	+	{net.ParseIP("2001:db8::7"), net.IPv6unspecified},
	60	+	} {
	61	+	t.Run(tt.server.String(), func(t *testing.T) {
	62	+	require.True(t, upstreamBindAddr(tt.server).IP.Equal(tt.want))
	63	+	})
	64	+	}
	65	+	}

@@ -0,0 +1,65 @@
	1	+	package quicnettest
	2	+
	3	+	import (
	4	+	"net"
	5	+	"testing"
	6	+	"time"
	7	+
	8	+	"github.com/stretchr/testify/require"
	9	+	)
	10	+
	11	+	func TestExplicitListenSurvivesRebind(t *testing.T) {
	12	+	for _, ip := range []net.IP{net.IPv4(127, 0, 0, 1), net.IPv6loopback} {
	13	+	t.Run(ip.String(), func(t *testing.T) {
	14	+	server, err := net.ListenUDP("udp", &net.UDPAddr{IP: ip})
	15	+	if err != nil && ip.To4() == nil {
	16	+	t.Skipf("IPv6 loopback unavailable: %v", err)
	17	+	}
	18	+	require.NoError(t, err)
	19	+	defer server.Close()
	20	+	proxy, err := New(Config{ServerAddr: server.LocalAddr().(*net.UDPAddr), ListenAddr: &net.UDPAddr{IP: ip}})
	21	+	require.NoError(t, err)
	22	+	defer proxy.Close()
	23	+	require.True(t, proxy.Addr().IP.Equal(ip), "requested listener address")
	24	+	client, err := net.ListenUDP("udp", &net.UDPAddr{IP: ip})
	25	+	require.NoError(t, err)
	26	+	defer client.Close()
	27	+	var sources []string
	28	+	for attempt := 0; attempt < 2; attempt++ {
	29	+	if attempt == 1 {
	30	+	require.NoError(t, proxy.Rebind())
	31	+	}
	32	+	_, err = client.WriteToUDP([]byte("fixture"), proxy.Addr())
	33	+	require.NoError(t, err)
	34	+	require.NoError(t, server.SetReadDeadline(time.Now().Add(time.Second)))
	35	+	data := make([]byte, 32)
	36	+	n, from, err := server.ReadFromUDP(data)
	37	+	require.NoError(t, err)
	38	+	require.Equal(t, "fixture", string(data[:n]))
	39	+	sources = append(sources, from.String())
	40	+	_, err = server.WriteToUDP(data[:n], from)
	41	+	require.NoError(t, err)
	42	+	require.NoError(t, client.SetReadDeadline(time.Now().Add(time.Second)))
	43	+	n, _, err = client.ReadFromUDP(data)
	44	+	require.NoError(t, err)
	45	+	require.Equal(t, "fixture", string(data[:n]))
	46	+	}
	47	+	require.NotEqual(t, sources[0], sources[1], "rebind must change the upstream source address")
	48	+	})
	49	+	}
	50	+	}
	51	+
	52	+	func TestUpstreamBindAddrMatchesServerFamily(t *testing.T) {
	53	+	for _, tt := range []struct {
	54	+	server, want net.IP
	55	+	}{
	56	+	{net.IPv4(127, 0, 0, 1), net.IPv4(127, 0, 0, 1)},
	57	+	{net.IPv4(192, 0, 2, 7), net.IPv4zero},
	58	+	{net.IPv6loopback, net.IPv6loopback},
	59	+	{net.ParseIP("2001:db8::7"), net.IPv6unspecified},
	60	+	} {
	61	+	t.Run(tt.server.String(), func(t *testing.T) {
	62	+	require.True(t, upstreamBindAddr(tt.server).IP.Equal(tt.want))
	63	+	})
	64	+	}
	65	+	}

M internal/adapters/quicnettest/proxy.go +41 -33


@@ -2,7 +2,6 @@ package quicnettest
2	2
3	3		import (
4	4		"container/heap"
5		-	"context"
6	5		"errors"
7	6		"fmt"
8	7		"net"

@@ -14,14 +13,31 @@ import (
14	13		// ErrClosed is returned by operations on a closed proxy.
15	14		var ErrClosed = errors.New("quicnettest: proxy is closed")
16	15
	16	+	// upstreamBindAddr returns the ephemeral upstream bind address in the server's
	17	+	// address family: loopback for a loopback server, unspecified otherwise.
	18	+	func upstreamBindAddr(server net.IP) *net.UDPAddr {
	19	+	switch {
	20	+	case server.To4() != nil && server.IsLoopback():
	21	+	return &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)}
	22	+	case server.To4() != nil:
	23	+	return &net.UDPAddr{IP: net.IPv4zero}
	24	+	case server.IsLoopback():
	25	+	return &net.UDPAddr{IP: net.IPv6loopback}
	26	+	default:
	27	+	return &net.UDPAddr{IP: net.IPv6unspecified}
	28	+	}
	29	+	}
	30	+
17	31		// Proxy is a bounded, deterministic UDP relay with per-direction impairment. It
18		-	// listens on an ephemeral loopback port for clients and forwards to
	32	+	// listens on Config.ListenAddr (default: ephemeral IPv4 loopback) for clients
	33	+	// and forwards to
19	34		// Config.ServerAddr from a separate upstream socket that can be rebound to a
20	35		// new source address at any time.
21	36		type Proxy struct {
22		-	cfg Config
23		-	clock Clock
24		-	server *net.UDPAddr
	37	+	cfg Config
	38	+	clock Clock
	39	+	server *net.UDPAddr
	40	+	upstreamAddr *net.UDPAddr
25	41
26	42		clientConn *net.UDPConn
27	43		upstream atomic.Pointer[net.UDPConn]

@@ -128,11 +144,16 @@ func New(cfg Config) (*Proxy, error) {
128	144		if err := cfg.validate(); err != nil {
129	145		return nil, err
130	146		}
131		-	clientConn, err := listenLoopback()
	147	+	listenAddr := cfg.ListenAddr
	148	+	if listenAddr == nil {
	149	+	listenAddr = &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)}
	150	+	}
	151	+	clientConn, err := net.ListenUDP("udp", listenAddr)
132	152		if err != nil {
133	153		return nil, fmt.Errorf("quicnettest: listen client socket: %w", err)
134	154		}
135		-	upstream, err := listenLoopback()
	155	+	upstreamAddr := upstreamBindAddr(cfg.ServerAddr.IP)
	156	+	upstream, err := net.ListenUDP("udp", upstreamAddr)
136	157		if err != nil {
137	158		_ = clientConn.Close()
138	159		return nil, fmt.Errorf("quicnettest: listen upstream socket: %w", err)

@@ -149,17 +170,18 @@ func New(cfg Config) (*Proxy, error) {
149	170		return newRNG(cfg.Seed, direction)
150	171		}
151	172		p := &Proxy{
152		-	cfg: cfg,
153		-	clock: cfg.Clock,
154		-	server: cfg.ServerAddr,
155		-	clientConn: clientConn,
156		-	done: make(chan struct{}),
157		-	slots: make(chan struct{}, cfg.QueueCapacity),
158		-	queue: make(chan packet, cfg.QueueCapacity),
159		-	start: start,
160		-	toServer: impairer{link: cfg.ToServer, rng: rngFor(ToServer), start: start},
161		-	toClient: impairer{link: cfg.ToClient, rng: rngFor(ToClient), start: start},
162		-	onPacket: cfg.OnPacket,
	173	+	cfg: cfg,
	174	+	clock: cfg.Clock,
	175	+	server: cfg.ServerAddr,
	176	+	upstreamAddr: upstreamAddr,
	177	+	clientConn: clientConn,
	178	+	done: make(chan struct{}),
	179	+	slots: make(chan struct{}, cfg.QueueCapacity),
	180	+	queue: make(chan packet, cfg.QueueCapacity),
	181	+	start: start,
	182	+	toServer: impairer{link: cfg.ToServer, rng: rngFor(ToServer), start: start},
	183	+	toClient: impairer{link: cfg.ToClient, rng: rngFor(ToClient), start: start},
	184	+	onPacket: cfg.OnPacket,
163	185		}
164	186		p.upstream.Store(upstream)
165	187		p.wg.Add(3)

@@ -206,7 +228,7 @@ func (p *Proxy) Rebind() error {
206	228		if p.stopping() {
207	229		return ErrClosed
208	230		}
209		-	conn, err := listenLoopback()
	231	+	conn, err := net.ListenUDP("udp", p.upstreamAddr)
210	232		if err != nil {
211	233		return fmt.Errorf("quicnettest: rebind: %w", err)
212	234		}

@@ -530,20 +552,6 @@ func directionOf(toServer bool) Direction {
530	552		return ToClient
531	553		}
532	554
533		-	func listenLoopback() (*net.UDPConn, error) {
534		-	var lc net.ListenConfig
535		-	conn, err := lc.ListenPacket(context.Background(), "udp", "127.0.0.1:0")
536		-	if err != nil {
537		-	return nil, err
538		-	}
539		-	udp, ok := conn.(*net.UDPConn)
540		-	if !ok {
541		-	_ = conn.Close()
542		-	return nil, fmt.Errorf("quicnettest: udp listener returned %T", conn)
543		-	}
544		-	return udp, nil
545		-	}
546		-
547	555		func sameUDPAddr(a, b *net.UDPAddr) bool {
548	556		return a != nil && b != nil && a.Port == b.Port && a.IP.Equal(b.IP)
549	557		}

 import (
 	"container/heap"
-	"context"
 	"errors"
 	"fmt"
 	"net"
 // ErrClosed is returned by operations on a closed proxy.
 var ErrClosed = errors.New("quicnettest: proxy is closed")
+// upstreamBindAddr returns the ephemeral upstream bind address in the server's
+// address family: loopback for a loopback server, unspecified otherwise.
+func upstreamBindAddr(server net.IP) *net.UDPAddr {
+	switch {
+	case server.To4() != nil && server.IsLoopback():
+		return &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)}
+	case server.To4() != nil:
+		return &net.UDPAddr{IP: net.IPv4zero}
+	case server.IsLoopback():
+		return &net.UDPAddr{IP: net.IPv6loopback}
+	default:
+		return &net.UDPAddr{IP: net.IPv6unspecified}
+	}
+}
 // Proxy is a bounded, deterministic UDP relay with per-direction impairment. It
-// listens on an ephemeral loopback port for clients and forwards to
+// listens on Config.ListenAddr (default: ephemeral IPv4 loopback) for clients
+// and forwards to
 // Config.ServerAddr from a separate upstream socket that can be rebound to a
 // new source address at any time.
 type Proxy struct {
-	cfg    Config
+	cfg          Config
-	clock  Clock
+	clock        Clock
-	server *net.UDPAddr
+	server       *net.UDPAddr
+	upstreamAddr *net.UDPAddr
 	clientConn *net.UDPConn
 	upstream   atomic.Pointer[net.UDPConn]
 	if err := cfg.validate(); err != nil {
 		return nil, err
 	}
-	clientConn, err := listenLoopback()
+	listenAddr := cfg.ListenAddr
+	if listenAddr == nil {
+		listenAddr = &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)}
+	}
+	clientConn, err := net.ListenUDP("udp", listenAddr)
 	if err != nil {
 		return nil, fmt.Errorf("quicnettest: listen client socket: %w", err)
 	}
-	upstream, err := listenLoopback()
+	upstreamAddr := upstreamBindAddr(cfg.ServerAddr.IP)
+	upstream, err := net.ListenUDP("udp", upstreamAddr)
 	if err != nil {
 		_ = clientConn.Close()
 		return nil, fmt.Errorf("quicnettest: listen upstream socket: %w", err)
 		return newRNG(cfg.Seed, direction)
 	}
 	p := &Proxy{
-		cfg:        cfg,
+		cfg:          cfg,
-		clock:      cfg.Clock,
+		clock:        cfg.Clock,
-		server:     cfg.ServerAddr,
+		server:       cfg.ServerAddr,
-		clientConn: clientConn,
+		upstreamAddr: upstreamAddr,
-		done:       make(chan struct{}),
+		clientConn:   clientConn,
-		slots:      make(chan struct{}, cfg.QueueCapacity),
+		done:         make(chan struct{}),
-		queue:      make(chan packet, cfg.QueueCapacity),
+		slots:        make(chan struct{}, cfg.QueueCapacity),
-		start:      start,
+		queue:        make(chan packet, cfg.QueueCapacity),
-		toServer:   impairer{link: cfg.ToServer, rng: rngFor(ToServer), start: start},
+		start:        start,
-		toClient:   impairer{link: cfg.ToClient, rng: rngFor(ToClient), start: start},
+		toServer:     impairer{link: cfg.ToServer, rng: rngFor(ToServer), start: start},
-		onPacket:   cfg.OnPacket,
+		toClient:     impairer{link: cfg.ToClient, rng: rngFor(ToClient), start: start},
+		onPacket:     cfg.OnPacket,
 	}
 	p.upstream.Store(upstream)
 	p.wg.Add(3)
 	if p.stopping() {
 		return ErrClosed
 	}
-	conn, err := listenLoopback()
+	conn, err := net.ListenUDP("udp", p.upstreamAddr)
 	if err != nil {
 		return fmt.Errorf("quicnettest: rebind: %w", err)
 	}
 	return ToClient
 }
-func listenLoopback() (*net.UDPConn, error) {
-	var lc net.ListenConfig
-	conn, err := lc.ListenPacket(context.Background(), "udp", "127.0.0.1:0")
-	if err != nil {
-		return nil, err
-	}
-	udp, ok := conn.(*net.UDPConn)
-	if !ok {
-		_ = conn.Close()
-		return nil, fmt.Errorf("quicnettest: udp listener returned %T", conn)
-	}
-	return udp, nil
-}
 func sameUDPAddr(a, b *net.UDPAddr) bool {
 	return a != nil && b != nil && a.Port == b.Port && a.IP.Equal(b.IP)
 }

M internal/adapters/quicnettest/simulation.go +3 -0


@@ -143,6 +143,9 @@ func (l LinkConfig) validate(direction Direction) error {
143	143
144	144		// Config configures a [Proxy].
145	145		type Config struct {
	146	+	// ListenAddr optionally selects the client-facing fixture address.
	147	+	// Nil retains the ephemeral loopback default.
	148	+	ListenAddr *net.UDPAddr
146	149		// ServerAddr is the UDP server the proxy forwards to. Required.
147	150		ServerAddr *net.UDPAddr
148	151		// Clock drives all scheduling. Defaults to [SystemClock].


@@ -143,6 +143,9 @@ func (l LinkConfig) validate(direction Direction) error {
143		143
144	// Config configures a [Proxy].	144		// Config configures a [Proxy].
145	type Config struct {	145		type Config struct {
		146	+	// ListenAddr optionally selects the client-facing fixture address.
		147	+	// Nil retains the ephemeral loopback default.
		148	+	ListenAddr *net.UDPAddr
146	// ServerAddr is the UDP server the proxy forwards to. Required.	149		// ServerAddr is the UDP server the proxy forwards to. Required.
147	ServerAddr *net.UDPAddr	150		ServerAddr *net.UDPAddr
148	// Clock drives all scheduling. Defaults to [SystemClock].	151		// Clock drives all scheduling. Defaults to [SystemClock].

M internal/usecase/client/attachment_input_test.go +14 -2


@@ -33,6 +33,7 @@ func (r *inputTestReader) Read(p []byte) (int, error) {
33	33
34	34		type inputHarness struct {
35	35		stream *sessionTestStream
	36	+	pump *terminalInputPump
36	37		term *workerTestTerminal
37	38		clock *supervisorTestClock
38	39		reader *inputTestReader

@@ -49,6 +50,16 @@ func startInputHarness(t testing.T, themes terminalThemeState) *inputHarness {
49	50
50	51		func startInputHarnessWithConfig(t testing.T, themes terminalThemeState, request ports.BrokerOpenStreamRequest, clipboard ports.ClipboardReader) *inputHarness {
51	52		t.Helper()
	53	+	return startInputHarnessOn(t, themes, request, clipboard, newSessionTestStream(), nil)
	54	+	}
	55	+
	56	+	// startInputHarnessOn runs the harness over stream; transport, when non-nil,
	57	+	// is what the worker writes through (a wrapper over stream).
	58	+	func startInputHarnessOn(t testing.T, themes terminalThemeState, request ports.BrokerOpenStreamRequest, clipboard ports.ClipboardReader, stream sessionTestStream, transport ports.BrokerLogicalConnection) inputHarness {
	59	+	t.Helper()
	60	+	if transport == nil {
	61	+	transport = stream
	62	+	}
52	63		reader := &inputTestReader{chunks: make(chan []byte, 16)}
53	64		pump := newTerminalInputPump(reader)
54	65		pump.start()

@@ -57,7 +68,8 @@ func startInputHarnessWithConfig(t testing.T, themes terminalThemeState, reque
57	68		close(reader.chunks)
58	69		})
59	70		h := &inputHarness{
60		-	stream: newSessionTestStream(),
	71	+	stream: stream,
	72	+	pump: pump,
61	73		term: newWorkerTestTerminal(),
62	74		clock: newSupervisorTestClock(),
63	75		reader: reader,

@@ -72,7 +84,7 @@ func startInputHarnessWithConfig(t testing.T, themes terminalThemeState, reque
72	84		worker, err := newSessionAttachmentWorker(cfg)
73	85		require.NoError(t, err)
74	86		go func() {
75		-	event, _ := host.Run(context.Background(), AttachmentToken{Generation: 4, Attempt: 1}, worker, h.stream)
	87	+	event, _ := host.Run(context.Background(), AttachmentToken{Generation: 4, Attempt: 1}, worker, transport)
76	88		h.events <- event
77	89		}()
78	90		awaitHello(t, h.stream)

 type inputHarness struct {
 	stream *sessionTestStream
+	pump   *terminalInputPump
 	term   *workerTestTerminal
 	clock  *supervisorTestClock
 	reader *inputTestReader
 func startInputHarnessWithConfig(t *testing.T, themes *terminalThemeState, request ports.BrokerOpenStreamRequest, clipboard ports.ClipboardReader) *inputHarness {
 	t.Helper()
+	return startInputHarnessOn(t, themes, request, clipboard, newSessionTestStream(), nil)
+}
+// startInputHarnessOn runs the harness over stream; transport, when non-nil,
+// is what the worker writes through (a wrapper over stream).
+func startInputHarnessOn(t *testing.T, themes *terminalThemeState, request ports.BrokerOpenStreamRequest, clipboard ports.ClipboardReader, stream *sessionTestStream, transport ports.BrokerLogicalConnection) *inputHarness {
+	t.Helper()
+	if transport == nil {
+		transport = stream
+	}
 	reader := &inputTestReader{chunks: make(chan []byte, 16)}
 	pump := newTerminalInputPump(reader)
 	pump.start()
 		close(reader.chunks)
 	})
 	h := &inputHarness{
-		stream: newSessionTestStream(),
+		stream: stream,
+		pump:   pump,
 		term:   newWorkerTestTerminal(),
 		clock:  newSupervisorTestClock(),
 		reader: reader,
 	worker, err := newSessionAttachmentWorker(cfg)
 	require.NoError(t, err)
 	go func() {
-		event, _ := host.Run(context.Background(), AttachmentToken{Generation: 4, Attempt: 1}, worker, h.stream)
+		event, _ := host.Run(context.Background(), AttachmentToken{Generation: 4, Attempt: 1}, worker, transport)
 		h.events <- event
 	}()
 	awaitHello(t, h.stream)

A internal/usecase/client/attachment_outbox.go +246 -0

@@ -0,0 +1,246 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"errors"
	6	+	"sync"
	7	+
	8	+	"github.com/bnema/vev/internal/ports"
	9	+	"github.com/bnema/vev/internal/protocol"
	10	+	)
	11	+
	12	+	// Outbox bounds. Past either, enqueue blocks until the sender makes progress
	13	+	// or the caller is cancelled. Input is acknowledged to the terminal pump only
	14	+	// once written, so the pump stops reading new keys long before these limits;
	15	+	// they bound acks, focus reports, and other control traffic.
	16	+	const (
	17	+	outboxMaxPending = 4096
	18	+	outboxMaxBytes = 4 << 20
	19	+	)
	20	+
	21	+	// errOutboxStopped is returned by SendClient after the outbox stopped.
	22	+	var errOutboxStopped = errors.New("client: attachment outbox stopped")
	23	+
	24	+	// attachmentOutbox decouples the attachment worker loop from the carriage:
	25	+	// enqueue returns once the message is accepted, and one sender goroutine
	26	+	// writes messages in order. A stalled link therefore no longer freezes key
	27	+	// handling, overlays, or painting. The first write failure is sticky: it is
	28	+	// reported on Failed and returned by every later enqueue.
	29	+	type attachmentOutbox struct {
	30	+	ports.BrokerLogicalConnection
	31	+
	32	+	mu sync.Mutex
	33	+	queue []protocol.ClientMessage
	34	+	bytes int // payload bytes queued
	35	+	queued uint64 // messages ever accepted
	36	+	written uint64 // messages written successfully
	37	+	err error
	38	+	stopped bool
	39	+	wake chan struct{} // sender wake-up, capacity 1
	40	+	space chan struct{} // closed-and-replaced after each write, closed on failure
	41	+	idle chan struct{} // closed-and-replaced when the queue drains
	42	+	failed chan struct{} // closed once on the first write failure
	43	+	done chan struct{} // closed when the sender exits
	44	+	}
	45	+
	46	+	func newAttachmentOutbox(stream ports.BrokerLogicalConnection) *attachmentOutbox {
	47	+	b := &attachmentOutbox{
	48	+	BrokerLogicalConnection: stream,
	49	+	wake: make(chan struct{}, 1),
	50	+	space: make(chan struct{}),
	51	+	idle: make(chan struct{}),
	52	+	failed: make(chan struct{}),
	53	+	done: make(chan struct{}),
	54	+	}
	55	+	close(b.idle)
	56	+	go b.run()
	57	+	return b
	58	+	}
	59	+
	60	+	// outboxPayload is the variable-size part of a message counted against
	61	+	// outboxMaxBytes.
	62	+	func outboxPayload(message protocol.ClientMessage) int {
	63	+	switch m := message.(type) {
	64	+	case protocol.Input:
	65	+	return len(m.Data)
	66	+	case protocol.ImagePush:
	67	+	return len(m.Data)
	68	+	}
	69	+	return 0
	70	+	}
	71	+
	72	+	// SendClient enqueues message without a cancellation source.
	73	+	func (b *attachmentOutbox) SendClient(message protocol.ClientMessage) error {
	74	+	return b.enqueue(context.Background(), nil, message)
	75	+	}
	76	+
	77	+	// enqueue accepts message for ordered transmission. It blocks only while the
	78	+	// outbox is full, until space frees, ctx ends, or cancel closes. It returns
	79	+	// the sticky failure once the sender failed.
	80	+	func (b *attachmentOutbox) enqueue(ctx context.Context, cancel <-chan struct{}, message protocol.ClientMessage) error {
	81	+	size := outboxPayload(message)
	82	+	b.mu.Lock()
	83	+	for {
	84	+	if b.err != nil {
	85	+	err := b.err
	86	+	b.mu.Unlock()
	87	+	return err
	88	+	}
	89	+	if b.stopped {
	90	+	b.mu.Unlock()
	91	+	return errOutboxStopped
	92	+	}
	93	+	// An empty queue always admits, so one oversized message still goes.
	94	+	if len(b.queue) == 0 \|\| (len(b.queue) < outboxMaxPending && b.bytes+size <= outboxMaxBytes) {
	95	+	break
	96	+	}
	97	+	space := b.space
	98	+	b.mu.Unlock()
	99	+	select {
	100	+	case <-space:
	101	+	case <-b.done:
	102	+	case <-ctx.Done():
	103	+	return ctx.Err()
	104	+	case <-cancel:
	105	+	return errAttachmentForegroundRevoked
	106	+	}
	107	+	b.mu.Lock()
	108	+	}
	109	+	if len(b.queue) == 0 {
	110	+	b.idle = make(chan struct{})
	111	+	}
	112	+	b.queue = append(b.queue, message)
	113	+	b.bytes += size
	114	+	b.queued++
	115	+	b.mu.Unlock()
	116	+	select {
	117	+	case b.wake <- struct{}{}:
	118	+	default:
	119	+	}
	120	+	return nil
	121	+	}
	122	+
	123	+	// Close stops the sender and closes the underlying stream, which unblocks a
	124	+	// write stalled on the link.
	125	+	func (b *attachmentOutbox) Close() error {
	126	+	b.stop()
	127	+	return b.BrokerLogicalConnection.Close()
	128	+	}
	129	+
	130	+	// stop ends the sender after its current write; queued messages are dropped.
	131	+	// Session input among them was never acknowledged to the terminal pump, so
	132	+	// the worker preserves it like a failed synchronous send.
	133	+	func (b *attachmentOutbox) stop() {
	134	+	b.mu.Lock()
	135	+	b.stopped = true
	136	+	b.mu.Unlock()
	137	+	select {
	138	+	case b.wake <- struct{}{}:
	139	+	default:
	140	+	}
	141	+	}
	142	+
	143	+	// unsentInput returns the session input bytes of accepted messages numbered
	144	+	// (from, to] that were never written. It is final only after join.
	145	+	func (b *attachmentOutbox) unsentInput(from, to uint64) []byte {
	146	+	b.mu.Lock()
	147	+	defer b.mu.Unlock()
	148	+	var data []byte
	149	+	// queue[i] is accepted message number written+1+i.
	150	+	for i, message := range b.queue {
	151	+	n := b.written + 1 + uint64(i)
	152	+	if n <= from \|\| n > to {
	153	+	continue
	154	+	}
	155	+	if input, ok := message.(protocol.Input); ok {
	156	+	data = append(data, input.Data...)
	157	+	}
	158	+	}
	159	+	return data
	160	+	}
	161	+
	162	+	// join stops the sender, closes the stream so a stalled write returns, and
	163	+	// waits for the sender to exit. Afterwards writtenCount is final.
	164	+	func (b *attachmentOutbox) join() {
	165	+	_ = b.Close()
	166	+	<-b.done
	167	+	}
	168	+
	169	+	// accepted reports how many messages were ever accepted.
	170	+	func (b *attachmentOutbox) accepted() uint64 {
	171	+	b.mu.Lock()
	172	+	defer b.mu.Unlock()
	173	+	return b.queued
	174	+	}
	175	+
	176	+	// writtenCount reports how many messages were written successfully.
	177	+	func (b *attachmentOutbox) writtenCount() uint64 {
	178	+	b.mu.Lock()
	179	+	defer b.mu.Unlock()
	180	+	return b.written
	181	+	}
	182	+
	183	+	// Progress is closed after the next successful write or on failure.
	184	+	func (b *attachmentOutbox) Progress() <-chan struct{} {
	185	+	b.mu.Lock()
	186	+	defer b.mu.Unlock()
	187	+	return b.space
	188	+	}
	189	+
	190	+	// Failed is closed once a write failed; failure returns the cause.
	191	+	func (b *attachmentOutbox) Failed() <-chan struct{} { return b.failed }
	192	+
	193	+	func (b *attachmentOutbox) failure() error {
	194	+	b.mu.Lock()
	195	+	defer b.mu.Unlock()
	196	+	return b.err
	197	+	}
	198	+
	199	+	// Idle is closed while nothing is queued or being written.
	200	+	func (b *attachmentOutbox) Idle() <-chan struct{} {
	201	+	b.mu.Lock()
	202	+	defer b.mu.Unlock()
	203	+	return b.idle
	204	+	}
	205	+
	206	+	func (b *attachmentOutbox) run() {
	207	+	defer close(b.done)
	208	+	for {
	209	+	b.mu.Lock()
	210	+	if b.stopped {
	211	+	b.mu.Unlock()
	212	+	return
	213	+	}
	214	+	if len(b.queue) == 0 {
	215	+	b.mu.Unlock()
	216	+	<-b.wake
	217	+	continue
	218	+	}
	219	+	message := b.queue[0]
	220	+	b.mu.Unlock()
	221	+
	222	+	err := b.BrokerLogicalConnection.SendClient(message)
	223	+
	224	+	b.mu.Lock()
	225	+	if err != nil {
	226	+	// The queue is kept, failed write first, so unsentInput can
	227	+	// hand undelivered keys back to the terminal pump.
	228	+	b.err = err
	229	+	close(b.failed)
	230	+	close(b.idle)
	231	+	close(b.space)
	232	+	b.mu.Unlock()
	233	+	return
	234	+	}
	235	+	b.queue[0] = nil
	236	+	b.queue = b.queue[1:]
	237	+	b.bytes -= outboxPayload(message)
	238	+	b.written++
	239	+	close(b.space)
	240	+	b.space = make(chan struct{})
	241	+	if len(b.queue) == 0 {
	242	+	close(b.idle)
	243	+	}
	244	+	b.mu.Unlock()
	245	+	}
	246	+	}

@@ -0,0 +1,246 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"errors"
	6	+	"sync"
	7	+
	8	+	"github.com/bnema/vev/internal/ports"
	9	+	"github.com/bnema/vev/internal/protocol"
	10	+	)
	11	+
	12	+	// Outbox bounds. Past either, enqueue blocks until the sender makes progress
	13	+	// or the caller is cancelled. Input is acknowledged to the terminal pump only
	14	+	// once written, so the pump stops reading new keys long before these limits;
	15	+	// they bound acks, focus reports, and other control traffic.
	16	+	const (
	17	+	outboxMaxPending = 4096
	18	+	outboxMaxBytes = 4 << 20
	19	+	)
	20	+
	21	+	// errOutboxStopped is returned by SendClient after the outbox stopped.
	22	+	var errOutboxStopped = errors.New("client: attachment outbox stopped")
	23	+
	24	+	// attachmentOutbox decouples the attachment worker loop from the carriage:
	25	+	// enqueue returns once the message is accepted, and one sender goroutine
	26	+	// writes messages in order. A stalled link therefore no longer freezes key
	27	+	// handling, overlays, or painting. The first write failure is sticky: it is
	28	+	// reported on Failed and returned by every later enqueue.
	29	+	type attachmentOutbox struct {
	30	+	ports.BrokerLogicalConnection
	31	+
	32	+	mu sync.Mutex
	33	+	queue []protocol.ClientMessage
	34	+	bytes int // payload bytes queued
	35	+	queued uint64 // messages ever accepted
	36	+	written uint64 // messages written successfully
	37	+	err error
	38	+	stopped bool
	39	+	wake chan struct{} // sender wake-up, capacity 1
	40	+	space chan struct{} // closed-and-replaced after each write, closed on failure
	41	+	idle chan struct{} // closed-and-replaced when the queue drains
	42	+	failed chan struct{} // closed once on the first write failure
	43	+	done chan struct{} // closed when the sender exits
	44	+	}
	45	+
	46	+	func newAttachmentOutbox(stream ports.BrokerLogicalConnection) *attachmentOutbox {
	47	+	b := &attachmentOutbox{
	48	+	BrokerLogicalConnection: stream,
	49	+	wake: make(chan struct{}, 1),
	50	+	space: make(chan struct{}),
	51	+	idle: make(chan struct{}),
	52	+	failed: make(chan struct{}),
	53	+	done: make(chan struct{}),
	54	+	}
	55	+	close(b.idle)
	56	+	go b.run()
	57	+	return b
	58	+	}
	59	+
	60	+	// outboxPayload is the variable-size part of a message counted against
	61	+	// outboxMaxBytes.
	62	+	func outboxPayload(message protocol.ClientMessage) int {
	63	+	switch m := message.(type) {
	64	+	case protocol.Input:
	65	+	return len(m.Data)
	66	+	case protocol.ImagePush:
	67	+	return len(m.Data)
	68	+	}
	69	+	return 0
	70	+	}
	71	+
	72	+	// SendClient enqueues message without a cancellation source.
	73	+	func (b *attachmentOutbox) SendClient(message protocol.ClientMessage) error {
	74	+	return b.enqueue(context.Background(), nil, message)
	75	+	}
	76	+
	77	+	// enqueue accepts message for ordered transmission. It blocks only while the
	78	+	// outbox is full, until space frees, ctx ends, or cancel closes. It returns
	79	+	// the sticky failure once the sender failed.
	80	+	func (b *attachmentOutbox) enqueue(ctx context.Context, cancel <-chan struct{}, message protocol.ClientMessage) error {
	81	+	size := outboxPayload(message)
	82	+	b.mu.Lock()
	83	+	for {
	84	+	if b.err != nil {
	85	+	err := b.err
	86	+	b.mu.Unlock()
	87	+	return err
	88	+	}
	89	+	if b.stopped {
	90	+	b.mu.Unlock()
	91	+	return errOutboxStopped
	92	+	}
	93	+	// An empty queue always admits, so one oversized message still goes.
	94	+	if len(b.queue) == 0 \|\| (len(b.queue) < outboxMaxPending && b.bytes+size <= outboxMaxBytes) {
	95	+	break
	96	+	}
	97	+	space := b.space
	98	+	b.mu.Unlock()
	99	+	select {
	100	+	case <-space:
	101	+	case <-b.done:
	102	+	case <-ctx.Done():
	103	+	return ctx.Err()
	104	+	case <-cancel:
	105	+	return errAttachmentForegroundRevoked
	106	+	}
	107	+	b.mu.Lock()
	108	+	}
	109	+	if len(b.queue) == 0 {
	110	+	b.idle = make(chan struct{})
	111	+	}
	112	+	b.queue = append(b.queue, message)
	113	+	b.bytes += size
	114	+	b.queued++
	115	+	b.mu.Unlock()
	116	+	select {
	117	+	case b.wake <- struct{}{}:
	118	+	default:
	119	+	}
	120	+	return nil
	121	+	}
	122	+
	123	+	// Close stops the sender and closes the underlying stream, which unblocks a
	124	+	// write stalled on the link.
	125	+	func (b *attachmentOutbox) Close() error {
	126	+	b.stop()
	127	+	return b.BrokerLogicalConnection.Close()
	128	+	}
	129	+
	130	+	// stop ends the sender after its current write; queued messages are dropped.
	131	+	// Session input among them was never acknowledged to the terminal pump, so
	132	+	// the worker preserves it like a failed synchronous send.
	133	+	func (b *attachmentOutbox) stop() {
	134	+	b.mu.Lock()
	135	+	b.stopped = true
	136	+	b.mu.Unlock()
	137	+	select {
	138	+	case b.wake <- struct{}{}:
	139	+	default:
	140	+	}
	141	+	}
	142	+
	143	+	// unsentInput returns the session input bytes of accepted messages numbered
	144	+	// (from, to] that were never written. It is final only after join.
	145	+	func (b *attachmentOutbox) unsentInput(from, to uint64) []byte {
	146	+	b.mu.Lock()
	147	+	defer b.mu.Unlock()
	148	+	var data []byte
	149	+	// queue[i] is accepted message number written+1+i.
	150	+	for i, message := range b.queue {
	151	+	n := b.written + 1 + uint64(i)
	152	+	if n <= from \|\| n > to {
	153	+	continue
	154	+	}
	155	+	if input, ok := message.(protocol.Input); ok {
	156	+	data = append(data, input.Data...)
	157	+	}
	158	+	}
	159	+	return data
	160	+	}
	161	+
	162	+	// join stops the sender, closes the stream so a stalled write returns, and
	163	+	// waits for the sender to exit. Afterwards writtenCount is final.
	164	+	func (b *attachmentOutbox) join() {
	165	+	_ = b.Close()
	166	+	<-b.done
	167	+	}
	168	+
	169	+	// accepted reports how many messages were ever accepted.
	170	+	func (b *attachmentOutbox) accepted() uint64 {
	171	+	b.mu.Lock()
	172	+	defer b.mu.Unlock()
	173	+	return b.queued
	174	+	}
	175	+
	176	+	// writtenCount reports how many messages were written successfully.
	177	+	func (b *attachmentOutbox) writtenCount() uint64 {
	178	+	b.mu.Lock()
	179	+	defer b.mu.Unlock()
	180	+	return b.written
	181	+	}
	182	+
	183	+	// Progress is closed after the next successful write or on failure.
	184	+	func (b *attachmentOutbox) Progress() <-chan struct{} {
	185	+	b.mu.Lock()
	186	+	defer b.mu.Unlock()
	187	+	return b.space
	188	+	}
	189	+
	190	+	// Failed is closed once a write failed; failure returns the cause.
	191	+	func (b *attachmentOutbox) Failed() <-chan struct{} { return b.failed }
	192	+
	193	+	func (b *attachmentOutbox) failure() error {
	194	+	b.mu.Lock()
	195	+	defer b.mu.Unlock()
	196	+	return b.err
	197	+	}
	198	+
	199	+	// Idle is closed while nothing is queued or being written.
	200	+	func (b *attachmentOutbox) Idle() <-chan struct{} {
	201	+	b.mu.Lock()
	202	+	defer b.mu.Unlock()
	203	+	return b.idle
	204	+	}
	205	+
	206	+	func (b *attachmentOutbox) run() {
	207	+	defer close(b.done)
	208	+	for {
	209	+	b.mu.Lock()
	210	+	if b.stopped {
	211	+	b.mu.Unlock()
	212	+	return
	213	+	}
	214	+	if len(b.queue) == 0 {
	215	+	b.mu.Unlock()
	216	+	<-b.wake
	217	+	continue
	218	+	}
	219	+	message := b.queue[0]
	220	+	b.mu.Unlock()
	221	+
	222	+	err := b.BrokerLogicalConnection.SendClient(message)
	223	+
	224	+	b.mu.Lock()
	225	+	if err != nil {
	226	+	// The queue is kept, failed write first, so unsentInput can
	227	+	// hand undelivered keys back to the terminal pump.
	228	+	b.err = err
	229	+	close(b.failed)
	230	+	close(b.idle)
	231	+	close(b.space)
	232	+	b.mu.Unlock()
	233	+	return
	234	+	}
	235	+	b.queue[0] = nil
	236	+	b.queue = b.queue[1:]
	237	+	b.bytes -= outboxPayload(message)
	238	+	b.written++
	239	+	close(b.space)
	240	+	b.space = make(chan struct{})
	241	+	if len(b.queue) == 0 {
	242	+	close(b.idle)
	243	+	}
	244	+	b.mu.Unlock()
	245	+	}
	246	+	}

A internal/usecase/client/attachment_outbox_test.go +398 -0

@@ -0,0 +1,398 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"bytes"
	5	+	"errors"
	6	+	"strings"
	7	+	"sync/atomic"
	8	+	"testing"
	9	+	"time"
	10	+
	11	+	"github.com/stretchr/testify/require"
	12	+
	13	+	"github.com/bnema/vev/internal/protocol"
	14	+	)
	15	+
	16	+	// gatedSessionStream blocks every write until release is closed, like a
	17	+	// carriage stalled on a dead link, and can fail writes after release.
	18	+	type gatedSessionStream struct {
	19	+	*sessionTestStream
	20	+	release chan struct{}
	21	+	fail error
	22	+	}
	23	+
	24	+	func (s *gatedSessionStream) SendClient(message protocol.ClientMessage) error {
	25	+	select {
	26	+	case <-s.release:
	27	+	case <-s.closed:
	28	+	return errSessionTestClosed
	29	+	}
	30	+	if s.fail != nil {
	31	+	return s.fail
	32	+	}
	33	+	return s.sessionTestStream.SendClient(message)
	34	+	}
	35	+
	36	+	func TestAttachmentOutbox(t *testing.T) {
	37	+	writeErr := errors.New("link lost")
	38	+	tests := []struct {
	39	+	name string
	40	+	fail error
	41	+	wantErr error
	42	+	}{
	43	+	{name: "stalled link does not block enqueue and preserves order"},
	44	+	{name: "write failure is sticky and signalled", fail: writeErr, wantErr: writeErr},
	45	+	}
	46	+	for _, tt := range tests {
	47	+	t.Run(tt.name, func(t *testing.T) {
	48	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{}), fail: tt.fail}
	49	+	outbox := newAttachmentOutbox(stream)
	50	+	defer outbox.stop()
	51	+
	52	+	messages := []protocol.ClientMessage{
	53	+	protocol.Input{InputSeq: 1, Data: []byte("a")},
	54	+	protocol.Input{InputSeq: 2, Data: []byte("b")},
	55	+	protocol.Input{InputSeq: 3, Data: []byte("c")},
	56	+	}
	57	+	enqueued := make(chan struct{})
	58	+	go func() {
	59	+	for _, m := range messages {
	60	+	if err := outbox.SendClient(m); err != nil {
	61	+	t.Errorf("SendClient: %v", err)
	62	+	}
	63	+	}
	64	+	close(enqueued)
	65	+	}()
	66	+	select {
	67	+	case <-enqueued:
	68	+	case <-time.After(5 * time.Second):
	69	+	t.Fatal("SendClient blocked on a stalled link")
	70	+	}
	71	+
	72	+	close(stream.release)
	73	+	if tt.wantErr != nil {
	74	+	<-outbox.Failed()
	75	+	require.ErrorIs(t, outbox.failure(), tt.wantErr)
	76	+	require.ErrorIs(t, outbox.SendClient(protocol.Detach{}), tt.wantErr)
	77	+	return
	78	+	}
	79	+	<-outbox.Idle()
	80	+	stream.mu.Lock()
	81	+	defer stream.mu.Unlock()
	82	+	require.Equal(t, messages, stream.sent)
	83	+	})
	84	+	}
	85	+	}
	86	+
	87	+	func TestAttachmentOutboxCloseUnblocksStalledWrite(t *testing.T) {
	88	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	89	+	outbox := newAttachmentOutbox(stream)
	90	+	require.NoError(t, outbox.SendClient(protocol.Detach{}))
	91	+
	92	+	require.NoError(t, outbox.Close())
	93	+	select {
	94	+	case <-outbox.done:
	95	+	case <-time.After(5 * time.Second):
	96	+	t.Fatal("sender did not exit after Close")
	97	+	}
	98	+	}
	99	+
	100	+	// TestResumeWatch proves the resume backoff reads the terminal: Ctrl-C or a
	101	+	// lone Esc cancels, anything else is replayed to the next owner in order.
	102	+	func TestResumeWatch(t *testing.T) {
	103	+	tests := []struct {
	104	+	name string
	105	+	reads []string
	106	+	wantCancel bool
	107	+	fireEscape bool
	108	+	wantResidual string
	109	+	}{
	110	+	{name: "ctrl-c cancels and drops held keys", reads: []string{"ls", "\x03"}, wantCancel: true},
	111	+	{name: "lone esc cancels", reads: []string{"\x1b"}, wantCancel: true, fireEscape: true},
	112	+	{name: "kitty esc cancels", reads: []string{"\x1b[27u"}, wantCancel: true},
	113	+	{name: "kitty ctrl-c cancels", reads: []string{"\x1b[99;5u"}, wantCancel: true},
	114	+	{name: "kitty ctrl-c with lock bits cancels", reads: []string{"\x1b[99;69u"}, wantCancel: true},
	115	+	{name: "escape sequence is kept", reads: []string{"\x1b[A"}, wantResidual: "\x1b[A"},
	116	+	{name: "typed keys are kept in order", reads: []string{"l", "s", "\r"}, wantResidual: "ls\r"},
	117	+	}
	118	+	for _, tt := range tests {
	119	+	t.Run(tt.name, func(t *testing.T) {
	120	+	reader := &inputTestReader{chunks: make(chan []byte, len(tt.reads))}
	121	+	pump := newTerminalInputPump(reader)
	122	+	pump.start()
	123	+	t.Cleanup(func() {
	124	+	pump.stop()
	125	+	close(reader.chunks)
	126	+	})
	127	+	lifetime := &terminalInputLifetime{eof: make(chan error, 1), pump: pump}
	128	+	clock := newSupervisorTestClock()
	129	+	watch := lifetime.watchResume(clock)
	130	+	for _, read := range tt.reads {
	131	+	reader.chunks <- []byte(read)
	132	+	}
	133	+	if tt.wantCancel {
	134	+	if tt.fireEscape {
	135	+	clock.awaitTimer(t).fire()
	136	+	}
	137	+	select {
	138	+	case <-watch.cancelled:
	139	+	case <-time.After(5 * time.Second):
	140	+	t.Fatal("cancel key ignored")
	141	+	}
	142	+	} else {
	143	+	require.Eventually(t, func() bool { return watch.heldLen() == len(tt.wantResidual) }, 5*time.Second, time.Millisecond)
	144	+	}
	145	+	watch.release()
	146	+	pump.mu.Lock()
	147	+	defer pump.mu.Unlock()
	148	+	require.Equal(t, tt.wantResidual, string(pump.residual))
	149	+	require.Zero(t, pump.consumer, "release returns the claim")
	150	+	})
	151	+	}
	152	+	}
	153	+
	154	+	// TestResumeWatchKeepsBacklogVerbatim starts a watch while session input the
	155	+	// lost attachment kept is still waiting: a Ctrl-C or trailing Esc in it was
	156	+	// typed for the remote shell, so it is replayed, not read as a cancel key.
	157	+	// Keys typed after the watch started still cancel.
	158	+	func TestResumeWatchKeepsBacklogVerbatim(t *testing.T) {
	159	+	for _, tt := range []struct {
	160	+	name, residual, pending, typed string
	161	+	wantCancel bool
	162	+	wantResidual string
	163	+	}{
	164	+	{name: "kept ctrl-c", residual: "\x03", wantResidual: "\x03"},
	165	+	{name: "kept trailing esc", residual: "ihello\x1b", wantResidual: "ihello\x1b"},
	166	+	{name: "pending ctrl-c", pending: "x\x03", wantResidual: "x\x03"},
	167	+	{name: "residual then pending", residual: "a", pending: "\x03", wantResidual: "a\x03"},
	168	+	{name: "new ctrl-c still cancels", residual: "\x03", typed: "\x03", wantCancel: true},
	169	+	} {
	170	+	t.Run(tt.name, func(t *testing.T) {
	171	+	reader := &inputTestReader{chunks: make(chan []byte, 2)}
	172	+	pump := newTerminalInputPump(reader)
	173	+	pump.start()
	174	+	t.Cleanup(func() {
	175	+	pump.stop()
	176	+	close(reader.chunks)
	177	+	})
	178	+	pump.mu.Lock()
	179	+	pump.residual = []byte(tt.residual)
	180	+	pump.mu.Unlock()
	181	+	if tt.pending != "" {
	182	+	reader.chunks <- []byte(tt.pending)
	183	+	require.Eventually(t, func() bool {
	184	+	pump.mu.Lock()
	185	+	defer pump.mu.Unlock()
	186	+	return pump.pending != nil
	187	+	}, 5*time.Second, time.Millisecond)
	188	+	}
	189	+	lifetime := &terminalInputLifetime{eof: make(chan error, 1), pump: pump}
	190	+	watch := lifetime.watchResume(newSupervisorTestClock())
	191	+	require.Eventually(t, func() bool { return watch.heldLen() == len(tt.residual)+len(tt.pending) }, 5*time.Second, time.Millisecond)
	192	+	if tt.typed != "" {
	193	+	reader.chunks <- []byte(tt.typed)
	194	+	}
	195	+	if tt.wantCancel {
	196	+	select {
	197	+	case <-watch.cancelled:
	198	+	case <-time.After(5 * time.Second):
	199	+	t.Fatal("new cancel key ignored")
	200	+	}
	201	+	}
	202	+	watch.release()
	203	+	select {
	204	+	case <-watch.cancelled:
	205	+	require.True(t, tt.wantCancel, "backlog cancelled the resume")
	206	+	default:
	207	+	}
	208	+	pump.mu.Lock()
	209	+	defer pump.mu.Unlock()
	210	+	require.Equal(t, tt.wantResidual, string(pump.residual))
	211	+	})
	212	+	}
	213	+	}
	214	+
	215	+	// TestResumeWatchResolvesEscapeAtRelease presses Esc just before the phase
	216	+	// ends: release waits for the escape delay, so a lone Esc still cancels and an
	217	+	// escape sequence completed in time is kept for the session.
	218	+	func TestResumeWatchResolvesEscapeAtRelease(t *testing.T) {
	219	+	for _, tt := range []struct {
	220	+	name, after string
	221	+	wantCancel bool
	222	+	wantResidual string
	223	+	}{
	224	+	{name: "lone esc cancels", wantCancel: true},
	225	+	{name: "arrow key is kept", after: "[A", wantResidual: "\x1b[A"},
	226	+	} {
	227	+	t.Run(tt.name, func(t *testing.T) {
	228	+	reader := &inputTestReader{chunks: make(chan []byte, 2)}
	229	+	pump := newTerminalInputPump(reader)
	230	+	pump.start()
	231	+	t.Cleanup(func() {
	232	+	pump.stop()
	233	+	close(reader.chunks)
	234	+	})
	235	+	lifetime := &terminalInputLifetime{eof: make(chan error, 1), pump: pump}
	236	+	clock := newSupervisorTestClock()
	237	+	watch := lifetime.watchResume(clock)
	238	+	reader.chunks <- []byte("\x1b")
	239	+	escape := clock.awaitTimer(t)
	240	+	released := make(chan struct{})
	241	+	go func() { watch.release(); close(released) }()
	242	+	select {
	243	+	case <-released:
	244	+	t.Fatal("release returned before the escape was resolved")
	245	+	case <-time.After(50 * time.Millisecond):
	246	+	}
	247	+	if tt.after != "" {
	248	+	reader.chunks <- []byte(tt.after)
	249	+	} else {
	250	+	escape.fire()
	251	+	}
	252	+	select {
	253	+	case <-released:
	254	+	case <-time.After(5 * time.Second):
	255	+	t.Fatal("release never returned")
	256	+	}
	257	+	select {
	258	+	case <-watch.cancelled:
	259	+	require.True(t, tt.wantCancel, "escape sequence cancelled the resume")
	260	+	default:
	261	+	require.False(t, tt.wantCancel, "lone esc at release did not cancel")
	262	+	}
	263	+	pump.mu.Lock()
	264	+	defer pump.mu.Unlock()
	265	+	require.Equal(t, tt.wantResidual, string(pump.residual))
	266	+	})
	267	+	}
	268	+	}
	269	+
	270	+	func TestResumeWatchHeldOverflowIsSticky(t *testing.T) {
	271	+	w := &resumeWatch{}
	272	+	w.hold(bytes.Repeat([]byte("a"), resumeHeldInputLimit-1))
	273	+	w.hold([]byte("bb"))
	274	+	w.hold([]byte("c"))
	275	+	require.Equal(t, resumeHeldInputLimit-1, w.heldLen(), "keys after an overflow must be dropped too")
	276	+	}
	277	+
	278	+	// stallingSessionStream passes writes through until stall is set; stalled
	279	+	// writes block until release closes, then fail with failErr when set.
	280	+	type stallingSessionStream struct {
	281	+	*sessionTestStream
	282	+	stall atomic.Bool
	283	+	entered chan struct{}
	284	+	release chan struct{}
	285	+	failErr error
	286	+	}
	287	+
	288	+	func (s *stallingSessionStream) SendClient(message protocol.ClientMessage) error {
	289	+	if s.stall.Load() {
	290	+	select {
	291	+	case s.entered <- struct{}{}:
	292	+	default:
	293	+	}
	294	+	select {
	295	+	case <-s.release:
	296	+	case <-s.closed:
	297	+	return errSessionTestClosed
	298	+	}
	299	+	if s.failErr != nil {
	300	+	return s.failErr
	301	+	}
	302	+	}
	303	+	return s.sessionTestStream.SendClient(message)
	304	+	}
	305	+
	306	+	// TestAttachmentWorkerStalledLink drives the real worker over a link that
	307	+	// stops accepting writes after attach.
	308	+	func TestAttachmentWorkerStalledLink(t *testing.T) {
	309	+	linkErr := errors.New("link lost")
	310	+	tests := []struct {
	311	+	name string
	312	+	run func(t testing.T, h inputHarness, s *stallingSessionStream)
	313	+	}{
	314	+	{
	315	+	name: "output keeps painting while a write is stalled",
	316	+	run: func(t testing.T, h inputHarness, s *stallingSessionStream) {
	317	+	h.send("a")
	318	+	<-s.entered
	319	+	later := sessionTestOutput(2, "\x1b[Hlater")
	320	+	later.Epoch = 2
	321	+	h.stream.deliver(later)
	322	+	require.Eventually(t, func() bool { return strings.Contains(h.term.written(), "later") }, 5*time.Second, time.Millisecond, "a stalled write blocked painting")
	323	+	close(s.release)
	324	+	require.Equal(t, []string{"a"}, h.awaitInputs(t, 1))
	325	+	h.end(t)
	326	+	},
	327	+	},
	328	+	{
	329	+	name: "a read that queues nothing is acked while an earlier write is stalled",
	330	+	run: func(t testing.T, h inputHarness, s *stallingSessionStream) {
	331	+	// A queued output Ack stalls behind the link.
	332	+	next := sessionTestOutput(2, "\x1b[Hmore")
	333	+	next.Epoch = 2
	334	+	h.stream.deliver(next)
	335	+	<-s.entered
	336	+	// An open bracketed paste is held by the coalescer and queues
	337	+	// nothing, so there is no write to wait for.
	338	+	h.send("\x1b[200~abc")
	339	+	h.send("def")
	340	+	require.Eventually(t, func() bool {
	341	+	h.pump.mu.Lock()
	342	+	defer h.pump.mu.Unlock()
	343	+	return h.pump.pending == nil && h.pump.delivering == 0 && len(h.reader.chunks) == 0
	344	+	}, 5*time.Second, time.Millisecond, "an unqueued read held the pump behind a stalled write")
	345	+	close(s.release)
	346	+	h.end(t)
	347	+	},
	348	+	},
	349	+	{
	350	+	name: "keys behind a failed write are preserved for the next attempt",
	351	+	run: func(t testing.T, h inputHarness, s *stallingSessionStream) {
	352	+	s.failErr = linkErr
	353	+	h.send("a")
	354	+	<-s.entered
	355	+	close(s.release)
	356	+	select {
	357	+	case event := <-h.events:
	358	+	require.Equal(t, AttachmentEventFailed, event.Kind)
	359	+	require.ErrorIs(t, event.Err, linkErr)
	360	+	case <-time.After(5 * time.Second):
	361	+	t.Fatal("the failed write never settled the run")
	362	+	}
	363	+	h.pump.mu.Lock()
	364	+	defer h.pump.mu.Unlock()
	365	+	require.Equal(t, "a", string(h.pump.residual), "the unsent key is kept for the resume")
	366	+	},
	367	+	},
	368	+	{
	369	+	name: "detach on a stalled link is bounded",
	370	+	run: func(t testing.T, h inputHarness, s *stallingSessionStream) {
	371	+	h.send("a")
	372	+	<-s.entered
	373	+	h.stream.deliver(protocol.Detached{Reason: protocol.ReasonDetach})
	374	+	select {
	375	+	case event := <-h.events:
	376	+	require.Equal(t, AttachmentEventEnded, event.Kind)
	377	+	case <-time.After(5 * time.Second):
	378	+	t.Fatal("the daemon-confirmed detach never settled")
	379	+	}
	380	+	h.pump.mu.Lock()
	381	+	defer h.pump.mu.Unlock()
	382	+	require.Equal(t, "a", string(h.pump.residual), "a key the link never took is not lost")
	383	+	},
	384	+	},
	385	+	}
	386	+	for _, tt := range tests {
	387	+	t.Run(tt.name, func(t *testing.T) {
	388	+	base := newSessionTestStream()
	389	+	s := &stallingSessionStream{sessionTestStream: base, entered: make(chan struct{}, 1), release: make(chan struct{})}
	390	+	h := startInputHarnessOn(t, nil, sessionTestRequest(true), nil, base, s)
	391	+	require.Eventually(t, func() bool { return strings.Contains(h.term.written(), "ready") }, 5*time.Second, time.Millisecond)
	392	+	// Let the initial Ack and focus report through, then stall.
	393	+	require.Eventually(t, func() bool { return len(base.messages()) >= 2 }, 5*time.Second, time.Millisecond)
	394	+	s.stall.Store(true)
	395	+	tt.run(t, h, s)
	396	+	})
	397	+	}
	398	+	}

@@ -0,0 +1,398 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"bytes"
	5	+	"errors"
	6	+	"strings"
	7	+	"sync/atomic"
	8	+	"testing"
	9	+	"time"
	10	+
	11	+	"github.com/stretchr/testify/require"
	12	+
	13	+	"github.com/bnema/vev/internal/protocol"
	14	+	)
	15	+
	16	+	// gatedSessionStream blocks every write until release is closed, like a
	17	+	// carriage stalled on a dead link, and can fail writes after release.
	18	+	type gatedSessionStream struct {
	19	+	*sessionTestStream
	20	+	release chan struct{}
	21	+	fail error
	22	+	}
	23	+
	24	+	func (s *gatedSessionStream) SendClient(message protocol.ClientMessage) error {
	25	+	select {
	26	+	case <-s.release:
	27	+	case <-s.closed:
	28	+	return errSessionTestClosed
	29	+	}
	30	+	if s.fail != nil {
	31	+	return s.fail
	32	+	}
	33	+	return s.sessionTestStream.SendClient(message)
	34	+	}
	35	+
	36	+	func TestAttachmentOutbox(t *testing.T) {
	37	+	writeErr := errors.New("link lost")
	38	+	tests := []struct {
	39	+	name string
	40	+	fail error
	41	+	wantErr error
	42	+	}{
	43	+	{name: "stalled link does not block enqueue and preserves order"},
	44	+	{name: "write failure is sticky and signalled", fail: writeErr, wantErr: writeErr},
	45	+	}
	46	+	for _, tt := range tests {
	47	+	t.Run(tt.name, func(t *testing.T) {
	48	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{}), fail: tt.fail}
	49	+	outbox := newAttachmentOutbox(stream)
	50	+	defer outbox.stop()
	51	+
	52	+	messages := []protocol.ClientMessage{
	53	+	protocol.Input{InputSeq: 1, Data: []byte("a")},
	54	+	protocol.Input{InputSeq: 2, Data: []byte("b")},
	55	+	protocol.Input{InputSeq: 3, Data: []byte("c")},
	56	+	}
	57	+	enqueued := make(chan struct{})
	58	+	go func() {
	59	+	for _, m := range messages {
	60	+	if err := outbox.SendClient(m); err != nil {
	61	+	t.Errorf("SendClient: %v", err)
	62	+	}
	63	+	}
	64	+	close(enqueued)
	65	+	}()
	66	+	select {
	67	+	case <-enqueued:
	68	+	case <-time.After(5 * time.Second):
	69	+	t.Fatal("SendClient blocked on a stalled link")
	70	+	}
	71	+
	72	+	close(stream.release)
	73	+	if tt.wantErr != nil {
	74	+	<-outbox.Failed()
	75	+	require.ErrorIs(t, outbox.failure(), tt.wantErr)
	76	+	require.ErrorIs(t, outbox.SendClient(protocol.Detach{}), tt.wantErr)
	77	+	return
	78	+	}
	79	+	<-outbox.Idle()
	80	+	stream.mu.Lock()
	81	+	defer stream.mu.Unlock()
	82	+	require.Equal(t, messages, stream.sent)
	83	+	})
	84	+	}
	85	+	}
	86	+
	87	+	func TestAttachmentOutboxCloseUnblocksStalledWrite(t *testing.T) {
	88	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	89	+	outbox := newAttachmentOutbox(stream)
	90	+	require.NoError(t, outbox.SendClient(protocol.Detach{}))
	91	+
	92	+	require.NoError(t, outbox.Close())
	93	+	select {
	94	+	case <-outbox.done:
	95	+	case <-time.After(5 * time.Second):
	96	+	t.Fatal("sender did not exit after Close")
	97	+	}
	98	+	}
	99	+
	100	+	// TestResumeWatch proves the resume backoff reads the terminal: Ctrl-C or a
	101	+	// lone Esc cancels, anything else is replayed to the next owner in order.
	102	+	func TestResumeWatch(t *testing.T) {
	103	+	tests := []struct {
	104	+	name string
	105	+	reads []string
	106	+	wantCancel bool
	107	+	fireEscape bool
	108	+	wantResidual string
	109	+	}{
	110	+	{name: "ctrl-c cancels and drops held keys", reads: []string{"ls", "\x03"}, wantCancel: true},
	111	+	{name: "lone esc cancels", reads: []string{"\x1b"}, wantCancel: true, fireEscape: true},
	112	+	{name: "kitty esc cancels", reads: []string{"\x1b[27u"}, wantCancel: true},
	113	+	{name: "kitty ctrl-c cancels", reads: []string{"\x1b[99;5u"}, wantCancel: true},
	114	+	{name: "kitty ctrl-c with lock bits cancels", reads: []string{"\x1b[99;69u"}, wantCancel: true},
	115	+	{name: "escape sequence is kept", reads: []string{"\x1b[A"}, wantResidual: "\x1b[A"},
	116	+	{name: "typed keys are kept in order", reads: []string{"l", "s", "\r"}, wantResidual: "ls\r"},
	117	+	}
	118	+	for _, tt := range tests {
	119	+	t.Run(tt.name, func(t *testing.T) {
	120	+	reader := &inputTestReader{chunks: make(chan []byte, len(tt.reads))}
	121	+	pump := newTerminalInputPump(reader)
	122	+	pump.start()
	123	+	t.Cleanup(func() {
	124	+	pump.stop()
	125	+	close(reader.chunks)
	126	+	})
	127	+	lifetime := &terminalInputLifetime{eof: make(chan error, 1), pump: pump}
	128	+	clock := newSupervisorTestClock()
	129	+	watch := lifetime.watchResume(clock)
	130	+	for _, read := range tt.reads {
	131	+	reader.chunks <- []byte(read)
	132	+	}
	133	+	if tt.wantCancel {
	134	+	if tt.fireEscape {
	135	+	clock.awaitTimer(t).fire()
	136	+	}
	137	+	select {
	138	+	case <-watch.cancelled:
	139	+	case <-time.After(5 * time.Second):
	140	+	t.Fatal("cancel key ignored")
	141	+	}
	142	+	} else {
	143	+	require.Eventually(t, func() bool { return watch.heldLen() == len(tt.wantResidual) }, 5*time.Second, time.Millisecond)
	144	+	}
	145	+	watch.release()
	146	+	pump.mu.Lock()
	147	+	defer pump.mu.Unlock()
	148	+	require.Equal(t, tt.wantResidual, string(pump.residual))
	149	+	require.Zero(t, pump.consumer, "release returns the claim")
	150	+	})
	151	+	}
	152	+	}
	153	+
	154	+	// TestResumeWatchKeepsBacklogVerbatim starts a watch while session input the
	155	+	// lost attachment kept is still waiting: a Ctrl-C or trailing Esc in it was
	156	+	// typed for the remote shell, so it is replayed, not read as a cancel key.
	157	+	// Keys typed after the watch started still cancel.
	158	+	func TestResumeWatchKeepsBacklogVerbatim(t *testing.T) {
	159	+	for _, tt := range []struct {
	160	+	name, residual, pending, typed string
	161	+	wantCancel bool
	162	+	wantResidual string
	163	+	}{
	164	+	{name: "kept ctrl-c", residual: "\x03", wantResidual: "\x03"},
	165	+	{name: "kept trailing esc", residual: "ihello\x1b", wantResidual: "ihello\x1b"},
	166	+	{name: "pending ctrl-c", pending: "x\x03", wantResidual: "x\x03"},
	167	+	{name: "residual then pending", residual: "a", pending: "\x03", wantResidual: "a\x03"},
	168	+	{name: "new ctrl-c still cancels", residual: "\x03", typed: "\x03", wantCancel: true},
	169	+	} {
	170	+	t.Run(tt.name, func(t *testing.T) {
	171	+	reader := &inputTestReader{chunks: make(chan []byte, 2)}
	172	+	pump := newTerminalInputPump(reader)
	173	+	pump.start()
	174	+	t.Cleanup(func() {
	175	+	pump.stop()
	176	+	close(reader.chunks)
	177	+	})
	178	+	pump.mu.Lock()
	179	+	pump.residual = []byte(tt.residual)
	180	+	pump.mu.Unlock()
	181	+	if tt.pending != "" {
	182	+	reader.chunks <- []byte(tt.pending)
	183	+	require.Eventually(t, func() bool {
	184	+	pump.mu.Lock()
	185	+	defer pump.mu.Unlock()
	186	+	return pump.pending != nil
	187	+	}, 5*time.Second, time.Millisecond)
	188	+	}
	189	+	lifetime := &terminalInputLifetime{eof: make(chan error, 1), pump: pump}
	190	+	watch := lifetime.watchResume(newSupervisorTestClock())
	191	+	require.Eventually(t, func() bool { return watch.heldLen() == len(tt.residual)+len(tt.pending) }, 5*time.Second, time.Millisecond)
	192	+	if tt.typed != "" {
	193	+	reader.chunks <- []byte(tt.typed)
	194	+	}
	195	+	if tt.wantCancel {
	196	+	select {
	197	+	case <-watch.cancelled:
	198	+	case <-time.After(5 * time.Second):
	199	+	t.Fatal("new cancel key ignored")
	200	+	}
	201	+	}
	202	+	watch.release()
	203	+	select {
	204	+	case <-watch.cancelled:
	205	+	require.True(t, tt.wantCancel, "backlog cancelled the resume")
	206	+	default:
	207	+	}
	208	+	pump.mu.Lock()
	209	+	defer pump.mu.Unlock()
	210	+	require.Equal(t, tt.wantResidual, string(pump.residual))
	211	+	})
	212	+	}
	213	+	}
	214	+
	215	+	// TestResumeWatchResolvesEscapeAtRelease presses Esc just before the phase
	216	+	// ends: release waits for the escape delay, so a lone Esc still cancels and an
	217	+	// escape sequence completed in time is kept for the session.
	218	+	func TestResumeWatchResolvesEscapeAtRelease(t *testing.T) {
	219	+	for _, tt := range []struct {
	220	+	name, after string
	221	+	wantCancel bool
	222	+	wantResidual string
	223	+	}{
	224	+	{name: "lone esc cancels", wantCancel: true},
	225	+	{name: "arrow key is kept", after: "[A", wantResidual: "\x1b[A"},
	226	+	} {
	227	+	t.Run(tt.name, func(t *testing.T) {
	228	+	reader := &inputTestReader{chunks: make(chan []byte, 2)}
	229	+	pump := newTerminalInputPump(reader)
	230	+	pump.start()
	231	+	t.Cleanup(func() {
	232	+	pump.stop()
	233	+	close(reader.chunks)
	234	+	})
	235	+	lifetime := &terminalInputLifetime{eof: make(chan error, 1), pump: pump}
	236	+	clock := newSupervisorTestClock()
	237	+	watch := lifetime.watchResume(clock)
	238	+	reader.chunks <- []byte("\x1b")
	239	+	escape := clock.awaitTimer(t)
	240	+	released := make(chan struct{})
	241	+	go func() { watch.release(); close(released) }()
	242	+	select {
	243	+	case <-released:
	244	+	t.Fatal("release returned before the escape was resolved")
	245	+	case <-time.After(50 * time.Millisecond):
	246	+	}
	247	+	if tt.after != "" {
	248	+	reader.chunks <- []byte(tt.after)
	249	+	} else {
	250	+	escape.fire()
	251	+	}
	252	+	select {
	253	+	case <-released:
	254	+	case <-time.After(5 * time.Second):
	255	+	t.Fatal("release never returned")
	256	+	}
	257	+	select {
	258	+	case <-watch.cancelled:
	259	+	require.True(t, tt.wantCancel, "escape sequence cancelled the resume")
	260	+	default:
	261	+	require.False(t, tt.wantCancel, "lone esc at release did not cancel")
	262	+	}
	263	+	pump.mu.Lock()
	264	+	defer pump.mu.Unlock()
	265	+	require.Equal(t, tt.wantResidual, string(pump.residual))
	266	+	})
	267	+	}
	268	+	}
	269	+
	270	+	func TestResumeWatchHeldOverflowIsSticky(t *testing.T) {
	271	+	w := &resumeWatch{}
	272	+	w.hold(bytes.Repeat([]byte("a"), resumeHeldInputLimit-1))
	273	+	w.hold([]byte("bb"))
	274	+	w.hold([]byte("c"))
	275	+	require.Equal(t, resumeHeldInputLimit-1, w.heldLen(), "keys after an overflow must be dropped too")
	276	+	}
	277	+
	278	+	// stallingSessionStream passes writes through until stall is set; stalled
	279	+	// writes block until release closes, then fail with failErr when set.
	280	+	type stallingSessionStream struct {
	281	+	*sessionTestStream
	282	+	stall atomic.Bool
	283	+	entered chan struct{}
	284	+	release chan struct{}
	285	+	failErr error
	286	+	}
	287	+
	288	+	func (s *stallingSessionStream) SendClient(message protocol.ClientMessage) error {
	289	+	if s.stall.Load() {
	290	+	select {
	291	+	case s.entered <- struct{}{}:
	292	+	default:
	293	+	}
	294	+	select {
	295	+	case <-s.release:
	296	+	case <-s.closed:
	297	+	return errSessionTestClosed
	298	+	}
	299	+	if s.failErr != nil {
	300	+	return s.failErr
	301	+	}
	302	+	}
	303	+	return s.sessionTestStream.SendClient(message)
	304	+	}
	305	+
	306	+	// TestAttachmentWorkerStalledLink drives the real worker over a link that
	307	+	// stops accepting writes after attach.
	308	+	func TestAttachmentWorkerStalledLink(t *testing.T) {
	309	+	linkErr := errors.New("link lost")
	310	+	tests := []struct {
	311	+	name string
	312	+	run func(t testing.T, h inputHarness, s *stallingSessionStream)
	313	+	}{
	314	+	{
	315	+	name: "output keeps painting while a write is stalled",
	316	+	run: func(t testing.T, h inputHarness, s *stallingSessionStream) {
	317	+	h.send("a")
	318	+	<-s.entered
	319	+	later := sessionTestOutput(2, "\x1b[Hlater")
	320	+	later.Epoch = 2
	321	+	h.stream.deliver(later)
	322	+	require.Eventually(t, func() bool { return strings.Contains(h.term.written(), "later") }, 5*time.Second, time.Millisecond, "a stalled write blocked painting")
	323	+	close(s.release)
	324	+	require.Equal(t, []string{"a"}, h.awaitInputs(t, 1))
	325	+	h.end(t)
	326	+	},
	327	+	},
	328	+	{
	329	+	name: "a read that queues nothing is acked while an earlier write is stalled",
	330	+	run: func(t testing.T, h inputHarness, s *stallingSessionStream) {
	331	+	// A queued output Ack stalls behind the link.
	332	+	next := sessionTestOutput(2, "\x1b[Hmore")
	333	+	next.Epoch = 2
	334	+	h.stream.deliver(next)
	335	+	<-s.entered
	336	+	// An open bracketed paste is held by the coalescer and queues
	337	+	// nothing, so there is no write to wait for.
	338	+	h.send("\x1b[200~abc")
	339	+	h.send("def")
	340	+	require.Eventually(t, func() bool {
	341	+	h.pump.mu.Lock()
	342	+	defer h.pump.mu.Unlock()
	343	+	return h.pump.pending == nil && h.pump.delivering == 0 && len(h.reader.chunks) == 0
	344	+	}, 5*time.Second, time.Millisecond, "an unqueued read held the pump behind a stalled write")
	345	+	close(s.release)
	346	+	h.end(t)
	347	+	},
	348	+	},
	349	+	{
	350	+	name: "keys behind a failed write are preserved for the next attempt",
	351	+	run: func(t testing.T, h inputHarness, s *stallingSessionStream) {
	352	+	s.failErr = linkErr
	353	+	h.send("a")
	354	+	<-s.entered
	355	+	close(s.release)
	356	+	select {
	357	+	case event := <-h.events:
	358	+	require.Equal(t, AttachmentEventFailed, event.Kind)
	359	+	require.ErrorIs(t, event.Err, linkErr)
	360	+	case <-time.After(5 * time.Second):
	361	+	t.Fatal("the failed write never settled the run")
	362	+	}
	363	+	h.pump.mu.Lock()
	364	+	defer h.pump.mu.Unlock()
	365	+	require.Equal(t, "a", string(h.pump.residual), "the unsent key is kept for the resume")
	366	+	},
	367	+	},
	368	+	{
	369	+	name: "detach on a stalled link is bounded",
	370	+	run: func(t testing.T, h inputHarness, s *stallingSessionStream) {
	371	+	h.send("a")
	372	+	<-s.entered
	373	+	h.stream.deliver(protocol.Detached{Reason: protocol.ReasonDetach})
	374	+	select {
	375	+	case event := <-h.events:
	376	+	require.Equal(t, AttachmentEventEnded, event.Kind)
	377	+	case <-time.After(5 * time.Second):
	378	+	t.Fatal("the daemon-confirmed detach never settled")
	379	+	}
	380	+	h.pump.mu.Lock()
	381	+	defer h.pump.mu.Unlock()
	382	+	require.Equal(t, "a", string(h.pump.residual), "a key the link never took is not lost")
	383	+	},
	384	+	},
	385	+	}
	386	+	for _, tt := range tests {
	387	+	t.Run(tt.name, func(t *testing.T) {
	388	+	base := newSessionTestStream()
	389	+	s := &stallingSessionStream{sessionTestStream: base, entered: make(chan struct{}, 1), release: make(chan struct{})}
	390	+	h := startInputHarnessOn(t, nil, sessionTestRequest(true), nil, base, s)
	391	+	require.Eventually(t, func() bool { return strings.Contains(h.term.written(), "ready") }, 5*time.Second, time.Millisecond)
	392	+	// Let the initial Ack and focus report through, then stall.
	393	+	require.Eventually(t, func() bool { return len(base.messages()) >= 2 }, 5*time.Second, time.Millisecond)
	394	+	s.stall.Store(true)
	395	+	tt.run(t, h, s)
	396	+	})
	397	+	}
	398	+	}

M internal/usecase/client/attachment_session.go +141 -7


@@ -301,9 +301,13 @@ var (
301	301
302	302		// pumpAttached publishes output and concurrently forwards the foreground's
303	303		// authorized input, latest geometry and explicit lifecycle decision.
304		-	func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg AttachmentForeground, stream ports.BrokerLogicalConnection, token AttachmentToken, state outputApplyState, incoming <-chan serverReceive) AttachmentEvent {
	304	+	func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg AttachmentForeground, raw ports.BrokerLogicalConnection, token AttachmentToken, state outputApplyState, incoming <-chan serverReceive) AttachmentEvent {
305	305		pumpCtx, cancel := context.WithCancel(ctx)
306	306		defer cancel()
	307	+	// Every send while attached goes through the outbox so a stalled link
	308	+	// never blocks key handling, overlays, or painting.
	309	+	stream := newAttachmentOutbox(raw)
	310	+	defer stream.stop()
307	311		outgoing := make(chan attachmentClientEvent)
308	312		go pumpAttachmentInput(pumpCtx, fg, outgoing)
309	313		go pumpAttachmentGeometry(pumpCtx, fg, outgoing)

@@ -345,7 +349,73 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
345	349		if err := sendFocus(); err != nil {
346	350		return w.settle(ctx, fg, stream, token, err)
347	351		}
	352	+	// pending is the one undecided input delivery waiting on the outbox.
	353	+	var pending *pendingInputAck
	354	+	defer func() {
	355	+	if pending == nil {
	356	+	return
	357	+	}
	358	+	// Join the sender before deciding which input reached the carriage.
	359	+	// A written automation action remains OutcomeUnknown if teardown
	360	+	// prevents its UIFence receipt; acknowledging input is not proof
	361	+	// that the daemon processed the action.
	362	+	stream.join()
	363	+	if unsent := stream.unsentInput(pending.from, pending.mark); len(unsent) != 0 {
	364	+	fg.PreserveInput(unsent)
	365	+	return
	366	+	}
	367	+	fg.AckInput()
	368	+	}()
	369	+	// ackWritten commits pending once the outbox wrote it, then fences its
	370	+	// UI action after it.
	371	+	ackWritten := func() error {
	372	+	if pending == nil \|\| stream.writtenCount() < pending.mark {
	373	+	return nil
	374	+	}
	375	+	actionID := pending.actionID
	376	+	pending = nil
	377	+	fg.AckInput()
	378	+	if actionID != 0 {
	379	+	return w.send(ctx, fg, stream, protocol.UIFence{ActionID: actionID})
	380	+	}
	381	+	return nil
	382	+	}
	383	+	// await makes the delivery just routed pending until the outbox wrote
	384	+	// every message accepted since from.
	385	+	await := func(from, actionID uint64) error {
	386	+	mark := stream.accepted()
	387	+	if mark == from {
	388	+	// Nothing was queued (an overlay consumed the keys), so there is
	389	+	// no write to wait for: an earlier stalled message must not hold
	390	+	// the overlay's next key.
	391	+	fg.AckInput()
	392	+	if actionID != 0 {
	393	+	return w.send(ctx, fg, stream, protocol.UIFence{ActionID: actionID})
	394	+	}
	395	+	return nil
	396	+	}
	397	+	pending = &pendingInputAck{from: from, mark: mark, actionID: actionID}
	398	+	return ackWritten()
	399	+	}
348	400		for {
	401	+	// Failure also closes progress. Settle it before rearming that
	402	+	// channel, rather than spinning on an acknowledgement that cannot
	403	+	// advance.
	404	+	select {
	405	+	case <-stream.Failed():
	406	+	return w.settle(ctx, fg, stream, token, stream.failure())
	407	+	default:
	408	+	}
	409	+	var progress <-chan struct{}
	410	+	if pending != nil {
	411	+	progress = stream.Progress()
	412	+	if err := ackWritten(); err != nil {
	413	+	return w.settle(ctx, fg, stream, token, err)
	414	+	}
	415	+	if pending == nil {
	416	+	progress = nil
	417	+	}
	418	+	}
349	419		select {
350	420		case <-focus.changed:
351	421		if err := sendFocus(); err != nil {

@@ -517,13 +587,16 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
517	587		if event.input != nil && event.input.actionID == 0 {
518	588		// A physical read: replies are stripped and ordinary bytes
519	589		// reach the overlay or the session inside read.
	590	+	from := stream.accepted()
520	591		if err := input.read(ctx, state, event.input.Data); err != nil {
521	592		// Part of the read may already be delivered or held, so
522	593		// it is committed rather than replayed.
523	594		fg.AckInput()
524	595		return w.settle(ctx, fg, stream, token, err)
525	596		}
526		-	fg.AckInput()
	597	+	if err := await(from, 0); err != nil {
	598	+	return w.settle(ctx, fg, stream, token, err)
	599	+	}
527	600		continue
528	601		}
529	602		if event.input != nil {

@@ -557,6 +630,7 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
557	630		return w.settle(ctx, fg, stream, token, err)
558	631		}
559	632		}
	633	+	from := stream.accepted()
560	634		if err := w.send(ctx, fg, stream, event.message); err != nil {
561	635		if event.input != nil {
562	636		fg.PreserveInput(event.input.Data)

@@ -564,16 +638,28 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
564	638		return w.settle(ctx, fg, stream, token, err)
565	639		}
566	640		if event.input != nil {
567		-	fg.AckInput()
568		-	if event.input.actionID != 0 {
569		-	if err := w.send(ctx, fg, stream, protocol.UIFence{ActionID: event.input.actionID}); err != nil {
570		-	return w.settle(ctx, fg, stream, token, err)
571		-	}
	641	+	// The delivery stays undecided until the outbox wrote it, so
	642	+	// a link failure preserves it for the next attempt exactly as
	643	+	// a failed synchronous send did. The pump hands out no newer
	644	+	// delivery meanwhile, which bounds queued input.
	645	+	if err := await(from, event.input.actionID); err != nil {
	646	+	return w.settle(ctx, fg, stream, token, err)
572	647		}
573	648		}
574	649		if event.action != 0 {
	650	+	// The lifecycle message must reach the daemon before the
	651	+	// supervisor closes the stream.
	652	+	if err := w.drainOutbox(ctx, fg, stream); err != nil {
	653	+	return w.settle(ctx, fg, stream, token, err)
	654	+	}
575	655		return attachmentLifecycleEnd(token, event.action)
576	656		}
	657	+	case <-progress:
	658	+	if err := ackWritten(); err != nil {
	659	+	return w.settle(ctx, fg, stream, token, err)
	660	+	}
	661	+	case <-stream.Failed():
	662	+	return w.settle(ctx, fg, stream, token, stream.failure())
577	663		case <-ctx.Done():
578	664		return w.settle(ctx, fg, stream, token, ctx.Err())
579	665		case <-fg.Done():

@@ -582,6 +668,14 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
582	668		}
583	669		}
584	670
	671	+	// pendingInputAck is an input delivery whose messages the outbox accepted but
	672	+	// has not written yet: accepted messages numbered (from, mark].
	673	+	type pendingInputAck struct {
	674	+	from uint64
	675	+	mark uint64
	676	+	actionID uint64
	677	+	}
	678	+
585	679		// attachmentLifecycleEnd gives local and daemon-triggered lifecycle actions
586	680		// the same settlement path: drain/revoke foreground before publishing picker.
587	681		func attachmentLifecycleEnd(token AttachmentToken, action AttachmentLifecycleActionKind) AttachmentEvent {

@@ -800,6 +894,15 @@ func (w *sessionAttachmentWorker) hello(stream ports.BrokerLogicalConnection) pr
800	894		// wins a blocked send, closing the stream interrupts the carriage and the
801	895		// worker joins the send before returning, so no per-send goroutine is leaked.
802	896		func (w *sessionAttachmentWorker) send(ctx context.Context, fg AttachmentForeground, stream ports.BrokerLogicalConnection, message protocol.ClientMessage) error {
	897	+	if outbox, ok := stream.(*attachmentOutbox); ok {
	898	+	// Enqueue returns at once unless the outbox is full, and then still
	899	+	// honours cancellation, so no helper goroutine is needed.
	900	+	err := outbox.enqueue(ctx, fg.Done(), message)
	901	+	if errors.Is(err, errAttachmentForegroundRevoked) \|\| (ctx.Err() != nil && errors.Is(err, ctx.Err())) {
	902	+	_ = outbox.Close()
	903	+	}
	904	+	return err
	905	+	}
803	906		sent := make(chan error, 1)
804	907		go func() { sent <- stream.SendClient(message) }()
805	908		select {

@@ -816,6 +919,32 @@ func (w *sessionAttachmentWorker) send(ctx context.Context, fg AttachmentForegro
816	919		}
817	920		}
818	921
	922	+	// drainOutboxTimeout bounds how long a lifecycle exit waits for queued
	923	+	// messages on a stalled link. Past it the client leaves anyway; the daemon
	924	+	// parks the attachment when the detach never arrives.
	925	+	const drainOutboxTimeout = 2 * time.Second
	926	+
	927	+	// drainOutbox waits until every queued message was written, the bound
	928	+	// elapses, or the run ends.
	929	+	func (w sessionAttachmentWorker) drainOutbox(ctx context.Context, fg AttachmentForeground, outbox attachmentOutbox) error {
	930	+	timer := w.cfg.Clock.NewTimer(drainOutboxTimeout)
	931	+	defer stopSupervisorTimer(timer)
	932	+	select {
	933	+	case <-timer.C():
	934	+	return nil
	935	+	case <-outbox.Idle():
	936	+	return outbox.failure()
	937	+	case <-outbox.Failed():
	938	+	return outbox.failure()
	939	+	case <-ctx.Done():
	940	+	_ = outbox.Close()
	941	+	return ctx.Err()
	942	+	case <-fg.Done():
	943	+	_ = outbox.Close()
	944	+	return errAttachmentForegroundRevoked
	945	+	}
	946	+	}
	947	+
819	948		// awaitServer waits for the next server message while the run is live.
820	949		func (w *sessionAttachmentWorker) awaitServer(ctx context.Context, fg AttachmentForeground, incoming <-chan serverReceive) (protocol.ServerMessage, error) {
821	950		select {

@@ -861,6 +990,11 @@ func (w *sessionAttachmentWorker) sendClosed(stream ports.BrokerLogicalConnectio
861	990		if supervisorNil(stream) {
862	991		return
863	992		}
	993	+	if outbox, ok := stream.(*attachmentOutbox); ok {
	994	+	// Bypass the queue: the run is ending and its sender stops with it.
	995	+	outbox.stop()
	996	+	stream = outbox.BrokerLogicalConnection
	997	+	}
864	998		sent := make(chan error, 1)
865	999		go func() { sent <- stream.SendClient(protocol.Detach{Closed: true}) }()
866	1000		timer := w.cfg.Clock.NewTimer(closedDetachTimeout)


@@ -301,9 +301,13 @@ var (
301			301
302		// pumpAttached publishes output and concurrently forwards the foreground's	302		// pumpAttached publishes output and concurrently forwards the foreground's
303		// authorized input, latest geometry and explicit lifecycle decision.	303		// authorized input, latest geometry and explicit lifecycle decision.
304	-	func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg AttachmentForeground, stream ports.BrokerLogicalConnection, token AttachmentToken, state outputApplyState, incoming <-chan serverReceive) AttachmentEvent {	304	+	func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg AttachmentForeground, raw ports.BrokerLogicalConnection, token AttachmentToken, state outputApplyState, incoming <-chan serverReceive) AttachmentEvent {
305		pumpCtx, cancel := context.WithCancel(ctx)	305		pumpCtx, cancel := context.WithCancel(ctx)
306		defer cancel()	306		defer cancel()
			307	+	// Every send while attached goes through the outbox so a stalled link
			308	+	// never blocks key handling, overlays, or painting.
			309	+	stream := newAttachmentOutbox(raw)
			310	+	defer stream.stop()
307		outgoing := make(chan attachmentClientEvent)	311		outgoing := make(chan attachmentClientEvent)
308		go pumpAttachmentInput(pumpCtx, fg, outgoing)	312		go pumpAttachmentInput(pumpCtx, fg, outgoing)
309		go pumpAttachmentGeometry(pumpCtx, fg, outgoing)	313		go pumpAttachmentGeometry(pumpCtx, fg, outgoing)

@@ -345,7 +349,73 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
345		if err := sendFocus(); err != nil {	349		if err := sendFocus(); err != nil {
346		return w.settle(ctx, fg, stream, token, err)	350		return w.settle(ctx, fg, stream, token, err)
347		}	351		}
			352	+	// pending is the one undecided input delivery waiting on the outbox.
			353	+	var pending *pendingInputAck
			354	+	defer func() {
			355	+	if pending == nil {
			356	+	return
			357	+	}
			358	+	// Join the sender before deciding which input reached the carriage.
			359	+	// A written automation action remains OutcomeUnknown if teardown
			360	+	// prevents its UIFence receipt; acknowledging input is not proof
			361	+	// that the daemon processed the action.
			362	+	stream.join()
			363	+	if unsent := stream.unsentInput(pending.from, pending.mark); len(unsent) != 0 {
			364	+	fg.PreserveInput(unsent)
			365	+	return
			366	+	}
			367	+	fg.AckInput()
			368	+	}()
			369	+	// ackWritten commits pending once the outbox wrote it, then fences its
			370	+	// UI action after it.
			371	+	ackWritten := func() error {
			372	+	if pending == nil \|\| stream.writtenCount() < pending.mark {
			373	+	return nil
			374	+	}
			375	+	actionID := pending.actionID
			376	+	pending = nil
			377	+	fg.AckInput()
			378	+	if actionID != 0 {
			379	+	return w.send(ctx, fg, stream, protocol.UIFence{ActionID: actionID})
			380	+	}
			381	+	return nil
			382	+	}
			383	+	// await makes the delivery just routed pending until the outbox wrote
			384	+	// every message accepted since from.
			385	+	await := func(from, actionID uint64) error {
			386	+	mark := stream.accepted()
			387	+	if mark == from {
			388	+	// Nothing was queued (an overlay consumed the keys), so there is
			389	+	// no write to wait for: an earlier stalled message must not hold
			390	+	// the overlay's next key.
			391	+	fg.AckInput()
			392	+	if actionID != 0 {
			393	+	return w.send(ctx, fg, stream, protocol.UIFence{ActionID: actionID})
			394	+	}
			395	+	return nil
			396	+	}
			397	+	pending = &pendingInputAck{from: from, mark: mark, actionID: actionID}
			398	+	return ackWritten()
			399	+	}
348		for {	400		for {
			401	+	// Failure also closes progress. Settle it before rearming that
			402	+	// channel, rather than spinning on an acknowledgement that cannot
			403	+	// advance.
			404	+	select {
			405	+	case <-stream.Failed():
			406	+	return w.settle(ctx, fg, stream, token, stream.failure())
			407	+	default:
			408	+	}
			409	+	var progress <-chan struct{}
			410	+	if pending != nil {
			411	+	progress = stream.Progress()
			412	+	if err := ackWritten(); err != nil {
			413	+	return w.settle(ctx, fg, stream, token, err)
			414	+	}
			415	+	if pending == nil {
			416	+	progress = nil
			417	+	}
			418	+	}
349		select {	419		select {
350		case <-focus.changed:	420		case <-focus.changed:
351		if err := sendFocus(); err != nil {	421		if err := sendFocus(); err != nil {

@@ -517,13 +587,16 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
517		if event.input != nil && event.input.actionID == 0 {	587		if event.input != nil && event.input.actionID == 0 {
518		// A physical read: replies are stripped and ordinary bytes	588		// A physical read: replies are stripped and ordinary bytes
519		// reach the overlay or the session inside read.	589		// reach the overlay or the session inside read.
			590	+	from := stream.accepted()
520		if err := input.read(ctx, state, event.input.Data); err != nil {	591		if err := input.read(ctx, state, event.input.Data); err != nil {
521		// Part of the read may already be delivered or held, so	592		// Part of the read may already be delivered or held, so
522		// it is committed rather than replayed.	593		// it is committed rather than replayed.
523		fg.AckInput()	594		fg.AckInput()
524		return w.settle(ctx, fg, stream, token, err)	595		return w.settle(ctx, fg, stream, token, err)
525		}	596		}
526	-	fg.AckInput()	597	+	if err := await(from, 0); err != nil {
			598	+	return w.settle(ctx, fg, stream, token, err)
			599	+	}
527		continue	600		continue
528		}	601		}
529		if event.input != nil {	602		if event.input != nil {

@@ -557,6 +630,7 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
557		return w.settle(ctx, fg, stream, token, err)	630		return w.settle(ctx, fg, stream, token, err)
558		}	631		}
559		}	632		}
			633	+	from := stream.accepted()
560		if err := w.send(ctx, fg, stream, event.message); err != nil {	634		if err := w.send(ctx, fg, stream, event.message); err != nil {
561		if event.input != nil {	635		if event.input != nil {
562		fg.PreserveInput(event.input.Data)	636		fg.PreserveInput(event.input.Data)

@@ -564,16 +638,28 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
564		return w.settle(ctx, fg, stream, token, err)	638		return w.settle(ctx, fg, stream, token, err)
565		}	639		}
566		if event.input != nil {	640		if event.input != nil {
567	-	fg.AckInput()	641	+	// The delivery stays undecided until the outbox wrote it, so
568	-	if event.input.actionID != 0 {	642	+	// a link failure preserves it for the next attempt exactly as
569	-	if err := w.send(ctx, fg, stream, protocol.UIFence{ActionID: event.input.actionID}); err != nil {	643	+	// a failed synchronous send did. The pump hands out no newer
570	-	return w.settle(ctx, fg, stream, token, err)	644	+	// delivery meanwhile, which bounds queued input.
571	-	}	645	+	if err := await(from, event.input.actionID); err != nil {
			646	+	return w.settle(ctx, fg, stream, token, err)
572		}	647		}
573		}	648		}
574		if event.action != 0 {	649		if event.action != 0 {
			650	+	// The lifecycle message must reach the daemon before the
			651	+	// supervisor closes the stream.
			652	+	if err := w.drainOutbox(ctx, fg, stream); err != nil {
			653	+	return w.settle(ctx, fg, stream, token, err)
			654	+	}
575		return attachmentLifecycleEnd(token, event.action)	655		return attachmentLifecycleEnd(token, event.action)
576		}	656		}
			657	+	case <-progress:
			658	+	if err := ackWritten(); err != nil {
			659	+	return w.settle(ctx, fg, stream, token, err)
			660	+	}
			661	+	case <-stream.Failed():
			662	+	return w.settle(ctx, fg, stream, token, stream.failure())
577		case <-ctx.Done():	663		case <-ctx.Done():
578		return w.settle(ctx, fg, stream, token, ctx.Err())	664		return w.settle(ctx, fg, stream, token, ctx.Err())
579		case <-fg.Done():	665		case <-fg.Done():

@@ -582,6 +668,14 @@ func (w *sessionAttachmentWorker) pumpAttached(ctx context.Context, fg Attachmen
582		}	668		}
583		}	669		}
584			670
			671	+	// pendingInputAck is an input delivery whose messages the outbox accepted but
			672	+	// has not written yet: accepted messages numbered (from, mark].
			673	+	type pendingInputAck struct {
			674	+	from uint64
			675	+	mark uint64
			676	+	actionID uint64
			677	+	}
			678	+
585		// attachmentLifecycleEnd gives local and daemon-triggered lifecycle actions	679		// attachmentLifecycleEnd gives local and daemon-triggered lifecycle actions
586		// the same settlement path: drain/revoke foreground before publishing picker.	680		// the same settlement path: drain/revoke foreground before publishing picker.
587		func attachmentLifecycleEnd(token AttachmentToken, action AttachmentLifecycleActionKind) AttachmentEvent {	681		func attachmentLifecycleEnd(token AttachmentToken, action AttachmentLifecycleActionKind) AttachmentEvent {

@@ -800,6 +894,15 @@ func (w *sessionAttachmentWorker) hello(stream ports.BrokerLogicalConnection) pr
800		// wins a blocked send, closing the stream interrupts the carriage and the	894		// wins a blocked send, closing the stream interrupts the carriage and the
801		// worker joins the send before returning, so no per-send goroutine is leaked.	895		// worker joins the send before returning, so no per-send goroutine is leaked.
802		func (w *sessionAttachmentWorker) send(ctx context.Context, fg AttachmentForeground, stream ports.BrokerLogicalConnection, message protocol.ClientMessage) error {	896		func (w *sessionAttachmentWorker) send(ctx context.Context, fg AttachmentForeground, stream ports.BrokerLogicalConnection, message protocol.ClientMessage) error {
			897	+	if outbox, ok := stream.(*attachmentOutbox); ok {
			898	+	// Enqueue returns at once unless the outbox is full, and then still
			899	+	// honours cancellation, so no helper goroutine is needed.
			900	+	err := outbox.enqueue(ctx, fg.Done(), message)
			901	+	if errors.Is(err, errAttachmentForegroundRevoked) \|\| (ctx.Err() != nil && errors.Is(err, ctx.Err())) {
			902	+	_ = outbox.Close()
			903	+	}
			904	+	return err
			905	+	}
803		sent := make(chan error, 1)	906		sent := make(chan error, 1)
804		go func() { sent <- stream.SendClient(message) }()	907		go func() { sent <- stream.SendClient(message) }()
805		select {	908		select {

@@ -816,6 +919,32 @@ func (w *sessionAttachmentWorker) send(ctx context.Context, fg AttachmentForegro
816		}	919		}
817		}	920		}
818			921
			922	+	// drainOutboxTimeout bounds how long a lifecycle exit waits for queued
			923	+	// messages on a stalled link. Past it the client leaves anyway; the daemon
			924	+	// parks the attachment when the detach never arrives.
			925	+	const drainOutboxTimeout = 2 * time.Second
			926	+
			927	+	// drainOutbox waits until every queued message was written, the bound
			928	+	// elapses, or the run ends.
			929	+	func (w sessionAttachmentWorker) drainOutbox(ctx context.Context, fg AttachmentForeground, outbox attachmentOutbox) error {
			930	+	timer := w.cfg.Clock.NewTimer(drainOutboxTimeout)
			931	+	defer stopSupervisorTimer(timer)
			932	+	select {
			933	+	case <-timer.C():
			934	+	return nil
			935	+	case <-outbox.Idle():
			936	+	return outbox.failure()
			937	+	case <-outbox.Failed():
			938	+	return outbox.failure()
			939	+	case <-ctx.Done():
			940	+	_ = outbox.Close()
			941	+	return ctx.Err()
			942	+	case <-fg.Done():
			943	+	_ = outbox.Close()
			944	+	return errAttachmentForegroundRevoked
			945	+	}
			946	+	}
			947	+
819		// awaitServer waits for the next server message while the run is live.	948		// awaitServer waits for the next server message while the run is live.
820		func (w *sessionAttachmentWorker) awaitServer(ctx context.Context, fg AttachmentForeground, incoming <-chan serverReceive) (protocol.ServerMessage, error) {	949		func (w *sessionAttachmentWorker) awaitServer(ctx context.Context, fg AttachmentForeground, incoming <-chan serverReceive) (protocol.ServerMessage, error) {
821		select {	950		select {

@@ -861,6 +990,11 @@ func (w *sessionAttachmentWorker) sendClosed(stream ports.BrokerLogicalConnectio
861		if supervisorNil(stream) {	990		if supervisorNil(stream) {
862		return	991		return
863		}	992		}
			993	+	if outbox, ok := stream.(*attachmentOutbox); ok {
			994	+	// Bypass the queue: the run is ending and its sender stops with it.
			995	+	outbox.stop()
			996	+	stream = outbox.BrokerLogicalConnection
			997	+	}
864		sent := make(chan error, 1)	998		sent := make(chan error, 1)
865		go func() { sent <- stream.SendClient(protocol.Detach{Closed: true}) }()	999		go func() { sent <- stream.SendClient(protocol.Detach{Closed: true}) }()
866		timer := w.cfg.Clock.NewTimer(closedDetachTimeout)	1000		timer := w.cfg.Clock.NewTimer(closedDetachTimeout)

M internal/usecase/client/attachment_worker.go +4 -6


@@ -1231,12 +1231,10 @@ func (f *attachmentForeground) finish() {
1231	1231		return
1232	1232		}
1233	1233		f.finishedOnce.Do(func() {
1234		-	// Autonomous supervisor foregrounds have a host-lifetime geometry
1235		-	// collector. At that owner-class boundary, no attachment byte may reach
1236		-	// the picker. Attachment-to-attachment replacement instead delivers pending input.
1237		-	if f.input != nil && f.host != nil && f.host.ownerBoundary {
1238		-	f.input.dropOwned(f.consumer)
1239		-	}
	1234	+	// Finish is lossless: input kept for a resume must survive it. The
	1235	+	// supervisor drops unclaimed bytes (dropUnclaimed) when the terminal
	1236	+	// returns to the picker, the owner-class boundary no attachment byte
	1237	+	// may cross.
1240	1238		f.authority.revoke(f)
1241	1239		})
1242	1240		}

 		return
 	}
 	f.finishedOnce.Do(func() {
-		// Autonomous supervisor foregrounds have a host-lifetime geometry
+		// Finish is lossless: input kept for a resume must survive it. The
-		// collector. At that owner-class boundary, no attachment byte may reach
+		// supervisor drops unclaimed bytes (dropUnclaimed) when the terminal
-		// the picker. Attachment-to-attachment replacement instead delivers pending input.
+		// returns to the picker, the owner-class boundary no attachment byte
-		if f.input != nil && f.host != nil && f.host.ownerBoundary {
+		// may cross.
-			f.input.dropOwned(f.consumer)
-		}
 		f.authority.revoke(f)
 	})
 }

M internal/usecase/client/lifecycle_p54a_test.go +32 -0


@@ -311,6 +311,9 @@ func TestP54AttachmentFinalizeDropsUnleasedPendingBeforePicker(t *testing.T) {
311	311		return lifetime.pump.pending != nil && lifetime.pump.delivering == 0
312	312		}, 5*time.Second, time.Millisecond)
313	313		fg.finish()
	314	+	// The supervisor drops unclaimed session input as the terminal returns to
	315	+	// the picker; finish itself stays lossless for a resume.
	316	+	lifetime.pump.dropUnclaimed()
314	317		lifetime.acquirePicker()
315	318		assertNoPickerRead(t, consumer.reads)
316	319		}

@@ -330,6 +333,9 @@ func TestP54AttachmentFinalizeDropsPreservedResidualBeforePicker(t *testing.T) {
330	333		require.True(t, ok)
331	334		fg.PreserveInput(event.Data)
332	335		fg.finish()
	336	+	// The supervisor drops unclaimed session input as the terminal returns to
	337	+	// the picker; finish itself stays lossless for a resume.
	338	+	lifetime.pump.dropUnclaimed()
333	339		lifetime.acquirePicker()
334	340		assertNoPickerRead(t, consumer.reads)
335	341		}

@@ -468,3 +474,29 @@ func TestP54LifecycleInterleavingsRace(t *testing.T) {
468	474		host.stopGeometry()
469	475		}
470	476		}
	477	+
	478	+	// TestResumeKeysSurviveFinish runs the production owner boundary: input kept
	479	+	// for a resume survives the attachment's finish and reaches the next
	480	+	// foreground. TestResumeHeldKeysNeverReachThePicker covers the picker side.
	481	+	func TestResumeKeysSurviveFinish(t *testing.T) {
	482	+	reader := newAttachTestReader()
	483	+	consumer := &recordingPickerConsumer{reads: make(chan []byte, 1)}
	484	+	lifetime := startTerminalInputLifetime(reader, consumer)
	485	+	defer lifetime.stop()
	486	+	lifetime.releasePicker()
	487	+	host := newAttachmentHost(attachmentHostConfig{Input: lifetime.pump})
	488	+	host.ownerBoundary = true
	489	+	fg := host.newForeground(AttachmentToken{Generation: 1}, newSessionTestStream())
	490	+	require.True(t, host.authority.grant(fg))
	491	+	reader.push([]byte("ls\r"))
	492	+	event, ok := fg.Input(context.Background())
	493	+	require.True(t, ok)
	494	+	fg.PreserveInput(event.Data)
	495	+	fg.finish()
	496	+
	497	+	next := host.newForeground(AttachmentToken{Generation: 2}, newSessionTestStream())
	498	+	require.True(t, host.authority.grant(next))
	499	+	replayed, ok := next.Input(context.Background())
	500	+	require.True(t, ok)
	501	+	require.Equal(t, []byte("ls\r"), replayed.Data)
	502	+	}


@@ -311,6 +311,9 @@ func TestP54AttachmentFinalizeDropsUnleasedPendingBeforePicker(t *testing.T) {
311	return lifetime.pump.pending != nil && lifetime.pump.delivering == 0	311		return lifetime.pump.pending != nil && lifetime.pump.delivering == 0
312	}, 5*time.Second, time.Millisecond)	312		}, 5*time.Second, time.Millisecond)
313	fg.finish()	313		fg.finish()
		314	+	// The supervisor drops unclaimed session input as the terminal returns to
		315	+	// the picker; finish itself stays lossless for a resume.
		316	+	lifetime.pump.dropUnclaimed()
314	lifetime.acquirePicker()	317		lifetime.acquirePicker()
315	assertNoPickerRead(t, consumer.reads)	318		assertNoPickerRead(t, consumer.reads)
316	}	319		}

@@ -330,6 +333,9 @@ func TestP54AttachmentFinalizeDropsPreservedResidualBeforePicker(t *testing.T) {
330	require.True(t, ok)	333		require.True(t, ok)
331	fg.PreserveInput(event.Data)	334		fg.PreserveInput(event.Data)
332	fg.finish()	335		fg.finish()
		336	+	// The supervisor drops unclaimed session input as the terminal returns to
		337	+	// the picker; finish itself stays lossless for a resume.
		338	+	lifetime.pump.dropUnclaimed()
333	lifetime.acquirePicker()	339		lifetime.acquirePicker()
334	assertNoPickerRead(t, consumer.reads)	340		assertNoPickerRead(t, consumer.reads)
335	}	341		}

@@ -468,3 +474,29 @@ func TestP54LifecycleInterleavingsRace(t *testing.T) {
468	host.stopGeometry()	474		host.stopGeometry()
469	}	475		}
470	}	476		}
		477	+
		478	+	// TestResumeKeysSurviveFinish runs the production owner boundary: input kept
		479	+	// for a resume survives the attachment's finish and reaches the next
		480	+	// foreground. TestResumeHeldKeysNeverReachThePicker covers the picker side.
		481	+	func TestResumeKeysSurviveFinish(t *testing.T) {
		482	+	reader := newAttachTestReader()
		483	+	consumer := &recordingPickerConsumer{reads: make(chan []byte, 1)}
		484	+	lifetime := startTerminalInputLifetime(reader, consumer)
		485	+	defer lifetime.stop()
		486	+	lifetime.releasePicker()
		487	+	host := newAttachmentHost(attachmentHostConfig{Input: lifetime.pump})
		488	+	host.ownerBoundary = true
		489	+	fg := host.newForeground(AttachmentToken{Generation: 1}, newSessionTestStream())
		490	+	require.True(t, host.authority.grant(fg))
		491	+	reader.push([]byte("ls\r"))
		492	+	event, ok := fg.Input(context.Background())
		493	+	require.True(t, ok)
		494	+	fg.PreserveInput(event.Data)
		495	+	fg.finish()
		496	+
		497	+	next := host.newForeground(AttachmentToken{Generation: 2}, newSessionTestStream())
		498	+	require.True(t, host.authority.grant(next))
		499	+	replayed, ok := next.Input(context.Background())
		500	+	require.True(t, ok)
		501	+	require.Equal(t, []byte("ls\r"), replayed.Data)
		502	+	}

A internal/usecase/client/outbox_bounds_test.go +80 -0

@@ -0,0 +1,80 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"testing"
	6	+	"time"
	7	+
	8	+	"github.com/bnema/vev/internal/protocol"
	9	+	"github.com/stretchr/testify/require"
	10	+	)
	11	+
	12	+	func TestOutboxFullCancellation(t *testing.T) {
	13	+	for _, foreground := range []bool{false, true} {
	14	+	t.Run(map[bool]string{false: "context", true: "foreground"}[foreground], func(t *testing.T) {
	15	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	16	+	box := newAttachmentOutbox(stream)
	17	+	defer box.join()
	18	+	for range outboxMaxPending {
	19	+	require.NoError(t, box.SendClient(protocol.Ack{}))
	20	+	}
	21	+	ctx, cancel := context.WithCancel(t.Context())
	22	+	defer cancel()
	23	+	revoked := make(chan struct{})
	24	+	result := make(chan error, 1)
	25	+	go func() { result <- box.enqueue(ctx, revoked, protocol.Ack{}) }()
	26	+	if foreground {
	27	+	close(revoked)
	28	+	} else {
	29	+	cancel()
	30	+	}
	31	+	select {
	32	+	case err := <-result:
	33	+	if foreground {
	34	+	require.ErrorIs(t, err, errAttachmentForegroundRevoked)
	35	+	} else {
	36	+	require.ErrorIs(t, err, context.Canceled)
	37	+	}
	38	+	case <-time.After(time.Second):
	39	+	t.Fatal("full outbox ignored cancellation")
	40	+	}
	41	+	})
	42	+	}
	43	+	}
	44	+
	45	+	// TestOutboxFullAdmitsAfterProgress checks that a caller blocked on a full
	46	+	// outbox is admitted once the sender writes and frees space.
	47	+	func TestOutboxFullAdmitsAfterProgress(t *testing.T) {
	48	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	49	+	box := newAttachmentOutbox(stream)
	50	+	defer box.join()
	51	+	for range outboxMaxPending {
	52	+	require.NoError(t, box.SendClient(protocol.Ack{}))
	53	+	}
	54	+	result := make(chan error, 1)
	55	+	go func() { result <- box.enqueue(t.Context(), nil, protocol.Detach{}) }()
	56	+	select {
	57	+	case err := <-result:
	58	+	t.Fatalf("full outbox admitted before progress: %v", err)
	59	+	case <-time.After(50 * time.Millisecond):
	60	+	}
	61	+	close(stream.release)
	62	+	select {
	63	+	case err := <-result:
	64	+	require.NoError(t, err)
	65	+	case <-time.After(time.Second):
	66	+	t.Fatal("blocked caller was not admitted after progress")
	67	+	}
	68	+	require.Eventually(t, func() bool { return len(stream.messages()) == outboxMaxPending+1 }, time.Second, time.Millisecond)
	69	+	}
	70	+
	71	+	func TestOutboxMixedUnsentInput(t *testing.T) {
	72	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	73	+	box := newAttachmentOutbox(stream)
	74	+	defer box.join()
	75	+	for _, message := range []protocol.ClientMessage{protocol.Ack{}, protocol.Input{Data: []byte("a")}, protocol.TerminalFocus{}, protocol.Input{Data: []byte("b")}} {
	76	+	require.NoError(t, box.SendClient(message))
	77	+	}
	78	+	require.Equal(t, []byte("ab"), box.unsentInput(0, box.accepted()))
	79	+	require.Equal(t, []byte("b"), box.unsentInput(2, box.accepted()))
	80	+	}

@@ -0,0 +1,80 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"testing"
	6	+	"time"
	7	+
	8	+	"github.com/bnema/vev/internal/protocol"
	9	+	"github.com/stretchr/testify/require"
	10	+	)
	11	+
	12	+	func TestOutboxFullCancellation(t *testing.T) {
	13	+	for _, foreground := range []bool{false, true} {
	14	+	t.Run(map[bool]string{false: "context", true: "foreground"}[foreground], func(t *testing.T) {
	15	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	16	+	box := newAttachmentOutbox(stream)
	17	+	defer box.join()
	18	+	for range outboxMaxPending {
	19	+	require.NoError(t, box.SendClient(protocol.Ack{}))
	20	+	}
	21	+	ctx, cancel := context.WithCancel(t.Context())
	22	+	defer cancel()
	23	+	revoked := make(chan struct{})
	24	+	result := make(chan error, 1)
	25	+	go func() { result <- box.enqueue(ctx, revoked, protocol.Ack{}) }()
	26	+	if foreground {
	27	+	close(revoked)
	28	+	} else {
	29	+	cancel()
	30	+	}
	31	+	select {
	32	+	case err := <-result:
	33	+	if foreground {
	34	+	require.ErrorIs(t, err, errAttachmentForegroundRevoked)
	35	+	} else {
	36	+	require.ErrorIs(t, err, context.Canceled)
	37	+	}
	38	+	case <-time.After(time.Second):
	39	+	t.Fatal("full outbox ignored cancellation")
	40	+	}
	41	+	})
	42	+	}
	43	+	}
	44	+
	45	+	// TestOutboxFullAdmitsAfterProgress checks that a caller blocked on a full
	46	+	// outbox is admitted once the sender writes and frees space.
	47	+	func TestOutboxFullAdmitsAfterProgress(t *testing.T) {
	48	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	49	+	box := newAttachmentOutbox(stream)
	50	+	defer box.join()
	51	+	for range outboxMaxPending {
	52	+	require.NoError(t, box.SendClient(protocol.Ack{}))
	53	+	}
	54	+	result := make(chan error, 1)
	55	+	go func() { result <- box.enqueue(t.Context(), nil, protocol.Detach{}) }()
	56	+	select {
	57	+	case err := <-result:
	58	+	t.Fatalf("full outbox admitted before progress: %v", err)
	59	+	case <-time.After(50 * time.Millisecond):
	60	+	}
	61	+	close(stream.release)
	62	+	select {
	63	+	case err := <-result:
	64	+	require.NoError(t, err)
	65	+	case <-time.After(time.Second):
	66	+	t.Fatal("blocked caller was not admitted after progress")
	67	+	}
	68	+	require.Eventually(t, func() bool { return len(stream.messages()) == outboxMaxPending+1 }, time.Second, time.Millisecond)
	69	+	}
	70	+
	71	+	func TestOutboxMixedUnsentInput(t *testing.T) {
	72	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	73	+	box := newAttachmentOutbox(stream)
	74	+	defer box.join()
	75	+	for _, message := range []protocol.ClientMessage{protocol.Ack{}, protocol.Input{Data: []byte("a")}, protocol.TerminalFocus{}, protocol.Input{Data: []byte("b")}} {
	76	+	require.NoError(t, box.SendClient(message))
	77	+	}
	78	+	require.Equal(t, []byte("ab"), box.unsentInput(0, box.accepted()))
	79	+	require.Equal(t, []byte("b"), box.unsentInput(2, box.accepted()))
	80	+	}

A internal/usecase/client/outbox_drain_test.go +64 -0

@@ -0,0 +1,64 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"errors"
	6	+	"testing"
	7	+	"time"
	8	+
	9	+	"github.com/bnema/vev/internal/protocol"
	10	+	"github.com/stretchr/testify/require"
	11	+	)
	12	+
	13	+	func TestOutboxDrain(t *testing.T) {
	14	+	linkErr := errors.New("link lost")
	15	+	for _, mode := range []string{"written", "timeout", "cancelled", "failed", "revoked"} {
	16	+	t.Run(mode, func(t *testing.T) {
	17	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	18	+	box := newAttachmentOutbox(stream)
	19	+	defer box.join()
	20	+	require.NoError(t, box.SendClient(protocol.Detach{}))
	21	+	clock := newSupervisorTestClock()
	22	+	worker := &sessionAttachmentWorker{cfg: sessionAttachmentConfig{Clock: clock}}
	23	+	host := newAttachmentHost(attachmentHostConfig{})
	24	+	fg := host.newForeground(AttachmentToken{Generation: 1}, stream)
	25	+	ctx, cancel := context.WithCancel(t.Context())
	26	+	defer cancel()
	27	+	result := make(chan error, 1)
	28	+	go func() { result <- worker.drainOutbox(ctx, fg, box) }()
	29	+	timer := clock.awaitTimer(t)
	30	+	require.Equal(t, 2*time.Second, timer.delay)
	31	+	switch mode {
	32	+	case "written":
	33	+	close(stream.release)
	34	+	case "timeout":
	35	+	timer.fire()
	36	+	case "cancelled":
	37	+	cancel()
	38	+	case "failed":
	39	+	stream.fail = linkErr
	40	+	close(stream.release)
	41	+	case "revoked":
	42	+	fg.finalize()
	43	+	}
	44	+	select {
	45	+	case err := <-result:
	46	+	switch mode {
	47	+	case "cancelled":
	48	+	require.ErrorIs(t, err, context.Canceled)
	49	+	case "failed":
	50	+	require.ErrorIs(t, err, linkErr)
	51	+	case "revoked":
	52	+	require.ErrorIs(t, err, errAttachmentForegroundRevoked)
	53	+	default:
	54	+	require.NoError(t, err)
	55	+	}
	56	+	case <-time.After(time.Second):
	57	+	t.Fatal("drain did not settle")
	58	+	}
	59	+	if mode == "written" {
	60	+	require.Len(t, stream.messages(), 1)
	61	+	}
	62	+	})
	63	+	}
	64	+	}

@@ -0,0 +1,64 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"errors"
	6	+	"testing"
	7	+	"time"
	8	+
	9	+	"github.com/bnema/vev/internal/protocol"
	10	+	"github.com/stretchr/testify/require"
	11	+	)
	12	+
	13	+	func TestOutboxDrain(t *testing.T) {
	14	+	linkErr := errors.New("link lost")
	15	+	for _, mode := range []string{"written", "timeout", "cancelled", "failed", "revoked"} {
	16	+	t.Run(mode, func(t *testing.T) {
	17	+	stream := &gatedSessionStream{sessionTestStream: newSessionTestStream(), release: make(chan struct{})}
	18	+	box := newAttachmentOutbox(stream)
	19	+	defer box.join()
	20	+	require.NoError(t, box.SendClient(protocol.Detach{}))
	21	+	clock := newSupervisorTestClock()
	22	+	worker := &sessionAttachmentWorker{cfg: sessionAttachmentConfig{Clock: clock}}
	23	+	host := newAttachmentHost(attachmentHostConfig{})
	24	+	fg := host.newForeground(AttachmentToken{Generation: 1}, stream)
	25	+	ctx, cancel := context.WithCancel(t.Context())
	26	+	defer cancel()
	27	+	result := make(chan error, 1)
	28	+	go func() { result <- worker.drainOutbox(ctx, fg, box) }()
	29	+	timer := clock.awaitTimer(t)
	30	+	require.Equal(t, 2*time.Second, timer.delay)
	31	+	switch mode {
	32	+	case "written":
	33	+	close(stream.release)
	34	+	case "timeout":
	35	+	timer.fire()
	36	+	case "cancelled":
	37	+	cancel()
	38	+	case "failed":
	39	+	stream.fail = linkErr
	40	+	close(stream.release)
	41	+	case "revoked":
	42	+	fg.finalize()
	43	+	}
	44	+	select {
	45	+	case err := <-result:
	46	+	switch mode {
	47	+	case "cancelled":
	48	+	require.ErrorIs(t, err, context.Canceled)
	49	+	case "failed":
	50	+	require.ErrorIs(t, err, linkErr)
	51	+	case "revoked":
	52	+	require.ErrorIs(t, err, errAttachmentForegroundRevoked)
	53	+	default:
	54	+	require.NoError(t, err)
	55	+	}
	56	+	case <-time.After(time.Second):
	57	+	t.Fatal("drain did not settle")
	58	+	}
	59	+	if mode == "written" {
	60	+	require.Len(t, stream.messages(), 1)
	61	+	}
	62	+	})
	63	+	}
	64	+	}

A internal/usecase/client/resume_held_input_test.go +115 -0

@@ -0,0 +1,115 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"strings"
	6	+	"testing"
	7	+	"time"
	8	+
	9	+	"github.com/bnema/vev/internal/ports"
	10	+	"github.com/bnema/vev/internal/protocol"
	11	+	"github.com/stretchr/testify/require"
	12	+	)
	13	+
	14	+	// sentInput joins every Input message a stream carried, in order.
	15	+	func sentInput(stream *sessionTestStream) string {
	16	+	var out strings.Builder
	17	+	for _, message := range stream.messages() {
	18	+	if input, ok := message.(protocol.Input); ok {
	19	+	out.Write(input.Data)
	20	+	}
	21	+	}
	22	+	return out.String()
	23	+	}
	24	+
	25	+	// pushKeys feeds the terminal one byte per read, like a keyboard, and waits
	26	+	// until the shared reader consumed them all.
	27	+	func pushKeys(t testing.T, h attachTestHarness, keys string) {
	28	+	t.Helper()
	29	+	for _, b := range []byte(keys) {
	30	+	h.reader.push([]byte{b})
	31	+	}
	32	+	require.Eventually(t, func() bool { return len(h.reader.chunks) == 0 }, 5*time.Second, time.Millisecond)
	33	+	}
	34	+
	35	+	// TestResumeReplaysKeysHeldWhileConnecting types while a lost attachment waits
	36	+	// for its backoff and while the resume stream opens (including a failed open
	37	+	// that is retried), then proves the resumed session receives every key, once,
	38	+	// in order, and the lost attachment none.
	39	+	func TestResumeReplaysKeysHeldWhileConnecting(t *testing.T) {
	40	+	for _, tt := range []struct {
	41	+	name string
	42	+	backoffKeys string
	43	+	openKeys string
	44	+	failFirst bool
	45	+	retryKeys string
	46	+	}{
	47	+	{name: "typed during backoff", backoffKeys: "HELD_ONE"},
	48	+	{name: "typed during stream open", openKeys: "HELD_TWO\r"},
	49	+	{name: "typed during backoff and open", backoffKeys: "HELD_ONE", openKeys: "HELD_TWO\r"},
	50	+	{name: "kept across a failed open", openKeys: "HELD_ONE", failFirst: true, retryKeys: "HELD_TWO\r"},
	51	+	} {
	52	+	t.Run(tt.name, func(t *testing.T) {
	53	+	picker := newAttachTestPicker()
	54	+	h := startAttachHarnessConfig(t, picker, func(cfg *SupervisorConfig) { cfg.Jitter = resumeTestJitter })
	55	+	first := newSessionTestStream()
	56	+	second := newSessionTestStream()
	57	+	opened := make(chan int, 4)
	58	+	failed := make(chan struct{})
	59	+	release := make(chan struct{})
	60	+	count := 0
	61	+	h.service.setOpenStream(func(ctx context.Context, _ ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
	62	+	count++
	63	+	switch {
	64	+	case count == 1:
	65	+	return first, nil
	66	+	case tt.failFirst && count == 2:
	67	+	opened <- count
	68	+	select {
	69	+	case <-failed:
	70	+	case <-ctx.Done():
	71	+	}
	72	+	return nil, ports.BrokerError{Code: ports.BrokerErrorUnavailable, Text: "route down"}
	73	+	}
	74	+	opened <- count
	75	+	select {
	76	+	case <-release:
	77	+	case <-ctx.Done():
	78	+	return nil, ctx.Err()
	79	+	}
	80	+	return second, nil
	81	+	})
	82	+	awaitOpen := func() {
	83	+	select {
	84	+	case <-opened:
	85	+	case <-time.After(5 * time.Second):
	86	+	t.Fatal("resume did not open")
	87	+	}
	88	+	}
	89	+	picker.commit(sessionTestRequest(false))
	90	+	deliverReadyStream(t, first)
	91	+	awaitAttachedState(t, h.sup)
	92	+	first.fail(testStreamLoss())
	93	+
	94	+	require.Eventually(t, func() bool { return h.sup.State().Presentation == PresentConnecting }, 5*time.Second, time.Millisecond)
	95	+	pushKeys(t, h, tt.backoffKeys)
	96	+	fireResumeTimerAfter(t, h.clock, 1, func() {})
	97	+	awaitOpen()
	98	+	pushKeys(t, h, tt.openKeys)
	99	+	if tt.failFirst {
	100	+	close(failed)
	101	+	fireResumeTimerAfter(t, h.clock, 2, func() {})
	102	+	awaitOpen()
	103	+	pushKeys(t, h, tt.retryKeys)
	104	+	}
	105	+	close(release)
	106	+	deliverReadyStream(t, second)
	107	+	awaitAttachedState(t, h.sup)
	108	+
	109	+	want := tt.backoffKeys + tt.openKeys + tt.retryKeys
	110	+	require.Eventually(t, func() bool { return sentInput(second) == want }, 5*time.Second, time.Millisecond,
	111	+	"resumed session input = %q, want %q", sentInput(second), want)
	112	+	require.Empty(t, sentInput(first))
	113	+	})
	114	+	}
	115	+	}

@@ -0,0 +1,115 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"strings"
	6	+	"testing"
	7	+	"time"
	8	+
	9	+	"github.com/bnema/vev/internal/ports"
	10	+	"github.com/bnema/vev/internal/protocol"
	11	+	"github.com/stretchr/testify/require"
	12	+	)
	13	+
	14	+	// sentInput joins every Input message a stream carried, in order.
	15	+	func sentInput(stream *sessionTestStream) string {
	16	+	var out strings.Builder
	17	+	for _, message := range stream.messages() {
	18	+	if input, ok := message.(protocol.Input); ok {
	19	+	out.Write(input.Data)
	20	+	}
	21	+	}
	22	+	return out.String()
	23	+	}
	24	+
	25	+	// pushKeys feeds the terminal one byte per read, like a keyboard, and waits
	26	+	// until the shared reader consumed them all.
	27	+	func pushKeys(t testing.T, h attachTestHarness, keys string) {
	28	+	t.Helper()
	29	+	for _, b := range []byte(keys) {
	30	+	h.reader.push([]byte{b})
	31	+	}
	32	+	require.Eventually(t, func() bool { return len(h.reader.chunks) == 0 }, 5*time.Second, time.Millisecond)
	33	+	}
	34	+
	35	+	// TestResumeReplaysKeysHeldWhileConnecting types while a lost attachment waits
	36	+	// for its backoff and while the resume stream opens (including a failed open
	37	+	// that is retried), then proves the resumed session receives every key, once,
	38	+	// in order, and the lost attachment none.
	39	+	func TestResumeReplaysKeysHeldWhileConnecting(t *testing.T) {
	40	+	for _, tt := range []struct {
	41	+	name string
	42	+	backoffKeys string
	43	+	openKeys string
	44	+	failFirst bool
	45	+	retryKeys string
	46	+	}{
	47	+	{name: "typed during backoff", backoffKeys: "HELD_ONE"},
	48	+	{name: "typed during stream open", openKeys: "HELD_TWO\r"},
	49	+	{name: "typed during backoff and open", backoffKeys: "HELD_ONE", openKeys: "HELD_TWO\r"},
	50	+	{name: "kept across a failed open", openKeys: "HELD_ONE", failFirst: true, retryKeys: "HELD_TWO\r"},
	51	+	} {
	52	+	t.Run(tt.name, func(t *testing.T) {
	53	+	picker := newAttachTestPicker()
	54	+	h := startAttachHarnessConfig(t, picker, func(cfg *SupervisorConfig) { cfg.Jitter = resumeTestJitter })
	55	+	first := newSessionTestStream()
	56	+	second := newSessionTestStream()
	57	+	opened := make(chan int, 4)
	58	+	failed := make(chan struct{})
	59	+	release := make(chan struct{})
	60	+	count := 0
	61	+	h.service.setOpenStream(func(ctx context.Context, _ ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
	62	+	count++
	63	+	switch {
	64	+	case count == 1:
	65	+	return first, nil
	66	+	case tt.failFirst && count == 2:
	67	+	opened <- count
	68	+	select {
	69	+	case <-failed:
	70	+	case <-ctx.Done():
	71	+	}
	72	+	return nil, ports.BrokerError{Code: ports.BrokerErrorUnavailable, Text: "route down"}
	73	+	}
	74	+	opened <- count
	75	+	select {
	76	+	case <-release:
	77	+	case <-ctx.Done():
	78	+	return nil, ctx.Err()
	79	+	}
	80	+	return second, nil
	81	+	})
	82	+	awaitOpen := func() {
	83	+	select {
	84	+	case <-opened:
	85	+	case <-time.After(5 * time.Second):
	86	+	t.Fatal("resume did not open")
	87	+	}
	88	+	}
	89	+	picker.commit(sessionTestRequest(false))
	90	+	deliverReadyStream(t, first)
	91	+	awaitAttachedState(t, h.sup)
	92	+	first.fail(testStreamLoss())
	93	+
	94	+	require.Eventually(t, func() bool { return h.sup.State().Presentation == PresentConnecting }, 5*time.Second, time.Millisecond)
	95	+	pushKeys(t, h, tt.backoffKeys)
	96	+	fireResumeTimerAfter(t, h.clock, 1, func() {})
	97	+	awaitOpen()
	98	+	pushKeys(t, h, tt.openKeys)
	99	+	if tt.failFirst {
	100	+	close(failed)
	101	+	fireResumeTimerAfter(t, h.clock, 2, func() {})
	102	+	awaitOpen()
	103	+	pushKeys(t, h, tt.retryKeys)
	104	+	}
	105	+	close(release)
	106	+	deliverReadyStream(t, second)
	107	+	awaitAttachedState(t, h.sup)
	108	+
	109	+	want := tt.backoffKeys + tt.openKeys + tt.retryKeys
	110	+	require.Eventually(t, func() bool { return sentInput(second) == want }, 5*time.Second, time.Millisecond,
	111	+	"resumed session input = %q, want %q", sentInput(second), want)
	112	+	require.Empty(t, sentInput(first))
	113	+	})
	114	+	}
	115	+	}

A internal/usecase/client/resume_input.go +96 -0

@@ -0,0 +1,96 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"bytes"
	5	+	"time"
	6	+
	7	+	"github.com/bnema/vev/internal/usecase/keys/kittykey"
	8	+	)
	9	+
	10	+	const resumeEscapeLimit = 256
	11	+
	12	+	// resumeEscapeDelay is how long a lone Esc stays undecided before it counts
	13	+	// as a cancel rather than the start of an escape sequence.
	14	+	const resumeEscapeDelay = 50 * time.Millisecond
	15	+
	16	+	// resumeInputDecoder retains only an undecided escape prefix. Held bytes stay
	17	+	// in their original terminal encoding; paste content is never a cancel key.
	18	+	type resumeInputDecoder struct {
	19	+	prefix []byte
	20	+	paste bool
	21	+	}
	22	+
	23	+	func (d *resumeInputDecoder) feed(data []byte) (held []byte, cancel bool) {
	24	+	for _, b := range data {
	25	+	if len(d.prefix) == 0 {
	26	+	if b == 0x1b {
	27	+	d.prefix = append(d.prefix, b)
	28	+	continue
	29	+	}
	30	+	if b == 3 && !d.paste {
	31	+	return held, true
	32	+	}
	33	+	held = append(held, b)
	34	+	continue
	35	+	}
	36	+	if b == 0x1b \|\| (b < 0x20 && !d.paste) {
	37	+	held = append(held, d.prefix...)
	38	+	d.prefix = d.prefix[:0]
	39	+	if b == 3 {
	40	+	return held, true
	41	+	}
	42	+	if b == 0x1b {
	43	+	d.prefix = append(d.prefix, b)
	44	+	} else {
	45	+	held = append(held, b)
	46	+	}
	47	+	continue
	48	+	}
	49	+	d.prefix = append(d.prefix, b)
	50	+	if d.paste {
	51	+	end := []byte("\x1b[201~")
	52	+	if bytes.Equal(d.prefix, end) {
	53	+	d.paste = false
	54	+	} else if bytes.HasPrefix(end, d.prefix) {
	55	+	continue
	56	+	}
	57	+	} else {
	58	+	start := []byte("\x1b[200~")
	59	+	if bytes.Equal(d.prefix, start) {
	60	+	d.paste = true
	61	+	} else if bytes.HasPrefix(start, d.prefix) {
	62	+	continue
	63	+	} else {
	64	+	_, n, ok, partial := kittykey.Parse(d.prefix)
	65	+	if ok {
	66	+	translated := kittykey.Translate(d.prefix[:n], 0)
	67	+	if len(translated) == 1 && (translated[0] == 3 \|\| translated[0] == 0x1b) {
	68	+	return held, true
	69	+	}
	70	+	} else if partial && len(d.prefix) < resumeEscapeLimit {
	71	+	continue
	72	+	} else if len(d.prefix) >= 2 && (d.prefix[1] == '[' \|\| d.prefix[1] == 'O') && b >= 0x20 && b < 0x40 && len(d.prefix) < resumeEscapeLimit {
	73	+	continue
	74	+	}
	75	+	}
	76	+	}
	77	+	held = append(held, d.prefix...)
	78	+	d.prefix = d.prefix[:0]
	79	+	}
	80	+	return held, false
	81	+	}
	82	+
	83	+	// undecided reports whether an escape prefix is still waiting for more bytes.
	84	+	func (d *resumeInputDecoder) undecided() bool {
	85	+	return len(d.prefix) != 0
	86	+	}
	87	+
	88	+	func (d *resumeInputDecoder) loneEscape() bool {
	89	+	return !d.paste && bytes.Equal(d.prefix, []byte{0x1b})
	90	+	}
	91	+
	92	+	func (d *resumeInputDecoder) flush() []byte {
	93	+	data := d.prefix
	94	+	d.prefix = nil
	95	+	return data
	96	+	}

@@ -0,0 +1,96 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"bytes"
	5	+	"time"
	6	+
	7	+	"github.com/bnema/vev/internal/usecase/keys/kittykey"
	8	+	)
	9	+
	10	+	const resumeEscapeLimit = 256
	11	+
	12	+	// resumeEscapeDelay is how long a lone Esc stays undecided before it counts
	13	+	// as a cancel rather than the start of an escape sequence.
	14	+	const resumeEscapeDelay = 50 * time.Millisecond
	15	+
	16	+	// resumeInputDecoder retains only an undecided escape prefix. Held bytes stay
	17	+	// in their original terminal encoding; paste content is never a cancel key.
	18	+	type resumeInputDecoder struct {
	19	+	prefix []byte
	20	+	paste bool
	21	+	}
	22	+
	23	+	func (d *resumeInputDecoder) feed(data []byte) (held []byte, cancel bool) {
	24	+	for _, b := range data {
	25	+	if len(d.prefix) == 0 {
	26	+	if b == 0x1b {
	27	+	d.prefix = append(d.prefix, b)
	28	+	continue
	29	+	}
	30	+	if b == 3 && !d.paste {
	31	+	return held, true
	32	+	}
	33	+	held = append(held, b)
	34	+	continue
	35	+	}
	36	+	if b == 0x1b \|\| (b < 0x20 && !d.paste) {
	37	+	held = append(held, d.prefix...)
	38	+	d.prefix = d.prefix[:0]
	39	+	if b == 3 {
	40	+	return held, true
	41	+	}
	42	+	if b == 0x1b {
	43	+	d.prefix = append(d.prefix, b)
	44	+	} else {
	45	+	held = append(held, b)
	46	+	}
	47	+	continue
	48	+	}
	49	+	d.prefix = append(d.prefix, b)
	50	+	if d.paste {
	51	+	end := []byte("\x1b[201~")
	52	+	if bytes.Equal(d.prefix, end) {
	53	+	d.paste = false
	54	+	} else if bytes.HasPrefix(end, d.prefix) {
	55	+	continue
	56	+	}
	57	+	} else {
	58	+	start := []byte("\x1b[200~")
	59	+	if bytes.Equal(d.prefix, start) {
	60	+	d.paste = true
	61	+	} else if bytes.HasPrefix(start, d.prefix) {
	62	+	continue
	63	+	} else {
	64	+	_, n, ok, partial := kittykey.Parse(d.prefix)
	65	+	if ok {
	66	+	translated := kittykey.Translate(d.prefix[:n], 0)
	67	+	if len(translated) == 1 && (translated[0] == 3 \|\| translated[0] == 0x1b) {
	68	+	return held, true
	69	+	}
	70	+	} else if partial && len(d.prefix) < resumeEscapeLimit {
	71	+	continue
	72	+	} else if len(d.prefix) >= 2 && (d.prefix[1] == '[' \|\| d.prefix[1] == 'O') && b >= 0x20 && b < 0x40 && len(d.prefix) < resumeEscapeLimit {
	73	+	continue
	74	+	}
	75	+	}
	76	+	}
	77	+	held = append(held, d.prefix...)
	78	+	d.prefix = d.prefix[:0]
	79	+	}
	80	+	return held, false
	81	+	}
	82	+
	83	+	// undecided reports whether an escape prefix is still waiting for more bytes.
	84	+	func (d *resumeInputDecoder) undecided() bool {
	85	+	return len(d.prefix) != 0
	86	+	}
	87	+
	88	+	func (d *resumeInputDecoder) loneEscape() bool {
	89	+	return !d.paste && bytes.Equal(d.prefix, []byte{0x1b})
	90	+	}
	91	+
	92	+	func (d *resumeInputDecoder) flush() []byte {
	93	+	data := d.prefix
	94	+	d.prefix = nil
	95	+	return data
	96	+	}

A internal/usecase/client/resume_input_test.go +54 -0

@@ -0,0 +1,54 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"testing"
	5	+
	6	+	"github.com/stretchr/testify/require"
	7	+	)
	8	+
	9	+	func TestResumeInputDecoder(t *testing.T) {
	10	+	for _, tt := range []struct {
	11	+	name, input string
	12	+	cancel bool
	13	+	}{
	14	+	{"grouped ctrl-c", "text\x03tail", true},
	15	+	{"paste escape followed by cancel", "\x1b[200~a\x1b\x1b[201~\x03", true},
	16	+	{"back to back escape", "\x1b\x1b[27u", true},
	17	+	{"ctrl-c in unfinished CSI", "\x1b[\x03", true},
	18	+	{"kitty ctrl-c", "\x1b[99;5u", true},
	19	+	{"kitty escape", "\x1b[27u", true},
	20	+	{"arrow", "\x1b[A", false},
	21	+	{"function", "\x1bOP", false},
	22	+	{"paste", "\x1b[200~a\x03\x1b[27u\x1b[201~", false},
	23	+	{"text", "hello", false},
	24	+	{"alt enter", "\x1b\r", false},
	25	+	{"alt tab", "\x1b\t", false},
	26	+	} {
	27	+	t.Run(tt.name, func(t *testing.T) {
	28	+	for split := 0; split <= len(tt.input); split++ {
	29	+	var d resumeInputDecoder
	30	+	first, cancelled := d.feed([]byte(tt.input[:split]))
	31	+	second := []byte(nil)
	32	+	if !cancelled {
	33	+	second, cancelled = d.feed([]byte(tt.input[split:]))
	34	+	}
	35	+	require.Equal(t, tt.cancel, cancelled, "split %d", split)
	36	+	if !tt.cancel {
	37	+	got := append(first, second...)
	38	+	got = append(got, d.flush()...)
	39	+	require.Equal(t, tt.input, string(got), "split %d", split)
	40	+	}
	41	+	}
	42	+	})
	43	+	}
	44	+	}
	45	+
	46	+	func TestResumeInputDecoderUndecidedEscape(t *testing.T) {
	47	+	var d resumeInputDecoder
	48	+	held, cancel := d.feed([]byte("\x1b"))
	49	+	require.Empty(t, held)
	50	+	require.False(t, cancel)
	51	+	require.True(t, d.loneEscape())
	52	+	require.Equal(t, []byte{0x1b}, d.flush())
	53	+	require.False(t, d.loneEscape())
	54	+	}

@@ -0,0 +1,54 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"testing"
	5	+
	6	+	"github.com/stretchr/testify/require"
	7	+	)
	8	+
	9	+	func TestResumeInputDecoder(t *testing.T) {
	10	+	for _, tt := range []struct {
	11	+	name, input string
	12	+	cancel bool
	13	+	}{
	14	+	{"grouped ctrl-c", "text\x03tail", true},
	15	+	{"paste escape followed by cancel", "\x1b[200~a\x1b\x1b[201~\x03", true},
	16	+	{"back to back escape", "\x1b\x1b[27u", true},
	17	+	{"ctrl-c in unfinished CSI", "\x1b[\x03", true},
	18	+	{"kitty ctrl-c", "\x1b[99;5u", true},
	19	+	{"kitty escape", "\x1b[27u", true},
	20	+	{"arrow", "\x1b[A", false},
	21	+	{"function", "\x1bOP", false},
	22	+	{"paste", "\x1b[200~a\x03\x1b[27u\x1b[201~", false},
	23	+	{"text", "hello", false},
	24	+	{"alt enter", "\x1b\r", false},
	25	+	{"alt tab", "\x1b\t", false},
	26	+	} {
	27	+	t.Run(tt.name, func(t *testing.T) {
	28	+	for split := 0; split <= len(tt.input); split++ {
	29	+	var d resumeInputDecoder
	30	+	first, cancelled := d.feed([]byte(tt.input[:split]))
	31	+	second := []byte(nil)
	32	+	if !cancelled {
	33	+	second, cancelled = d.feed([]byte(tt.input[split:]))
	34	+	}
	35	+	require.Equal(t, tt.cancel, cancelled, "split %d", split)
	36	+	if !tt.cancel {
	37	+	got := append(first, second...)
	38	+	got = append(got, d.flush()...)
	39	+	require.Equal(t, tt.input, string(got), "split %d", split)
	40	+	}
	41	+	}
	42	+	})
	43	+	}
	44	+	}
	45	+
	46	+	func TestResumeInputDecoderUndecidedEscape(t *testing.T) {
	47	+	var d resumeInputDecoder
	48	+	held, cancel := d.feed([]byte("\x1b"))
	49	+	require.Empty(t, held)
	50	+	require.False(t, cancel)
	51	+	require.True(t, d.loneEscape())
	52	+	require.Equal(t, []byte{0x1b}, d.flush())
	53	+	require.False(t, d.loneEscape())
	54	+	}

A internal/usecase/client/resume_open_test.go +134 -0

@@ -0,0 +1,134 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"testing"
	6	+	"time"
	7	+
	8	+	"github.com/bnema/vev/internal/ports"
	9	+	"github.com/stretchr/testify/require"
	10	+	)
	11	+
	12	+	func TestResumeCancelDuringStreamOpen(t *testing.T) {
	13	+	for _, tt := range []struct {
	14	+	name, input string
	15	+	escape bool
	16	+	}{
	17	+	{name: "ctrl-c", input: "\x03"},
	18	+	{name: "kitty escape", input: "\x1b[27u"},
	19	+	{name: "legacy escape", input: "\x1b", escape: true},
	20	+	} {
	21	+	t.Run(tt.name, func(t *testing.T) {
	22	+	picker := newAttachTestPicker()
	23	+	h := startAttachHarnessConfig(t, picker, func(cfg *SupervisorConfig) { cfg.Jitter = resumeTestJitter })
	24	+	first := newSessionTestStream()
	25	+	opened := make(chan struct{})
	26	+	cancelled := make(chan struct{})
	27	+	count := 0
	28	+	h.service.setOpenStream(func(ctx context.Context, _ ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
	29	+	count++
	30	+	if count == 1 {
	31	+	return first, nil
	32	+	}
	33	+	close(opened)
	34	+	<-ctx.Done()
	35	+	close(cancelled)
	36	+	return nil, ctx.Err()
	37	+	})
	38	+	picker.commit(sessionTestRequest(false))
	39	+	deliverReadyStream(t, first)
	40	+	awaitAttachedState(t, h.sup)
	41	+	first.fail(testStreamLoss())
	42	+	fireResumeTimerAfter(t, h.clock, 1, func() {})
	43	+	select {
	44	+	case <-opened:
	45	+	case <-time.After(5 * time.Second):
	46	+	t.Fatal("resume did not open")
	47	+	}
	48	+	h.reader.push([]byte(tt.input))
	49	+	if tt.escape {
	50	+	for {
	51	+	timer := h.clock.awaitTimer(t)
	52	+	if timer.delay == 50*time.Millisecond {
	53	+	timer.fire()
	54	+	break
	55	+	}
	56	+	}
	57	+	}
	58	+	select {
	59	+	case <-cancelled:
	60	+	case <-time.After(5 * time.Second):
	61	+	t.Fatal("cancel did not reach open context")
	62	+	}
	63	+	require.Eventually(t, picker.owns, 5*time.Second, time.Millisecond)
	64	+	require.Equal(t, 2, len(h.service.openedRequests()))
	65	+	})
	66	+	}
	67	+	}
	68	+
	69	+	// TestResumeHeldKeysNeverReachThePicker types while a lost attachment
	70	+	// resumes, then returns to the picker either by cancelling or because the
	71	+	// resume fails for good, and proves the picker never receives the held keys
	72	+	// while it still receives keys typed after it took the terminal back.
	73	+	func TestResumeHeldKeysNeverReachThePicker(t *testing.T) {
	74	+	for _, tt := range []struct {
	75	+	name string
	76	+	cancel bool
	77	+	}{
	78	+	{name: "cancelled", cancel: true},
	79	+	{name: "resume refused"},
	80	+	} {
	81	+	t.Run(tt.name, func(t *testing.T) {
	82	+	assertHeldKeysDroppedAtPicker(t, tt.cancel)
	83	+	})
	84	+	}
	85	+	}
	86	+
	87	+	func assertHeldKeysDroppedAtPicker(t *testing.T, cancel bool) {
	88	+	picker := newAttachTestPicker()
	89	+	h := startAttachHarnessConfig(t, picker, func(cfg *SupervisorConfig) { cfg.Jitter = resumeTestJitter })
	90	+	first := newSessionTestStream()
	91	+	opened := make(chan struct{}, 1)
	92	+	refuse := make(chan struct{})
	93	+	count := 0
	94	+	h.service.setOpenStream(func(ctx context.Context, _ ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
	95	+	count++
	96	+	if count == 1 {
	97	+	return first, nil
	98	+	}
	99	+	opened <- struct{}{}
	100	+	select {
	101	+	case <-ctx.Done():
	102	+	return nil, ctx.Err()
	103	+	case <-refuse:
	104	+	return nil, ports.BrokerError{Code: ports.BrokerErrorIncompatible, Text: "refused"}
	105	+	}
	106	+	})
	107	+	picker.commit(sessionTestRequest(false))
	108	+	deliverReadyStream(t, first)
	109	+	awaitAttachedState(t, h.sup)
	110	+	first.fail(testStreamLoss())
	111	+	require.Eventually(t, func() bool { return h.sup.State().Presentation == PresentConnecting }, 5*time.Second, time.Millisecond)
	112	+	pushKeys(t, h, "HELD\r")
	113	+	if cancel {
	114	+	h.reader.push([]byte("\x03"))
	115	+	} else {
	116	+	fireResumeTimerAfter(t, h.clock, 1, func() {})
	117	+	select {
	118	+	case <-opened:
	119	+	case <-time.After(5 * time.Second):
	120	+	t.Fatal("resume did not open")
	121	+	}
	122	+	close(refuse)
	123	+	}
	124	+	require.Eventually(t, picker.owns, 5*time.Second, time.Millisecond)
	125	+	h.reader.push([]byte("j"))
	126	+	require.Eventually(t, func() bool {
	127	+	picker.mu.Lock()
	128	+	defer picker.mu.Unlock()
	129	+	return len(picker.consumedBytes) > 0
	130	+	}, 5*time.Second, time.Millisecond)
	131	+	picker.mu.Lock()
	132	+	defer picker.mu.Unlock()
	133	+	require.Equal(t, "j", string(picker.consumedBytes), "held resume keys reached the picker")
	134	+	}

@@ -0,0 +1,134 @@
	1	+	package client
	2	+
	3	+	import (
	4	+	"context"
	5	+	"testing"
	6	+	"time"
	7	+
	8	+	"github.com/bnema/vev/internal/ports"
	9	+	"github.com/stretchr/testify/require"
	10	+	)
	11	+
	12	+	func TestResumeCancelDuringStreamOpen(t *testing.T) {
	13	+	for _, tt := range []struct {
	14	+	name, input string
	15	+	escape bool
	16	+	}{
	17	+	{name: "ctrl-c", input: "\x03"},
	18	+	{name: "kitty escape", input: "\x1b[27u"},
	19	+	{name: "legacy escape", input: "\x1b", escape: true},
	20	+	} {
	21	+	t.Run(tt.name, func(t *testing.T) {
	22	+	picker := newAttachTestPicker()
	23	+	h := startAttachHarnessConfig(t, picker, func(cfg *SupervisorConfig) { cfg.Jitter = resumeTestJitter })
	24	+	first := newSessionTestStream()
	25	+	opened := make(chan struct{})
	26	+	cancelled := make(chan struct{})
	27	+	count := 0
	28	+	h.service.setOpenStream(func(ctx context.Context, _ ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
	29	+	count++
	30	+	if count == 1 {
	31	+	return first, nil
	32	+	}
	33	+	close(opened)
	34	+	<-ctx.Done()
	35	+	close(cancelled)
	36	+	return nil, ctx.Err()
	37	+	})
	38	+	picker.commit(sessionTestRequest(false))
	39	+	deliverReadyStream(t, first)
	40	+	awaitAttachedState(t, h.sup)
	41	+	first.fail(testStreamLoss())
	42	+	fireResumeTimerAfter(t, h.clock, 1, func() {})
	43	+	select {
	44	+	case <-opened:
	45	+	case <-time.After(5 * time.Second):
	46	+	t.Fatal("resume did not open")
	47	+	}
	48	+	h.reader.push([]byte(tt.input))
	49	+	if tt.escape {
	50	+	for {
	51	+	timer := h.clock.awaitTimer(t)
	52	+	if timer.delay == 50*time.Millisecond {
	53	+	timer.fire()
	54	+	break
	55	+	}
	56	+	}
	57	+	}
	58	+	select {
	59	+	case <-cancelled:
	60	+	case <-time.After(5 * time.Second):
	61	+	t.Fatal("cancel did not reach open context")
	62	+	}
	63	+	require.Eventually(t, picker.owns, 5*time.Second, time.Millisecond)
	64	+	require.Equal(t, 2, len(h.service.openedRequests()))
	65	+	})
	66	+	}
	67	+	}
	68	+
	69	+	// TestResumeHeldKeysNeverReachThePicker types while a lost attachment
	70	+	// resumes, then returns to the picker either by cancelling or because the
	71	+	// resume fails for good, and proves the picker never receives the held keys
	72	+	// while it still receives keys typed after it took the terminal back.
	73	+	func TestResumeHeldKeysNeverReachThePicker(t *testing.T) {
	74	+	for _, tt := range []struct {
	75	+	name string
	76	+	cancel bool
	77	+	}{
	78	+	{name: "cancelled", cancel: true},
	79	+	{name: "resume refused"},
	80	+	} {
	81	+	t.Run(tt.name, func(t *testing.T) {
	82	+	assertHeldKeysDroppedAtPicker(t, tt.cancel)
	83	+	})
	84	+	}
	85	+	}
	86	+
	87	+	func assertHeldKeysDroppedAtPicker(t *testing.T, cancel bool) {
	88	+	picker := newAttachTestPicker()
	89	+	h := startAttachHarnessConfig(t, picker, func(cfg *SupervisorConfig) { cfg.Jitter = resumeTestJitter })
	90	+	first := newSessionTestStream()
	91	+	opened := make(chan struct{}, 1)
	92	+	refuse := make(chan struct{})
	93	+	count := 0
	94	+	h.service.setOpenStream(func(ctx context.Context, _ ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
	95	+	count++
	96	+	if count == 1 {
	97	+	return first, nil
	98	+	}
	99	+	opened <- struct{}{}
	100	+	select {
	101	+	case <-ctx.Done():
	102	+	return nil, ctx.Err()
	103	+	case <-refuse:
	104	+	return nil, ports.BrokerError{Code: ports.BrokerErrorIncompatible, Text: "refused"}
	105	+	}
	106	+	})
	107	+	picker.commit(sessionTestRequest(false))
	108	+	deliverReadyStream(t, first)
	109	+	awaitAttachedState(t, h.sup)
	110	+	first.fail(testStreamLoss())
	111	+	require.Eventually(t, func() bool { return h.sup.State().Presentation == PresentConnecting }, 5*time.Second, time.Millisecond)
	112	+	pushKeys(t, h, "HELD\r")
	113	+	if cancel {
	114	+	h.reader.push([]byte("\x03"))
	115	+	} else {
	116	+	fireResumeTimerAfter(t, h.clock, 1, func() {})
	117	+	select {
	118	+	case <-opened:
	119	+	case <-time.After(5 * time.Second):
	120	+	t.Fatal("resume did not open")
	121	+	}
	122	+	close(refuse)
	123	+	}
	124	+	require.Eventually(t, picker.owns, 5*time.Second, time.Millisecond)
	125	+	h.reader.push([]byte("j"))
	126	+	require.Eventually(t, func() bool {
	127	+	picker.mu.Lock()
	128	+	defer picker.mu.Unlock()
	129	+	return len(picker.consumedBytes) > 0
	130	+	}, 5*time.Second, time.Millisecond)
	131	+	picker.mu.Lock()
	132	+	defer picker.mu.Unlock()
	133	+	require.Equal(t, "j", string(picker.consumedBytes), "held resume keys reached the picker")
	134	+	}

M internal/usecase/client/supervisor.go +165 -0


@@ -1490,6 +1490,171 @@ func (l *terminalInputLifetime) stop() {
1490	1490		l.pump.suspend()
1491	1491		}
1492	1492
	1493	+	// resumeHeldInputLimit bounds keys kept while a lost attachment resumes.
	1494	+	// Past it further keys are dropped, like a full terminal buffer.
	1495	+	// TODO: the overflow flag is per resume watch; a later phase can accept keys
	1496	+	// again and leave a gap. Make it sticky on the input lifetime until attach.
	1497	+	const resumeHeldInputLimit = 64 << 10
	1498	+
	1499	+	// resumeWatch reads the terminal while a lost attachment waits to resume.
	1500	+	// cancelled closes on Ctrl-C or a lone Esc; any other bytes are handed back
	1501	+	// to the pump on release, so the resumed attachment receives them in order.
	1502	+	type resumeWatch struct {
	1503	+	cancelled chan struct{}
	1504	+	stop chan struct{}
	1505	+	done chan struct{}
	1506	+	lifetime *terminalInputLifetime
	1507	+	id uint64
	1508	+	clock ports.Clock
	1509	+
	1510	+	// backlog counts deliveries already waiting at claim: session input the
	1511	+	// lost attachment kept, held verbatim and never read as a cancel key.
	1512	+	backlog int
	1513	+	releaseOnce sync.Once
	1514	+
	1515	+	mu sync.Mutex
	1516	+	held []byte
	1517	+	// overflowed drops every key after the first one past the bound, so the
	1518	+	// resumed session never receives input with a gap in the middle.
	1519	+	overflowed bool
	1520	+	}
	1521	+
	1522	+	// heldLen reports how many kept bytes the watch holds.
	1523	+	func (w *resumeWatch) heldLen() int {
	1524	+	w.mu.Lock()
	1525	+	defer w.mu.Unlock()
	1526	+	return len(w.held)
	1527	+	}
	1528	+
	1529	+	// hold keeps data within the bound.
	1530	+	func (w *resumeWatch) hold(data []byte) {
	1531	+	w.mu.Lock()
	1532	+	defer w.mu.Unlock()
	1533	+	if w.overflowed \|\| len(w.held)+len(data) > resumeHeldInputLimit {
	1534	+	w.overflowed = w.overflowed \|\| len(data) != 0
	1535	+	return
	1536	+	}
	1537	+	w.held = append(w.held, data...)
	1538	+	}
	1539	+
	1540	+	// watchResume claims terminal input for one resume backoff. When the claim
	1541	+	// is unavailable it returns a watch that never cancels.
	1542	+	func (l terminalInputLifetime) watchResume(clock ports.Clock) resumeWatch {
	1543	+	w := &resumeWatch{cancelled: make(chan struct{}), stop: make(chan struct{}), done: make(chan struct{}), lifetime: l, clock: clock}
	1544	+	if l == nil \|\| l.pump == nil {
	1545	+	close(w.done)
	1546	+	return w
	1547	+	}
	1548	+	id, backlog, ok := l.pump.claimWithBacklog()
	1549	+	if !ok {
	1550	+	close(w.done)
	1551	+	return w
	1552	+	}
	1553	+	w.id = id
	1554	+	w.backlog = backlog
	1555	+	go w.run()
	1556	+	return w
	1557	+	}
	1558	+
	1559	+	func (w *resumeWatch) run() {
	1560	+	defer close(w.done)
	1561	+	pump := w.lifetime.pump
	1562	+	var decoder resumeInputDecoder
	1563	+	var escape ports.Timer
	1564	+	defer func() {
	1565	+	if escape != nil {
	1566	+	escape.Stop()
	1567	+	}
	1568	+	w.hold(decoder.flush())
	1569	+	}()
	1570	+	stop := w.stop
	1571	+	for {
	1572	+	var escapeC <-chan time.Time
	1573	+	if escape != nil {
	1574	+	escapeC = escape.C()
	1575	+	}
	1576	+	result, ok := pump.take(context.Background(), w.id)
	1577	+	if !ok {
	1578	+	select {
	1579	+	case <-pump.readyFor(w.id):
	1580	+	case <-escapeC:
	1581	+	if decoder.loneEscape() {
	1582	+	close(w.cancelled)
	1583	+	}
	1584	+	return
	1585	+	case <-stop:
	1586	+	if !decoder.undecided() {
	1587	+	return
	1588	+	}
	1589	+	// Resolve an escape typed just before the phase ended: the
	1590	+	// next watch would hold it as backlog, so a cancel pressed at
	1591	+	// the boundary would reach the session instead.
	1592	+	stop = nil
	1593	+	if escape == nil {
	1594	+	escape = w.clock.NewTimer(resumeEscapeDelay)
	1595	+	}
	1596	+	case <-pump.done:
	1597	+	return
	1598	+	}
	1599	+	continue
	1600	+	}
	1601	+	if result.err != nil {
	1602	+	// Leave the final read for the terminal EOF watcher.
	1603	+	pump.ack(w.id)
	1604	+	w.hold(decoder.flush())
	1605	+	w.hold(result.data)
	1606	+	w.lifetime.finish(result.err)
	1607	+	return
	1608	+	}
	1609	+	pump.ack(w.id)
	1610	+	if w.backlog > 0 {
	1611	+	w.backlog--
	1612	+	w.hold(result.data)
	1613	+	continue
	1614	+	}
	1615	+	held, cancel := decoder.feed(result.data)
	1616	+	w.hold(held)
	1617	+	if cancel {
	1618	+	close(w.cancelled)
	1619	+	return
	1620	+	}
	1621	+	if escape != nil {
	1622	+	escape.Stop()
	1623	+	escape = nil
	1624	+	}
	1625	+	if stop == nil && !decoder.undecided() {
	1626	+	return
	1627	+	}
	1628	+	if decoder.loneEscape() \|\| stop == nil {
	1629	+	escape = w.clock.NewTimer(resumeEscapeDelay)
	1630	+	}
	1631	+	}
	1632	+	}
	1633	+
	1634	+	// release stops the watch and hands kept keys back to the pump, unless the
	1635	+	// user cancelled, in which case they are dropped with the session. An escape
	1636	+	// still undecided is resolved first, so it may cancel. Later calls are no-ops.
	1637	+	func (w *resumeWatch) release() {
	1638	+	if w.id == 0 {
	1639	+	return
	1640	+	}
	1641	+	w.releaseOnce.Do(w.releaseClaim)
	1642	+	}
	1643	+
	1644	+	func (w *resumeWatch) releaseClaim() {
	1645	+	close(w.stop)
	1646	+	<-w.done
	1647	+	pump := w.lifetime.pump
	1648	+	select {
	1649	+	case <-w.cancelled:
	1650	+	default:
	1651	+	w.mu.Lock()
	1652	+	pump.appendResidual(w.id, w.held, resumeHeldInputLimit)
	1653	+	w.mu.Unlock()
	1654	+	}
	1655	+	pump.revoke(w.id)
	1656	+	}
	1657	+
1493	1658		// finish publishes the terminal cause exactly once.
1494	1659		func (l *terminalInputLifetime) finish(err error) {
1495	1660		l.once.Do(func() { l.eof <- err })


@@ -1490,6 +1490,171 @@ func (l *terminalInputLifetime) stop() {
1490	l.pump.suspend()	1490		l.pump.suspend()
1491	}	1491		}
1492		1492
		1493	+	// resumeHeldInputLimit bounds keys kept while a lost attachment resumes.
		1494	+	// Past it further keys are dropped, like a full terminal buffer.
		1495	+	// TODO: the overflow flag is per resume watch; a later phase can accept keys
		1496	+	// again and leave a gap. Make it sticky on the input lifetime until attach.
		1497	+	const resumeHeldInputLimit = 64 << 10
		1498	+
		1499	+	// resumeWatch reads the terminal while a lost attachment waits to resume.
		1500	+	// cancelled closes on Ctrl-C or a lone Esc; any other bytes are handed back
		1501	+	// to the pump on release, so the resumed attachment receives them in order.
		1502	+	type resumeWatch struct {
		1503	+	cancelled chan struct{}
		1504	+	stop chan struct{}
		1505	+	done chan struct{}
		1506	+	lifetime *terminalInputLifetime
		1507	+	id uint64
		1508	+	clock ports.Clock
		1509	+
		1510	+	// backlog counts deliveries already waiting at claim: session input the
		1511	+	// lost attachment kept, held verbatim and never read as a cancel key.
		1512	+	backlog int
		1513	+	releaseOnce sync.Once
		1514	+
		1515	+	mu sync.Mutex
		1516	+	held []byte
		1517	+	// overflowed drops every key after the first one past the bound, so the
		1518	+	// resumed session never receives input with a gap in the middle.
		1519	+	overflowed bool
		1520	+	}
		1521	+
		1522	+	// heldLen reports how many kept bytes the watch holds.
		1523	+	func (w *resumeWatch) heldLen() int {
		1524	+	w.mu.Lock()
		1525	+	defer w.mu.Unlock()
		1526	+	return len(w.held)
		1527	+	}
		1528	+
		1529	+	// hold keeps data within the bound.
		1530	+	func (w *resumeWatch) hold(data []byte) {
		1531	+	w.mu.Lock()
		1532	+	defer w.mu.Unlock()
		1533	+	if w.overflowed \|\| len(w.held)+len(data) > resumeHeldInputLimit {
		1534	+	w.overflowed = w.overflowed \|\| len(data) != 0
		1535	+	return
		1536	+	}
		1537	+	w.held = append(w.held, data...)
		1538	+	}
		1539	+
		1540	+	// watchResume claims terminal input for one resume backoff. When the claim
		1541	+	// is unavailable it returns a watch that never cancels.
		1542	+	func (l terminalInputLifetime) watchResume(clock ports.Clock) resumeWatch {
		1543	+	w := &resumeWatch{cancelled: make(chan struct{}), stop: make(chan struct{}), done: make(chan struct{}), lifetime: l, clock: clock}
		1544	+	if l == nil \|\| l.pump == nil {
		1545	+	close(w.done)
		1546	+	return w
		1547	+	}
		1548	+	id, backlog, ok := l.pump.claimWithBacklog()
		1549	+	if !ok {
		1550	+	close(w.done)
		1551	+	return w
		1552	+	}
		1553	+	w.id = id
		1554	+	w.backlog = backlog
		1555	+	go w.run()
		1556	+	return w
		1557	+	}
		1558	+
		1559	+	func (w *resumeWatch) run() {
		1560	+	defer close(w.done)
		1561	+	pump := w.lifetime.pump
		1562	+	var decoder resumeInputDecoder
		1563	+	var escape ports.Timer
		1564	+	defer func() {
		1565	+	if escape != nil {
		1566	+	escape.Stop()
		1567	+	}
		1568	+	w.hold(decoder.flush())
		1569	+	}()
		1570	+	stop := w.stop
		1571	+	for {
		1572	+	var escapeC <-chan time.Time
		1573	+	if escape != nil {
		1574	+	escapeC = escape.C()
		1575	+	}
		1576	+	result, ok := pump.take(context.Background(), w.id)
		1577	+	if !ok {
		1578	+	select {
		1579	+	case <-pump.readyFor(w.id):
		1580	+	case <-escapeC:
		1581	+	if decoder.loneEscape() {
		1582	+	close(w.cancelled)
		1583	+	}
		1584	+	return
		1585	+	case <-stop:
		1586	+	if !decoder.undecided() {
		1587	+	return
		1588	+	}
		1589	+	// Resolve an escape typed just before the phase ended: the
		1590	+	// next watch would hold it as backlog, so a cancel pressed at
		1591	+	// the boundary would reach the session instead.
		1592	+	stop = nil
		1593	+	if escape == nil {
		1594	+	escape = w.clock.NewTimer(resumeEscapeDelay)
		1595	+	}
		1596	+	case <-pump.done:
		1597	+	return
		1598	+	}
		1599	+	continue
		1600	+	}
		1601	+	if result.err != nil {
		1602	+	// Leave the final read for the terminal EOF watcher.
		1603	+	pump.ack(w.id)
		1604	+	w.hold(decoder.flush())
		1605	+	w.hold(result.data)
		1606	+	w.lifetime.finish(result.err)
		1607	+	return
		1608	+	}
		1609	+	pump.ack(w.id)
		1610	+	if w.backlog > 0 {
		1611	+	w.backlog--
		1612	+	w.hold(result.data)
		1613	+	continue
		1614	+	}
		1615	+	held, cancel := decoder.feed(result.data)
		1616	+	w.hold(held)
		1617	+	if cancel {
		1618	+	close(w.cancelled)
		1619	+	return
		1620	+	}
		1621	+	if escape != nil {
		1622	+	escape.Stop()
		1623	+	escape = nil
		1624	+	}
		1625	+	if stop == nil && !decoder.undecided() {
		1626	+	return
		1627	+	}
		1628	+	if decoder.loneEscape() \|\| stop == nil {
		1629	+	escape = w.clock.NewTimer(resumeEscapeDelay)
		1630	+	}
		1631	+	}
		1632	+	}
		1633	+
		1634	+	// release stops the watch and hands kept keys back to the pump, unless the
		1635	+	// user cancelled, in which case they are dropped with the session. An escape
		1636	+	// still undecided is resolved first, so it may cancel. Later calls are no-ops.
		1637	+	func (w *resumeWatch) release() {
		1638	+	if w.id == 0 {
		1639	+	return
		1640	+	}
		1641	+	w.releaseOnce.Do(w.releaseClaim)
		1642	+	}
		1643	+
		1644	+	func (w *resumeWatch) releaseClaim() {
		1645	+	close(w.stop)
		1646	+	<-w.done
		1647	+	pump := w.lifetime.pump
		1648	+	select {
		1649	+	case <-w.cancelled:
		1650	+	default:
		1651	+	w.mu.Lock()
		1652	+	pump.appendResidual(w.id, w.held, resumeHeldInputLimit)
		1653	+	w.mu.Unlock()
		1654	+	}
		1655	+	pump.revoke(w.id)
		1656	+	}
		1657	+
1493	// finish publishes the terminal cause exactly once.	1658		// finish publishes the terminal cause exactly once.
1494	func (l *terminalInputLifetime) finish(err error) {	1659		func (l *terminalInputLifetime) finish(err error) {
1495	l.once.Do(func() { l.eof <- err })	1660		l.once.Do(func() { l.eof <- err })

M internal/usecase/client/supervisor_attach.go +101 -14


@@ -583,6 +583,9 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
583	583		// picker catalogue.
584	584		picker.ApplySnapshot(service.Snapshot())
585	585		picker.SetOwnsInput(true)
	586	+	if input != nil && input.pump != nil {
	587	+	input.pump.dropUnclaimed()
	588	+	}
586	589		input.acquirePicker()
587	590		}()
588	591		// budget.resuming is true while the loop reconnects a lost attachment:

@@ -604,6 +607,11 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
604	607		// attachment detached cleanly, and the swap goes through Connecting
605	608		// exactly like any committed selection.
606	609		budget = attachmentResumeBudget{}
	610	+	// Keys kept for the previous session must not run in the next one;
	611	+	// only a resume of the same target replays them.
	612	+	if input != nil && input.pump != nil {
	613	+	input.pump.dropUnclaimed()
	614	+	}
607	615		next, ok := s.withFreshStream(service, outcome.target)
608	616		if !ok {
609	617		return false, nil

@@ -616,7 +624,8 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
616	624		// same target again.
617	625		outcome.target = target
618	626		}
619		-	attempt, ok := budget.spend(outcome.kind, s.cfg.Clock.Now().Sub(attemptStart))
	627	+	now := s.cfg.Clock.Now()
	628	+	attempt, ok := budget.spend(outcome.kind, now.Sub(attemptStart), now)
620	629		if !ok {
621	630		s.logger.Warn("client_attachment_resume_exhausted", "endpoint", outcome.target.request.Endpoint, "session", outcome.target.request.Target.SessionName, "attempts", attempt-1, "error", outcome.cause)
622	631		s.transition(supervisorEvent{kind: supervisorAttachEnded, err: outcome.cause})

@@ -624,7 +633,7 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
624	633		s.notifyAttachment(outcome.cause)
625	634		return false, nil
626	635		}
627		-	if terminated, termErr, brokerLost := s.waitResume(ctx, input, service, attempt); terminated \|\| brokerLost {
	636	+	if terminated, termErr, stop := s.waitResume(ctx, input, service, attempt); terminated \|\| stop {
628	637		return terminated, termErr
629	638		}
630	639		next, ok := s.withFreshStream(service, outcome.target)

@@ -640,6 +649,12 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
640	649		type attachmentResumeBudget struct {
641	650		used int
642	651		resuming bool
	652	+	// flaps counts attachments lost before they became stable. They prove
	653	+	// the route works but the session drops, so they are capped by count
	654	+	// regardless of the outage window.
	655	+	flaps int
	656	+	// since is when the current outage began; zero before the first spend.
	657	+	since time.Time
643	658		}
644	659
645	660		// spend records one resume after an attempt of the given kind that ran for

@@ -647,14 +662,29 @@ type attachmentResumeBudget struct {
647	662		// fresh budget; one that drops right after attaching keeps counting, so a
648	663		// flapping session gives up.
649	664		// It returns the 1-based resume attempt, and false once the budget is
650		-	// exhausted.
651		-	func (b *attachmentResumeBudget) spend(kind attachmentOutcomeKind, up time.Duration) (int, bool) {
652		-	if kind == attachmentResume && up >= resumeStableAttachment {
653		-	b.used = 0
	665	+	// exhausted: more than maxAttachmentResumes flaps, or more than
	666	+	// maxAttachmentResumes attempts once resumeOutageWindow has passed since the
	667	+	// outage began. A route that stays down therefore keeps resuming for the
	668	+	// whole window, while a session that keeps dropping right after it attaches
	669	+	// gives up quickly.
	670	+	func (b *attachmentResumeBudget) spend(kind attachmentOutcomeKind, up time.Duration, now time.Time) (int, bool) {
	671	+	if kind == attachmentResume {
	672	+	if up >= resumeStableAttachment {
	673	+	b.used = 0
	674	+	b.flaps = 0
	675	+	} else {
	676	+	b.flaps++
	677	+	}
	678	+	}
	679	+	if b.used == 0 {
	680	+	b.since = now
654	681		}
655	682		b.used++
656	683		b.resuming = true
657		-	return b.used, b.used <= maxAttachmentResumes
	684	+	if b.flaps > maxAttachmentResumes {
	685	+	return b.used, false
	686	+	}
	687	+	return b.used, b.used <= maxAttachmentResumes \|\| now.Sub(b.since) < resumeOutageWindow
658	688		}
659	689
660	690		// withFreshStream gives target a newly allocated stream identity. It is

@@ -696,7 +726,39 @@ func (s Supervisor) attachResolved(ctx context.Context, input terminalInputLif
696	726		request.Env = append([]string(nil), sessionEnv.Env...)
697	727
698	728		s.logger.Debug("client_stream_open_begin", "local", request.Local, "endpoint", request.Endpoint, "admission", request.Admission, "generation", s.State().Generation)
699		-	stream, err := service.OpenStream(deadline.Context(), request)
	729	+	openCtx := deadline.Context()
	730	+	var keys *resumeWatch
	731	+	var cancelOpen context.CancelFunc
	732	+	var watchDone chan struct{}
	733	+	if resuming {
	734	+	keys = input.watchResume(s.cfg.Clock)
	735	+	openCtx, cancelOpen = context.WithCancel(openCtx)
	736	+	watchDone = make(chan struct{})
	737	+	go func() {
	738	+	defer close(watchDone)
	739	+	select {
	740	+	case <-keys.cancelled:
	741	+	cancelOpen()
	742	+	case <-openCtx.Done():
	743	+	}
	744	+	}()
	745	+	}
	746	+	stream, err := service.OpenStream(openCtx, request)
	747	+	if keys != nil {
	748	+	keys.release()
	749	+	cancelOpen()
	750	+	<-watchDone
	751	+	select {
	752	+	case <-keys.cancelled:
	753	+	s.logger.Info("client_attachment_resume_cancelled", "phase", "stream_open")
	754	+	if !supervisorNil(stream) {
	755	+	_ = stream.Close()
	756	+	}
	757	+	s.transition(supervisorEvent{kind: supervisorAttachEnded})
	758	+	return attachmentEndedOutcome()
	759	+	default:
	760	+	}
	761	+	}
700	762		if err != nil {
701	763		s.logger.Warn("client_stream_open_failed", "local", request.Local, "endpoint", request.Endpoint, "code", brokerErrorCode(err), "error", err, "cause", errors.Unwrap(err))
702	764		return s.attemptFailed(deadline.cause(err), resuming)

@@ -898,11 +960,17 @@ settlement:
898	960		return attachmentEndedOutcome()
899	961		}
900	962
901		-	// maxAttachmentResumes bounds how many times one lost attachment reconnects
902		-	// before the client gives up and returns to the picker. With the supervisor
903		-	// backoff this spans a few seconds, enough to ride out a transient drop.
	963	+	// maxAttachmentResumes is the minimum number of resume attempts before the
	964	+	// client may give up on a lost attachment. Giving up also requires the outage
	965	+	// to have lasted resumeOutageWindow, so a flaky mobile link keeps resuming
	966	+	// instead of dropping to the picker after a few seconds.
904	967		const maxAttachmentResumes = 5
905	968
	969	+	// resumeOutageWindow is how long one outage keeps resuming. It matches the
	970	+	// daemon's park window for a detached remote attachment: past it the session
	971	+	// view is gone anyway.
	972	+	const resumeOutageWindow = 15 * time.Minute
	973	+
906	974		// resumeStableAttachment is how long a resumed attachment must stay up before
907	975		// its next loss gets a fresh resume budget.
908	976		const resumeStableAttachment = 30 * time.Second

@@ -924,16 +992,35 @@ func resumeTarget(request ports.BrokerOpenStreamRequest, fg *attachmentForegroun
924	992
925	993		// waitResume presents Connecting and waits one backoff interval before the
926	994		// next resume attempt. It settles early on cancellation or terminal EOF
927		-	// (terminated), or on broker loss, which ends the resume and leaves the ready
928		-	// loop to observe and retire the connection.
929		-	func (s Supervisor) waitResume(ctx context.Context, input terminalInputLifetime, service ports.BrokerNavigator, attempt int) (terminated bool, termErr error, brokerLost bool) {
	995	+	// (terminated), or with stop on broker loss, which leaves the ready loop to
	996	+	// observe and retire the connection, or when the user cancels the resume
	997	+	// with Ctrl-C or Esc, which returns to the picker. Other keys typed during
	998	+	// the outage are kept and replayed into the session once it resumes.
	999	+	func (s Supervisor) waitResume(ctx context.Context, input terminalInputLifetime, service ports.BrokerNavigator, attempt int) (terminated bool, termErr error, stop bool) {
930	1000		s.transition(supervisorEvent{kind: supervisorAttachBegin})
931	1001		delay := supervisorBackoffDelay(uint64(attempt), s.cfg.Jitter)
932	1002		timer := s.cfg.Clock.NewTimer(delay)
933	1003		defer stopSupervisorTimer(timer)
	1004	+	keys := input.watchResume(s.cfg.Clock)
	1005	+	defer keys.release()
934	1006		for {
935	1007		select {
	1008	+	case <-keys.cancelled:
	1009	+	// The user gave up on the outage: back to the picker, quietly.
	1010	+	s.logger.Info("client_attachment_resume_cancelled", "attempt", attempt)
	1011	+	s.transition(supervisorEvent{kind: supervisorAttachEnded})
	1012	+	return false, nil, true
936	1013		case <-timer.C():
	1014	+	// Release resolves an Esc typed just before the backoff ended,
	1015	+	// which may still cancel.
	1016	+	keys.release()
	1017	+	select {
	1018	+	case <-keys.cancelled:
	1019	+	s.logger.Info("client_attachment_resume_cancelled", "attempt", attempt)
	1020	+	s.transition(supervisorEvent{kind: supervisorAttachEnded})
	1021	+	return false, nil, true
	1022	+	default:
	1023	+	}
937	1024		return false, nil, false
938	1025		case <-ctx.Done():
939	1026		return true, ctx.Err(), false


@@ -583,6 +583,9 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
583		// picker catalogue.	583		// picker catalogue.
584		picker.ApplySnapshot(service.Snapshot())	584		picker.ApplySnapshot(service.Snapshot())
585		picker.SetOwnsInput(true)	585		picker.SetOwnsInput(true)
			586	+	if input != nil && input.pump != nil {
			587	+	input.pump.dropUnclaimed()
			588	+	}
586		input.acquirePicker()	589		input.acquirePicker()
587		}()	590		}()
588		// budget.resuming is true while the loop reconnects a lost attachment:	591		// budget.resuming is true while the loop reconnects a lost attachment:

@@ -604,6 +607,11 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
604		// attachment detached cleanly, and the swap goes through Connecting	607		// attachment detached cleanly, and the swap goes through Connecting
605		// exactly like any committed selection.	608		// exactly like any committed selection.
606		budget = attachmentResumeBudget{}	609		budget = attachmentResumeBudget{}
			610	+	// Keys kept for the previous session must not run in the next one;
			611	+	// only a resume of the same target replays them.
			612	+	if input != nil && input.pump != nil {
			613	+	input.pump.dropUnclaimed()
			614	+	}
607		next, ok := s.withFreshStream(service, outcome.target)	615		next, ok := s.withFreshStream(service, outcome.target)
608		if !ok {	616		if !ok {
609		return false, nil	617		return false, nil

@@ -616,7 +624,8 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
616		// same target again.	624		// same target again.
617		outcome.target = target	625		outcome.target = target
618		}	626		}
619	-	attempt, ok := budget.spend(outcome.kind, s.cfg.Clock.Now().Sub(attemptStart))	627	+	now := s.cfg.Clock.Now()
			628	+	attempt, ok := budget.spend(outcome.kind, now.Sub(attemptStart), now)
620		if !ok {	629		if !ok {
621		s.logger.Warn("client_attachment_resume_exhausted", "endpoint", outcome.target.request.Endpoint, "session", outcome.target.request.Target.SessionName, "attempts", attempt-1, "error", outcome.cause)	630		s.logger.Warn("client_attachment_resume_exhausted", "endpoint", outcome.target.request.Endpoint, "session", outcome.target.request.Target.SessionName, "attempts", attempt-1, "error", outcome.cause)
622		s.transition(supervisorEvent{kind: supervisorAttachEnded, err: outcome.cause})	631		s.transition(supervisorEvent{kind: supervisorAttachEnded, err: outcome.cause})

@@ -624,7 +633,7 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
624		s.notifyAttachment(outcome.cause)	633		s.notifyAttachment(outcome.cause)
625		return false, nil	634		return false, nil
626		}	635		}
627	-	if terminated, termErr, brokerLost := s.waitResume(ctx, input, service, attempt); terminated \|\| brokerLost {	636	+	if terminated, termErr, stop := s.waitResume(ctx, input, service, attempt); terminated \|\| stop {
628		return terminated, termErr	637		return terminated, termErr
629		}	638		}
630		next, ok := s.withFreshStream(service, outcome.target)	639		next, ok := s.withFreshStream(service, outcome.target)

@@ -640,6 +649,12 @@ func (s Supervisor) runResolvedAttachment(ctx context.Context, input terminalI
640		type attachmentResumeBudget struct {	649		type attachmentResumeBudget struct {
641		used int	650		used int
642		resuming bool	651		resuming bool
			652	+	// flaps counts attachments lost before they became stable. They prove
			653	+	// the route works but the session drops, so they are capped by count
			654	+	// regardless of the outage window.
			655	+	flaps int
			656	+	// since is when the current outage began; zero before the first spend.
			657	+	since time.Time
643		}	658		}
644			659
645		// spend records one resume after an attempt of the given kind that ran for	660		// spend records one resume after an attempt of the given kind that ran for

@@ -647,14 +662,29 @@ type attachmentResumeBudget struct {
647		// fresh budget; one that drops right after attaching keeps counting, so a	662		// fresh budget; one that drops right after attaching keeps counting, so a
648		// flapping session gives up.	663		// flapping session gives up.
649		// It returns the 1-based resume attempt, and false once the budget is	664		// It returns the 1-based resume attempt, and false once the budget is
650	-	// exhausted.	665	+	// exhausted: more than maxAttachmentResumes flaps, or more than
651	-	func (b *attachmentResumeBudget) spend(kind attachmentOutcomeKind, up time.Duration) (int, bool) {	666	+	// maxAttachmentResumes attempts once resumeOutageWindow has passed since the
652	-	if kind == attachmentResume && up >= resumeStableAttachment {	667	+	// outage began. A route that stays down therefore keeps resuming for the
653	-	b.used = 0	668	+	// whole window, while a session that keeps dropping right after it attaches
			669	+	// gives up quickly.
			670	+	func (b *attachmentResumeBudget) spend(kind attachmentOutcomeKind, up time.Duration, now time.Time) (int, bool) {
			671	+	if kind == attachmentResume {
			672	+	if up >= resumeStableAttachment {
			673	+	b.used = 0
			674	+	b.flaps = 0
			675	+	} else {
			676	+	b.flaps++
			677	+	}
			678	+	}
			679	+	if b.used == 0 {
			680	+	b.since = now
654		}	681		}
655		b.used++	682		b.used++
656		b.resuming = true	683		b.resuming = true
657	-	return b.used, b.used <= maxAttachmentResumes	684	+	if b.flaps > maxAttachmentResumes {
			685	+	return b.used, false
			686	+	}
			687	+	return b.used, b.used <= maxAttachmentResumes \|\| now.Sub(b.since) < resumeOutageWindow
658		}	688		}
659			689
660		// withFreshStream gives target a newly allocated stream identity. It is	690		// withFreshStream gives target a newly allocated stream identity. It is

@@ -696,7 +726,39 @@ func (s Supervisor) attachResolved(ctx context.Context, input terminalInputLif
696		request.Env = append([]string(nil), sessionEnv.Env...)	726		request.Env = append([]string(nil), sessionEnv.Env...)
697			727
698		s.logger.Debug("client_stream_open_begin", "local", request.Local, "endpoint", request.Endpoint, "admission", request.Admission, "generation", s.State().Generation)	728		s.logger.Debug("client_stream_open_begin", "local", request.Local, "endpoint", request.Endpoint, "admission", request.Admission, "generation", s.State().Generation)
699	-	stream, err := service.OpenStream(deadline.Context(), request)	729	+	openCtx := deadline.Context()
			730	+	var keys *resumeWatch
			731	+	var cancelOpen context.CancelFunc
			732	+	var watchDone chan struct{}
			733	+	if resuming {
			734	+	keys = input.watchResume(s.cfg.Clock)
			735	+	openCtx, cancelOpen = context.WithCancel(openCtx)
			736	+	watchDone = make(chan struct{})
			737	+	go func() {
			738	+	defer close(watchDone)
			739	+	select {
			740	+	case <-keys.cancelled:
			741	+	cancelOpen()
			742	+	case <-openCtx.Done():
			743	+	}
			744	+	}()
			745	+	}
			746	+	stream, err := service.OpenStream(openCtx, request)
			747	+	if keys != nil {
			748	+	keys.release()
			749	+	cancelOpen()
			750	+	<-watchDone
			751	+	select {
			752	+	case <-keys.cancelled:
			753	+	s.logger.Info("client_attachment_resume_cancelled", "phase", "stream_open")
			754	+	if !supervisorNil(stream) {
			755	+	_ = stream.Close()
			756	+	}
			757	+	s.transition(supervisorEvent{kind: supervisorAttachEnded})
			758	+	return attachmentEndedOutcome()
			759	+	default:
			760	+	}
			761	+	}
700		if err != nil {	762		if err != nil {
701		s.logger.Warn("client_stream_open_failed", "local", request.Local, "endpoint", request.Endpoint, "code", brokerErrorCode(err), "error", err, "cause", errors.Unwrap(err))	763		s.logger.Warn("client_stream_open_failed", "local", request.Local, "endpoint", request.Endpoint, "code", brokerErrorCode(err), "error", err, "cause", errors.Unwrap(err))
702		return s.attemptFailed(deadline.cause(err), resuming)	764		return s.attemptFailed(deadline.cause(err), resuming)

@@ -898,11 +960,17 @@ settlement:
898		return attachmentEndedOutcome()	960		return attachmentEndedOutcome()
899		}	961		}
900			962
901	-	// maxAttachmentResumes bounds how many times one lost attachment reconnects	963	+	// maxAttachmentResumes is the minimum number of resume attempts before the
902	-	// before the client gives up and returns to the picker. With the supervisor	964	+	// client may give up on a lost attachment. Giving up also requires the outage
903	-	// backoff this spans a few seconds, enough to ride out a transient drop.	965	+	// to have lasted resumeOutageWindow, so a flaky mobile link keeps resuming
			966	+	// instead of dropping to the picker after a few seconds.
904		const maxAttachmentResumes = 5	967		const maxAttachmentResumes = 5
905			968
			969	+	// resumeOutageWindow is how long one outage keeps resuming. It matches the
			970	+	// daemon's park window for a detached remote attachment: past it the session
			971	+	// view is gone anyway.
			972	+	const resumeOutageWindow = 15 * time.Minute
			973	+
906		// resumeStableAttachment is how long a resumed attachment must stay up before	974		// resumeStableAttachment is how long a resumed attachment must stay up before
907		// its next loss gets a fresh resume budget.	975		// its next loss gets a fresh resume budget.
908		const resumeStableAttachment = 30 * time.Second	976		const resumeStableAttachment = 30 * time.Second

@@ -924,16 +992,35 @@ func resumeTarget(request ports.BrokerOpenStreamRequest, fg *attachmentForegroun
924			992
925		// waitResume presents Connecting and waits one backoff interval before the	993		// waitResume presents Connecting and waits one backoff interval before the
926		// next resume attempt. It settles early on cancellation or terminal EOF	994		// next resume attempt. It settles early on cancellation or terminal EOF
927	-	// (terminated), or on broker loss, which ends the resume and leaves the ready	995	+	// (terminated), or with stop on broker loss, which leaves the ready loop to
928	-	// loop to observe and retire the connection.	996	+	// observe and retire the connection, or when the user cancels the resume
929	-	func (s Supervisor) waitResume(ctx context.Context, input terminalInputLifetime, service ports.BrokerNavigator, attempt int) (terminated bool, termErr error, brokerLost bool) {	997	+	// with Ctrl-C or Esc, which returns to the picker. Other keys typed during
			998	+	// the outage are kept and replayed into the session once it resumes.
			999	+	func (s Supervisor) waitResume(ctx context.Context, input terminalInputLifetime, service ports.BrokerNavigator, attempt int) (terminated bool, termErr error, stop bool) {
930		s.transition(supervisorEvent{kind: supervisorAttachBegin})	1000		s.transition(supervisorEvent{kind: supervisorAttachBegin})
931		delay := supervisorBackoffDelay(uint64(attempt), s.cfg.Jitter)	1001		delay := supervisorBackoffDelay(uint64(attempt), s.cfg.Jitter)
932		timer := s.cfg.Clock.NewTimer(delay)	1002		timer := s.cfg.Clock.NewTimer(delay)
933		defer stopSupervisorTimer(timer)	1003		defer stopSupervisorTimer(timer)
			1004	+	keys := input.watchResume(s.cfg.Clock)
			1005	+	defer keys.release()
934		for {	1006		for {
935		select {	1007		select {
			1008	+	case <-keys.cancelled:
			1009	+	// The user gave up on the outage: back to the picker, quietly.
			1010	+	s.logger.Info("client_attachment_resume_cancelled", "attempt", attempt)
			1011	+	s.transition(supervisorEvent{kind: supervisorAttachEnded})
			1012	+	return false, nil, true
936		case <-timer.C():	1013		case <-timer.C():
			1014	+	// Release resolves an Esc typed just before the backoff ended,
			1015	+	// which may still cancel.
			1016	+	keys.release()
			1017	+	select {
			1018	+	case <-keys.cancelled:
			1019	+	s.logger.Info("client_attachment_resume_cancelled", "attempt", attempt)
			1020	+	s.transition(supervisorEvent{kind: supervisorAttachEnded})
			1021	+	return false, nil, true
			1022	+	default:
			1023	+	}
937		return false, nil, false	1024		return false, nil, false
938		case <-ctx.Done():	1025		case <-ctx.Done():
939		return true, ctx.Err(), false	1026		return true, ctx.Err(), false

M internal/usecase/client/supervisor_attach_test.go +82 -12


@@ -265,8 +265,10 @@ type attachTestPicker struct {
265	265		opsReady chan struct{}
266	266		ownsInput bool
267	267		consumed int
268		-	op pickerOp
269		-	key string
	268	+	// consumedBytes records every byte the picker consumed, in order.
	269	+	consumedBytes []byte
	270	+	op pickerOp
	271	+	key string
270	272		// resolveErr makes ResolveCommit refuse locally, which is how a test reaches
271	273		// the local-refusal classification without dialing anything.
272	274		resolveErr error

@@ -288,13 +290,14 @@ func newAttachTestPicker() *attachTestPicker {
288	290
289	291		func (p *attachTestPicker) ApplySnapshot(ports.BrokerSnapshot) {}
290	292
291		-	func (p *attachTestPicker) ConsumeTerminalRead([]byte) bool {
	293	+	func (p *attachTestPicker) ConsumeTerminalRead(data []byte) bool {
292	294		p.mu.Lock()
293	295		defer p.mu.Unlock()
294	296		if !p.ownsInput {
295	297		return false
296	298		}
297	299		p.consumed++
	300	+	p.consumedBytes = append(p.consumedBytes, data...)
298	301		if p.consumeOp != (pickerOp{}) {
299	302		p.op, p.key, p.request = p.consumeOp, "row", p.consumeRequest
300	303		p.consumeOp = pickerOp{}

@@ -601,6 +604,56 @@ func TestSupervisorPendingSwapContinuesRatherThanExiting(t *testing.T) {
601	604		}
602	605		}
603	606
	607	+	// TestSupervisorSwapDropsKeysKeptForPreviousSession types into a session whose
	608	+	// link stalls, then swaps to another session: the stalled keys were meant for
	609	+	// the first session and must never run in the second.
	610	+	func TestSupervisorSwapDropsKeysKeptForPreviousSession(t *testing.T) {
	611	+	picker := newAttachTestPicker()
	612	+	harness := startAttachHarness(t, picker)
	613	+	base := newSessionTestStream()
	614	+	first := &stallingSessionStream{sessionTestStream: base, entered: make(chan struct{}, 1), release: make(chan struct{})}
	615	+	harness.service.setOpenStream(func(context.Context, ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
	616	+	return first, nil
	617	+	})
	618	+	picker.commit(sessionTestRequest(true))
	619	+	deliverReadyStream(t, base)
	620	+	awaitAttachedState(t, harness.sup)
	621	+	require.Eventually(t, func() bool { return len(base.messages()) >= 2 }, 5*time.Second, time.Millisecond)
	622	+	first.stall.Store(true)
	623	+	harness.reader.push([]byte("rm -rf build\r"))
	624	+	<-first.entered
	625	+
	626	+	var mu sync.Mutex
	627	+	second := newSessionTestStream()
	628	+	harness.service.setOpenStream(func(context.Context, ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
	629	+	return second, nil
	630	+	})
	631	+	base.deliver(navigationOffer(1))
	632	+	awaitPresentation(t, harness.sup, PresentAttachedPicker)
	633	+	other := sessionTestRequest(true)
	634	+	other.Target = protocol.ExactSessionTarget{LifecycleID: domain.SessionLifecycleID{2}, SessionName: "beta"}
	635	+	picker.recordOp(pickerOp{commit: true}, "row", other)
	636	+	// The Detach queues behind the stalled write; let the bounded drain expire
	637	+	// so the unsent keys are preserved, as on a dead link.
	638	+	for {
	639	+	timer := harness.clock.awaitTimer(t)
	640	+	if timer.delay == drainOutboxTimeout {
	641	+	timer.fire()
	642	+	break
	643	+	}
	644	+	}
	645	+	awaitStreamHello(t, &mu, &second)
	646	+	second.deliver(protocol.Welcome{SessionName: "beta"})
	647	+	output := sessionTestOutput(1, "\x1b[Hbeta")
	648	+	output.Context.Route.Target = other.Target
	649	+	second.deliver(output)
	650	+	awaitAttachedState(t, harness.sup)
	651	+
	652	+	harness.reader.push([]byte("z"))
	653	+	require.Eventually(t, func() bool { return sentInput(second) == "z" }, 5*time.Second, time.Millisecond,
	654	+	"second session input = %q", sentInput(second))
	655	+	}
	656	+
604	657		// awaitStreamHello waits until the admitted stream has sent its first client
605	658		// message, polling the guarded stream pointer rather than a stale snapshot.
606	659		func awaitStreamHello(t testing.T, mu sync.Mutex, stream **sessionTestStream) {

@@ -1304,14 +1357,15 @@ func testStreamLoss() ports.BrokerStreamLost {
1304	1357		// 15s attachment deadline and the 200ms palette deadline).
1305	1358		func resumeTestJitter() float64 { return 0.5 }
1306	1359
1307		-	// fireResumeTimer fires the resume backoff timer for one attempt. Other timers
1308		-	// share the clock, so it matches the exact backoff delay.
1309		-	func fireResumeTimer(t testing.T, clock supervisorTestClock, attempt int) {
	1360	+	// fireResumeTimerAfter waits for the attempt's backoff timer, runs before, then
	1361	+	// fires it.
	1362	+	func fireResumeTimerAfter(t testing.T, clock supervisorTestClock, attempt int, before func()) {
1310	1363		t.Helper()
1311	1364		want := supervisorBackoffDelay(uint64(attempt), resumeTestJitter)
1312	1365		for {
1313	1366		timer := clock.awaitTimer(t)
1314	1367		if timer.delay == want {
	1368	+	before()
1315	1369		timer.fire()
1316	1370		return
1317	1371		}

@@ -1320,22 +1374,30 @@ func fireResumeTimer(t testing.T, clock supervisorTestClock, attempt int) {
1320	1374
1321	1375		func TestAttachmentResumeBudget(t *testing.T) {
1322	1376		brief := time.Second
	1377	+	// step is the clock advance between spends: past the outage window, the
	1378	+	// attempt count alone decides; within it, resuming never gives up.
	1379	+	past := resumeOutageWindow / maxAttachmentResumes
1323	1380		tests := []struct {
1324	1381		name string
1325	1382		steps []attachmentOutcomeKind
1326	1383		ups []time.Duration
	1384	+	step time.Duration
1327	1385		want []bool
1328	1386		}{
1329		-	{name: "flapping resumes exhaust", steps: repeatKinds(attachmentResume, maxAttachmentResumes+1), ups: repeatDurations(brief, maxAttachmentResumes+1), want: append(repeatBools(true, maxAttachmentResumes), false)},
1330		-	{name: "stable attachment restores the budget", steps: repeatKinds(attachmentResume, maxAttachmentResumes+2), ups: repeatDurations(resumeStableAttachment, maxAttachmentResumes+2), want: repeatBools(true, maxAttachmentResumes+2)},
1331		-	{name: "retries never restore the budget", steps: repeatKinds(attachmentRetry, maxAttachmentResumes+1), ups: repeatDurations(resumeStableAttachment, maxAttachmentResumes+1), want: append(repeatBools(true, maxAttachmentResumes), false)},
	1387	+	{name: "flapping resumes exhaust", steps: repeatKinds(attachmentResume, maxAttachmentResumes+1), ups: repeatDurations(brief, maxAttachmentResumes+1), step: past, want: append(repeatBools(true, maxAttachmentResumes), false)},
	1388	+	{name: "stable attachment restores the budget", steps: repeatKinds(attachmentResume, maxAttachmentResumes+2), ups: repeatDurations(resumeStableAttachment, maxAttachmentResumes+2), step: past, want: repeatBools(true, maxAttachmentResumes+2)},
	1389	+	{name: "retries never restore the budget", steps: repeatKinds(attachmentRetry, maxAttachmentResumes+1), ups: repeatDurations(resumeStableAttachment, maxAttachmentResumes+1), step: past, want: append(repeatBools(true, maxAttachmentResumes), false)},
	1390	+	{name: "flaps exhaust inside the outage window", steps: repeatKinds(attachmentResume, maxAttachmentResumes+1), ups: repeatDurations(brief, maxAttachmentResumes+1), step: time.Second, want: append(repeatBools(true, maxAttachmentResumes), false)},
	1391	+	{name: "outage window keeps resuming", steps: repeatKinds(attachmentRetry, 3maxAttachmentResumes), ups: repeatDurations(brief, 3maxAttachmentResumes), step: time.Second, want: repeatBools(true, 3*maxAttachmentResumes)},
1332	1392		}
1333	1393		for _, tt := range tests {
1334	1394		t.Run(tt.name, func(t *testing.T) {
1335	1395		var budget attachmentResumeBudget
1336	1396		require.False(t, budget.resuming)
	1397	+	now := time.Unix(0, 0)
1337	1398		for i, kind := range tt.steps {
1338		-	_, ok := budget.spend(kind, tt.ups[i])
	1399	+	_, ok := budget.spend(kind, tt.ups[i], now)
	1400	+	now = now.Add(tt.step)
1339	1401		require.Equal(t, tt.want[i], ok, "step %d", i)
1340	1402		require.True(t, budget.resuming)
1341	1403		}

@@ -1443,7 +1505,7 @@ func TestSupervisorAttachmentLossResumesSameSession(t *testing.T) {
1443	1505		first.fail(testStreamLoss())
1444	1506
1445	1507		attempt := 1
1446		-	for _, s := range tc.steps {
	1508	+	for i, s := range tc.steps {
1447	1509		if s == cancelDuringBackoff {
1448	1510		backoff := supervisorBackoffDelay(1, resumeTestJitter)
1449	1511		for harness.clock.awaitTimer(t).delay != backoff {

@@ -1456,7 +1518,15 @@ func TestSupervisorAttachmentLossResumesSameSession(t *testing.T) {
1456	1518		refusing.Store(s == refuse)
1457	1519		unavailable.Store(s == openUnavailable)
1458	1520		opened := len(harness.service.openedRequests())
1459		-	fireResumeTimer(t, harness.clock, attempt)
	1521	+	// Giving up also requires the outage window to pass. The
	1522	+	// clock moves only once the last attempt's backoff is
	1523	+	// scheduled, so no attempt measures it as uptime.
	1524	+	last := tc.wantPicker && i == len(tc.steps)-1
	1525	+	fireResumeTimerAfter(t, harness.clock, attempt, func() {
	1526	+	if last {
	1527	+	advancePickerClock(harness.clock, resumeOutageWindow)
	1528	+	}
	1529	+	})
1460	1530		if s == refuse {
1461	1531		break
1462	1532		}

 	opsReady  chan struct{}
 	ownsInput bool
 	consumed  int
-	op        pickerOp
+	// consumedBytes records every byte the picker consumed, in order.
-	key       string
+	consumedBytes []byte
+	op            pickerOp
+	key           string
 	// resolveErr makes ResolveCommit refuse locally, which is how a test reaches
 	// the local-refusal classification without dialing anything.
 	resolveErr error
 func (p *attachTestPicker) ApplySnapshot(ports.BrokerSnapshot) {}
-func (p *attachTestPicker) ConsumeTerminalRead([]byte) bool {
+func (p *attachTestPicker) ConsumeTerminalRead(data []byte) bool {
 	p.mu.Lock()
 	defer p.mu.Unlock()
 	if !p.ownsInput {
 		return false
 	}
 	p.consumed++
+	p.consumedBytes = append(p.consumedBytes, data...)
 	if p.consumeOp != (pickerOp{}) {
 		p.op, p.key, p.request = p.consumeOp, "row", p.consumeRequest
 		p.consumeOp = pickerOp{}
 	}
 }
+// TestSupervisorSwapDropsKeysKeptForPreviousSession types into a session whose
+// link stalls, then swaps to another session: the stalled keys were meant for
+// the first session and must never run in the second.
+func TestSupervisorSwapDropsKeysKeptForPreviousSession(t *testing.T) {
+	picker := newAttachTestPicker()
+	harness := startAttachHarness(t, picker)
+	base := newSessionTestStream()
+	first := &stallingSessionStream{sessionTestStream: base, entered: make(chan struct{}, 1), release: make(chan struct{})}
+	harness.service.setOpenStream(func(context.Context, ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
+		return first, nil
+	})
+	picker.commit(sessionTestRequest(true))
+	deliverReadyStream(t, base)
+	awaitAttachedState(t, harness.sup)
+	require.Eventually(t, func() bool { return len(base.messages()) >= 2 }, 5*time.Second, time.Millisecond)
+	first.stall.Store(true)
+	harness.reader.push([]byte("rm -rf build\r"))
+	<-first.entered
+	var mu sync.Mutex
+	second := newSessionTestStream()
+	harness.service.setOpenStream(func(context.Context, ports.BrokerOpenStreamRequest) (ports.BrokerLogicalConnection, error) {
+		return second, nil
+	})
+	base.deliver(navigationOffer(1))
+	awaitPresentation(t, harness.sup, PresentAttachedPicker)
+	other := sessionTestRequest(true)
+	other.Target = protocol.ExactSessionTarget{LifecycleID: domain.SessionLifecycleID{2}, SessionName: "beta"}
+	picker.recordOp(pickerOp{commit: true}, "row", other)
+	// The Detach queues behind the stalled write; let the bounded drain expire
+	// so the unsent keys are preserved, as on a dead link.
+	for {
+		timer := harness.clock.awaitTimer(t)
+		if timer.delay == drainOutboxTimeout {
+			timer.fire()
+			break
+		}
+	}
+	awaitStreamHello(t, &mu, &second)
+	second.deliver(protocol.Welcome{SessionName: "beta"})
+	output := sessionTestOutput(1, "\x1b[Hbeta")
+	output.Context.Route.Target = other.Target
+	second.deliver(output)
+	awaitAttachedState(t, harness.sup)
+	harness.reader.push([]byte("z"))
+	require.Eventually(t, func() bool { return sentInput(second) == "z" }, 5*time.Second, time.Millisecond,
+		"second session input = %q", sentInput(second))
+}
 // awaitStreamHello waits until the admitted stream has sent its first client
 // message, polling the guarded stream pointer rather than a stale snapshot.
 func awaitStreamHello(t *testing.T, mu *sync.Mutex, stream **sessionTestStream) {
 // 15s attachment deadline and the 200ms palette deadline).
 func resumeTestJitter() float64 { return 0.5 }
-// fireResumeTimer fires the resume backoff timer for one attempt. Other timers
+// fireResumeTimerAfter waits for the attempt's backoff timer, runs before, then
-// share the clock, so it matches the exact backoff delay.
+// fires it.
-func fireResumeTimer(t *testing.T, clock *supervisorTestClock, attempt int) {
+func fireResumeTimerAfter(t *testing.T, clock *supervisorTestClock, attempt int, before func()) {
 	t.Helper()
 	want := supervisorBackoffDelay(uint64(attempt), resumeTestJitter)
 	for {
 		timer := clock.awaitTimer(t)
 		if timer.delay == want {
+			before()
 			timer.fire()
 			return
 		}
 func TestAttachmentResumeBudget(t *testing.T) {
 	brief := time.Second
+	// step is the clock advance between spends: past the outage window, the
+	// attempt count alone decides; within it, resuming never gives up.
+	past := resumeOutageWindow / maxAttachmentResumes
 	tests := []struct {
 		name  string
 		steps []attachmentOutcomeKind
 		ups   []time.Duration
+		step  time.Duration
 		want  []bool
 	}{
-		{name: "flapping resumes exhaust", steps: repeatKinds(attachmentResume, maxAttachmentResumes+1), ups: repeatDurations(brief, maxAttachmentResumes+1), want: append(repeatBools(true, maxAttachmentResumes), false)},
+		{name: "flapping resumes exhaust", steps: repeatKinds(attachmentResume, maxAttachmentResumes+1), ups: repeatDurations(brief, maxAttachmentResumes+1), step: past, want: append(repeatBools(true, maxAttachmentResumes), false)},
-		{name: "stable attachment restores the budget", steps: repeatKinds(attachmentResume, maxAttachmentResumes+2), ups: repeatDurations(resumeStableAttachment, maxAttachmentResumes+2), want: repeatBools(true, maxAttachmentResumes+2)},
+		{name: "stable attachment restores the budget", steps: repeatKinds(attachmentResume, maxAttachmentResumes+2), ups: repeatDurations(resumeStableAttachment, maxAttachmentResumes+2), step: past, want: repeatBools(true, maxAttachmentResumes+2)},
-		{name: "retries never restore the budget", steps: repeatKinds(attachmentRetry, maxAttachmentResumes+1), ups: repeatDurations(resumeStableAttachment, maxAttachmentResumes+1), want: append(repeatBools(true, maxAttachmentResumes), false)},
+		{name: "retries never restore the budget", steps: repeatKinds(attachmentRetry, maxAttachmentResumes+1), ups: repeatDurations(resumeStableAttachment, maxAttachmentResumes+1), step: past, want: append(repeatBools(true, maxAttachmentResumes), false)},
+		{name: "flaps exhaust inside the outage window", steps: repeatKinds(attachmentResume, maxAttachmentResumes+1), ups: repeatDurations(brief, maxAttachmentResumes+1), step: time.Second, want: append(repeatBools(true, maxAttachmentResumes), false)},
+		{name: "outage window keeps resuming", steps: repeatKinds(attachmentRetry, 3*maxAttachmentResumes), ups: repeatDurations(brief, 3*maxAttachmentResumes), step: time.Second, want: repeatBools(true, 3*maxAttachmentResumes)},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
 			var budget attachmentResumeBudget
 			require.False(t, budget.resuming)
+			now := time.Unix(0, 0)
 			for i, kind := range tt.steps {
-				_, ok := budget.spend(kind, tt.ups[i])
+				_, ok := budget.spend(kind, tt.ups[i], now)
+				now = now.Add(tt.step)
 				require.Equal(t, tt.want[i], ok, "step %d", i)
 				require.True(t, budget.resuming)
 			}
 			first.fail(testStreamLoss())
 			attempt := 1
-			for _, s := range tc.steps {
+			for i, s := range tc.steps {
 				if s == cancelDuringBackoff {
 					backoff := supervisorBackoffDelay(1, resumeTestJitter)
 					for harness.clock.awaitTimer(t).delay != backoff {
 				refusing.Store(s == refuse)
 				unavailable.Store(s == openUnavailable)
 				opened := len(harness.service.openedRequests())
-				fireResumeTimer(t, harness.clock, attempt)
+				// Giving up also requires the outage window to pass. The
+				// clock moves only once the last attempt's backoff is
+				// scheduled, so no attempt measures it as uptime.
+				last := tc.wantPicker && i == len(tc.steps)-1
+				fireResumeTimerAfter(t, harness.clock, attempt, func() {
+					if last {
+						advancePickerClock(harness.clock, resumeOutageWindow)
+					}
+				})
 				if s == refuse {
 					break
 				}

M internal/usecase/client/terminal_input_pump.go +51 -2


@@ -105,10 +105,25 @@ func newTerminalInputPump(in io.Reader) *terminalInputPump {
105	105
106	106		// tryClaim lets competing foreground owners decline admission without panicking.
107	107		func (p *terminalInputPump) tryClaim() (uint64, bool) {
	108	+	consumer, _, ok := p.claimWithBacklog()
	109	+	return consumer, ok
	110	+	}
	111	+
	112	+	// claimWithBacklog claims the pump and reports, atomically with the claim, how
	113	+	// many deliveries were already waiting: the residual, then the pending read.
	114	+	// They were read for an earlier owner, not typed for this one.
	115	+	func (p *terminalInputPump) claimWithBacklog() (uint64, int, bool) {
108	116		p.mu.Lock()
109	117		if p.consumer != 0 {
110	118		p.mu.Unlock()
111		-	return 0, false
	119	+	return 0, 0, false
	120	+	}
	121	+	backlog := 0
	122	+	if len(p.residual) != 0 {
	123	+	backlog++
	124	+	}
	125	+	if p.pending != nil {
	126	+	backlog++
112	127		}
113	128		p.nextID++
114	129		p.consumer = p.nextID

@@ -122,7 +137,7 @@ func (p *terminalInputPump) tryClaim() (uint64, bool) {
122	137		if ready {
123	138		p.signalReady(consumer)
124	139		}
125		-	return consumer, true
	140	+	return consumer, backlog, true
126	141		}
127	142
128	143		// revoke invalidates an attempt before its replacement is allowed to claim

@@ -242,6 +257,19 @@ func (p *terminalInputPump) preserveResidual(consumer uint64, data []byte) {
242	257		p.residual = append(p.residual[:0], data...)
243	258		}
244	259
	260	+	// appendResidual appends bytes the consumer took but did not deliver, so the
	261	+	// next owner replays them after any earlier residual. It reports false when
	262	+	// the bound would be exceeded; the bytes are then dropped.
	263	+	func (p *terminalInputPump) appendResidual(consumer uint64, data []byte, limit int) bool {
	264	+	p.mu.Lock()
	265	+	defer p.mu.Unlock()
	266	+	if p.consumer != consumer \|\| len(p.residual)+len(data) > limit {
	267	+	return false
	268	+	}
	269	+	p.residual = append(p.residual, data...)
	270	+	return true
	271	+	}
	272	+
245	273		// ack commits scanner delivery of a leased read and lets the lifecycle reader
246	274		// accept the next result. Only the consumer that holds the lease can ack it.
247	275		// dropOwned disposes every undecided byte at an autonomous owner-class

@@ -271,6 +299,27 @@ func (p *terminalInputPump) dropOwned(consumer uint64) {
271	299		}
272	300		}
273	301
	302	+	// dropUnclaimed disposes pending and preserved bytes while no consumer holds
	303	+	// the claim. The supervisor calls it when the terminal goes back to the
	304	+	// picker, so session input kept for a resume never reaches the picker.
	305	+	func (p *terminalInputPump) dropUnclaimed() {
	306	+	p.mu.Lock()
	307	+	if p.consumer != 0 {
	308	+	p.mu.Unlock()
	309	+	return
	310	+	}
	311	+	hadPending := p.pending != nil
	312	+	p.pending = nil
	313	+	p.residual = nil
	314	+	p.mu.Unlock()
	315	+	if hadPending {
	316	+	select {
	317	+	case p.space <- struct{}{}:
	318	+	default:
	319	+	}
	320	+	}
	321	+	}
	322	+
274	323		func (p *terminalInputPump) ack(consumer uint64) {
275	324		p.mu.Lock()
276	325		if p.consumer != consumer \|\| p.delivering != consumer {


@@ -105,10 +105,25 @@ func newTerminalInputPump(in io.Reader) *terminalInputPump {
105			105
106		// tryClaim lets competing foreground owners decline admission without panicking.	106		// tryClaim lets competing foreground owners decline admission without panicking.
107		func (p *terminalInputPump) tryClaim() (uint64, bool) {	107		func (p *terminalInputPump) tryClaim() (uint64, bool) {
			108	+	consumer, _, ok := p.claimWithBacklog()
			109	+	return consumer, ok
			110	+	}
			111	+
			112	+	// claimWithBacklog claims the pump and reports, atomically with the claim, how
			113	+	// many deliveries were already waiting: the residual, then the pending read.
			114	+	// They were read for an earlier owner, not typed for this one.
			115	+	func (p *terminalInputPump) claimWithBacklog() (uint64, int, bool) {
108		p.mu.Lock()	116		p.mu.Lock()
109		if p.consumer != 0 {	117		if p.consumer != 0 {
110		p.mu.Unlock()	118		p.mu.Unlock()
111	-	return 0, false	119	+	return 0, 0, false
			120	+	}
			121	+	backlog := 0
			122	+	if len(p.residual) != 0 {
			123	+	backlog++
			124	+	}
			125	+	if p.pending != nil {
			126	+	backlog++
112		}	127		}
113		p.nextID++	128		p.nextID++
114		p.consumer = p.nextID	129		p.consumer = p.nextID

@@ -122,7 +137,7 @@ func (p *terminalInputPump) tryClaim() (uint64, bool) {
122		if ready {	137		if ready {
123		p.signalReady(consumer)	138		p.signalReady(consumer)
124		}	139		}
125	-	return consumer, true	140	+	return consumer, backlog, true
126		}	141		}
127			142
128		// revoke invalidates an attempt before its replacement is allowed to claim	143		// revoke invalidates an attempt before its replacement is allowed to claim

@@ -242,6 +257,19 @@ func (p *terminalInputPump) preserveResidual(consumer uint64, data []byte) {
242		p.residual = append(p.residual[:0], data...)	257		p.residual = append(p.residual[:0], data...)
243		}	258		}
244			259
			260	+	// appendResidual appends bytes the consumer took but did not deliver, so the
			261	+	// next owner replays them after any earlier residual. It reports false when
			262	+	// the bound would be exceeded; the bytes are then dropped.
			263	+	func (p *terminalInputPump) appendResidual(consumer uint64, data []byte, limit int) bool {
			264	+	p.mu.Lock()
			265	+	defer p.mu.Unlock()
			266	+	if p.consumer != consumer \|\| len(p.residual)+len(data) > limit {
			267	+	return false
			268	+	}
			269	+	p.residual = append(p.residual, data...)
			270	+	return true
			271	+	}
			272	+
245		// ack commits scanner delivery of a leased read and lets the lifecycle reader	273		// ack commits scanner delivery of a leased read and lets the lifecycle reader
246		// accept the next result. Only the consumer that holds the lease can ack it.	274		// accept the next result. Only the consumer that holds the lease can ack it.
247		// dropOwned disposes every undecided byte at an autonomous owner-class	275		// dropOwned disposes every undecided byte at an autonomous owner-class

@@ -271,6 +299,27 @@ func (p *terminalInputPump) dropOwned(consumer uint64) {
271		}	299		}
272		}	300		}
273			301
			302	+	// dropUnclaimed disposes pending and preserved bytes while no consumer holds
			303	+	// the claim. The supervisor calls it when the terminal goes back to the
			304	+	// picker, so session input kept for a resume never reaches the picker.
			305	+	func (p *terminalInputPump) dropUnclaimed() {
			306	+	p.mu.Lock()
			307	+	if p.consumer != 0 {
			308	+	p.mu.Unlock()
			309	+	return
			310	+	}
			311	+	hadPending := p.pending != nil
			312	+	p.pending = nil
			313	+	p.residual = nil
			314	+	p.mu.Unlock()
			315	+	if hadPending {
			316	+	select {
			317	+	case p.space <- struct{}{}:
			318	+	default:
			319	+	}
			320	+	}
			321	+	}
			322	+
274		func (p *terminalInputPump) ack(consumer uint64) {	323		func (p *terminalInputPump) ack(consumer uint64) {
275		p.mu.Lock()	324		p.mu.Lock()
276		if p.consumer != consumer \|\| p.delivering != consumer {	325		if p.consumer != consumer \|\| p.delivering != consumer {

M internal/usecase/daemon/render_pipeline.go +6 -1


@@ -335,7 +335,12 @@ func captureOverlayLayers(state capturedRenderState, snap overlayRenderSnapsho
335	335		modal := promptModalFor(snap.promptModel.Title())
336	336		presentation := modal.Resolve(size)
337	337		o.prompt = capturedModal{active: true, title: modal.Title, presentation: presentation, focused: true}
338		-	o.prompt.inner = snap.promptModel.RenderStyled(rectSize(presentation.Inner), prompt.RenderStyles{Base: styles.PromptBase, Selection: styles.SurfaceActive})
	338	+	selection := styles.Selection
	339	+	// Reduced copy selection is plain reverse; prompts retain bold emphasis.
	340	+	if styles.Reduced {
	341	+	selection = styles.SurfaceActive
	342	+	}
	343	+	o.prompt.inner = snap.promptModel.RenderStyled(rectSize(presentation.Inner), prompt.RenderStyles{Base: styles.PromptBase, Selection: selection})
339	344		}
340	345		state.cursor.hiddenByOverlay = o.active()
341	346		}

 		modal := promptModalFor(snap.promptModel.Title())
 		presentation := modal.Resolve(size)
 		o.prompt = capturedModal{active: true, title: modal.Title, presentation: presentation, focused: true}
-		o.prompt.inner = snap.promptModel.RenderStyled(rectSize(presentation.Inner), prompt.RenderStyles{Base: styles.PromptBase, Selection: styles.SurfaceActive})
+		selection := styles.Selection
+		// Reduced copy selection is plain reverse; prompts retain bold emphasis.
+		if styles.Reduced {
+			selection = styles.SurfaceActive
+		}
+		o.prompt.inner = snap.promptModel.RenderStyled(rectSize(presentation.Inner), prompt.RenderStyles{Base: styles.PromptBase, Selection: selection})
 	}
 	state.cursor.hiddenByOverlay = o.active()
 }

M internal/usecase/daemon/theme_integration_test.go +2 -2


@@ -161,12 +161,12 @@ func TestAccentLifecycleNeutralChromeByteSnapshots(t *testing.T) {
161	161		{
162	162		name: "forced dark",
163	163		config: domain.Config{Theme: domain.ThemeDark},
164		-	want: "\x1b[1;1H\x1b[0;38;2;216;216;216;48;2;47;47;47mABC\x1b[0;38;2;216;216;216;48;2;72;72;72mD\x1b[0;38;2;139;139;139mEF\x1b[0m",
	164	+	want: "\x1b[1;1H\x1b[0;38;2;216;216;216;48;2;47;47;47mABC\x1b[0;38;2;24;24;24;48;2;216;216;216mD\x1b[0;38;2;139;139;139mEF\x1b[0m",
165	165		},
166	166		{
167	167		name: "forced light",
168	168		config: domain.Config{Theme: domain.ThemeLight},
169		-	want: "\x1b[1;1H\x1b[0;38;2;32;32;32;48;2;222;222;222mABC\x1b[0;38;2;32;32;32;48;2;194;194;194mD\x1b[0;38;2;118;118;118mEF\x1b[0m",
	169	+	want: "\x1b[1;1H\x1b[0;38;2;32;32;32;48;2;222;222;222mABC\x1b[0;38;2;248;248;248;48;2;32;32;32mD\x1b[0;38;2;118;118;118mEF\x1b[0m",
170	170		},
171	171		}
172	172		for _, tt := range tests {


@@ -161,12 +161,12 @@ func TestAccentLifecycleNeutralChromeByteSnapshots(t *testing.T) {
161		{	161		{
162		name: "forced dark",	162		name: "forced dark",
163		config: domain.Config{Theme: domain.ThemeDark},	163		config: domain.Config{Theme: domain.ThemeDark},
164	-	want: "\x1b[1;1H\x1b[0;38;2;216;216;216;48;2;47;47;47mABC\x1b[0;38;2;216;216;216;48;2;72;72;72mD\x1b[0;38;2;139;139;139mEF\x1b[0m",	164	+	want: "\x1b[1;1H\x1b[0;38;2;216;216;216;48;2;47;47;47mABC\x1b[0;38;2;24;24;24;48;2;216;216;216mD\x1b[0;38;2;139;139;139mEF\x1b[0m",
165		},	165		},
166		{	166		{
167		name: "forced light",	167		name: "forced light",
168		config: domain.Config{Theme: domain.ThemeLight},	168		config: domain.Config{Theme: domain.ThemeLight},
169	-	want: "\x1b[1;1H\x1b[0;38;2;32;32;32;48;2;222;222;222mABC\x1b[0;38;2;32;32;32;48;2;194;194;194mD\x1b[0;38;2;118;118;118mEF\x1b[0m",	169	+	want: "\x1b[1;1H\x1b[0;38;2;32;32;32;48;2;222;222;222mABC\x1b[0;38;2;248;248;248;48;2;32;32;32mD\x1b[0;38;2;118;118;118mEF\x1b[0m",
170		},	170		},
171		}	171		}
172		for _, tt := range tests {	172		for _, tt := range tests {

A internal/usecase/theme/neutral_session_test.go +40 -0

@@ -0,0 +1,40 @@
	1	+	package theme
	2	+
	3	+	import (
	4	+	"testing"
	5	+
	6	+	renderer "github.com/bnema/vev-vt"
	7	+	"github.com/stretchr/testify/require"
	8	+	)
	9	+
	10	+	func TestNeutralSessionSurfaceHierarchy(t *testing.T) {
	11	+	black := renderer.RGB{}
	12	+	white := renderer.RGB{R: 255, G: 255, B: 255}
	13	+	for _, tt := range []struct {
	14	+	name string
	15	+	foreground, background renderer.RGB
	16	+	}{
	17	+	{name: "dark", foreground: white, background: black},
	18	+	{name: "light", foreground: black, background: white},
	19	+	} {
	20	+	t.Run(tt.name, func(t *testing.T) {
	21	+	theme := Theme{Known: true, Foreground: tt.foreground, Background: tt.background, HasFG: true, HasBG: true}
	22	+	styles := neutralStyles(theme)
	23	+	require.Equal(t, tt.foreground, styles.SurfaceActive.BackgroundRGB)
	24	+	require.Equal(t, tt.background, styles.SurfaceActive.ForegroundRGB)
	25	+	require.Equal(t, styles.TabActive.BackgroundRGB, styles.SurfaceActive.BackgroundRGB)
	26	+	require.Equal(t, styles.SurfaceActive.BackgroundRGB, styles.TabActiveTitle.BackgroundRGB)
	27	+	soft := Blend(tt.background, tt.foreground, neutralAccentBlend)
	28	+	for _, selection := range []renderer.Style{styles.PickerSelection, styles.Selection, styles.SearchSelection} {
	29	+	require.Equal(t, soft, selection.BackgroundRGB)
	30	+	}
	31	+	require.Equal(t, Blend(styles.SurfaceBar.BackgroundRGB, tt.foreground, 0.5), styles.MRUStyle(0, 3).BackgroundRGB)
	32	+	for count := 1; count <= 9; count++ {
	33	+	for index := range count {
	34	+	want := Blend(styles.SurfaceActive.BackgroundRGB, styles.SurfaceBar.BackgroundRGB, 1-mruWeight(index, count))
	35	+	require.Equal(t, want, styles.MRUStyle(index, count).BackgroundRGB)
	36	+	}
	37	+	}
	38	+	})
	39	+	}
	40	+	}

@@ -0,0 +1,40 @@
	1	+	package theme
	2	+
	3	+	import (
	4	+	"testing"
	5	+
	6	+	renderer "github.com/bnema/vev-vt"
	7	+	"github.com/stretchr/testify/require"
	8	+	)
	9	+
	10	+	func TestNeutralSessionSurfaceHierarchy(t *testing.T) {
	11	+	black := renderer.RGB{}
	12	+	white := renderer.RGB{R: 255, G: 255, B: 255}
	13	+	for _, tt := range []struct {
	14	+	name string
	15	+	foreground, background renderer.RGB
	16	+	}{
	17	+	{name: "dark", foreground: white, background: black},
	18	+	{name: "light", foreground: black, background: white},
	19	+	} {
	20	+	t.Run(tt.name, func(t *testing.T) {
	21	+	theme := Theme{Known: true, Foreground: tt.foreground, Background: tt.background, HasFG: true, HasBG: true}
	22	+	styles := neutralStyles(theme)
	23	+	require.Equal(t, tt.foreground, styles.SurfaceActive.BackgroundRGB)
	24	+	require.Equal(t, tt.background, styles.SurfaceActive.ForegroundRGB)
	25	+	require.Equal(t, styles.TabActive.BackgroundRGB, styles.SurfaceActive.BackgroundRGB)
	26	+	require.Equal(t, styles.SurfaceActive.BackgroundRGB, styles.TabActiveTitle.BackgroundRGB)
	27	+	soft := Blend(tt.background, tt.foreground, neutralAccentBlend)
	28	+	for _, selection := range []renderer.Style{styles.PickerSelection, styles.Selection, styles.SearchSelection} {
	29	+	require.Equal(t, soft, selection.BackgroundRGB)
	30	+	}
	31	+	require.Equal(t, Blend(styles.SurfaceBar.BackgroundRGB, tt.foreground, 0.5), styles.MRUStyle(0, 3).BackgroundRGB)
	32	+	for count := 1; count <= 9; count++ {
	33	+	for index := range count {
	34	+	want := Blend(styles.SurfaceActive.BackgroundRGB, styles.SurfaceBar.BackgroundRGB, 1-mruWeight(index, count))
	35	+	require.Equal(t, want, styles.MRUStyle(index, count).BackgroundRGB)
	36	+	}
	37	+	}
	38	+	})
	39	+	}
	40	+	}

M internal/usecase/theme/phase3_review_test.go +3 -1


@@ -120,8 +120,10 @@ func TestLegacyAliasesPaletteOffAndForcedThemesRemainExact(t *testing.T) {
120	120		// PickerSeparator is now a semantic role with the SurfaceBar
121	121		// background; it was never a compatibility alias.
122	122		delete(want, "PickerSeparator")
123		-	// Active neutral tab names use a high-contrast history anchor.
	123	+	// Active neutral sessions and tab names share the history anchor.
	124	+	want["Accent"] = rgbSurface(tt.theme.Background, tt.theme.Foreground)
124	125		want["TabNameActive"] = EmphasisStyle(rgbSurface(tt.theme.Background, tt.theme.Foreground), tt.theme)
	126	+	want["TabTitleActive"] = MutedVariantStyle(rgbSurface(tt.theme.Background, tt.theme.Foreground), tt.theme)
125	127		for name, style := range want {
126	128		require.Equalf(t, style, legacyAndSemanticStyles(got)[name], "%s", name)
127	129		}

 			// PickerSeparator is now a semantic role with the SurfaceBar
 			// background; it was never a compatibility alias.
 			delete(want, "PickerSeparator")
-			// Active neutral tab names use a high-contrast history anchor.
+			// Active neutral sessions and tab names share the history anchor.
+			want["Accent"] = rgbSurface(tt.theme.Background, tt.theme.Foreground)
 			want["TabNameActive"] = EmphasisStyle(rgbSurface(tt.theme.Background, tt.theme.Foreground), tt.theme)
+			want["TabTitleActive"] = MutedVariantStyle(rgbSurface(tt.theme.Background, tt.theme.Foreground), tt.theme)
 			for name, style := range want {
 				require.Equalf(t, style, legacyAndSemanticStyles(got)[name], "%s", name)
 			}

M internal/usecase/theme/resolved_styles.go +9 -3


@@ -117,8 +117,8 @@ func indexedForeground(style renderer.Style, slot uint8) renderer.Style {
117	117		return style
118	118		}
119	119
120		-	// neutralStyles deliberately preserves the pre-accent neutral hierarchy. It
121		-	// is used for palette-off, forced themes, incomplete defaults, and RGB-less
	120	+	// neutralStyles uses full-strength session surfaces and softer overlay selections.
	121	+	// It is used for palette-off, forced themes, incomplete defaults, and RGB-less
122	122		// indexed fallbacks; it never performs a palette slot lookup.
123	123		func neutralStyles(t Theme) Styles {
124	124		status := renderer.DefaultStyle()

@@ -162,8 +162,14 @@ func neutralStyles(t Theme) Styles {
162	162		styles.PaletteDesc = legacyMutedText(t)
163	163		styles = withMRUStyles(styles, Ramp{SurfaceBar: styles.SurfaceBar, SurfaceRecent: styles.SurfaceRecent})
164	164		if usable(t) {
165		-	styles.TabActive = EmphasisStyle(rgbSurface(t.Background, t.Foreground), t)
	165	+	// Sessions share the full-strength neutral endpoint with active tabs;
	166	+	// overlay selections retain their softer neutral surface.
	167	+	styles.SurfaceActive = rgbSurface(t.Background, t.Foreground)
	168	+	styles.Accent = styles.SurfaceActive
	169	+	styles.TabActive = EmphasisStyle(styles.SurfaceActive, t)
166	170		styles.TabNameActive = styles.TabActive
	171	+	styles.TabActiveTitle = MutedVariantStyle(styles.SurfaceActive, t)
	172	+	styles.TabTitleActive = styles.TabActiveTitle
167	173		// Without a palette accent the history uses the same mruWeight
168	174		// placement as MRUStyle, between the active tab and the bar.
169	175		for count := 1; count <= len(styles.mruStyles); count++ {

 	return style
 }
-// neutralStyles deliberately preserves the pre-accent neutral hierarchy. It
+// neutralStyles uses full-strength session surfaces and softer overlay selections.
-// is used for palette-off, forced themes, incomplete defaults, and RGB-less
+// It is used for palette-off, forced themes, incomplete defaults, and RGB-less
 // indexed fallbacks; it never performs a palette slot lookup.
 func neutralStyles(t Theme) Styles {
 	status := renderer.DefaultStyle()
 	styles.PaletteDesc = legacyMutedText(t)
 	styles = withMRUStyles(styles, Ramp{SurfaceBar: styles.SurfaceBar, SurfaceRecent: styles.SurfaceRecent})
 	if usable(t) {
-		styles.TabActive = EmphasisStyle(rgbSurface(t.Background, t.Foreground), t)
+		// Sessions share the full-strength neutral endpoint with active tabs;
+		// overlay selections retain their softer neutral surface.
+		styles.SurfaceActive = rgbSurface(t.Background, t.Foreground)
+		styles.Accent = styles.SurfaceActive
+		styles.TabActive = EmphasisStyle(styles.SurfaceActive, t)
 		styles.TabNameActive = styles.TabActive
+		styles.TabActiveTitle = MutedVariantStyle(styles.SurfaceActive, t)
+		styles.TabTitleActive = styles.TabActiveTitle
 		// Without a palette accent the history uses the same mruWeight
 		// placement as MRUStyle, between the active tab and the bar.
 		for count := 1; count <= len(styles.mruStyles); count++ {

M internal/usecase/theme/theme_test.go +1 -1


@@ -592,7 +592,7 @@ func TestStyleHelpersFallbackAndThemed(t *testing.T) {
592	592		t.Fatalf("status themed=%+v", status)
593	593		}
594	594		accent := AccentStyle(theme)
595		-	if accent.Inverse \|\| !accent.HasBackgroundRGB \|\| accent.BackgroundRGB != (renderer.RGB{R: 58, G: 65, B: 73}) {
	595	+	if accent.Inverse \|\| !accent.HasBackgroundRGB \|\| accent.BackgroundRGB != theme.Foreground \|\| accent.ForegroundRGB != theme.Background {
596	596		t.Fatalf("accent themed=%+v", accent)
597	597		}
598	598		border := BorderStyle(theme)


@@ -592,7 +592,7 @@ func TestStyleHelpersFallbackAndThemed(t *testing.T) {
592		t.Fatalf("status themed=%+v", status)	592		t.Fatalf("status themed=%+v", status)
593		}	593		}
594		accent := AccentStyle(theme)	594		accent := AccentStyle(theme)
595	-	if accent.Inverse \|\| !accent.HasBackgroundRGB \|\| accent.BackgroundRGB != (renderer.RGB{R: 58, G: 65, B: 73}) {	595	+	if accent.Inverse \|\| !accent.HasBackgroundRGB \|\| accent.BackgroundRGB != theme.Foreground \|\| accent.ForegroundRGB != theme.Background {
596		t.Fatalf("accent themed=%+v", accent)	596		t.Fatalf("accent themed=%+v", accent)
597		}	597		}
598		border := BorderStyle(theme)	598		border := BorderStyle(theme)

M scripts/demo/Dockerfile +1 -1


@@ -7,7 +7,7 @@ RUN --mount=type=cache,target=/var/cache/pacman/pkg,sharing=locked \
7	7		vhs ttyd ffmpeg chromium \
8	8		htop fastfetch gifsicle imagemagick \
9	9		eza fzf neovim man-db man-pages coreutils \
10		-	ncurses openssh util-linux \
	10	+	ncurses openssh util-linux python \
11	11		ttf-jetbrains-mono-nerd
12	12		RUN useradd -m -u 1000 demo
13	13		# P8.2 uses the exact worktree binary built by sandbox-run.sh, never a


@@ -7,7 +7,7 @@ RUN --mount=type=cache,target=/var/cache/pacman/pkg,sharing=locked \
7		vhs ttyd ffmpeg chromium \	7		vhs ttyd ffmpeg chromium \
8		htop fastfetch gifsicle imagemagick \	8		htop fastfetch gifsicle imagemagick \
9		eza fzf neovim man-db man-pages coreutils \	9		eza fzf neovim man-db man-pages coreutils \
10	-	ncurses openssh util-linux \	10	+	ncurses openssh util-linux python \
11		ttf-jetbrains-mono-nerd	11		ttf-jetbrains-mono-nerd
12		RUN useradd -m -u 1000 demo	12		RUN useradd -m -u 1000 demo
13		# P8.2 uses the exact worktree binary built by sandbox-run.sh, never a	13		# P8.2 uses the exact worktree binary built by sandbox-run.sh, never a

A scripts/remote-resilience-harness/README.md +25 -0

@@ -0,0 +1,25 @@
	1	+	# Visual remote resilience acceptance
	2	+
	3	+	`make remote-resilience-visual` runs a disposable remote session in a real foot terminal under NeferWL headless. Physical keys enter through the compositor. PNG differences and passive vev captures jointly verify local picker response within two seconds. A stopped client is the negative control.
	4	+
	5	+	Prerequisites: Go 1.27, rootless Docker, Python 3, ssh-keygen, foot, and a GPU supported by NeferWL. Set `NEFERWL_SOURCE` to the compositor checkout or `NEFERWL_BIN` to its built binary. No desktop Wayland socket, user credentials or host network configuration is shared with containers.
	6	+
	7	+	```sh
	8	+	make remote-resilience-visual
	9	+	VEV_RESILIENCE_TRANSPORT=ssh make remote-resilience-visual
	10	+	VEV_RESILIENCE_DEGRADED=1 make remote-resilience-visual
	11	+	VEV_RESILIENCE_TRANSPORT=ssh VEV_RESILIENCE_DEGRADED=1 make remote-resilience-visual
	12	+	VEV_RESILIENCE_LINK=udp make remote-resilience-visual
	13	+	VEV_RESILIENCE_LINK=tcp make remote-resilience-visual
	14	+	VEV_RESILIENCE_LONG=1 make remote-resilience-visual
	15	+	VEV_RESILIENCE_LONG=1 VEV_RESILIENCE_RESUME=1 make remote-resilience-visual
	16	+	VEV_RESILIENCE_LONG=1 VEV_RESILIENCE_CANCEL=Escape make remote-resilience-visual
	17	+	```
	18	+
	19	+	QUIC is routed through the existing bounded seeded UDP simulator. A fixture-only SSH wrapper preserves authentication fields and rewrites the dynamic bootstrap port to its relay. SSH runs through a TCP relay. The degraded profile applies 150 ms delay, up to 50 ms jitter and 5% datagram loss to UDP; TCP waits 150 ms per forwarded read and is paced to 64 KiB/s, without dropping stream bytes; many small reads therefore add up to more than 150 ms. Both transports can be blacked out together without privileges.
	20	+
	21	+	The short scenario opens the client picker while connected, edits it during blackout, restores the route and checks the named session identity and recovery output. Opening the daemon-rendered palette during blackout is not a local responsiveness guarantee. The optional long scenario asserts actual Connecting state and Ctrl-C or Esc cancellation. It cannot tell backoff from a blocked stream open; fake-clock client tests cover cancellation during stream opening. It fails if cancellation cannot reach the resume owner.
	22	+
	23	+	Artifacts are printed at exit: captures, pixel counts, observation contexts, relay counters and fixture logs. Containers, image, network and temporary credentials are removed. Unsupported rendering and failed assertions exit nonzero.
	24	+
	25	+	This is not a mobile-network emulator: TCP impairments model stream backpressure, not packet retransmissions; IP migration, CGNAT and exactly-once replay are not validated. `VEV_RESILIENCE_LINK=udp\|tcp` isolates short asymmetric blackouts. `VEV_RESILIENCE_RESUME=1` restores a long outage and checks two held fixture commands in order. Detach timing is covered by fake-clock client tests, not this visual entrypoint.

@@ -0,0 +1,25 @@
	1	+	# Visual remote resilience acceptance
	2	+
	3	+	`make remote-resilience-visual` runs a disposable remote session in a real foot terminal under NeferWL headless. Physical keys enter through the compositor. PNG differences and passive vev captures jointly verify local picker response within two seconds. A stopped client is the negative control.
	4	+
	5	+	Prerequisites: Go 1.27, rootless Docker, Python 3, ssh-keygen, foot, and a GPU supported by NeferWL. Set `NEFERWL_SOURCE` to the compositor checkout or `NEFERWL_BIN` to its built binary. No desktop Wayland socket, user credentials or host network configuration is shared with containers.
	6	+
	7	+	```sh
	8	+	make remote-resilience-visual
	9	+	VEV_RESILIENCE_TRANSPORT=ssh make remote-resilience-visual
	10	+	VEV_RESILIENCE_DEGRADED=1 make remote-resilience-visual
	11	+	VEV_RESILIENCE_TRANSPORT=ssh VEV_RESILIENCE_DEGRADED=1 make remote-resilience-visual
	12	+	VEV_RESILIENCE_LINK=udp make remote-resilience-visual
	13	+	VEV_RESILIENCE_LINK=tcp make remote-resilience-visual
	14	+	VEV_RESILIENCE_LONG=1 make remote-resilience-visual
	15	+	VEV_RESILIENCE_LONG=1 VEV_RESILIENCE_RESUME=1 make remote-resilience-visual
	16	+	VEV_RESILIENCE_LONG=1 VEV_RESILIENCE_CANCEL=Escape make remote-resilience-visual
	17	+	```
	18	+
	19	+	QUIC is routed through the existing bounded seeded UDP simulator. A fixture-only SSH wrapper preserves authentication fields and rewrites the dynamic bootstrap port to its relay. SSH runs through a TCP relay. The degraded profile applies 150 ms delay, up to 50 ms jitter and 5% datagram loss to UDP; TCP waits 150 ms per forwarded read and is paced to 64 KiB/s, without dropping stream bytes; many small reads therefore add up to more than 150 ms. Both transports can be blacked out together without privileges.
	20	+
	21	+	The short scenario opens the client picker while connected, edits it during blackout, restores the route and checks the named session identity and recovery output. Opening the daemon-rendered palette during blackout is not a local responsiveness guarantee. The optional long scenario asserts actual Connecting state and Ctrl-C or Esc cancellation. It cannot tell backoff from a blocked stream open; fake-clock client tests cover cancellation during stream opening. It fails if cancellation cannot reach the resume owner.
	22	+
	23	+	Artifacts are printed at exit: captures, pixel counts, observation contexts, relay counters and fixture logs. Containers, image, network and temporary credentials are removed. Unsupported rendering and failed assertions exit nonzero.
	24	+
	25	+	This is not a mobile-network emulator: TCP impairments model stream backpressure, not packet retransmissions; IP migration, CGNAT and exactly-once replay are not validated. `VEV_RESILIENCE_LINK=udp\|tcp` isolates short asymmetric blackouts. `VEV_RESILIENCE_RESUME=1` restores a long outage and checks two held fixture commands in order. Detach timing is covered by fake-clock client tests, not this visual entrypoint.

A scripts/remote-resilience-harness/pixels/main.go +77 -0

@@ -0,0 +1,77 @@
	1	+	// Command pixels compares the fixed 1280x720 fixture's terminal content.
	2	+	package main
	3	+
	4	+	import (
	5	+	"encoding/json"
	6	+	"errors"
	7	+	"fmt"
	8	+	"image"
	9	+	_ "image/png"
	10	+	"os"
	11	+	)
	12	+
	13	+	const fixtureMargin = 100
	14	+	const minimumChangedPixels = 20
	15	+
	16	+	func load(path string) (image.Image, error) {
	17	+	f, err := os.Open(path)
	18	+	if err != nil {
	19	+	return nil, err
	20	+	}
	21	+	defer f.Close()
	22	+	im, _, err := image.Decode(f)
	23	+	return im, err
	24	+	}
	25	+
	26	+	func compare(a, b image.Image) (int, error) {
	27	+	if a.Bounds() != b.Bounds() {
	28	+	return 0, errors.New("capture dimensions changed")
	29	+	}
	30	+	bounds := a.Bounds()
	31	+	if bounds.Dx() <= 2fixtureMargin \|\| bounds.Dy() <= 2fixtureMargin {
	32	+	return 0, errors.New("capture has no comparison region")
	33	+	}
	34	+	changed := 0
	35	+	for y := bounds.Min.Y + fixtureMargin; y < bounds.Max.Y-fixtureMargin; y++ {
	36	+	for x := bounds.Min.X + fixtureMargin; x < bounds.Max.X-fixtureMargin; x++ {
	37	+	ar, ag, ab, _ := a.At(x, y).RGBA()
	38	+	br, bg, bb, _ := b.At(x, y).RGBA()
	39	+	if ar != br \|\| ag != bg \|\| ab != bb {
	40	+	changed++
	41	+	}
	42	+	}
	43	+	}
	44	+	return changed, nil
	45	+	}
	46	+
	47	+	func execute(args []string) (int, error) {
	48	+	if len(args) != 2 {
	49	+	return 2, errors.New("usage: pixels before.png after.png")
	50	+	}
	51	+	a, err := load(args[0])
	52	+	if err != nil {
	53	+	return 2, err
	54	+	}
	55	+	b, err := load(args[1])
	56	+	if err != nil {
	57	+	return 2, err
	58	+	}
	59	+	changed, err := compare(a, b)
	60	+	if err != nil {
	61	+	return 2, err
	62	+	}
	63	+	if err := json.NewEncoder(os.Stdout).Encode(map[string]int{"changed_pixels": changed}); err != nil {
	64	+	return 2, err
	65	+	}
	66	+	if changed < minimumChangedPixels {
	67	+	return 1, nil
	68	+	}
	69	+	return 0, nil
	70	+	}
	71	+	func main() {
	72	+	code, err := execute(os.Args[1:])
	73	+	if err != nil {
	74	+	fmt.Fprintln(os.Stderr, err)
	75	+	}
	76	+	os.Exit(code)
	77	+	}

@@ -0,0 +1,77 @@
	1	+	// Command pixels compares the fixed 1280x720 fixture's terminal content.
	2	+	package main
	3	+
	4	+	import (
	5	+	"encoding/json"
	6	+	"errors"
	7	+	"fmt"
	8	+	"image"
	9	+	_ "image/png"
	10	+	"os"
	11	+	)
	12	+
	13	+	const fixtureMargin = 100
	14	+	const minimumChangedPixels = 20
	15	+
	16	+	func load(path string) (image.Image, error) {
	17	+	f, err := os.Open(path)
	18	+	if err != nil {
	19	+	return nil, err
	20	+	}
	21	+	defer f.Close()
	22	+	im, _, err := image.Decode(f)
	23	+	return im, err
	24	+	}
	25	+
	26	+	func compare(a, b image.Image) (int, error) {
	27	+	if a.Bounds() != b.Bounds() {
	28	+	return 0, errors.New("capture dimensions changed")
	29	+	}
	30	+	bounds := a.Bounds()
	31	+	if bounds.Dx() <= 2fixtureMargin \|\| bounds.Dy() <= 2fixtureMargin {
	32	+	return 0, errors.New("capture has no comparison region")
	33	+	}
	34	+	changed := 0
	35	+	for y := bounds.Min.Y + fixtureMargin; y < bounds.Max.Y-fixtureMargin; y++ {
	36	+	for x := bounds.Min.X + fixtureMargin; x < bounds.Max.X-fixtureMargin; x++ {
	37	+	ar, ag, ab, _ := a.At(x, y).RGBA()
	38	+	br, bg, bb, _ := b.At(x, y).RGBA()
	39	+	if ar != br \|\| ag != bg \|\| ab != bb {
	40	+	changed++
	41	+	}
	42	+	}
	43	+	}
	44	+	return changed, nil
	45	+	}
	46	+
	47	+	func execute(args []string) (int, error) {
	48	+	if len(args) != 2 {
	49	+	return 2, errors.New("usage: pixels before.png after.png")
	50	+	}
	51	+	a, err := load(args[0])
	52	+	if err != nil {
	53	+	return 2, err
	54	+	}
	55	+	b, err := load(args[1])
	56	+	if err != nil {
	57	+	return 2, err
	58	+	}
	59	+	changed, err := compare(a, b)
	60	+	if err != nil {
	61	+	return 2, err
	62	+	}
	63	+	if err := json.NewEncoder(os.Stdout).Encode(map[string]int{"changed_pixels": changed}); err != nil {
	64	+	return 2, err
	65	+	}
	66	+	if changed < minimumChangedPixels {
	67	+	return 1, nil
	68	+	}
	69	+	return 0, nil
	70	+	}
	71	+	func main() {
	72	+	code, err := execute(os.Args[1:])
	73	+	if err != nil {
	74	+	fmt.Fprintln(os.Stderr, err)
	75	+	}
	76	+	os.Exit(code)
	77	+	}

A scripts/remote-resilience-harness/pixels/main_test.go +28 -0

@@ -0,0 +1,28 @@
	1	+	package main
	2	+
	3	+	import (
	4	+	"image"
	5	+	"image/color"
	6	+	"testing"
	7	+
	8	+	"github.com/stretchr/testify/require"
	9	+	)
	10	+
	11	+	func TestCompare(t *testing.T) {
	12	+	for _, n := range []int{0, 19, 20} {
	13	+	a := image.NewRGBA(image.Rect(0, 0, 300, 300))
	14	+	b := image.NewRGBA(a.Bounds())
	15	+	for x := 0; x < n; x++ {
	16	+	b.Set(100+x, 100, color.White)
	17	+	}
	18	+	got, err := compare(a, b)
	19	+	require.NoError(t, err)
	20	+	require.Equal(t, n, got)
	21	+	}
	22	+	for _, bounds := range []image.Rectangle{image.Rect(0, 0, 200, 300), image.Rect(0, 0, 300, 200)} {
	23	+	_, err := compare(image.NewRGBA(bounds), image.NewRGBA(bounds))
	24	+	require.Error(t, err)
	25	+	}
	26	+	_, err := compare(image.NewRGBA(image.Rect(0, 0, 300, 300)), image.NewRGBA(image.Rect(0, 0, 400, 300)))
	27	+	require.Error(t, err)
	28	+	}

@@ -0,0 +1,28 @@
	1	+	package main
	2	+
	3	+	import (
	4	+	"image"
	5	+	"image/color"
	6	+	"testing"
	7	+
	8	+	"github.com/stretchr/testify/require"
	9	+	)
	10	+
	11	+	func TestCompare(t *testing.T) {
	12	+	for _, n := range []int{0, 19, 20} {
	13	+	a := image.NewRGBA(image.Rect(0, 0, 300, 300))
	14	+	b := image.NewRGBA(a.Bounds())
	15	+	for x := 0; x < n; x++ {
	16	+	b.Set(100+x, 100, color.White)
	17	+	}
	18	+	got, err := compare(a, b)
	19	+	require.NoError(t, err)
	20	+	require.Equal(t, n, got)
	21	+	}
	22	+	for _, bounds := range []image.Rectangle{image.Rect(0, 0, 200, 300), image.Rect(0, 0, 300, 200)} {
	23	+	_, err := compare(image.NewRGBA(bounds), image.NewRGBA(bounds))
	24	+	require.Error(t, err)
	25	+	}
	26	+	_, err := compare(image.NewRGBA(image.Rect(0, 0, 300, 300)), image.NewRGBA(image.Rect(0, 0, 400, 300)))
	27	+	require.Error(t, err)
	28	+	}

A scripts/remote-resilience-harness/relay/main.go +289 -0

@@ -0,0 +1,289 @@
	1	+	// Command relay is disposable acceptance infrastructure, never a production proxy.
	2	+	package main
	3	+
	4	+	import (
	5	+	"bufio"
	6	+	"bytes"
	7	+	"context"
	8	+	"encoding/json"
	9	+	"errors"
	10	+	"flag"
	11	+	"fmt"
	12	+	"io"
	13	+	"net"
	14	+	"os"
	15	+	"os/signal"
	16	+	"path/filepath"
	17	+	"sync"
	18	+	"sync/atomic"
	19	+	"syscall"
	20	+	"time"
	21	+
	22	+	"github.com/bnema/vev/internal/adapters/quicnettest"
	23	+	)
	24	+
	25	+	type command struct {
	26	+	ID uint64 `json:"id"`
	27	+	Op string `json:"op"`
	28	+	Port int `json:"port,omitempty"`
	29	+	UDP bool `json:"udp,omitempty"`
	30	+	TCP bool `json:"tcp,omitempty"`
	31	+	Active bool `json:"active,omitempty"`
	32	+	}
	33	+
	34	+	type relay struct {
	35	+	ctx context.Context
	36	+	cancel context.CancelFunc
	37	+	remote string
	38	+	degraded bool
	39	+	blackout atomic.Bool
	40	+	bytes atomic.Uint64
	41	+	mu sync.Mutex
	42	+	connections map[net.Conn]struct{}
	43	+	proxies []*quicnettest.Proxy
	44	+	udpBlackout bool
	45	+	wg sync.WaitGroup
	46	+	}
	47	+
	48	+	func main() {
	49	+	remote := flag.String("remote", "remote", "fixture upstream host")
	50	+	control := flag.String("control", "", "required private Unix control path")
	51	+	tcp := flag.String("tcp", ":2222", "SSH listener")
	52	+	degraded := flag.Bool("degraded", false, "enable seeded degraded profile")
	53	+	flag.Parse()
	54	+	if *control == "" {
	55	+	fmt.Fprintln(os.Stderr, "-control is required")
	56	+	os.Exit(2)
	57	+	}
	58	+	ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
	59	+	r := &relay{ctx: ctx, cancel: cancel, remote: remote, degraded: degraded, connections: make(map[net.Conn]struct{})}
	60	+	if err := r.serve(tcp, control); err != nil {
	61	+	fmt.Fprintln(os.Stderr, err)
	62	+	os.Exit(1)
	63	+	}
	64	+	}
	65	+
	66	+	func (r *relay) serve(addr, path string) error {
	67	+	listener, err := net.Listen("tcp", addr)
	68	+	if err != nil {
	69	+	return err
	70	+	}
	71	+	defer listener.Close()
	72	+	parent, err := os.Stat(filepath.Dir(path))
	73	+	if err != nil {
	74	+	return err
	75	+	}
	76	+	if !parent.IsDir() \|\| parent.Mode().Perm() != 0700 {
	77	+	return errors.New("control socket requires a private 0700 directory")
	78	+	}
	79	+	owner, ok := parent.Sys().(*syscall.Stat_t)
	80	+	if !ok \|\| owner.Uid != uint32(os.Getuid()) {
	81	+	return errors.New("control directory must belong to the current user")
	82	+	}
	83	+	control, err := net.Listen("unix", path)
	84	+	if err != nil {
	85	+	return err
	86	+	}
	87	+	defer control.Close()
	88	+	watchDone := make(chan struct{})
	89	+	defer close(watchDone)
	90	+	go func() {
	91	+	select {
	92	+	case <-r.ctx.Done():
	93	+	listener.Close()
	94	+	control.Close()
	95	+	case <-watchDone:
	96	+	}
	97	+	}()
	98	+	defer os.Remove(path)
	99	+	if err := os.Chmod(path, 0600); err != nil {
	100	+	return err
	101	+	}
	102	+	defer func() { listener.Close(); r.close() }()
	103	+	r.wg.Add(1)
	104	+	go func() {
	105	+	defer r.wg.Done()
	106	+	for {
	107	+	conn, err := listener.Accept()
	108	+	if err != nil {
	109	+	return
	110	+	}
	111	+	r.mu.Lock()
	112	+	if len(r.connections) >= 64 {
	113	+	r.mu.Unlock()
	114	+	conn.Close()
	115	+	continue
	116	+	}
	117	+	r.connections[conn] = struct{}{}
	118	+	r.mu.Unlock()
	119	+	r.wg.Add(1)
	120	+	go func() { defer r.wg.Done(); r.forward(conn) }()
	121	+	}
	122	+	}()
	123	+	for {
	124	+	conn, err := control.Accept()
	125	+	if err != nil {
	126	+	if r.ctx.Err() != nil {
	127	+	return nil
	128	+	}
	129	+	return err
	130	+	}
	131	+	// Commands are bounded and handled serially; a stalled caller gets 2s.
	132	+	conn.SetDeadline(time.Now().Add(2 * time.Second))
	133	+	scanner := bufio.NewScanner(conn)
	134	+	scanner.Buffer(make([]byte, 4096), 4096)
	135	+	if scanner.Scan() {
	136	+	var c command
	137	+	err := json.Unmarshal(scanner.Bytes(), &c)
	138	+	var result any
	139	+	if err == nil {
	140	+	result, err = r.apply(c)
	141	+	}
	142	+	reply := map[string]any{"id": c.ID, "result": result}
	143	+	if err != nil {
	144	+	reply["error"] = err.Error()
	145	+	}
	146	+	json.NewEncoder(conn).Encode(reply)
	147	+	}
	148	+	conn.Close()
	149	+	if r.ctx.Err() != nil {
	150	+	listener.Close()
	151	+	return nil
	152	+	}
	153	+	}
	154	+	}
	155	+
	156	+	func (r *relay) apply(c command) (any, error) {
	157	+	switch c.Op {
	158	+	case "udp":
	159	+	if c.Port <= 0 \|\| c.Port > 65535 {
	160	+	return nil, errors.New("invalid port")
	161	+	}
	162	+	upstream, err := net.ResolveUDPAddr("udp", net.JoinHostPort(r.remote, fmt.Sprint(c.Port)))
	163	+	if err != nil {
	164	+	return nil, err
	165	+	}
	166	+	link := quicnettest.LinkConfig{}
	167	+	if r.degraded {
	168	+	link = quicnettest.LinkConfig{BaseLatency: 150 * time.Millisecond, Jitter: 50 * time.Millisecond, LossPercent: 5}
	169	+	}
	170	+	r.mu.Lock()
	171	+	defer r.mu.Unlock()
	172	+	if len(r.proxies) >= 32 {
	173	+	return nil, errors.New("UDP listener limit")
	174	+	}
	175	+	p, err := quicnettest.New(quicnettest.Config{ServerAddr: upstream, ListenAddr: &net.UDPAddr{IP: net.IPv4zero}, Seed: 317, ToServer: link, ToClient: link})
	176	+	if err != nil {
	177	+	return nil, err
	178	+	}
	179	+	p.SetBlackout(r.udpBlackout)
	180	+	r.proxies = append(r.proxies, p)
	181	+	return p.Addr().Port, nil
	182	+	case "blackout":
	183	+	if c.TCP {
	184	+	r.blackout.Store(c.Active)
	185	+	}
	186	+	if c.UDP {
	187	+	r.mu.Lock()
	188	+	r.udpBlackout = c.Active
	189	+	for _, p := range r.proxies {
	190	+	p.SetBlackout(c.Active)
	191	+	}
	192	+	r.mu.Unlock()
	193	+	}
	194	+	return nil, nil
	195	+	case "disconnect":
	196	+	r.mu.Lock()
	197	+	for conn := range r.connections {
	198	+	conn.Close()
	199	+	}
	200	+	r.mu.Unlock()
	201	+	return nil, nil
	202	+	case "stats":
	203	+	r.mu.Lock()
	204	+	defer r.mu.Unlock()
	205	+	stats := make([]quicnettest.Stats, 0, len(r.proxies))
	206	+	for _, p := range r.proxies {
	207	+	stats = append(stats, p.Stats())
	208	+	}
	209	+	return map[string]any{"udp": stats, "tcp_bytes": r.bytes.Load()}, nil
	210	+	case "stop":
	211	+	r.cancel()
	212	+	return nil, nil
	213	+	default:
	214	+	return nil, errors.New("unknown command")
	215	+	}
	216	+	}
	217	+
	218	+	func (r *relay) forward(client net.Conn) {
	219	+	defer func() { client.Close(); r.mu.Lock(); delete(r.connections, client); r.mu.Unlock() }()
	220	+	server, err := (&net.Dialer{Timeout: 5 * time.Second}).DialContext(r.ctx, "tcp", net.JoinHostPort(r.remote, "22"))
	221	+	if err != nil {
	222	+	return
	223	+	}
	224	+	defer server.Close()
	225	+	r.mu.Lock()
	226	+	r.connections[server] = struct{}{}
	227	+	r.mu.Unlock()
	228	+	defer func() { r.mu.Lock(); delete(r.connections, server); r.mu.Unlock() }()
	229	+	done := make(chan struct{})
	230	+	go func() { r.copy(server, client); server.Close(); client.Close(); close(done) }()
	231	+	r.copy(client, server)
	232	+	server.Close()
	233	+	client.Close()
	234	+	<-done
	235	+	}
	236	+
	237	+	func (r *relay) wait(delay time.Duration) bool {
	238	+	timer := time.NewTimer(delay)
	239	+	defer timer.Stop()
	240	+	select {
	241	+	case <-timer.C:
	242	+	return true
	243	+	case <-r.ctx.Done():
	244	+	return false
	245	+	}
	246	+	}
	247	+
	248	+	func (r *relay) copy(dst, src net.Conn) {
	249	+	buf := make([]byte, 32<<10)
	250	+	for {
	251	+	for r.blackout.Load() {
	252	+	if !r.wait(10 * time.Millisecond) {
	253	+	return
	254	+	}
	255	+	}
	256	+	n, err := src.Read(buf)
	257	+	if n > 0 {
	258	+	for r.blackout.Load() {
	259	+	if !r.wait(10 * time.Millisecond) {
	260	+	return
	261	+	}
	262	+	}
	263	+	if r.degraded && !r.wait(150time.Millisecond+time.Duration(n)time.Second/(64<<10)) {
	264	+	return
	265	+	}
	266	+	written, writeErr := io.Copy(dst, bytes.NewReader(buf[:n]))
	267	+	r.bytes.Add(uint64(written))
	268	+	if writeErr != nil {
	269	+	return
	270	+	}
	271	+	}
	272	+	if err != nil {
	273	+	return
	274	+	}
	275	+	}
	276	+	}
	277	+
	278	+	func (r *relay) close() {
	279	+	r.cancel()
	280	+	r.mu.Lock()
	281	+	for conn := range r.connections {
	282	+	conn.Close()
	283	+	}
	284	+	for _, p := range r.proxies {
	285	+	p.Close()
	286	+	}
	287	+	r.mu.Unlock()
	288	+	r.wg.Wait()
	289	+	}

@@ -0,0 +1,289 @@
	1	+	// Command relay is disposable acceptance infrastructure, never a production proxy.
	2	+	package main
	3	+
	4	+	import (
	5	+	"bufio"
	6	+	"bytes"
	7	+	"context"
	8	+	"encoding/json"
	9	+	"errors"
	10	+	"flag"
	11	+	"fmt"
	12	+	"io"
	13	+	"net"
	14	+	"os"
	15	+	"os/signal"
	16	+	"path/filepath"
	17	+	"sync"
	18	+	"sync/atomic"
	19	+	"syscall"
	20	+	"time"
	21	+
	22	+	"github.com/bnema/vev/internal/adapters/quicnettest"
	23	+	)
	24	+
	25	+	type command struct {
	26	+	ID uint64 `json:"id"`
	27	+	Op string `json:"op"`
	28	+	Port int `json:"port,omitempty"`
	29	+	UDP bool `json:"udp,omitempty"`
	30	+	TCP bool `json:"tcp,omitempty"`
	31	+	Active bool `json:"active,omitempty"`
	32	+	}
	33	+
	34	+	type relay struct {
	35	+	ctx context.Context
	36	+	cancel context.CancelFunc
	37	+	remote string
	38	+	degraded bool
	39	+	blackout atomic.Bool
	40	+	bytes atomic.Uint64
	41	+	mu sync.Mutex
	42	+	connections map[net.Conn]struct{}
	43	+	proxies []*quicnettest.Proxy
	44	+	udpBlackout bool
	45	+	wg sync.WaitGroup
	46	+	}
	47	+
	48	+	func main() {
	49	+	remote := flag.String("remote", "remote", "fixture upstream host")
	50	+	control := flag.String("control", "", "required private Unix control path")
	51	+	tcp := flag.String("tcp", ":2222", "SSH listener")
	52	+	degraded := flag.Bool("degraded", false, "enable seeded degraded profile")
	53	+	flag.Parse()
	54	+	if *control == "" {
	55	+	fmt.Fprintln(os.Stderr, "-control is required")
	56	+	os.Exit(2)
	57	+	}
	58	+	ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
	59	+	r := &relay{ctx: ctx, cancel: cancel, remote: remote, degraded: degraded, connections: make(map[net.Conn]struct{})}
	60	+	if err := r.serve(tcp, control); err != nil {
	61	+	fmt.Fprintln(os.Stderr, err)
	62	+	os.Exit(1)
	63	+	}
	64	+	}
	65	+
	66	+	func (r *relay) serve(addr, path string) error {
	67	+	listener, err := net.Listen("tcp", addr)
	68	+	if err != nil {
	69	+	return err
	70	+	}
	71	+	defer listener.Close()
	72	+	parent, err := os.Stat(filepath.Dir(path))
	73	+	if err != nil {
	74	+	return err
	75	+	}
	76	+	if !parent.IsDir() \|\| parent.Mode().Perm() != 0700 {
	77	+	return errors.New("control socket requires a private 0700 directory")
	78	+	}
	79	+	owner, ok := parent.Sys().(*syscall.Stat_t)
	80	+	if !ok \|\| owner.Uid != uint32(os.Getuid()) {
	81	+	return errors.New("control directory must belong to the current user")
	82	+	}
	83	+	control, err := net.Listen("unix", path)
	84	+	if err != nil {
	85	+	return err
	86	+	}
	87	+	defer control.Close()
	88	+	watchDone := make(chan struct{})
	89	+	defer close(watchDone)
	90	+	go func() {
	91	+	select {
	92	+	case <-r.ctx.Done():
	93	+	listener.Close()
	94	+	control.Close()
	95	+	case <-watchDone:
	96	+	}
	97	+	}()
	98	+	defer os.Remove(path)
	99	+	if err := os.Chmod(path, 0600); err != nil {
	100	+	return err
	101	+	}
	102	+	defer func() { listener.Close(); r.close() }()
	103	+	r.wg.Add(1)
	104	+	go func() {
	105	+	defer r.wg.Done()
	106	+	for {
	107	+	conn, err := listener.Accept()
	108	+	if err != nil {
	109	+	return
	110	+	}
	111	+	r.mu.Lock()
	112	+	if len(r.connections) >= 64 {
	113	+	r.mu.Unlock()
	114	+	conn.Close()
	115	+	continue
	116	+	}
	117	+	r.connections[conn] = struct{}{}
	118	+	r.mu.Unlock()
	119	+	r.wg.Add(1)
	120	+	go func() { defer r.wg.Done(); r.forward(conn) }()
	121	+	}
	122	+	}()
	123	+	for {
	124	+	conn, err := control.Accept()
	125	+	if err != nil {
	126	+	if r.ctx.Err() != nil {
	127	+	return nil
	128	+	}
	129	+	return err
	130	+	}
	131	+	// Commands are bounded and handled serially; a stalled caller gets 2s.
	132	+	conn.SetDeadline(time.Now().Add(2 * time.Second))
	133	+	scanner := bufio.NewScanner(conn)
	134	+	scanner.Buffer(make([]byte, 4096), 4096)
	135	+	if scanner.Scan() {
	136	+	var c command
	137	+	err := json.Unmarshal(scanner.Bytes(), &c)
	138	+	var result any
	139	+	if err == nil {
	140	+	result, err = r.apply(c)
	141	+	}
	142	+	reply := map[string]any{"id": c.ID, "result": result}
	143	+	if err != nil {
	144	+	reply["error"] = err.Error()
	145	+	}
	146	+	json.NewEncoder(conn).Encode(reply)
	147	+	}
	148	+	conn.Close()
	149	+	if r.ctx.Err() != nil {
	150	+	listener.Close()
	151	+	return nil
	152	+	}
	153	+	}
	154	+	}
	155	+
	156	+	func (r *relay) apply(c command) (any, error) {
	157	+	switch c.Op {
	158	+	case "udp":
	159	+	if c.Port <= 0 \|\| c.Port > 65535 {
	160	+	return nil, errors.New("invalid port")
	161	+	}
	162	+	upstream, err := net.ResolveUDPAddr("udp", net.JoinHostPort(r.remote, fmt.Sprint(c.Port)))
	163	+	if err != nil {
	164	+	return nil, err
	165	+	}
	166	+	link := quicnettest.LinkConfig{}
	167	+	if r.degraded {
	168	+	link = quicnettest.LinkConfig{BaseLatency: 150 * time.Millisecond, Jitter: 50 * time.Millisecond, LossPercent: 5}
	169	+	}
	170	+	r.mu.Lock()
	171	+	defer r.mu.Unlock()
	172	+	if len(r.proxies) >= 32 {
	173	+	return nil, errors.New("UDP listener limit")
	174	+	}
	175	+	p, err := quicnettest.New(quicnettest.Config{ServerAddr: upstream, ListenAddr: &net.UDPAddr{IP: net.IPv4zero}, Seed: 317, ToServer: link, ToClient: link})
	176	+	if err != nil {
	177	+	return nil, err
	178	+	}
	179	+	p.SetBlackout(r.udpBlackout)
	180	+	r.proxies = append(r.proxies, p)
	181	+	return p.Addr().Port, nil
	182	+	case "blackout":
	183	+	if c.TCP {
	184	+	r.blackout.Store(c.Active)
	185	+	}
	186	+	if c.UDP {
	187	+	r.mu.Lock()
	188	+	r.udpBlackout = c.Active
	189	+	for _, p := range r.proxies {
	190	+	p.SetBlackout(c.Active)
	191	+	}
	192	+	r.mu.Unlock()
	193	+	}
	194	+	return nil, nil
	195	+	case "disconnect":
	196	+	r.mu.Lock()
	197	+	for conn := range r.connections {
	198	+	conn.Close()
	199	+	}
	200	+	r.mu.Unlock()
	201	+	return nil, nil
	202	+	case "stats":
	203	+	r.mu.Lock()
	204	+	defer r.mu.Unlock()
	205	+	stats := make([]quicnettest.Stats, 0, len(r.proxies))
	206	+	for _, p := range r.proxies {
	207	+	stats = append(stats, p.Stats())
	208	+	}
	209	+	return map[string]any{"udp": stats, "tcp_bytes": r.bytes.Load()}, nil
	210	+	case "stop":
	211	+	r.cancel()
	212	+	return nil, nil
	213	+	default:
	214	+	return nil, errors.New("unknown command")
	215	+	}
	216	+	}
	217	+
	218	+	func (r *relay) forward(client net.Conn) {
	219	+	defer func() { client.Close(); r.mu.Lock(); delete(r.connections, client); r.mu.Unlock() }()
	220	+	server, err := (&net.Dialer{Timeout: 5 * time.Second}).DialContext(r.ctx, "tcp", net.JoinHostPort(r.remote, "22"))
	221	+	if err != nil {
	222	+	return
	223	+	}
	224	+	defer server.Close()
	225	+	r.mu.Lock()
	226	+	r.connections[server] = struct{}{}
	227	+	r.mu.Unlock()
	228	+	defer func() { r.mu.Lock(); delete(r.connections, server); r.mu.Unlock() }()
	229	+	done := make(chan struct{})
	230	+	go func() { r.copy(server, client); server.Close(); client.Close(); close(done) }()
	231	+	r.copy(client, server)
	232	+	server.Close()
	233	+	client.Close()
	234	+	<-done
	235	+	}
	236	+
	237	+	func (r *relay) wait(delay time.Duration) bool {
	238	+	timer := time.NewTimer(delay)
	239	+	defer timer.Stop()
	240	+	select {
	241	+	case <-timer.C:
	242	+	return true
	243	+	case <-r.ctx.Done():
	244	+	return false
	245	+	}
	246	+	}
	247	+
	248	+	func (r *relay) copy(dst, src net.Conn) {
	249	+	buf := make([]byte, 32<<10)
	250	+	for {
	251	+	for r.blackout.Load() {
	252	+	if !r.wait(10 * time.Millisecond) {
	253	+	return
	254	+	}
	255	+	}
	256	+	n, err := src.Read(buf)
	257	+	if n > 0 {
	258	+	for r.blackout.Load() {
	259	+	if !r.wait(10 * time.Millisecond) {
	260	+	return
	261	+	}
	262	+	}
	263	+	if r.degraded && !r.wait(150time.Millisecond+time.Duration(n)time.Second/(64<<10)) {
	264	+	return
	265	+	}
	266	+	written, writeErr := io.Copy(dst, bytes.NewReader(buf[:n]))
	267	+	r.bytes.Add(uint64(written))
	268	+	if writeErr != nil {
	269	+	return
	270	+	}
	271	+	}
	272	+	if err != nil {
	273	+	return
	274	+	}
	275	+	}
	276	+	}
	277	+
	278	+	func (r *relay) close() {
	279	+	r.cancel()
	280	+	r.mu.Lock()
	281	+	for conn := range r.connections {
	282	+	conn.Close()
	283	+	}
	284	+	for _, p := range r.proxies {
	285	+	p.Close()
	286	+	}
	287	+	r.mu.Unlock()
	288	+	r.wg.Wait()
	289	+	}

A scripts/remote-resilience-harness/relay/main_test.go +200 -0

@@ -0,0 +1,200 @@
	1	+	package main
	2	+
	3	+	import (
	4	+	"bufio"
	5	+	"context"
	6	+	"encoding/json"
	7	+	"io"
	8	+	"net"
	9	+	"os"
	10	+	"path/filepath"
	11	+	"testing"
	12	+	"time"
	13	+
	14	+	"github.com/stretchr/testify/require"
	15	+	)
	16	+
	17	+	func TestTCPRelayPreservesBytesAndStops(t *testing.T) {
	18	+	ctx, cancel := context.WithCancel(t.Context())
	19	+	defer cancel()
	20	+	r := &relay{ctx: ctx, cancel: cancel}
	21	+	source, writer := net.Pipe()
	22	+	reader, destination := net.Pipe()
	23	+	defer source.Close()
	24	+	defer writer.Close()
	25	+	defer reader.Close()
	26	+	defer destination.Close()
	27	+	done := make(chan struct{})
	28	+	go func() { defer close(done); r.copy(destination, source) }()
	29	+	payload := []byte("fixture input in order")
	30	+	go func() { writer.Write(payload); writer.Close() }()
	31	+	reader.SetReadDeadline(time.Now().Add(time.Second))
	32	+	got := make([]byte, len(payload))
	33	+	_, err := io.ReadFull(reader, got)
	34	+	require.NoError(t, err)
	35	+	require.Equal(t, payload, got)
	36	+	select {
	37	+	case <-done:
	38	+	case <-time.After(time.Second):
	39	+	t.Fatal("relay did not stop after EOF")
	40	+	}
	41	+	require.Equal(t, uint64(len(payload)), r.bytes.Load())
	42	+	}
	43	+
	44	+	// controlSocketPath returns a socket path in a short private directory: the
	45	+	// macOS t.TempDir path exceeds the 104-byte AF_UNIX limit.
	46	+	func controlSocketPath(t *testing.T) string {
	47	+	t.Helper()
	48	+	directory, err := os.MkdirTemp("/tmp", "relay")
	49	+	require.NoError(t, err)
	50	+	t.Cleanup(func() { _ = os.RemoveAll(directory) })
	51	+	require.NoError(t, os.Chmod(directory, 0700))
	52	+	return filepath.Join(directory, "control.sock")
	53	+	}
	54	+
	55	+	func TestRelayShutdownClosesListeners(t *testing.T) {
	56	+	ctx, cancel := context.WithCancel(t.Context())
	57	+	r := &relay{ctx: ctx, cancel: cancel, connections: make(map[net.Conn]struct{})}
	58	+	path := controlSocketPath(t)
	59	+	done := make(chan error, 1)
	60	+	go func() { done <- r.serve("127.0.0.1:0", path) }()
	61	+	require.Eventually(t, func() bool { _, err := os.Stat(path); return err == nil }, time.Second, time.Millisecond)
	62	+	cancel()
	63	+	select {
	64	+	case err := <-done:
	65	+	require.NoError(t, err)
	66	+	case <-time.After(time.Second):
	67	+	t.Fatal("relay shutdown hung")
	68	+	}
	69	+	_, err := os.Stat(path)
	70	+	require.True(t, os.IsNotExist(err))
	71	+	}
	72	+
	73	+	func TestTCPRelayBlackoutHoldsBytes(t *testing.T) {
	74	+	ctx, cancel := context.WithCancel(t.Context())
	75	+	defer cancel()
	76	+	r := &relay{ctx: ctx, cancel: cancel}
	77	+	_, err := r.apply(command{Op: "blackout", TCP: true, Active: true})
	78	+	require.NoError(t, err)
	79	+	source, writer := net.Pipe()
	80	+	reader, destination := net.Pipe()
	81	+	defer source.Close()
	82	+	defer writer.Close()
	83	+	defer reader.Close()
	84	+	defer destination.Close()
	85	+	done := make(chan struct{})
	86	+	go func() { defer close(done); r.copy(destination, source) }()
	87	+	payload := []byte("held")
	88	+	wrote := make(chan error, 1)
	89	+	go func() { _, err := writer.Write(payload); wrote <- err }()
	90	+	select {
	91	+	case err := <-wrote:
	92	+	t.Fatalf("blackout read the source: %v", err)
	93	+	case <-time.After(50 * time.Millisecond):
	94	+	}
	95	+	require.Zero(t, r.bytes.Load())
	96	+	_, err = r.apply(command{Op: "blackout", TCP: true, Active: false})
	97	+	require.NoError(t, err)
	98	+	reader.SetReadDeadline(time.Now().Add(time.Second))
	99	+	got := make([]byte, len(payload))
	100	+	_, err = io.ReadFull(reader, got)
	101	+	require.NoError(t, err)
	102	+	require.Equal(t, payload, got)
	103	+	require.NoError(t, <-wrote)
	104	+	cancel()
	105	+	writer.Close()
	106	+	<-done
	107	+	}
	108	+
	109	+	// TestRelayUDPProxyInheritsActiveBlackout opens a UDP route while a blackout
	110	+	// is active and proves it drops datagrams until the blackout ends.
	111	+	func TestRelayUDPProxyInheritsActiveBlackout(t *testing.T) {
	112	+	ctx, cancel := context.WithCancel(t.Context())
	113	+	defer cancel()
	114	+	r := &relay{ctx: ctx, cancel: cancel, remote: "127.0.0.1"}
	115	+	defer r.close()
	116	+	upstream, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)})
	117	+	require.NoError(t, err)
	118	+	defer upstream.Close()
	119	+	_, err = r.apply(command{Op: "blackout", UDP: true, Active: true})
	120	+	require.NoError(t, err)
	121	+	port, err := r.apply(command{Op: "udp", Port: upstream.LocalAddr().(*net.UDPAddr).Port})
	122	+	require.NoError(t, err)
	123	+	client, err := net.DialUDP("udp", nil, &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1), Port: port.(int)})
	124	+	require.NoError(t, err)
	125	+	defer client.Close()
	126	+	_, err = client.Write([]byte("blocked"))
	127	+	require.NoError(t, err)
	128	+	require.Eventually(t, func() bool { return r.proxies[0].Stats().ToServer.BlackoutDrops == 1 }, time.Second, time.Millisecond)
	129	+	_, err = r.apply(command{Op: "blackout", UDP: true, Active: false})
	130	+	require.NoError(t, err)
	131	+	_, err = client.Write([]byte("open"))
	132	+	require.NoError(t, err)
	133	+	require.NoError(t, upstream.SetReadDeadline(time.Now().Add(time.Second)))
	134	+	data := make([]byte, 16)
	135	+	n, _, err := upstream.ReadFromUDP(data)
	136	+	require.NoError(t, err)
	137	+	require.Equal(t, "open", string(data[:n]))
	138	+	}
	139	+
	140	+	func TestRelayDisconnectClosesConnections(t *testing.T) {
	141	+	ctx, cancel := context.WithCancel(t.Context())
	142	+	defer cancel()
	143	+	r := &relay{ctx: ctx, cancel: cancel, connections: make(map[net.Conn]struct{})}
	144	+	local, peer := net.Pipe()
	145	+	defer peer.Close()
	146	+	r.connections[local] = struct{}{}
	147	+	_, err := r.apply(command{Op: "disconnect"})
	148	+	require.NoError(t, err)
	149	+	peer.SetReadDeadline(time.Now().Add(time.Second))
	150	+	_, err = peer.Read(make([]byte, 1))
	151	+	require.ErrorIs(t, err, io.EOF)
	152	+	}
	153	+
	154	+	func TestRelayControlRoundTrip(t *testing.T) {
	155	+	ctx, cancel := context.WithCancel(t.Context())
	156	+	defer cancel()
	157	+	r := &relay{ctx: ctx, cancel: cancel, connections: make(map[net.Conn]struct{})}
	158	+	path := controlSocketPath(t)
	159	+	done := make(chan error, 1)
	160	+	go func() { done <- r.serve("127.0.0.1:0", path) }()
	161	+	require.Eventually(t, func() bool { _, err := os.Stat(path); return err == nil }, time.Second, time.Millisecond)
	162	+	send := func(c command) map[string]any {
	163	+	conn, err := net.Dial("unix", path)
	164	+	require.NoError(t, err)
	165	+	defer conn.Close()
	166	+	require.NoError(t, json.NewEncoder(conn).Encode(c))
	167	+	line, err := bufio.NewReader(conn).ReadBytes('\n')
	168	+	require.NoError(t, err)
	169	+	var reply map[string]any
	170	+	require.NoError(t, json.Unmarshal(line, &reply))
	171	+	return reply
	172	+	}
	173	+	reply := send(command{ID: 7, Op: "blackout", TCP: true, Active: true})
	174	+	require.EqualValues(t, 7, reply["id"])
	175	+	require.NotContains(t, reply, "error")
	176	+	require.True(t, r.blackout.Load())
	177	+	reply = send(command{ID: 8, Op: "unknown"})
	178	+	require.Equal(t, "unknown command", reply["error"])
	179	+	reply = send(command{ID: 9, Op: "stop"})
	180	+	require.NotContains(t, reply, "error")
	181	+	select {
	182	+	case err := <-done:
	183	+	require.NoError(t, err)
	184	+	case <-time.After(time.Second):
	185	+	t.Fatal("stop command did not shut the relay down")
	186	+	}
	187	+	}
	188	+
	189	+	func TestRelayRejectsInvalidControl(t *testing.T) {
	190	+	ctx, cancel := context.WithCancel(t.Context())
	191	+	defer cancel()
	192	+	r := &relay{ctx: ctx, cancel: cancel}
	193	+	for _, c := range []command{{Op: "unknown"}, {Op: "udp", Port: 0}, {Op: "udp", Port: 65536}} {
	194	+	_, err := r.apply(c)
	195	+	require.Error(t, err)
	196	+	}
	197	+	_, err := r.apply(command{Op: "stop"})
	198	+	require.NoError(t, err)
	199	+	require.ErrorIs(t, ctx.Err(), context.Canceled)
	200	+	}

@@ -0,0 +1,200 @@
	1	+	package main
	2	+
	3	+	import (
	4	+	"bufio"
	5	+	"context"
	6	+	"encoding/json"
	7	+	"io"
	8	+	"net"
	9	+	"os"
	10	+	"path/filepath"
	11	+	"testing"
	12	+	"time"
	13	+
	14	+	"github.com/stretchr/testify/require"
	15	+	)
	16	+
	17	+	func TestTCPRelayPreservesBytesAndStops(t *testing.T) {
	18	+	ctx, cancel := context.WithCancel(t.Context())
	19	+	defer cancel()
	20	+	r := &relay{ctx: ctx, cancel: cancel}
	21	+	source, writer := net.Pipe()
	22	+	reader, destination := net.Pipe()
	23	+	defer source.Close()
	24	+	defer writer.Close()
	25	+	defer reader.Close()
	26	+	defer destination.Close()
	27	+	done := make(chan struct{})
	28	+	go func() { defer close(done); r.copy(destination, source) }()
	29	+	payload := []byte("fixture input in order")
	30	+	go func() { writer.Write(payload); writer.Close() }()
	31	+	reader.SetReadDeadline(time.Now().Add(time.Second))
	32	+	got := make([]byte, len(payload))
	33	+	_, err := io.ReadFull(reader, got)
	34	+	require.NoError(t, err)
	35	+	require.Equal(t, payload, got)
	36	+	select {
	37	+	case <-done:
	38	+	case <-time.After(time.Second):
	39	+	t.Fatal("relay did not stop after EOF")
	40	+	}
	41	+	require.Equal(t, uint64(len(payload)), r.bytes.Load())
	42	+	}
	43	+
	44	+	// controlSocketPath returns a socket path in a short private directory: the
	45	+	// macOS t.TempDir path exceeds the 104-byte AF_UNIX limit.
	46	+	func controlSocketPath(t *testing.T) string {
	47	+	t.Helper()
	48	+	directory, err := os.MkdirTemp("/tmp", "relay")
	49	+	require.NoError(t, err)
	50	+	t.Cleanup(func() { _ = os.RemoveAll(directory) })
	51	+	require.NoError(t, os.Chmod(directory, 0700))
	52	+	return filepath.Join(directory, "control.sock")
	53	+	}
	54	+
	55	+	func TestRelayShutdownClosesListeners(t *testing.T) {
	56	+	ctx, cancel := context.WithCancel(t.Context())
	57	+	r := &relay{ctx: ctx, cancel: cancel, connections: make(map[net.Conn]struct{})}
	58	+	path := controlSocketPath(t)
	59	+	done := make(chan error, 1)
	60	+	go func() { done <- r.serve("127.0.0.1:0", path) }()
	61	+	require.Eventually(t, func() bool { _, err := os.Stat(path); return err == nil }, time.Second, time.Millisecond)
	62	+	cancel()
	63	+	select {
	64	+	case err := <-done:
	65	+	require.NoError(t, err)
	66	+	case <-time.After(time.Second):
	67	+	t.Fatal("relay shutdown hung")
	68	+	}
	69	+	_, err := os.Stat(path)
	70	+	require.True(t, os.IsNotExist(err))
	71	+	}
	72	+
	73	+	func TestTCPRelayBlackoutHoldsBytes(t *testing.T) {
	74	+	ctx, cancel := context.WithCancel(t.Context())
	75	+	defer cancel()
	76	+	r := &relay{ctx: ctx, cancel: cancel}
	77	+	_, err := r.apply(command{Op: "blackout", TCP: true, Active: true})
	78	+	require.NoError(t, err)
	79	+	source, writer := net.Pipe()
	80	+	reader, destination := net.Pipe()
	81	+	defer source.Close()
	82	+	defer writer.Close()
	83	+	defer reader.Close()
	84	+	defer destination.Close()
	85	+	done := make(chan struct{})
	86	+	go func() { defer close(done); r.copy(destination, source) }()
	87	+	payload := []byte("held")
	88	+	wrote := make(chan error, 1)
	89	+	go func() { _, err := writer.Write(payload); wrote <- err }()
	90	+	select {
	91	+	case err := <-wrote:
	92	+	t.Fatalf("blackout read the source: %v", err)
	93	+	case <-time.After(50 * time.Millisecond):
	94	+	}
	95	+	require.Zero(t, r.bytes.Load())
	96	+	_, err = r.apply(command{Op: "blackout", TCP: true, Active: false})
	97	+	require.NoError(t, err)
	98	+	reader.SetReadDeadline(time.Now().Add(time.Second))
	99	+	got := make([]byte, len(payload))
	100	+	_, err = io.ReadFull(reader, got)
	101	+	require.NoError(t, err)
	102	+	require.Equal(t, payload, got)
	103	+	require.NoError(t, <-wrote)
	104	+	cancel()
	105	+	writer.Close()
	106	+	<-done
	107	+	}
	108	+
	109	+	// TestRelayUDPProxyInheritsActiveBlackout opens a UDP route while a blackout
	110	+	// is active and proves it drops datagrams until the blackout ends.
	111	+	func TestRelayUDPProxyInheritsActiveBlackout(t *testing.T) {
	112	+	ctx, cancel := context.WithCancel(t.Context())
	113	+	defer cancel()
	114	+	r := &relay{ctx: ctx, cancel: cancel, remote: "127.0.0.1"}
	115	+	defer r.close()
	116	+	upstream, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)})
	117	+	require.NoError(t, err)
	118	+	defer upstream.Close()
	119	+	_, err = r.apply(command{Op: "blackout", UDP: true, Active: true})
	120	+	require.NoError(t, err)
	121	+	port, err := r.apply(command{Op: "udp", Port: upstream.LocalAddr().(*net.UDPAddr).Port})
	122	+	require.NoError(t, err)
	123	+	client, err := net.DialUDP("udp", nil, &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1), Port: port.(int)})
	124	+	require.NoError(t, err)
	125	+	defer client.Close()
	126	+	_, err = client.Write([]byte("blocked"))
	127	+	require.NoError(t, err)
	128	+	require.Eventually(t, func() bool { return r.proxies[0].Stats().ToServer.BlackoutDrops == 1 }, time.Second, time.Millisecond)
	129	+	_, err = r.apply(command{Op: "blackout", UDP: true, Active: false})
	130	+	require.NoError(t, err)
	131	+	_, err = client.Write([]byte("open"))
	132	+	require.NoError(t, err)
	133	+	require.NoError(t, upstream.SetReadDeadline(time.Now().Add(time.Second)))
	134	+	data := make([]byte, 16)
	135	+	n, _, err := upstream.ReadFromUDP(data)
	136	+	require.NoError(t, err)
	137	+	require.Equal(t, "open", string(data[:n]))
	138	+	}
	139	+
	140	+	func TestRelayDisconnectClosesConnections(t *testing.T) {
	141	+	ctx, cancel := context.WithCancel(t.Context())
	142	+	defer cancel()
	143	+	r := &relay{ctx: ctx, cancel: cancel, connections: make(map[net.Conn]struct{})}
	144	+	local, peer := net.Pipe()
	145	+	defer peer.Close()
	146	+	r.connections[local] = struct{}{}
	147	+	_, err := r.apply(command{Op: "disconnect"})
	148	+	require.NoError(t, err)
	149	+	peer.SetReadDeadline(time.Now().Add(time.Second))
	150	+	_, err = peer.Read(make([]byte, 1))
	151	+	require.ErrorIs(t, err, io.EOF)
	152	+	}
	153	+
	154	+	func TestRelayControlRoundTrip(t *testing.T) {
	155	+	ctx, cancel := context.WithCancel(t.Context())
	156	+	defer cancel()
	157	+	r := &relay{ctx: ctx, cancel: cancel, connections: make(map[net.Conn]struct{})}
	158	+	path := controlSocketPath(t)
	159	+	done := make(chan error, 1)
	160	+	go func() { done <- r.serve("127.0.0.1:0", path) }()
	161	+	require.Eventually(t, func() bool { _, err := os.Stat(path); return err == nil }, time.Second, time.Millisecond)
	162	+	send := func(c command) map[string]any {
	163	+	conn, err := net.Dial("unix", path)
	164	+	require.NoError(t, err)
	165	+	defer conn.Close()
	166	+	require.NoError(t, json.NewEncoder(conn).Encode(c))
	167	+	line, err := bufio.NewReader(conn).ReadBytes('\n')
	168	+	require.NoError(t, err)
	169	+	var reply map[string]any
	170	+	require.NoError(t, json.Unmarshal(line, &reply))
	171	+	return reply
	172	+	}
	173	+	reply := send(command{ID: 7, Op: "blackout", TCP: true, Active: true})
	174	+	require.EqualValues(t, 7, reply["id"])
	175	+	require.NotContains(t, reply, "error")
	176	+	require.True(t, r.blackout.Load())
	177	+	reply = send(command{ID: 8, Op: "unknown"})
	178	+	require.Equal(t, "unknown command", reply["error"])
	179	+	reply = send(command{ID: 9, Op: "stop"})
	180	+	require.NotContains(t, reply, "error")
	181	+	select {
	182	+	case err := <-done:
	183	+	require.NoError(t, err)
	184	+	case <-time.After(time.Second):
	185	+	t.Fatal("stop command did not shut the relay down")
	186	+	}
	187	+	}
	188	+
	189	+	func TestRelayRejectsInvalidControl(t *testing.T) {
	190	+	ctx, cancel := context.WithCancel(t.Context())
	191	+	defer cancel()
	192	+	r := &relay{ctx: ctx, cancel: cancel}
	193	+	for _, c := range []command{{Op: "unknown"}, {Op: "udp", Port: 0}, {Op: "udp", Port: 65536}} {
	194	+	_, err := r.apply(c)
	195	+	require.Error(t, err)
	196	+	}
	197	+	_, err := r.apply(command{Op: "stop"})
	198	+	require.NoError(t, err)
	199	+	require.ErrorIs(t, ctx.Err(), context.Canceled)
	200	+	}

A scripts/remote-resilience-harness/ssh.py +58 -0

@@ -0,0 +1,58 @@
	1	+	#!/usr/bin/env python3
	2	+	"""Fixture-only SSH wrapper: pin SSH and rewrite only QUIC readiness port."""
	3	+	import json
	4	+	import os
	5	+	import socket
	6	+	import subprocess
	7	+	import sys
	8	+
	9	+
	10	+	def control(request):
	11	+	with socket.socket(socket.AF_UNIX) as conn:
	12	+	conn.settimeout(2)
	13	+	conn.connect(os.environ.get("RESILIENCE_CONTROL", "/home/demo/relay/control.sock"))
	14	+	conn.sendall(json.dumps(request).encode() + b"\n")
	15	+	reply = json.loads(conn.makefile("rb").readline(4096))
	16	+	if "error" in reply:
	17	+	raise RuntimeError(reply["error"])
	18	+	return reply["result"]
	19	+
	20	+
	21	+	def main():
	22	+	args = sys.argv[1:]
	23	+	command = ["/usr/bin/ssh-real", "-p", "2222", "-o", "HostKeyAlias=remote"] + args
	24	+	if not any("_broker-mux-quic-bootstrap" in arg for arg in args):
	25	+	os.execv(command[0], command)
	26	+	process = subprocess.Popen(command, stdin=sys.stdin.buffer, stdout=subprocess.PIPE)
	27	+	try:
	28	+	line = process.stdout.readline(4097)
	29	+	if len(line) > 4096 or not line.endswith(b"\n"):
	30	+	raise RuntimeError("invalid bounded bootstrap readiness")
	31	+	readiness = json.loads(line)
	32	+	port = readiness.get("port")
	33	+	if not isinstance(port, int) or not 0 < port <= 65535:
	34	+	raise RuntimeError("invalid bootstrap port")
	35	+	readiness["port"] = control({"id": 1, "op": "udp", "port": port})
	36	+	sys.stdout.buffer.write(json.dumps(readiness, separators=(",", ":")).encode() + b"\n")
	37	+	sys.stdout.buffer.flush()
	38	+	# No readiness or encrypted payload is recorded in artifacts.
	39	+	while chunk := process.stdout.read1(32768):
	40	+	sys.stdout.buffer.write(chunk)
	41	+	sys.stdout.buffer.flush()
	42	+	return process.wait()
	43	+	finally:
	44	+	if process.poll() is None:
	45	+	process.terminate()
	46	+	try:
	47	+	process.wait(timeout=2)
	48	+	except subprocess.TimeoutExpired:
	49	+	process.kill()
	50	+	process.wait()
	51	+
	52	+
	53	+	if __name__ == "__main__":
	54	+	try:
	55	+	sys.exit(main())
	56	+	except (OSError, ValueError, RuntimeError):
	57	+	print("resilience SSH fixture failed", file=sys.stderr)
	58	+	sys.exit(1)

@@ -0,0 +1,58 @@
	1	+	#!/usr/bin/env python3
	2	+	"""Fixture-only SSH wrapper: pin SSH and rewrite only QUIC readiness port."""
	3	+	import json
	4	+	import os
	5	+	import socket
	6	+	import subprocess
	7	+	import sys
	8	+
	9	+
	10	+	def control(request):
	11	+	with socket.socket(socket.AF_UNIX) as conn:
	12	+	conn.settimeout(2)
	13	+	conn.connect(os.environ.get("RESILIENCE_CONTROL", "/home/demo/relay/control.sock"))
	14	+	conn.sendall(json.dumps(request).encode() + b"\n")
	15	+	reply = json.loads(conn.makefile("rb").readline(4096))
	16	+	if "error" in reply:
	17	+	raise RuntimeError(reply["error"])
	18	+	return reply["result"]
	19	+
	20	+
	21	+	def main():
	22	+	args = sys.argv[1:]
	23	+	command = ["/usr/bin/ssh-real", "-p", "2222", "-o", "HostKeyAlias=remote"] + args
	24	+	if not any("_broker-mux-quic-bootstrap" in arg for arg in args):
	25	+	os.execv(command[0], command)
	26	+	process = subprocess.Popen(command, stdin=sys.stdin.buffer, stdout=subprocess.PIPE)
	27	+	try:
	28	+	line = process.stdout.readline(4097)
	29	+	if len(line) > 4096 or not line.endswith(b"\n"):
	30	+	raise RuntimeError("invalid bounded bootstrap readiness")
	31	+	readiness = json.loads(line)
	32	+	port = readiness.get("port")
	33	+	if not isinstance(port, int) or not 0 < port <= 65535:
	34	+	raise RuntimeError("invalid bootstrap port")
	35	+	readiness["port"] = control({"id": 1, "op": "udp", "port": port})
	36	+	sys.stdout.buffer.write(json.dumps(readiness, separators=(",", ":")).encode() + b"\n")
	37	+	sys.stdout.buffer.flush()
	38	+	# No readiness or encrypted payload is recorded in artifacts.
	39	+	while chunk := process.stdout.read1(32768):
	40	+	sys.stdout.buffer.write(chunk)
	41	+	sys.stdout.buffer.flush()
	42	+	return process.wait()
	43	+	finally:
	44	+	if process.poll() is None:
	45	+	process.terminate()
	46	+	try:
	47	+	process.wait(timeout=2)
	48	+	except subprocess.TimeoutExpired:
	49	+	process.kill()
	50	+	process.wait()
	51	+
	52	+
	53	+	if __name__ == "__main__":
	54	+	try:
	55	+	sys.exit(main())
	56	+	except (OSError, ValueError, RuntimeError):
	57	+	print("resilience SSH fixture failed", file=sys.stderr)
	58	+	sys.exit(1)

A scripts/remote-resilience-harness/ssh_test.py +67 -0

@@ -0,0 +1,67 @@
	1	+	"""Fixture readiness relay tests; no network or production credentials."""
	2	+	import importlib.util
	3	+	from pathlib import Path
	4	+	import unittest
	5	+	from unittest.mock import patch, MagicMock
	6	+
	7	+	spec = importlib.util.spec_from_file_location("fixture_ssh", Path(__file__).with_name("ssh.py"))
	8	+	wrapper = importlib.util.module_from_spec(spec)
	9	+	spec.loader.exec_module(wrapper)
	10	+
	11	+
	12	+	class ControlTests(unittest.TestCase):
	13	+	def test_control_failure_is_not_a_port(self):
	14	+	conn = MagicMock()
	15	+	conn.__enter__.return_value = conn
	16	+	conn.makefile.return_value.readline.return_value = b'{"id":1,"error":"listener limit"}\n'
	17	+	with patch.object(wrapper.socket, "socket", return_value=conn):
	18	+	with self.assertRaises(RuntimeError):
	19	+	wrapper.control({"id": 1, "op": "udp", "port": 1234})
	20	+
	21	+	def test_control_returns_fixture_port(self):
	22	+	conn = MagicMock()
	23	+	conn.__enter__.return_value = conn
	24	+	conn.makefile.return_value.readline.return_value = b'{"id":1,"result":4321}\n'
	25	+	with patch.object(wrapper.socket, "socket", return_value=conn):
	26	+	self.assertEqual(wrapper.control({"id": 1, "op": "udp", "port": 1234}), 4321)
	27	+
	28	+
	29	+	class MainTests(unittest.TestCase):
	30	+	BOOTSTRAP = ["-T", "--", "remote", "'vev' '_broker-mux-quic-bootstrap'"]
	31	+
	32	+	def run_main(self, readiness, rewritten=4321):
	33	+	process = MagicMock()
	34	+	process.stdout.readline.return_value = readiness
	35	+	process.stdout.read1.return_value = b""
	36	+	process.wait.return_value = 0
	37	+	process.poll.return_value = 0
	38	+	out = MagicMock()
	39	+	with patch.object(wrapper.sys, "argv", ["ssh"] + self.BOOTSTRAP), \
	40	+	patch.object(wrapper.subprocess, "Popen", return_value=process), \
	41	+	patch.object(wrapper, "control", return_value=rewritten) as control, \
	42	+	patch.object(wrapper.sys, "stdout", MagicMock(buffer=out)):
	43	+	code = wrapper.main()
	44	+	return code, control, out
	45	+
	46	+	def test_rewrites_only_the_port(self):
	47	+	code, control, out = self.run_main(b'{"port":1234,"fingerprint":"f"}\n')
	48	+	self.assertEqual(code, 0)
	49	+	control.assert_called_once_with({"id": 1, "op": "udp", "port": 1234})
	50	+	self.assertEqual(out.write.call_args_list[0].args[0], b'{"port":4321,"fingerprint":"f"}\n')
	51	+
	52	+	def test_rejects_invalid_readiness(self):
	53	+	for line in (b'{"port":1}', b"x" * 4097 + b"\n", b'{"port":0}\n', b'{"port":65536}\n', b'{"port":"22"}\n'):
	54	+	with self.subTest(line=line[:20]):
	55	+	with self.assertRaises(RuntimeError):
	56	+	self.run_main(line)
	57	+
	58	+	def test_other_commands_exec_ssh_unchanged(self):
	59	+	with patch.object(wrapper.sys, "argv", ["ssh", "remote", "true"]), \
	60	+	patch.object(wrapper.os, "execv", side_effect=SystemExit(0)) as execv:
	61	+	with self.assertRaises(SystemExit):
	62	+	wrapper.main()
	63	+	execv.assert_called_once_with("/usr/bin/ssh-real", ["/usr/bin/ssh-real", "-p", "2222", "-o", "HostKeyAlias=remote", "remote", "true"])
	64	+
	65	+
	66	+	if __name__ == "__main__":
	67	+	unittest.main()

@@ -0,0 +1,67 @@
	1	+	"""Fixture readiness relay tests; no network or production credentials."""
	2	+	import importlib.util
	3	+	from pathlib import Path
	4	+	import unittest
	5	+	from unittest.mock import patch, MagicMock
	6	+
	7	+	spec = importlib.util.spec_from_file_location("fixture_ssh", Path(__file__).with_name("ssh.py"))
	8	+	wrapper = importlib.util.module_from_spec(spec)
	9	+	spec.loader.exec_module(wrapper)
	10	+
	11	+
	12	+	class ControlTests(unittest.TestCase):
	13	+	def test_control_failure_is_not_a_port(self):
	14	+	conn = MagicMock()
	15	+	conn.__enter__.return_value = conn
	16	+	conn.makefile.return_value.readline.return_value = b'{"id":1,"error":"listener limit"}\n'
	17	+	with patch.object(wrapper.socket, "socket", return_value=conn):
	18	+	with self.assertRaises(RuntimeError):
	19	+	wrapper.control({"id": 1, "op": "udp", "port": 1234})
	20	+
	21	+	def test_control_returns_fixture_port(self):
	22	+	conn = MagicMock()
	23	+	conn.__enter__.return_value = conn
	24	+	conn.makefile.return_value.readline.return_value = b'{"id":1,"result":4321}\n'
	25	+	with patch.object(wrapper.socket, "socket", return_value=conn):
	26	+	self.assertEqual(wrapper.control({"id": 1, "op": "udp", "port": 1234}), 4321)
	27	+
	28	+
	29	+	class MainTests(unittest.TestCase):
	30	+	BOOTSTRAP = ["-T", "--", "remote", "'vev' '_broker-mux-quic-bootstrap'"]
	31	+
	32	+	def run_main(self, readiness, rewritten=4321):
	33	+	process = MagicMock()
	34	+	process.stdout.readline.return_value = readiness
	35	+	process.stdout.read1.return_value = b""
	36	+	process.wait.return_value = 0
	37	+	process.poll.return_value = 0
	38	+	out = MagicMock()
	39	+	with patch.object(wrapper.sys, "argv", ["ssh"] + self.BOOTSTRAP), \
	40	+	patch.object(wrapper.subprocess, "Popen", return_value=process), \
	41	+	patch.object(wrapper, "control", return_value=rewritten) as control, \
	42	+	patch.object(wrapper.sys, "stdout", MagicMock(buffer=out)):
	43	+	code = wrapper.main()
	44	+	return code, control, out
	45	+
	46	+	def test_rewrites_only_the_port(self):
	47	+	code, control, out = self.run_main(b'{"port":1234,"fingerprint":"f"}\n')
	48	+	self.assertEqual(code, 0)
	49	+	control.assert_called_once_with({"id": 1, "op": "udp", "port": 1234})
	50	+	self.assertEqual(out.write.call_args_list[0].args[0], b'{"port":4321,"fingerprint":"f"}\n')
	51	+
	52	+	def test_rejects_invalid_readiness(self):
	53	+	for line in (b'{"port":1}', b"x" * 4097 + b"\n", b'{"port":0}\n', b'{"port":65536}\n', b'{"port":"22"}\n'):
	54	+	with self.subTest(line=line[:20]):
	55	+	with self.assertRaises(RuntimeError):
	56	+	self.run_main(line)
	57	+
	58	+	def test_other_commands_exec_ssh_unchanged(self):
	59	+	with patch.object(wrapper.sys, "argv", ["ssh", "remote", "true"]), \
	60	+	patch.object(wrapper.os, "execv", side_effect=SystemExit(0)) as execv:
	61	+	with self.assertRaises(SystemExit):
	62	+	wrapper.main()
	63	+	execv.assert_called_once_with("/usr/bin/ssh-real", ["/usr/bin/ssh-real", "-p", "2222", "-o", "HostKeyAlias=remote", "remote", "true"])
	64	+
	65	+
	66	+	if __name__ == "__main__":
	67	+	unittest.main()

A scripts/remote-resilience-harness/visual.py +369 -0

@@ -0,0 +1,369 @@
	1	+	#!/usr/bin/env python3
	2	+	"""Disposable dual-transport fixture and physical Wayland input acceptance."""
	3	+	import importlib.util
	4	+	import json
	5	+	import os
	6	+	from pathlib import Path
	7	+	import shutil
	8	+	import signal
	9	+	import subprocess
	10	+	import tempfile
	11	+	import time
	12	+	import uuid
	13	+
	14	+	ROOT = Path(__file__).resolve().parents[2]
	15	+
	16	+
	17	+	def run(args, *kwargs):
	18	+	return subprocess.run(args, check=True, timeout=kwargs.pop("timeout", 60), **kwargs)
	19	+
	20	+
	21	+	def docker(args, *kwargs):
	22	+	return run("docker", args, *kwargs)
	23	+
	24	+
	25	+	def main():
	26	+	name = "vev-resilience-" + uuid.uuid4().hex[:10]
	27	+	artifacts = Path(tempfile.mkdtemp(prefix="vev-resilience-artifacts-"))
	28	+	processes = []
	29	+	containers = []
	30	+	network = False
	31	+	def interrupt(signum, frame):
	32	+	raise KeyboardInterrupt("fixture interrupted")
	33	+	signal.signal(signal.SIGTERM, interrupt)
	34	+	with tempfile.TemporaryDirectory(prefix="vev-resilience-private-") as directory:
	35	+	private = Path(directory)
	36	+	runtime = private / "runtime"
	37	+	runtime.mkdir(mode=0o700)
	38	+	try:
	39	+	for tool in ("docker", "foot", "ssh-keygen", "go"):
	40	+	if shutil.which(tool) is None:
	41	+	raise RuntimeError("missing prerequisite: " + tool)
	42	+	run("go", "build", "-o", str(ROOT / "build/vev"), ".", cwd=ROOT, timeout=600)
	43	+	for helper in ("relay", "pixels"):
	44	+	run("go", "build", "-o", str(ROOT / f"build/resilience-{helper}"), f"./scripts/remote-resilience-harness/{helper}", cwd=ROOT, timeout=600)
	45	+	nefer = os.environ.get("NEFERWL_BIN")
	46	+	if not nefer:
	47	+	nefer = str(private / "neferwl")
	48	+	common = run("git", "rev-parse", "--path-format=absolute", "--git-common-dir", cwd=ROOT, capture_output=True, text=True).stdout.strip()
	49	+	source = Path(os.environ.get("NEFERWL_SOURCE", str(Path(common).parent.parent / "neferwl")))
	50	+	run("go", "build", "-o", nefer, "./cmd/neferwl", cwd=source, env={**os.environ, "CGO_ENABLED": "0"}, timeout=600)
	51	+	docker("build", "-f", str(ROOT / "scripts/demo/Dockerfile"), "-t", name, str(ROOT), timeout=600)
	52	+	docker("network", "create", name)
	53	+	network = True
	54	+	for role in ("client", "remote"):
	55	+	container = name + "-" + role
	56	+	docker("run", "-d", "--name", container, "--network", name, "--network-alias", "upstream" if role == "remote" else "remote", "--entrypoint", "sleep", name, "infinity")
	57	+	containers.append(container)
	58	+	client, remote = containers
	59	+	for key in ("client", "host"):
	60	+	run("ssh-keygen", "-q", "-t", "ed25519", "-N", "", "-C", "fixture", "-f", str(private / key))
	61	+	docker("cp", str(private / "host"), remote + ":/etc/ssh/ssh_host_ed25519_key")
	62	+	docker("cp", str(private / "host.pub"), remote + ":/etc/ssh/ssh_host_ed25519_key.pub")
	63	+	docker("cp", str(private / "client.pub"), remote + ":/home/demo/.ssh/authorized_keys")
	64	+	docker("exec", "--user", "root", remote, "sh", "-c", "rm -f /home/demo/.ssh/id_ed25519*; chmod 600 /etc/ssh/ssh_host_ed25519_key /home/demo/.ssh/authorized_keys; chown demo:demo /home/demo/.ssh/authorized_keys; mkdir -p /run/sshd; /usr/sbin/sshd")
	65	+	docker("exec", "--user", "root", client, "sh", "-c", "rm -f /home/demo/.ssh/id_ed25519*")
	66	+	docker("cp", str(private / "client"), client + ":/home/demo/.ssh/id_ed25519")
	67	+	public = (private / "host.pub").read_text().split()
	68	+	(private / "known_hosts").write_text("remote " + " ".join(public[:2]) + "\n")
	69	+	(private / "config").write_text("Host remote\n HostName remote\n Port 2222\n User demo\n IdentityFile ~/.ssh/id_ed25519\n IdentitiesOnly yes\n StrictHostKeyChecking yes\n HostKeyAlias remote\n UserKnownHostsFile ~/.ssh/known_hosts\n")
	70	+	for file in ("known_hosts", "config"):
	71	+	docker("cp", str(private / file), client + ":/home/demo/.ssh/" + file)
	72	+	docker("cp", str(ROOT / "build/resilience-relay"), client + ":/tmp/relay")
	73	+	docker("cp", str(ROOT / "scripts/remote-resilience-harness/ssh.py"), client + ":/tmp/ssh")
	74	+	docker("exec", "--user", "root", client, "sh", "-c", "chown demo:demo /home/demo/.ssh/; chmod 600 /home/demo/.ssh/; chmod 755 /tmp/ssh; install -d -m 700 -o demo -g demo /home/demo/relay; mkdir -p /tmp/bin; ln -s /tmp/ssh /tmp/bin/ssh; mv /usr/bin/ssh /usr/bin/ssh-real; ln -s /tmp/ssh /usr/bin/ssh")
	75	+	transport = os.environ.get("VEV_RESILIENCE_TRANSPORT", "quic")
	76	+	if transport not in ("quic", "ssh"):
	77	+	raise ValueError("VEV_RESILIENCE_TRANSPORT must be quic or ssh")
	78	+	relay_args = ["-degraded"] if os.environ.get("VEV_RESILIENCE_DEGRADED") == "1" else []
	79	+	relay = subprocess.Popen(["docker", "exec", client, "/tmp/relay", "-remote", "upstream", "-control", "/home/demo/relay/control.sock", *relay_args])
	80	+	processes.append(relay)
	81	+	control_id = 0
	82	+	def control(op, **fields):
	83	+	nonlocal control_id
	84	+	control_id += 1
	85	+	request = {"id": control_id, "op": op, **fields}
	86	+	code = "import socket,json; s=socket.socket(socket.AF_UNIX); s.settimeout(3); s.connect('/home/demo/relay/control.sock'); s.sendall(" + repr(json.dumps(request).encode() + b"\n") + "); print(s.makefile().readline())"
	87	+	result = docker("exec", client, "python3", "-c", code, capture_output=True, text=True)
	88	+	reply = json.loads(result.stdout)
	89	+	assert reply["id"] == control_id and "error" not in reply, reply
	90	+	return reply["result"]
	91	+	# Fixture image must explicitly supply Python rather than using host tools.
	92	+	docker("exec", "--user", "root", client, "sh", "-c", "command -v python3", capture_output=True)
	93	+	deadline = time.monotonic() + 5
	94	+	while True:
	95	+	ready = subprocess.run(["docker", "exec", client, "test", "-S", "/home/demo/relay/control.sock"], capture_output=True, timeout=3)
	96	+	if ready.returncode == 0:
	97	+	break
	98	+	if time.monotonic() > deadline:
	99	+	raise RuntimeError("relay control socket did not become ready")
	100	+	time.sleep(0.05)
	101	+	probe = subprocess.run(["docker", "exec", client, "/usr/bin/ssh", "-v", "remote", "true"], capture_output=True, text=True, timeout=10)
	102	+	if probe.returncode:
	103	+	(artifacts / "ssh-probe.log").write_text(probe.stderr)
	104	+	raise RuntimeError("fixture SSH authentication failed")
	105	+	env_args = ["-e", "PATH=/tmp/bin:/usr/local/bin:/usr/bin", "-e", "VEV_LOG=debug", "-e", "SSH_AUTH_SOCK="]
	106	+	spec = importlib.util.spec_from_file_location("acceptance", ROOT / "scripts/remote-picker-harness/acceptance.py")
	107	+	acceptance = importlib.util.module_from_spec(spec)
	108	+	spec.loader.exec_module(acceptance)
	109	+	fixture = acceptance.Driver(remote, ["--session", "resilience"])
	110	+	fixture.close()
	111	+	docker("exec", *env_args, client, "vev", "host", "add", "--transport", transport, "remote")
	112	+	# Wait for the broker publication before resolving direct attach.
	113	+	deadline = time.monotonic() + 30
	114	+	while True:
	115	+	inventory = docker("exec", *env_args, client, "vev", "ls", "remote", capture_output=True, text=True)
	116	+	if "resilience" in inventory.stdout:
	117	+	break
	118	+	(artifacts / "inventory.txt").write_text(inventory.stdout + inventory.stderr)
	119	+	if time.monotonic() > deadline:
	120	+	raise RuntimeError("remote fixture was not published")
	121	+	time.sleep(0.2)
	122	+	shots = artifacts / "shots"
	123	+	shots.mkdir()
	124	+	docker_endpoint = run("docker", "context", "inspect", "--format", "{{.Endpoints.docker.Host}}", capture_output=True, text=True).stdout.strip()
	125	+	env = {**os.environ, "DOCKER_HOST": docker_endpoint, "XDG_RUNTIME_DIR": str(runtime), "XDG_CONFIG_HOME": str(private / "config-home"), "XDG_STATE_HOME": str(private / "state"), "HOME": str(private)}
	126	+	env["TERMINAL"] = "foot --config=/dev/null --override=cursor.blink=no --log-level=info --hold -- docker --host=" + docker_endpoint + " exec -it " + " ".join(env_args) + " " + client + " vev attach demo@remote:resilience --ui-observe --ui-socket /home/demo/visual/ui.sock"
	127	+	log = (artifacts / "compositor.log").open("w")
	128	+	compositor = subprocess.Popen([nefer, "--backend=headless", "--size=1280x720", "--input=-", "--screenshot", str(shots), "--timeout", "180s"], stdin=subprocess.PIPE, stdout=log, stderr=log, text=True, env=env)
	129	+	processes.append(compositor)
	130	+	deadline = time.monotonic() + 15
	131	+	while not list(runtime.glob("wayland-*")):
	132	+	if compositor.poll() is not None or time.monotonic() > deadline:
	133	+	raise RuntimeError("headless compositor did not create socket")
	134	+	time.sleep(0.05)
	135	+	sockets = [p for p in runtime.glob("wayland-*") if not p.name.endswith(".lock")]
	136	+	env["WAYLAND_DISPLAY"] = str(sockets[0])
	137	+	env.pop("WAYLAND_SOCKET", None)
	138	+	env.pop("DISPLAY", None)
	139	+	def capture():
	140	+	bridge = subprocess.Popen(["docker", "exec", "-i", client, "vev", "--ui-driver", "--socket", "/home/demo/visual/ui.sock"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
	141	+	try:
	142	+	import select
	143	+	if not select.select([bridge.stdout], [], [], 1)[0]:
	144	+	raise RuntimeError("observation discovery timed out")
	145	+	discovery = json.loads(bridge.stdout.readline())
	146	+	request = {"version": 1, "id": 1, "op": "capture", "attachment": discovery["result"]["attachment"]}
	147	+	output, error = bridge.communicate(json.dumps(request) + "\n", timeout=1)
	148	+	envelopes = [json.loads(line) for line in output.splitlines()]
	149	+	if not envelopes or "error" in envelopes[-1]:
	150	+	raise RuntimeError("observation capture failed: " + str(envelopes) + error)
	151	+	return envelopes[-1]["result"]
	152	+	finally:
	153	+	if bridge.poll() is None:
	154	+	bridge.kill()
	155	+	bridge.communicate()
	156	+	compositor.stdin.write("sleep 1s\nkey Return\n")
	157	+	compositor.stdin.flush()
	158	+	deadline = time.monotonic() + 30
	159	+	while True:
	160	+	try:
	161	+	state = capture()
	162	+	if state.get("context", {}).get("status") == "attached" and state.get("context", {}).get("session", {}).get("session_name") == "resilience":
	163	+	break
	164	+	except (subprocess.SubprocessError, ValueError, RuntimeError) as error:
	165	+	(artifacts / "capture-error.txt").write_text(str(error) + "\n" + str(getattr(error, "stderr", "")))
	166	+	if time.monotonic() > deadline:
	167	+	diagnosis = subprocess.run(["docker", "exec", client, "sh", "-c", "ps -ef; ls -l /home/demo/visual/ui.sock"], capture_output=True, text=True, timeout=5)
	168	+	(artifacts / "client-processes.txt").write_text(diagnosis.stdout + diagnosis.stderr)
	169	+	(artifacts / "relay-stats.json").write_text(json.dumps(control("stats")))
	170	+	(artifacts / "last-capture.json").write_text(json.dumps(locals().get("state")))
	171	+	raise RuntimeError("visual client did not attach")
	172	+	time.sleep(0.1)
	173	+	(artifacts / "initial.json").write_text(json.dumps(state))
	174	+	def key(command):
	175	+	compositor.stdin.write(command + "\n")
	176	+	compositor.stdin.flush()
	177	+	def responsive(label):
	178	+	latest = shots / "latest.png"
	179	+	before = artifacts / (label + "-before.png")
	180	+	shutil.copyfile(latest, before)
	181	+	start = time.monotonic()
	182	+	stamp = latest.stat().st_mtime_ns
	183	+	key("key Alt+space")
	184	+	key("type SSP")
	185	+	key("key Return")
	186	+	while time.monotonic() - start < 2:
	187	+	if latest.stat().st_mtime_ns != stamp:
	188	+	after = artifacts / (label + "-after.png")
	189	+	shutil.copyfile(latest, after)
	190	+	changed = subprocess.run([str(ROOT / "build/resilience-pixels"), str(before), str(after)], capture_output=True, text=True)
	191	+	observed = capture()
	192	+	(artifacts / (label + "-observed.json")).write_text(json.dumps(observed))
	193	+	if changed.returncode == 0 and "Sessions" in observed.get("text", "") and "Esc close" in observed.get("text", ""):
	194	+	key("key Escape")
	195	+	return {"elapsed": time.monotonic()-start, "pixels": json.loads(changed.stdout), "context": observed["context"]}
	196	+	time.sleep(0.05)
	197	+	raise RuntimeError(label + ": local picker failed 2s response gate")
	198	+	evidence = {"healthy": responsive("healthy")}
	199	+	time.sleep(0.5)
	200	+	key("key Alt+space")
	201	+	key("type SSP")
	202	+	key("key Return")
	203	+	deadline = time.monotonic() + 5
	204	+	while "Sessions" not in capture().get("text", ""):
	205	+	if time.monotonic() > deadline:
	206	+	raise RuntimeError("picker did not open before blackout")
	207	+	time.sleep(0.1)
	208	+	before = artifacts / "blackout-before.png"
	209	+	shutil.copyfile(shots / "latest.png", before)
	210	+	assert "Sessions" in capture().get("text", ""), "picker was not open before blackout"
	211	+	impairment = os.environ.get("VEV_RESILIENCE_LINK", "both")
	212	+	if impairment not in ("both", "udp", "tcp"):
	213	+	raise ValueError("VEV_RESILIENCE_LINK must be both, udp or tcp")
	214	+	control("blackout", udp=impairment != "tcp", tcp=impairment != "udp", active=True)
	215	+	start = time.monotonic()
	216	+	key("type /resilience")
	217	+	deadline = start + 2
	218	+	while time.monotonic() < deadline:
	219	+	after = artifacts / "blackout-after.png"
	220	+	shutil.copyfile(shots / "latest.png", after)
	221	+	changed = subprocess.run([str(ROOT / "build/resilience-pixels"), str(before), str(after)], capture_output=True, text=True)
	222	+	observed = capture()
	223	+	if changed.returncode == 0 and "1 matches" in observed.get("text", ""):
	224	+	evidence["blackout"] = {"elapsed": time.monotonic()-start, "pixels": json.loads(changed.stdout)}
	225	+	break
	226	+	time.sleep(0.05)
	227	+	else:
	228	+	raise RuntimeError("open picker search froze during blackout")
	229	+	time.sleep(3)
	230	+	control("blackout", udp=True, tcp=True, active=False)
	231	+	key("key Escape")
	232	+	time.sleep(0.2)
	233	+	key("key Escape")
	234	+	deadline = time.monotonic() + 5
	235	+	while "Sessions" in capture().get("text", ""):
	236	+	if time.monotonic() > deadline:
	237	+	raise RuntimeError("picker did not close after restoration")
	238	+	time.sleep(0.1)
	239	+	key("key Ctrl+c")
	240	+	time.sleep(1)
	241	+	initial_session = state["context"]["session"]
	242	+	key("type printf 'RESILIENCE_RECOVERED\\n'")
	243	+	key("key Return")
	244	+	deadline = time.monotonic() + 10
	245	+	while time.monotonic() < deadline:
	246	+	recovered = capture()
	247	+	if "\nRESILIENCE_RECOVERED" in recovered.get("text", "") and recovered["context"]["session"] == initial_session:
	248	+	evidence["recovered"] = recovered["context"]
	249	+	break
	250	+	time.sleep(0.1)
	251	+	else:
	252	+	raise RuntimeError("session did not recover after blackout")
	253	+	# Negative control: compositor can still render while client is
	254	+	# stopped. No client observation is allowed to satisfy the gate.
	255	+	pid_result = docker("exec", client, "pgrep", "-f", "^vev attach", capture_output=True, text=True)
	256	+	pid = pid_result.stdout.strip().splitlines()[0]
	257	+	docker("exec", client, "kill", "-STOP", pid)
	258	+	frozen = artifacts / "frozen.png"
	259	+	shutil.copyfile(shots / "latest.png", frozen)
	260	+	try:
	261	+	frozen_at = (shots / "latest.png").stat().st_mtime_ns
	262	+	key("key Alt+space")
	263	+	# The compositor must keep capturing, or an unchanged frame
	264	+	# proves nothing about the stopped client.
	265	+	deadline = time.monotonic() + 3
	266	+	while (shots / "latest.png").stat().st_mtime_ns == frozen_at:
	267	+	if time.monotonic() > deadline:
	268	+	raise RuntimeError("compositor stopped capturing during the negative control")
	269	+	time.sleep(0.05)
	270	+	time.sleep(0.3)
	271	+	check = subprocess.run([str(ROOT / "build/resilience-pixels"), str(frozen), str(shots / "latest.png")], capture_output=True, text=True)
	272	+	assert check.returncode == 1, "visual gate accepted a stopped client"
	273	+	evidence["negative_control"] = "stopped client rejected"
	274	+	finally:
	275	+	docker("exec", client, "kill", "-CONT", pid)
	276	+	if os.environ.get("VEV_RESILIENCE_LONG") == "1":
	277	+	# The negative control's Alt+space was buffered while the
	278	+	# client was stopped and opens the palette once it resumes.
	279	+	# Close it on observed state only: a blind extra Escape would
	280	+	# reach the shell as a lone ESC, which readline keeps as a
	281	+	# Meta prefix and uses to eat the first replayed key.
	282	+	deadline = time.monotonic() + 5
	283	+	while "Commands" not in capture().get("text", ""):
	284	+	if time.monotonic() > deadline:
	285	+	raise RuntimeError("palette did not open after the stopped client resumed")
	286	+	time.sleep(0.1)
	287	+	key("key Escape")
	288	+	deadline = time.monotonic() + 5
	289	+	while "Commands" in capture().get("text", ""):
	290	+	if time.monotonic() > deadline:
	291	+	raise RuntimeError("palette did not close before the long outage")
	292	+	time.sleep(0.1)
	293	+	time.sleep(0.5)
	294	+	control("blackout", udp=True, tcp=True, active=True)
	295	+	if transport == "ssh":
	296	+	control("disconnect")
	297	+	deadline = time.monotonic() + 90
	298	+	while time.monotonic() < deadline:
	299	+	observed = capture()
	300	+	if observed.get("context", {}).get("status") == "connecting":
	301	+	break
	302	+	time.sleep(0.2)
	303	+	else:
	304	+	raise RuntimeError("long outage did not enter resume")
	305	+	if os.environ.get("VEV_RESILIENCE_RESUME") == "1":
	306	+	key("type echo HELD_ONE; echo HELD_TWO")
	307	+	key("key Return")
	308	+	# Keep the route down until the compositor has delivered
	309	+	# the complete physical input script to the resume owner.
	310	+	time.sleep(2)
	311	+	control("blackout", udp=True, tcp=True, active=False)
	312	+	deadline = time.monotonic() + 30
	313	+	while time.monotonic() < deadline:
	314	+	resumed = capture()
	315	+	text = resumed.get("text", "")
	316	+	if "\nHELD_ONE" in text and "\nHELD_TWO" in text and resumed.get("context", {}).get("session") == initial_session:
	317	+	assert text.index("\nHELD_ONE") < text.index("\nHELD_TWO"), "held input reordered"
	318	+	# The echoed command line proves the first held
	319	+	# byte was not swallowed before the shell saw it.
	320	+	assert "$ echo HELD_ONE; echo HELD_TWO" in text, "held input replayed incompletely"
	321	+	evidence["resumed"] = resumed["context"]
	322	+	break
	323	+	time.sleep(0.1)
	324	+	else:
	325	+	raise RuntimeError("long outage did not replay held input")
	326	+	else:
	327	+	start = time.monotonic()
	328	+	cancel_key = os.environ.get("VEV_RESILIENCE_CANCEL", "Ctrl+c")
	329	+	if cancel_key not in ("Ctrl+c", "Escape"):
	330	+	raise ValueError("cancel must be Ctrl+c or Escape")
	331	+	key("key " + cancel_key)
	332	+	deadline = start + 2
	333	+	while time.monotonic() < deadline:
	334	+	observed = capture()
	335	+	if observed.get("context", {}).get("status") == "picker":
	336	+	evidence["resume_cancel"] = {"elapsed": time.monotonic()-start}
	337	+	break
	338	+	time.sleep(0.05)
	339	+	else:
	340	+	raise RuntimeError(cancel_key + " did not cancel resume")
	341	+	control("blackout", udp=True, tcp=True, active=False)
	342	+	evidence["stats"] = control("stats")
	343	+	if transport == "quic":
	344	+	assert evidence["stats"]["udp"], "QUIC bypassed impairment relay"
	345	+	assert evidence["stats"]["tcp_bytes"] > 0, "SSH bypassed impairment relay"
	346	+	(artifacts / "evidence.json").write_text(json.dumps(evidence, indent=2))
	347	+	control("stop")
	348	+	finally:
	349	+	for process in reversed(processes):
	350	+	if process.poll() is None:
	351	+	process.terminate()
	352	+	try:
	353	+	process.wait(timeout=5)
	354	+	except subprocess.TimeoutExpired:
	355	+	process.kill()
	356	+	process.wait()
	357	+	for container in containers:
	358	+	role = container.rsplit("-", 1)[-1]
	359	+	for filename in ("vev-client.log", "vev-daemon.log", "broker/log/vev-broker.log"):
	360	+	subprocess.run(["docker", "cp", container + ":/home/demo/.local/state/vev/" + filename, str(artifacts / (role + "-" + filename.replace("/", "-")))], timeout=30, capture_output=True)
	361	+	subprocess.run(["docker", "rm", "-f", container], timeout=30, capture_output=True)
	362	+	if network:
	363	+	subprocess.run(["docker", "network", "rm", name], timeout=30, capture_output=True)
	364	+	subprocess.run(["docker", "image", "rm", name], timeout=30, capture_output=True)
	365	+	print("artifacts:", artifacts)
	366	+
	367	+
	368	+	if __name__ == "__main__":
	369	+	main()

@@ -0,0 +1,369 @@
	1	+	#!/usr/bin/env python3
	2	+	"""Disposable dual-transport fixture and physical Wayland input acceptance."""
	3	+	import importlib.util
	4	+	import json
	5	+	import os
	6	+	from pathlib import Path
	7	+	import shutil
	8	+	import signal
	9	+	import subprocess
	10	+	import tempfile
	11	+	import time
	12	+	import uuid
	13	+
	14	+	ROOT = Path(__file__).resolve().parents[2]
	15	+
	16	+
	17	+	def run(args, *kwargs):
	18	+	return subprocess.run(args, check=True, timeout=kwargs.pop("timeout", 60), **kwargs)
	19	+
	20	+
	21	+	def docker(args, *kwargs):
	22	+	return run("docker", args, *kwargs)
	23	+
	24	+
	25	+	def main():
	26	+	name = "vev-resilience-" + uuid.uuid4().hex[:10]
	27	+	artifacts = Path(tempfile.mkdtemp(prefix="vev-resilience-artifacts-"))
	28	+	processes = []
	29	+	containers = []
	30	+	network = False
	31	+	def interrupt(signum, frame):
	32	+	raise KeyboardInterrupt("fixture interrupted")
	33	+	signal.signal(signal.SIGTERM, interrupt)
	34	+	with tempfile.TemporaryDirectory(prefix="vev-resilience-private-") as directory:
	35	+	private = Path(directory)
	36	+	runtime = private / "runtime"
	37	+	runtime.mkdir(mode=0o700)
	38	+	try:
	39	+	for tool in ("docker", "foot", "ssh-keygen", "go"):
	40	+	if shutil.which(tool) is None:
	41	+	raise RuntimeError("missing prerequisite: " + tool)
	42	+	run("go", "build", "-o", str(ROOT / "build/vev"), ".", cwd=ROOT, timeout=600)
	43	+	for helper in ("relay", "pixels"):
	44	+	run("go", "build", "-o", str(ROOT / f"build/resilience-{helper}"), f"./scripts/remote-resilience-harness/{helper}", cwd=ROOT, timeout=600)
	45	+	nefer = os.environ.get("NEFERWL_BIN")
	46	+	if not nefer:
	47	+	nefer = str(private / "neferwl")
	48	+	common = run("git", "rev-parse", "--path-format=absolute", "--git-common-dir", cwd=ROOT, capture_output=True, text=True).stdout.strip()
	49	+	source = Path(os.environ.get("NEFERWL_SOURCE", str(Path(common).parent.parent / "neferwl")))
	50	+	run("go", "build", "-o", nefer, "./cmd/neferwl", cwd=source, env={**os.environ, "CGO_ENABLED": "0"}, timeout=600)
	51	+	docker("build", "-f", str(ROOT / "scripts/demo/Dockerfile"), "-t", name, str(ROOT), timeout=600)
	52	+	docker("network", "create", name)
	53	+	network = True
	54	+	for role in ("client", "remote"):
	55	+	container = name + "-" + role
	56	+	docker("run", "-d", "--name", container, "--network", name, "--network-alias", "upstream" if role == "remote" else "remote", "--entrypoint", "sleep", name, "infinity")
	57	+	containers.append(container)
	58	+	client, remote = containers
	59	+	for key in ("client", "host"):
	60	+	run("ssh-keygen", "-q", "-t", "ed25519", "-N", "", "-C", "fixture", "-f", str(private / key))
	61	+	docker("cp", str(private / "host"), remote + ":/etc/ssh/ssh_host_ed25519_key")
	62	+	docker("cp", str(private / "host.pub"), remote + ":/etc/ssh/ssh_host_ed25519_key.pub")
	63	+	docker("cp", str(private / "client.pub"), remote + ":/home/demo/.ssh/authorized_keys")
	64	+	docker("exec", "--user", "root", remote, "sh", "-c", "rm -f /home/demo/.ssh/id_ed25519*; chmod 600 /etc/ssh/ssh_host_ed25519_key /home/demo/.ssh/authorized_keys; chown demo:demo /home/demo/.ssh/authorized_keys; mkdir -p /run/sshd; /usr/sbin/sshd")
	65	+	docker("exec", "--user", "root", client, "sh", "-c", "rm -f /home/demo/.ssh/id_ed25519*")
	66	+	docker("cp", str(private / "client"), client + ":/home/demo/.ssh/id_ed25519")
	67	+	public = (private / "host.pub").read_text().split()
	68	+	(private / "known_hosts").write_text("remote " + " ".join(public[:2]) + "\n")
	69	+	(private / "config").write_text("Host remote\n HostName remote\n Port 2222\n User demo\n IdentityFile ~/.ssh/id_ed25519\n IdentitiesOnly yes\n StrictHostKeyChecking yes\n HostKeyAlias remote\n UserKnownHostsFile ~/.ssh/known_hosts\n")
	70	+	for file in ("known_hosts", "config"):
	71	+	docker("cp", str(private / file), client + ":/home/demo/.ssh/" + file)
	72	+	docker("cp", str(ROOT / "build/resilience-relay"), client + ":/tmp/relay")
	73	+	docker("cp", str(ROOT / "scripts/remote-resilience-harness/ssh.py"), client + ":/tmp/ssh")
	74	+	docker("exec", "--user", "root", client, "sh", "-c", "chown demo:demo /home/demo/.ssh/; chmod 600 /home/demo/.ssh/; chmod 755 /tmp/ssh; install -d -m 700 -o demo -g demo /home/demo/relay; mkdir -p /tmp/bin; ln -s /tmp/ssh /tmp/bin/ssh; mv /usr/bin/ssh /usr/bin/ssh-real; ln -s /tmp/ssh /usr/bin/ssh")
	75	+	transport = os.environ.get("VEV_RESILIENCE_TRANSPORT", "quic")
	76	+	if transport not in ("quic", "ssh"):
	77	+	raise ValueError("VEV_RESILIENCE_TRANSPORT must be quic or ssh")
	78	+	relay_args = ["-degraded"] if os.environ.get("VEV_RESILIENCE_DEGRADED") == "1" else []
	79	+	relay = subprocess.Popen(["docker", "exec", client, "/tmp/relay", "-remote", "upstream", "-control", "/home/demo/relay/control.sock", *relay_args])
	80	+	processes.append(relay)
	81	+	control_id = 0
	82	+	def control(op, **fields):
	83	+	nonlocal control_id
	84	+	control_id += 1
	85	+	request = {"id": control_id, "op": op, **fields}
	86	+	code = "import socket,json; s=socket.socket(socket.AF_UNIX); s.settimeout(3); s.connect('/home/demo/relay/control.sock'); s.sendall(" + repr(json.dumps(request).encode() + b"\n") + "); print(s.makefile().readline())"
	87	+	result = docker("exec", client, "python3", "-c", code, capture_output=True, text=True)
	88	+	reply = json.loads(result.stdout)
	89	+	assert reply["id"] == control_id and "error" not in reply, reply
	90	+	return reply["result"]
	91	+	# Fixture image must explicitly supply Python rather than using host tools.
	92	+	docker("exec", "--user", "root", client, "sh", "-c", "command -v python3", capture_output=True)
	93	+	deadline = time.monotonic() + 5
	94	+	while True:
	95	+	ready = subprocess.run(["docker", "exec", client, "test", "-S", "/home/demo/relay/control.sock"], capture_output=True, timeout=3)
	96	+	if ready.returncode == 0:
	97	+	break
	98	+	if time.monotonic() > deadline:
	99	+	raise RuntimeError("relay control socket did not become ready")
	100	+	time.sleep(0.05)
	101	+	probe = subprocess.run(["docker", "exec", client, "/usr/bin/ssh", "-v", "remote", "true"], capture_output=True, text=True, timeout=10)
	102	+	if probe.returncode:
	103	+	(artifacts / "ssh-probe.log").write_text(probe.stderr)
	104	+	raise RuntimeError("fixture SSH authentication failed")
	105	+	env_args = ["-e", "PATH=/tmp/bin:/usr/local/bin:/usr/bin", "-e", "VEV_LOG=debug", "-e", "SSH_AUTH_SOCK="]
	106	+	spec = importlib.util.spec_from_file_location("acceptance", ROOT / "scripts/remote-picker-harness/acceptance.py")
	107	+	acceptance = importlib.util.module_from_spec(spec)
	108	+	spec.loader.exec_module(acceptance)
	109	+	fixture = acceptance.Driver(remote, ["--session", "resilience"])
	110	+	fixture.close()
	111	+	docker("exec", *env_args, client, "vev", "host", "add", "--transport", transport, "remote")
	112	+	# Wait for the broker publication before resolving direct attach.
	113	+	deadline = time.monotonic() + 30
	114	+	while True:
	115	+	inventory = docker("exec", *env_args, client, "vev", "ls", "remote", capture_output=True, text=True)
	116	+	if "resilience" in inventory.stdout:
	117	+	break
	118	+	(artifacts / "inventory.txt").write_text(inventory.stdout + inventory.stderr)
	119	+	if time.monotonic() > deadline:
	120	+	raise RuntimeError("remote fixture was not published")
	121	+	time.sleep(0.2)
	122	+	shots = artifacts / "shots"
	123	+	shots.mkdir()
	124	+	docker_endpoint = run("docker", "context", "inspect", "--format", "{{.Endpoints.docker.Host}}", capture_output=True, text=True).stdout.strip()
	125	+	env = {**os.environ, "DOCKER_HOST": docker_endpoint, "XDG_RUNTIME_DIR": str(runtime), "XDG_CONFIG_HOME": str(private / "config-home"), "XDG_STATE_HOME": str(private / "state"), "HOME": str(private)}
	126	+	env["TERMINAL"] = "foot --config=/dev/null --override=cursor.blink=no --log-level=info --hold -- docker --host=" + docker_endpoint + " exec -it " + " ".join(env_args) + " " + client + " vev attach demo@remote:resilience --ui-observe --ui-socket /home/demo/visual/ui.sock"
	127	+	log = (artifacts / "compositor.log").open("w")
	128	+	compositor = subprocess.Popen([nefer, "--backend=headless", "--size=1280x720", "--input=-", "--screenshot", str(shots), "--timeout", "180s"], stdin=subprocess.PIPE, stdout=log, stderr=log, text=True, env=env)
	129	+	processes.append(compositor)
	130	+	deadline = time.monotonic() + 15
	131	+	while not list(runtime.glob("wayland-*")):
	132	+	if compositor.poll() is not None or time.monotonic() > deadline:
	133	+	raise RuntimeError("headless compositor did not create socket")
	134	+	time.sleep(0.05)
	135	+	sockets = [p for p in runtime.glob("wayland-*") if not p.name.endswith(".lock")]
	136	+	env["WAYLAND_DISPLAY"] = str(sockets[0])
	137	+	env.pop("WAYLAND_SOCKET", None)
	138	+	env.pop("DISPLAY", None)
	139	+	def capture():
	140	+	bridge = subprocess.Popen(["docker", "exec", "-i", client, "vev", "--ui-driver", "--socket", "/home/demo/visual/ui.sock"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
	141	+	try:
	142	+	import select
	143	+	if not select.select([bridge.stdout], [], [], 1)[0]:
	144	+	raise RuntimeError("observation discovery timed out")
	145	+	discovery = json.loads(bridge.stdout.readline())
	146	+	request = {"version": 1, "id": 1, "op": "capture", "attachment": discovery["result"]["attachment"]}
	147	+	output, error = bridge.communicate(json.dumps(request) + "\n", timeout=1)
	148	+	envelopes = [json.loads(line) for line in output.splitlines()]
	149	+	if not envelopes or "error" in envelopes[-1]:
	150	+	raise RuntimeError("observation capture failed: " + str(envelopes) + error)
	151	+	return envelopes[-1]["result"]
	152	+	finally:
	153	+	if bridge.poll() is None:
	154	+	bridge.kill()
	155	+	bridge.communicate()
	156	+	compositor.stdin.write("sleep 1s\nkey Return\n")
	157	+	compositor.stdin.flush()
	158	+	deadline = time.monotonic() + 30
	159	+	while True:
	160	+	try:
	161	+	state = capture()
	162	+	if state.get("context", {}).get("status") == "attached" and state.get("context", {}).get("session", {}).get("session_name") == "resilience":
	163	+	break
	164	+	except (subprocess.SubprocessError, ValueError, RuntimeError) as error:
	165	+	(artifacts / "capture-error.txt").write_text(str(error) + "\n" + str(getattr(error, "stderr", "")))
	166	+	if time.monotonic() > deadline:
	167	+	diagnosis = subprocess.run(["docker", "exec", client, "sh", "-c", "ps -ef; ls -l /home/demo/visual/ui.sock"], capture_output=True, text=True, timeout=5)
	168	+	(artifacts / "client-processes.txt").write_text(diagnosis.stdout + diagnosis.stderr)
	169	+	(artifacts / "relay-stats.json").write_text(json.dumps(control("stats")))
	170	+	(artifacts / "last-capture.json").write_text(json.dumps(locals().get("state")))
	171	+	raise RuntimeError("visual client did not attach")
	172	+	time.sleep(0.1)
	173	+	(artifacts / "initial.json").write_text(json.dumps(state))
	174	+	def key(command):
	175	+	compositor.stdin.write(command + "\n")
	176	+	compositor.stdin.flush()
	177	+	def responsive(label):
	178	+	latest = shots / "latest.png"
	179	+	before = artifacts / (label + "-before.png")
	180	+	shutil.copyfile(latest, before)
	181	+	start = time.monotonic()
	182	+	stamp = latest.stat().st_mtime_ns
	183	+	key("key Alt+space")
	184	+	key("type SSP")
	185	+	key("key Return")
	186	+	while time.monotonic() - start < 2:
	187	+	if latest.stat().st_mtime_ns != stamp:
	188	+	after = artifacts / (label + "-after.png")
	189	+	shutil.copyfile(latest, after)
	190	+	changed = subprocess.run([str(ROOT / "build/resilience-pixels"), str(before), str(after)], capture_output=True, text=True)
	191	+	observed = capture()
	192	+	(artifacts / (label + "-observed.json")).write_text(json.dumps(observed))
	193	+	if changed.returncode == 0 and "Sessions" in observed.get("text", "") and "Esc close" in observed.get("text", ""):
	194	+	key("key Escape")
	195	+	return {"elapsed": time.monotonic()-start, "pixels": json.loads(changed.stdout), "context": observed["context"]}
	196	+	time.sleep(0.05)
	197	+	raise RuntimeError(label + ": local picker failed 2s response gate")
	198	+	evidence = {"healthy": responsive("healthy")}
	199	+	time.sleep(0.5)
	200	+	key("key Alt+space")
	201	+	key("type SSP")
	202	+	key("key Return")
	203	+	deadline = time.monotonic() + 5
	204	+	while "Sessions" not in capture().get("text", ""):
	205	+	if time.monotonic() > deadline:
	206	+	raise RuntimeError("picker did not open before blackout")
	207	+	time.sleep(0.1)
	208	+	before = artifacts / "blackout-before.png"
	209	+	shutil.copyfile(shots / "latest.png", before)
	210	+	assert "Sessions" in capture().get("text", ""), "picker was not open before blackout"
	211	+	impairment = os.environ.get("VEV_RESILIENCE_LINK", "both")
	212	+	if impairment not in ("both", "udp", "tcp"):
	213	+	raise ValueError("VEV_RESILIENCE_LINK must be both, udp or tcp")
	214	+	control("blackout", udp=impairment != "tcp", tcp=impairment != "udp", active=True)
	215	+	start = time.monotonic()
	216	+	key("type /resilience")
	217	+	deadline = start + 2
	218	+	while time.monotonic() < deadline:
	219	+	after = artifacts / "blackout-after.png"
	220	+	shutil.copyfile(shots / "latest.png", after)
	221	+	changed = subprocess.run([str(ROOT / "build/resilience-pixels"), str(before), str(after)], capture_output=True, text=True)
	222	+	observed = capture()
	223	+	if changed.returncode == 0 and "1 matches" in observed.get("text", ""):
	224	+	evidence["blackout"] = {"elapsed": time.monotonic()-start, "pixels": json.loads(changed.stdout)}
	225	+	break
	226	+	time.sleep(0.05)
	227	+	else:
	228	+	raise RuntimeError("open picker search froze during blackout")
	229	+	time.sleep(3)
	230	+	control("blackout", udp=True, tcp=True, active=False)
	231	+	key("key Escape")
	232	+	time.sleep(0.2)
	233	+	key("key Escape")
	234	+	deadline = time.monotonic() + 5
	235	+	while "Sessions" in capture().get("text", ""):
	236	+	if time.monotonic() > deadline:
	237	+	raise RuntimeError("picker did not close after restoration")
	238	+	time.sleep(0.1)
	239	+	key("key Ctrl+c")
	240	+	time.sleep(1)
	241	+	initial_session = state["context"]["session"]
	242	+	key("type printf 'RESILIENCE_RECOVERED\\n'")
	243	+	key("key Return")
	244	+	deadline = time.monotonic() + 10
	245	+	while time.monotonic() < deadline:
	246	+	recovered = capture()
	247	+	if "\nRESILIENCE_RECOVERED" in recovered.get("text", "") and recovered["context"]["session"] == initial_session:
	248	+	evidence["recovered"] = recovered["context"]
	249	+	break
	250	+	time.sleep(0.1)
	251	+	else:
	252	+	raise RuntimeError("session did not recover after blackout")
	253	+	# Negative control: compositor can still render while client is
	254	+	# stopped. No client observation is allowed to satisfy the gate.
	255	+	pid_result = docker("exec", client, "pgrep", "-f", "^vev attach", capture_output=True, text=True)
	256	+	pid = pid_result.stdout.strip().splitlines()[0]
	257	+	docker("exec", client, "kill", "-STOP", pid)
	258	+	frozen = artifacts / "frozen.png"
	259	+	shutil.copyfile(shots / "latest.png", frozen)
	260	+	try:
	261	+	frozen_at = (shots / "latest.png").stat().st_mtime_ns
	262	+	key("key Alt+space")
	263	+	# The compositor must keep capturing, or an unchanged frame
	264	+	# proves nothing about the stopped client.
	265	+	deadline = time.monotonic() + 3
	266	+	while (shots / "latest.png").stat().st_mtime_ns == frozen_at:
	267	+	if time.monotonic() > deadline:
	268	+	raise RuntimeError("compositor stopped capturing during the negative control")
	269	+	time.sleep(0.05)
	270	+	time.sleep(0.3)
	271	+	check = subprocess.run([str(ROOT / "build/resilience-pixels"), str(frozen), str(shots / "latest.png")], capture_output=True, text=True)
	272	+	assert check.returncode == 1, "visual gate accepted a stopped client"
	273	+	evidence["negative_control"] = "stopped client rejected"
	274	+	finally:
	275	+	docker("exec", client, "kill", "-CONT", pid)
	276	+	if os.environ.get("VEV_RESILIENCE_LONG") == "1":
	277	+	# The negative control's Alt+space was buffered while the
	278	+	# client was stopped and opens the palette once it resumes.
	279	+	# Close it on observed state only: a blind extra Escape would
	280	+	# reach the shell as a lone ESC, which readline keeps as a
	281	+	# Meta prefix and uses to eat the first replayed key.
	282	+	deadline = time.monotonic() + 5
	283	+	while "Commands" not in capture().get("text", ""):
	284	+	if time.monotonic() > deadline:
	285	+	raise RuntimeError("palette did not open after the stopped client resumed")
	286	+	time.sleep(0.1)
	287	+	key("key Escape")
	288	+	deadline = time.monotonic() + 5
	289	+	while "Commands" in capture().get("text", ""):
	290	+	if time.monotonic() > deadline:
	291	+	raise RuntimeError("palette did not close before the long outage")
	292	+	time.sleep(0.1)
	293	+	time.sleep(0.5)
	294	+	control("blackout", udp=True, tcp=True, active=True)
	295	+	if transport == "ssh":
	296	+	control("disconnect")
	297	+	deadline = time.monotonic() + 90
	298	+	while time.monotonic() < deadline:
	299	+	observed = capture()
	300	+	if observed.get("context", {}).get("status") == "connecting":
	301	+	break
	302	+	time.sleep(0.2)
	303	+	else:
	304	+	raise RuntimeError("long outage did not enter resume")
	305	+	if os.environ.get("VEV_RESILIENCE_RESUME") == "1":
	306	+	key("type echo HELD_ONE; echo HELD_TWO")
	307	+	key("key Return")
	308	+	# Keep the route down until the compositor has delivered
	309	+	# the complete physical input script to the resume owner.
	310	+	time.sleep(2)
	311	+	control("blackout", udp=True, tcp=True, active=False)
	312	+	deadline = time.monotonic() + 30
	313	+	while time.monotonic() < deadline:
	314	+	resumed = capture()
	315	+	text = resumed.get("text", "")
	316	+	if "\nHELD_ONE" in text and "\nHELD_TWO" in text and resumed.get("context", {}).get("session") == initial_session:
	317	+	assert text.index("\nHELD_ONE") < text.index("\nHELD_TWO"), "held input reordered"
	318	+	# The echoed command line proves the first held
	319	+	# byte was not swallowed before the shell saw it.
	320	+	assert "$ echo HELD_ONE; echo HELD_TWO" in text, "held input replayed incompletely"
	321	+	evidence["resumed"] = resumed["context"]
	322	+	break
	323	+	time.sleep(0.1)
	324	+	else:
	325	+	raise RuntimeError("long outage did not replay held input")
	326	+	else:
	327	+	start = time.monotonic()
	328	+	cancel_key = os.environ.get("VEV_RESILIENCE_CANCEL", "Ctrl+c")
	329	+	if cancel_key not in ("Ctrl+c", "Escape"):
	330	+	raise ValueError("cancel must be Ctrl+c or Escape")
	331	+	key("key " + cancel_key)
	332	+	deadline = start + 2
	333	+	while time.monotonic() < deadline:
	334	+	observed = capture()
	335	+	if observed.get("context", {}).get("status") == "picker":
	336	+	evidence["resume_cancel"] = {"elapsed": time.monotonic()-start}
	337	+	break
	338	+	time.sleep(0.05)
	339	+	else:
	340	+	raise RuntimeError(cancel_key + " did not cancel resume")
	341	+	control("blackout", udp=True, tcp=True, active=False)
	342	+	evidence["stats"] = control("stats")
	343	+	if transport == "quic":
	344	+	assert evidence["stats"]["udp"], "QUIC bypassed impairment relay"
	345	+	assert evidence["stats"]["tcp_bytes"] > 0, "SSH bypassed impairment relay"
	346	+	(artifacts / "evidence.json").write_text(json.dumps(evidence, indent=2))
	347	+	control("stop")
	348	+	finally:
	349	+	for process in reversed(processes):
	350	+	if process.poll() is None:
	351	+	process.terminate()
	352	+	try:
	353	+	process.wait(timeout=5)
	354	+	except subprocess.TimeoutExpired:
	355	+	process.kill()
	356	+	process.wait()
	357	+	for container in containers:
	358	+	role = container.rsplit("-", 1)[-1]
	359	+	for filename in ("vev-client.log", "vev-daemon.log", "broker/log/vev-broker.log"):
	360	+	subprocess.run(["docker", "cp", container + ":/home/demo/.local/state/vev/" + filename, str(artifacts / (role + "-" + filename.replace("/", "-")))], timeout=30, capture_output=True)
	361	+	subprocess.run(["docker", "rm", "-f", container], timeout=30, capture_output=True)
	362	+	if network:
	363	+	subprocess.run(["docker", "network", "rm", name], timeout=30, capture_output=True)
	364	+	subprocess.run(["docker", "image", "rm", name], timeout=30, capture_output=True)
	365	+	print("artifacts:", artifacts)
	366	+
	367	+
	368	+	if __name__ == "__main__":
	369	+	main()