performance.md Raw History Folder

Performance measurement

For contributors. How to measure vev's latency end to end, and the budgets it should meet.

Quick start:

go build -o /tmp/vev ./
go run ./cmd/vev-perf-harness --vev-bin /tmp/vev \
  --manifest testdata/perf/manifest.json --out testdata/perf/results \
  --scenario 1x4-idle-local

Measurement never changes rendering, pacing, transports, or wire bytes.

Canonical matrix

The canonical matrix is testdata/perf/manifest.json:

topologies 1x4, 4x1, 4x4, and 8x1;
120x40 terminal content with 10,000 history rows per pane;
idle, active output, all/inactive output, interactive flood, copy search, resize sweep, snapshot/output/resize, and attach/restore/tab switch;
local Unix IPC and SSH stdio.

The matrix contains 72 scenarios (4 topologies × 9 workloads × 2 transports). QUIC clean/adverse transport evidence is owned by the real-QUIC integration suite described below rather than simulated by the process launcher. Inapplicable scenarios stay present with an explicit reason. Multiple-attachment scenarios measure shared session mutations together with independent attachment views and output streams.

A connection handshake is bounded by 15 seconds. A command request is bounded by 10 seconds. These limits apply during measurement as well as normal use.

Trace schema and clocks

Every process JSONL record has exactly these fields: schema, process_id, component, scenario, run, sequence, request_id, epoch, kind, tick, bytes, fragments, retransmits, pending, ack_rtt_nanos, and valid.

Schema 1 is represented by:

{"schema":1,"process_id":"p","component":"daemon","scenario":"s","run":1,"sequence":1,"request_id":1,"epoch":1,"kind":"diff_start","tick":0,"bytes":0,"fragments":0,"retransmits":0,"pending":0,"ack_rtt_nanos":0,"valid":true}

The concrete JSONL observer stamps tick with its process-local injected monotonic clock; producers supply no timestamp. Components in one OS process share that observer. The harness owns a separate monotonic clock for the input-injection to successfully-flushed-terminal-bytes boundary. Ticks from different processes or clock domains are never ordered or subtracted. Records correlate through scenario, run, sequence, request, and epoch IDs.

Process-local spans cover capture, compose, diff, queue wait, acknowledgement blocking, emit, adapter send, and adapter receive. Bytes, fragments, retransmits, pending work, and acknowledgement RTT remain diagnostics rather than duration inputs. Reports retain raw records, counts, p50/p95/p99, and max where applicable.

Running the matrix

Use a clean build and at least a 10-second warmup, a 30-second measured interval, and ten independent repetitions:

go build -o /tmp/vev ./
rm -rf testdata/perf/results
go run ./cmd/vev-perf-harness \
  --vev-bin /tmp/vev \
  --manifest testdata/perf/manifest.json \
  --out testdata/perf/results \
  --warmup 10s --duration 30s --repetitions 10

Each run writes manifest.json, raw-harness.jsonl, one trace per launched process, runs.json, and summary.json. The process manifest records process identity, role, scenario/run, clock domain, and its exclusive trace path.

The harness injects at most one measured event per second. A sample requires both input injection and successful terminal flush inside the measured interval; warmup, straddling, and failed events remain raw diagnostics. Reject results with missing or unmatched boundaries, invalid correlation, nonmonotonic same-process sequence, negative same-domain duration, fewer than 10 complete in-interval pairs, invalid denominators, cross-process tick math, less than 30 seconds measured, or fewer than 10 repetitions.

Before a full run, validate the manifest and harness:

jq -e '(.topologies|length)==4 and (.workloads|length)==9 and (.transports|length)==2 and (.scenarios|length)==72' testdata/perf/manifest.json
go test ./cmd/vev-perf-harness

For a bounded local check, add --scenario 1x4-idle-local. The harness still validates the complete manifest before selecting that scenario. Do not commit result directories; raw traces and run manifests are the measurement evidence.

Output snapshot encoding

Full state-bearing output snapshots larger than 1 KiB are encoded with standard-library zlib only when the compressed representation is smaller. Incremental output, small snapshots, and incompressible snapshots remain uncompressed. The wire carries a closed compression kind and exact decoded length; decoding rejects unknown kinds, truncation, trailing bytes, integrity failures, and output beyond the frame bound.

Measure the complete production Output path with:

go test ./internal/adapters/sessionwire -run '^$' \
  -bench '^BenchmarkOutputProductionRoundTrip$' -benchmem -count=5

It includes semantic conversion, compression policy, Protobuf marshal, strict wire scanning, unmarshal, decompression, validation, and semantic conversion back. Its incremental, compressible snapshot, and deterministic random incompressible snapshot fixtures are valid protocol.Output state chains. It is a codec benchmark, not a mobile-link latency measurement.

Protobuf and QUIC budgets

The production Output benchmark covers incremental, compressible snapshot, and incompressible snapshot traffic. The zlib writer pool is part of the production path because it materially reduces compression time and allocation volume. The harness boundary is successful writing of bytes read from the client PTY; it deliberately does not call file.Sync, which measures filesystem durability rather than terminal delivery.

The release budgets are:

local p95 input-to-terminal-write ratio ≤1.10× on calm and active-output work;
steady-state CPU/work ratio ≤1.15× when captured on the same host;
no unbounded heap, goroutine, or queue growth under slow-reader/adverse tests;
real QUIC must preserve ordered, duplicate-free typed traffic through the clean path and the train profile (25 ms base latency, 15 ms jitter, 4% loss, 6% duplication, 10% reordering, 500 ms blackout), and through NAT rebinding.

The real-QUIC integration tests are:

go test ./internal/adapters/quicnettest -race \
  -run 'TestRealQUICTypedConversationOverTrainProfile|TestRealQUICNATRebindKeepsTypedConversation' \
  -count=10

They exercise the production QUIC and sessionwire adapters. Latency under the train profile reflects configured impairment and QUIC recovery, so it is not compared to the local IPC budget.

The representative QUIC workload is Input → validated incremental Output → Ack, with 512-byte Input and Output bodies. The evidence test logs latency percentiles, process CPU, allocations, GC, retained heap, goroutines, wire amplification, blackout recovery, NAT-rebind recovery, and proxy accounting. Reproduce the bounded evidence and isolated benchmarks with:

go test ./internal/adapters/quicnettest -race \
  -run TestRealQUICPerformanceEvidence -count=3 -v
go test ./internal/adapters/quicnettest -run '^$' \
  -bench 'BenchmarkRealQUICExchange|BenchmarkRealQUICBlackoutRecovery|BenchmarkRealQUICNATRebindRecovery' \
  -benchtime=50x -benchmem

Process CPU is collected with getrusage(RUSAGE_SELF). It provides an absolute CPU/work signal for QUIC. Track it together with bounded retained heap and zero queue overflow; treat the train profile as a resilience workload rather than a local-latency gate.

Application recovery after irreversible QUIC loss is covered separately by TestRealQUICApplicationReconnectResumesTypedSession. It runs the real client use case over two distinct production QUIC connections and two sessionwire preambles. Closing the first accepted connection must trigger a second dial whose Hello uses IntentResume, preserves the client ID and previous resume token, receives a rotated token, accepts a fresh full Output, presents it, and returns the matching ACK without requesting an Output reset. Run it with:

go test ./internal/adapters/quicnettest -race \
  -run TestRealQUICApplicationReconnectResumesTypedSession -count=10

In-process checks

These commands measure local daemon or VT work only, not end-to-end transport latency:

go test ./internal/usecase/daemon -run '^$' -bench '^BenchmarkDaemonHistory' -benchtime=1x -benchmem
go test github.com/bnema/vev-vt/ansi github.com/bnema/vev-vt ./internal/adapters/ipc -run '^$' -bench=. -benchmem
go test ./internal/usecase/daemon -run '^$' -bench '^BenchmarkComposeCapturedFloatingFrameCached$' -benchmem

Focused benchmarks for specific paths:

Path	Command
Copy-mode search over 12k rows	`go test ./internal/usecase/copy -run '^$' -bench '^BenchmarkFindMatches12K$' -benchmem`
Snapshot publication bytes and time	`go test ./internal/usecase/daemon -run '^$' -bench '^BenchmarkDaemonIncrementalSnapshotRepository$' -benchmem`
UI mirror flush with `--ui-observe`/`--ui-control`	`go test ./internal/adapters/uiterm -run '^$' -bench '^BenchmarkTerminalMirrorFlush$' -benchmem`

Idle cost

An idle session should cost almost nothing:

The broker rewrites state.json only when durable state changes, not when an observation only refreshes its timestamps.
A routine refresh of a reachable daemon publishes one snapshot, not a transient "checking" snapshot first.
A session keeps one pre-started floating shell, not one per tab, so its process count is its panes plus one.
The UI mirror converts a frame to cells only when something reads it.

Mouse scroll rendering

Wheel input moves the immutable copy viewport, rather than walking the keyboard cursor to an edge before scrolling. Ordinary copy paints preserve the output epoch. For unchanged full-width viewports, composition rotates retained compact rows and paints only exposed rows and cursor highlights. The ANSI renderer checks the scroll against its own committed shadow before using a scroll region in either direction. Selection, search, narrow panes, floating panes, notices, and geometry/theme changes keep their conservative composition paths.

The unadorned live-pane cache stays separate from the displayed copy viewport. Neither committed frame is mutated while preparing a candidate; failed output cannot publish its viewport metadata. Copy exit refreshes live content.

Mouse animation is tested separately with injected clocks: first response, 16 ms pacing, burst accumulation, deceleration, reversal, cancellation, and the 120 ms tail deadline. The rendering benchmark deliberately bypasses those timers to measure work rather than sleep time:

go test ./internal/usecase/daemon -run '^$' \
  -bench '^BenchmarkDaemonHistoryCopyScroll$' -benchmem -count=3
go test ./internal/usecase/daemon ./internal/usecase/copy -race \
  -run 'TestCopyScrollAnimation|TestCopyWheel|TestScrollRows|TestRenderRowsRange'

source

1# Performance measurement
2
3For contributors. How to measure vev's latency end to end, and the budgets it should meet.
4
5**Quick start:**
6
7```sh
8go build -o /tmp/vev ./
9go run ./cmd/vev-perf-harness --vev-bin /tmp/vev \
10  --manifest testdata/perf/manifest.json --out testdata/perf/results \
11  --scenario 1x4-idle-local
12```
13
14Measurement never changes rendering, pacing, transports, or wire bytes.
15
16## Canonical matrix
17
18The canonical matrix is [`testdata/perf/manifest.json`](../testdata/perf/manifest.json):
19
20- topologies `1x4`, `4x1`, `4x4`, and `8x1`;
21- `120x40` terminal content with 10,000 history rows per pane;
22- idle, active output, all/inactive output, interactive flood, copy search,
23  resize sweep, snapshot/output/resize, and attach/restore/tab switch;
24- local Unix IPC and SSH stdio.
25
26The matrix contains 72 scenarios (4 topologies × 9 workloads × 2 transports).
27QUIC clean/adverse transport evidence is owned by the real-QUIC integration
28suite described below rather than simulated by the process launcher.
29Inapplicable scenarios stay present with an explicit reason.
30Multiple-attachment scenarios measure shared session mutations together with
31independent attachment views and output streams.
32
33A connection handshake is bounded by 15 seconds. A command request is bounded
34by 10 seconds. These limits apply during measurement as well as normal use.
35
36## Trace schema and clocks
37
38Every process JSONL record has exactly these fields:
39`schema`, `process_id`, `component`, `scenario`, `run`, `sequence`, `request_id`,
40`epoch`, `kind`, `tick`, `bytes`, `fragments`, `retransmits`, `pending`,
41`ack_rtt_nanos`, and `valid`.
42
43Schema 1 is represented by:
44
45```json
46{"schema":1,"process_id":"p","component":"daemon","scenario":"s","run":1,"sequence":1,"request_id":1,"epoch":1,"kind":"diff_start","tick":0,"bytes":0,"fragments":0,"retransmits":0,"pending":0,"ack_rtt_nanos":0,"valid":true}
47```
48
49The concrete JSONL observer stamps `tick` with its process-local injected
50monotonic clock; producers supply no timestamp. Components in one OS process
51share that observer. The harness owns a separate monotonic clock for the
52input-injection to successfully-flushed-terminal-bytes boundary. Ticks from
53different processes or clock domains are never ordered or subtracted. Records
54correlate through scenario, run, sequence, request, and epoch IDs.
55
56Process-local spans cover capture, compose, diff, queue wait, acknowledgement
57blocking, emit, adapter send, and adapter receive. Bytes, fragments,
58retransmits, pending work, and acknowledgement RTT remain diagnostics rather
59than duration inputs. Reports retain raw records, counts, p50/p95/p99, and max
60where applicable.
61
62## Running the matrix
63
64Use a clean build and at least a 10-second warmup, a 30-second measured
65interval, and ten independent repetitions:
66
67```sh
68go build -o /tmp/vev ./
69rm -rf testdata/perf/results
70go run ./cmd/vev-perf-harness \
71  --vev-bin /tmp/vev \
72  --manifest testdata/perf/manifest.json \
73  --out testdata/perf/results \
74  --warmup 10s --duration 30s --repetitions 10
75```
76
77Each run writes `manifest.json`, `raw-harness.jsonl`, one trace per launched
78process, `runs.json`, and `summary.json`. The process manifest records process
79identity, role, scenario/run, clock domain, and its exclusive trace path.
80
81The harness injects at most one measured event per second. A sample requires
82both input injection and successful terminal flush inside the measured
83interval; warmup, straddling, and failed events remain raw diagnostics. Reject
84results with missing or unmatched boundaries, invalid correlation,
85nonmonotonic same-process sequence, negative same-domain duration, fewer than
8610 complete in-interval pairs, invalid denominators, cross-process tick math,
87less than 30 seconds measured, or fewer than 10 repetitions.
88
89Before a full run, validate the manifest and harness:
90
91```sh
92jq -e '(.topologies|length)==4 and (.workloads|length)==9 and (.transports|length)==2 and (.scenarios|length)==72' testdata/perf/manifest.json
93go test ./cmd/vev-perf-harness
94```
95
96For a bounded local check, add `--scenario 1x4-idle-local`. The harness still
97validates the complete manifest before selecting that scenario. Do not commit
98result directories; raw traces and run manifests are the measurement evidence.
99
100## Output snapshot encoding
101
102Full state-bearing output snapshots larger than 1 KiB are encoded with
103standard-library zlib only when the compressed representation is smaller.
104Incremental output, small snapshots, and incompressible snapshots remain
105uncompressed. The wire carries a closed compression kind and exact decoded
106length; decoding rejects unknown kinds, truncation, trailing bytes, integrity
107failures, and output beyond the frame bound.
108
109Measure the complete production Output path with:
110
111```sh
112go test ./internal/adapters/sessionwire -run '^$' \
113  -bench '^BenchmarkOutputProductionRoundTrip$' -benchmem -count=5
114```
115
116It includes semantic conversion, compression policy, Protobuf marshal, strict
117wire scanning, unmarshal, decompression, validation, and semantic conversion
118back. Its incremental, compressible snapshot, and deterministic random
119incompressible snapshot fixtures are valid `protocol.Output` state chains.
120It is a codec benchmark, not a mobile-link latency measurement.
121
122## Protobuf and QUIC budgets
123
124The production Output benchmark covers incremental, compressible snapshot, and
125incompressible snapshot traffic. The zlib writer pool is part of the production
126path because it materially reduces compression time and allocation volume. The
127harness boundary is successful writing of bytes read from the client PTY; it
128deliberately does not call `file.Sync`, which measures filesystem durability
129rather than terminal delivery.
130
131The release budgets are:
132
133- local p95 input-to-terminal-write ratio ≤1.10× on calm and active-output work;
134- steady-state CPU/work ratio ≤1.15× when captured on the same host;
135- no unbounded heap, goroutine, or queue growth under slow-reader/adverse tests;
136- real QUIC must preserve ordered, duplicate-free typed traffic through the
137  clean path and the train profile (25 ms base latency, 15 ms jitter, 4% loss,
138  6% duplication, 10% reordering, 500 ms blackout), and through NAT rebinding.
139
140The real-QUIC integration tests are:
141
142```sh
143go test ./internal/adapters/quicnettest -race \
144  -run 'TestRealQUICTypedConversationOverTrainProfile|TestRealQUICNATRebindKeepsTypedConversation' \
145  -count=10
146```
147
148They exercise the production QUIC and sessionwire adapters. Latency under the
149train profile reflects configured impairment and QUIC recovery, so it is not
150compared to the local IPC budget.
151
152The representative QUIC workload is `Input` → validated incremental `Output` →
153`Ack`, with 512-byte Input and Output bodies. The evidence test logs latency
154percentiles, process CPU, allocations, GC, retained heap, goroutines, wire
155amplification, blackout recovery, NAT-rebind recovery, and proxy accounting.
156Reproduce the bounded evidence and isolated benchmarks with:
157
158```sh
159go test ./internal/adapters/quicnettest -race \
160  -run TestRealQUICPerformanceEvidence -count=3 -v
161go test ./internal/adapters/quicnettest -run '^$' \
162  -bench 'BenchmarkRealQUICExchange|BenchmarkRealQUICBlackoutRecovery|BenchmarkRealQUICNATRebindRecovery' \
163  -benchtime=50x -benchmem
164```
165
166Process CPU is collected with `getrusage(RUSAGE_SELF)`. It provides an absolute
167CPU/work signal for QUIC. Track it together with bounded retained heap and zero
168queue overflow; treat the train profile as a resilience workload rather than a
169local-latency gate.
170
171Application recovery after irreversible QUIC loss is covered separately by
172`TestRealQUICApplicationReconnectResumesTypedSession`. It runs the real client
173use case over two distinct production QUIC connections and two sessionwire
174preambles. Closing the first accepted connection must trigger a second dial
175whose `Hello` uses `IntentResume`, preserves the client ID and previous resume
176token, receives a rotated token, accepts a fresh full Output, presents it, and
177returns the matching ACK without requesting an Output reset. Run it with:
178
179```sh
180go test ./internal/adapters/quicnettest -race \
181  -run TestRealQUICApplicationReconnectResumesTypedSession -count=10
182```
183
184## In-process checks
185
186These commands measure local daemon or VT work only, not end-to-end transport
187latency:
188
189```sh
190go test ./internal/usecase/daemon -run '^$' -bench '^BenchmarkDaemonHistory' -benchtime=1x -benchmem
191go test github.com/bnema/vev-vt/ansi github.com/bnema/vev-vt ./internal/adapters/ipc -run '^$' -bench=. -benchmem
192go test ./internal/usecase/daemon -run '^$' -bench '^BenchmarkComposeCapturedFloatingFrameCached$' -benchmem
193```
194
195Focused benchmarks for specific paths:
196
197| Path | Command |
198|---|---|
199| Copy-mode search over 12k rows | `go test ./internal/usecase/copy -run '^$' -bench '^BenchmarkFindMatches12K$' -benchmem` |
200| Snapshot publication bytes and time | `go test ./internal/usecase/daemon -run '^$' -bench '^BenchmarkDaemonIncrementalSnapshotRepository$' -benchmem` |
201| UI mirror flush with `--ui-observe`/`--ui-control` | `go test ./internal/adapters/uiterm -run '^$' -bench '^BenchmarkTerminalMirrorFlush$' -benchmem` |
202
203## Idle cost
204
205An idle session should cost almost nothing:
206
207- The broker rewrites `state.json` only when durable state changes, not when
208  an observation only refreshes its timestamps.
209- A routine refresh of a reachable daemon publishes one snapshot, not a
210  transient "checking" snapshot first.
211- A session keeps one pre-started floating shell, not one per tab, so its
212  process count is its panes plus one.
213- The UI mirror converts a frame to cells only when something reads it.
214
215## Mouse scroll rendering
216
217Wheel input moves the immutable copy viewport, rather than walking the keyboard
218cursor to an edge before scrolling. Ordinary copy paints preserve the output
219epoch. For unchanged full-width viewports, composition rotates retained compact
220rows and paints only exposed rows and cursor highlights. The ANSI renderer checks
221the scroll against its own committed shadow before using a scroll region in
222either direction. Selection, search, narrow panes, floating panes, notices, and
223geometry/theme changes keep their conservative composition paths.
224
225The unadorned live-pane cache stays separate from the displayed copy viewport.
226Neither committed frame is mutated while preparing a candidate; failed output
227cannot publish its viewport metadata. Copy exit refreshes live content.
228
229Mouse animation is tested separately with injected clocks: first response,
23016 ms pacing, burst accumulation, deceleration, reversal, cancellation, and the
231120 ms tail deadline. The rendering benchmark deliberately bypasses those timers
232to measure work rather than sleep time:
233
234```sh
235go test ./internal/usecase/daemon -run '^$' \
236  -bench '^BenchmarkDaemonHistoryCopyScroll$' -benchmem -count=3
237go test ./internal/usecase/daemon ./internal/usecase/copy -race \
238  -run 'TestCopyScrollAnimation|TestCopyWheel|TestScrollRows|TestRenderRowsRange'
239```

performance.md RawHistoryFolder