1# Offline broker store (Plan 001 P2.3)
2
3`Open` is deliberately not wired into application composition. Supply a
4private destination directory, immutable copies of any legacy inputs, and an
5explicit policy for each imported endpoint. An empty input path means no input;
6a named missing, empty, corrupt, or unsupported file fails closed. The lifetime
7lock is `hosts.json.lock`, the exact lock file the legacy host writer takes, so
8a store opened on a directory that live writer also uses excludes it for the
9lifetime of the store. The legacy cache source has no such lock — its writer
10takes none — so a cache source must be immutable or quiesced before migration.
11Never point migration at a cache file that a writer can still mutate.
12
13Each source's recovery manifest entry records its generic role label
14(`membership` for the host registry, `observations` for the catalogue cache),
15whether the source was supplied at all, and the SHA-256 of its bytes. Presence
16is separate from the digest because an absent source and an empty one can share
17a digest; a record that claims a source was absent while retaining its bytes is
18rejected.
19
20## Transaction and recovery
21
22- Hold `hosts.json.lock` with Linux nonblocking exclusive `flock` until `Close`.
23  The name matches the legacy host writer's lock, so the store and that writer
24  exclude each other; the legacy catalog cache writer takes no lock at all.
25- Validate bounded legacy hosts v1/v2/v3 and cache v2/v3/v4 without calling their
26  existing writer adapters. Cache decoding freezes the existing exact legacy
27  DTOs here; count-only v2 inventories are deliberately unsupported.
28- Write `recovery.json`: original bytes, SHA-256 migration manifest with source
29  presence, policies, generated registration identities, and initial unified
30  state. Write, fsync, rename, then fsync the directory before attempting state
31  publication.
32- Write `state.json` with the same atomic sequence. Hosts, policies, snapshot,
33  and manifest share one rename commit point. Only then return a ready store.
34- If state is absent on restart, validate and finish the recovery record without
35  resampling identities or rereading changed sources. If state exists but is
36  corrupt, fail closed; never silently roll authority back.
37- Keep recovery and original inputs indefinitely. Rollback is an explicit
38  offline operator action using the original bytes, not automatic fallback.
39  P2.3 does not retire or delete production files.
40
41A failed mutation poisons the handle because rename may already have happened.
42Close and reopen to resolve that outcome; do not blindly retry. Fault tests cover
43both files at write/fsync/rename/directory-fsync boundaries. They simulate errors
44and process reopening, not hardware power-loss behavior of a particular disk.
45
46The file budget is 16 MiB per JSON file (including the base64 rollback inputs in
47the recovery file). Inputs are bounded before decoding, must be regular files,
48and cannot be symlinks. JSON rejects duplicate/unknown fields, trailing content,
49invalid UTF-8, excessive nesting, and invalid domain/catalogue values.
50
51## Authority
52
53`BrokerHostStore` separates durable membership from advisory observations.
54Host replacement uses a store revision CAS. Policy changes require a changed
55registration identity/generation; generations cannot regress within an
56incarnation. Full remove/re-add callers must supply a fresh incarnation.
57Snapshots cannot add hosts, overwrite retired registrations, regress revisions,
58or switch epochs after the first successful publication on an open handle.
59`NewRegistry` requires `BrokerHostStore` explicitly: durable membership is not
60an optional capability, and the registry never runs on a store that cannot
61answer `LoadHosts`. Startup intersects cached observations with authoritative
62membership, and creates unknown projections for registrations without
63observations. An observation is adopted only when it satisfies the same durable
64projection rules the store enforces (`ports.ValidateDurableHostProjection`):
65availability and failure kind inside their closed ranges, and a catalogue-valid
66session inventory for the exact incarnation and success time. Anything else is
67typed as an invalid response, so a published snapshot is never one the store
68would reject and silently never persist.
69
70Store failures are classified with `internal/ports` sentinels:
71`ErrBrokerHostConflict` (refresh the revision and retry),
72`ErrBrokerStoreLocked` (another owner is live), and
73`ErrBrokerStoreInvalidState` (resolve offline from the recovery record; state is
74never silently replaced or rolled back). Malformed caller membership passed to
75`ReplaceHosts` is validated before the CAS and returned as a plain caller error,
76never as `ErrBrokerStoreInvalidState`; the durable state is unchanged.
77
78Unbound cache v2/v3 records remain recoverable but cannot seed registration
79identity. A bound cache entry seeds only the exact surviving host incarnation;
80legacy cache schemas do not carry a registration generation. The imported host
81record remains authoritative for generation. Transient checking/error objects
82are never persisted. Retirement tombstones are process-local fencing state and
83are never durable: the registry's persistable copy and the store both clear
84`Snapshot.Removed` before writing, so reopening sees membership without stale
85tombstones. Policy is membership authority, not observation state, so the
86durable observation format omits it: the loader re-stamps every restored
87observation from the matching `ports.BrokerHostRecord`, keeping membership the
88single policy authority. A local daemon observation is process-local and never
89durable: the store rejects one outright. No environment, launch, or trust policy
90is guessed.
91
92Production cutover, lifecycle supervision, and IPC composition remain later
93phases. Close ordering must drain registry snapshot writers before releasing the
94store; the store does not own those workers.