docs/decisions.md
Ref: Size: 557.4 KiB History
# Decision log
Paths cited in entries before 2026-08-28 predate the folder move: what these
call `src/X.zig` now lives under `src/engine/`, `src/server/`, `src/client/`
or `src/tui/`. The entries are the record of what was decided and are left as
written.
## 2026-08-07 (M1)
- **Language: Zig.** The engine dependency (ghostty-vt) is a Zig module; a C
shim would add surface without adding capability.
- **Engine: upstream ghostty package, not a fork.** Pinned at commit
`853183e9` (1.3.2-dev), module `ghostty-vt`. The M1 kill criterion
("cannot extract grid without invasive forking") is moot: upstream ships
a headless VT library with plain/VT/HTML formatters and a RenderState
dirty-tracking API (relevant for M4 deltas). API is documented unstable;
the pin is load-bearing.
- **Toolchain: Zig 0.15.2, pinned in the Makefile.** ghostty's build hard-
requires 0.15.x (`requireZig` checks major.minor); the system default zig
is 0.17-dev and 0.16.0 also fails (ghostty uses pre-0.16 std.Build APIs).
Builds use LLVM+LLD (`use_llvm`/`use_lld` in build.zig) because Zig 0.15's
self-hosted x86_64 linker can't handle the `.sframe` sections gcc >= 16
emits in this system's crt1.o.
- **No code reuse from waystty** (user decision: not performant). ghostty-vt
API knowledge only. muxd uses blocking fds + a single-threaded poll loop.
- **M1 debug protocol:** one LF-terminated command per connection on
`$XDG_RUNTIME_DIR/muxd-debug.sock`, EOF-delimited reply. Throwaway;
M2 replaces it and claims `muxd.sock` for the real protocol.
- **TERM=xterm-256color** in the child, not xterm-ghostty: terminfo
availability beats capability advertising for a prototype.
- **Scrollback: engine-native.** ghostty-vt's max_scrollback (10k lines)
is the ring buffer; no separate structure in muxd.
- **Poll-loop lesson:** a pipe/FIFO stdin at EOF reports POLLHUP without
POLLIN; the loop must attempt a read on any revents or it busy-loops at
100% CPU. Found during the M1 demo, fixed and verified at 0% idle.
## 2026-08-07 (M2)
- **Snapshot = canonical VT state serialization.** `TerminalFormatter` with
`extra = .all` (palette, modes incl. active screen, cursor, styles), plus
a trailing CUP appended by `dumpState`: upstream emits DECSTBM (homes the
cursor) and tabstop HTS walks *after* the screen section's CUP, so the
dump's final cursor position is wrong without it. The client rebuilds its
replica with fullReset + feed. State sync via the engine's canonical
form — not a replay of session history.
- **Render dump is side-effect-free.** The client paints `dumpVt` with
`extra = .none` (content + inline SGR only): the default `.styles` extra
emits the full OSC 4 palette, which would overwrite the host terminal's
theme on every repaint (~5KB/frame).
- **Alt-screen snapshot limitation:** the snapshot carries only the active
screen's content. A replica that leaves the alt screen (TUI exits after a
reattach) reveals a blank primary screen instead of the shell history.
Locked by an engine test; M3 owes a two-screen snapshot or lazy fetch.
- **Wire format: 1-byte type + u32 LE length frames, M2 only.** The
handoff's "msgpack or protobuf, do not invent one" is owed at M4, where
structured payloads (damage regions, cell runs) first appear; every M2
payload is a byte blob or two u16s. Recorded so M4 doesn't inherit this
by inertia.
- **Single interactive client + dump-only observers.** Unattached
connections may `debug_dump` (keeps `muxd dump` working alongside a live
client). A second `attach` *takes over*: the old client gets `taken_over`
and exits with a message. Chosen over refusal after real use showed
refusal exits silently and stale clients wedge the session. M5 replaces
this with real multi-client.
- **Daemon shuts down cleanly on SIGINT/SIGTERM** (removes socket file,
reaps shell). Found via first-touch use: Ctrl-C on a foreground `muxd
run` left a stale socket, making every later `mux`/`dump` fail with a
confusing connection error.
- **Blocking frame I/O.** One local client on a Unix socket; a stuck client
can stall the daemon. Buffered nonblocking I/O is owed by M4 (network).
**Superseded in M6** for the write half (queued per-client sends with a
cap); reads stay blocking and POLLIN-gated, deliberately — see the M6
section.
- **Detach chord: Ctrl-\ (0x1c).** No keybinding layer in the prototype.
**Superseded** once session commands needed room: in a live session
Ctrl-\ is a prefix and the key after it selects the command (`d` or a
second Ctrl-\ detach; anything else is dropped with the prefix, which
costs nothing — a literal 0x1c never reached the pty). A bare press
still aborts a dial or a reconnect, where there is no session to
command.
- **Render: home + ED(2) full repaint under CSI ?2026 sync.** Wasteful by
design (deltas are M4); sync-output makes it artifact-free on modern
terminals.
- **Degenerate sizes rejected.** A pty can report 0x0 (`script` with piped
stdin); a zero-sized grid trips engine asserts. Client treats <2x2 as
"unknown" (falls back 80x24); daemon ignores <2x2 attach/resize.
- **systemd socket activation supported** via LISTEN_PID/LISTEN_FDS (fd 3);
units in contrib/. Verified with systemd-socket-activate.
## 2026-08-07 (M3)
- **Scrollback addressing: screen-space row index** (0 = oldest retained
row). Positions drift as history evicts at max_scrollback; scroll
positions are ephemeral, so drift is acceptable. Fetch is pull-only:
snapshots carry a u32 history_rows count, never history content.
- **Scroll UX: Shift+PageUp/PageDown** (`\x1b[5;2~`/`\x1b[6;2~`), page at a
time, remote-paged with no client cache (a page fetch is ~2KB over a
local socket). Any other key snaps back to live and is swallowed. Live
snapshots keep applying while scrolled but paint only on return.
- **Dual-screen snapshots.** When the alt screen is active, dumpState
prepends the primary screen's visible content before the full-state dump
(whose mode section performs the alt switch). Primary saved-cursor lands
at end-of-content, not the exact pre-TUI position — accepted.
- **SIGPIPE ignored in both processes.** A kill -9'd client previously
killed the daemon via SIGPIPE on the next snapshot write — found by the
M3 kill-reattach scenario, now covered by a server test and e2e.
- **SGR canonicalization note:** the formatter emits palette colors in
indexed form (31 -> 38;5;1). Semantically identical; tests assert the
canonical form.
## 2026-08-07 (M4)
- **Deltas are row-granular.** Damage = set of viewport rows whose styled
dump hash (Wyhash) changed, sent as self-contained row repaints
(CUP+EL+content composed by protocol.composeDelta on the client). Chosen
over cell-level damage rects: rows are the natural unit of the canonical
formatter, and the measured win already clears the kill criterion.
- **Wire format: still hand-rolled, deliberately.** The handoff says
"msgpack or protobuf, do not invent one"; M4's payloads are row-keyed
byte blobs plus fixed-width integers, and no serialization library is
proven on Zig 0.15.2. The judgment flips when payloads become truly
structured (cell runs, multi-rect damage, negotiation) — msgpack is due
then, and this entry is the tripwire.
- **Snapshot-vs-delta threshold:** a client is served a delta iff its
have_seq is >= the tracker's reset_seq (last resize/screen-switch/init
discontinuity), <= the current seq, and the viewport size matches;
otherwise snapshot. Per-row last-change seqs advance even while
detached — once per coalesced PTY read burst; nothing at all while idle
— so reattach-after-a-gap resolves by delta when nothing discontinuous
happened. All OOM paths in the delta machinery collapse to snapshot
resync (retry-until-works, no flags).
- **Diffing is content-hash based,** not ghostty RenderState dirty
tracking: deterministic, no new engine API surface, O(rows) styled row
dumps per coalesced update. RenderState remains the optimization path if
hashing ever shows up in a profile.
- **row_count is authoritative and validated.** composeDelta rejects any
delta whose header count disagrees with the rows present; a client
receiving a malformed delta resyncs via re-attach rather than skipping
it (a skipped delta would desync the replica permanently).
- **DECOM is a known gap.** Origin mode (CSI ?6h) is not treated as a
discontinuity, and under it the replica's row addressing would shift
relative to the daemon's. Rare in practice — vim/less/htop are
unaffected because CUP is absolute under DECSTBM — so it is recorded
rather than fixed.
- **Measured (bench.sh, 120-char typing workload):** delta bytes were
7751 against a snapshot-equivalent of 689053 — 1%. The workload
produced exactly one delta per typed character (120 deltas, ~65 bytes
each) against one initial snapshot. The M4 kill criterion (delta
traffic must scale down meaningfully) is cleared.
- **The win survives 100% damage.** Adversarial check: sustained
full-screen scrolling (`seq` over 400k lines, all 24 rows changing
every update) still measured ~366-byte deltas against ~5775-byte
snapshot-equivalents, ~6%. Row repaints beat snapshots even with no
unchanged rows to skip, because a snapshot also carries palette and
mode state that a row repaint does not.
- **No session epoch in the protocol.** have_seq is meaningful only
within one daemon instance; a client reconnecting to a restarted daemon
with overlapping counters would accept a delta belonging to a different
session. Unreachable today — the shipped client always attaches with
have_seq=0 — but a generation/epoch field is due before any real
network transport. **Superseded in M6:** attach and the snapshot prefix
now carry an epoch; the M6 section records the design.
- **Stats counterfactual costs a dumpState per delta send** while a client
is attached — deliberate prototype instrumentation; gate behind an
option if it ever matters for performance comparisons.
- **Banked cleanup (post-M4), from the review cycle:** shared frame
applier in protocol.zig so the server fidelity test exercises the real
client apply path; Client struct decomposition of client.zig's attach
loop; resync_pending bound on the client's malformed-delta re-attach;
consistent malformed-frame policy (short snapshots currently ignored,
bad deltas resync); timer-based test deadlines + generalized poll
helper in server tests; DeltaTracker split into src/delta.zig; naming
(resyncSnapshot/sendResync, DeltaTracker.update); paintDelta golden
test; putU32/getU32 now-dead exports; client-side got_state refusal
branch lacks automated coverage; serviceClient's resync guard asymmetry;
automated coverage of the resize trigger for latest-wins with two
clients attached (the attach trigger is pinned by a test, the resize
trigger only by the manual demo).
## 2026-08-07 (M5)
- **Resize policy: latest wins.** The authoritative grid follows the most
recently *active* client: attaching, resizing, or typing claims it
(modern tmux `window-size latest`). **Amended post-M6 (2026-08-07):**
until then the triggers were attach and resize only, and real two-client
use showed the tmux comparison overpromised — tmux's "latest" follows
the active client, so typing at a console it had not resized left our
grid at the other client's size. Input is now a trigger, applying the
size and broadcasting down the same discontinuity path a resize takes:
one broadcast on the first keystroke after a switch, nothing on the
keystrokes after that, since the sizes then already match. A size the
daemon refuses (degenerate, or an engine resize that fails) is neither
recorded as that client's size nor broadcast — the grid never moved, so
there is nothing to repaint anyone for, and a client whose size was
never accepted claims nothing when it types. Scroll
fetches are deliberately excluded from "activity" — paging history from
a small terminal must not yank the grid away from the client actually
working; nor are stats/dump/detach, which are not somebody using the
terminal. Rejected: smallest-wins (punishes the larger
screen for the smaller one's presence — the handoff calls the result
"widely disliked"); per-client reflow (needs a per-client engine or
reflow pass, contradicting the single-authoritative-grid architecture —
reconsider post-prototype only with a concrete need). Non-matching
clients render the grid clipped: autowrap off (DECAWM), rows beyond the
tty skipped, cursor clamped, no border art. Snapshots carry the grid
size (16-byte prefix); size changes always travel as snapshots, so
deltas stay size-free.
- **Multi-client: broadcast, up to 8.** Every attached client receives
every delta/snapshot; per-client write failure drops that client only.
Takeover is retired — attach joins; a full session refuses with
exit_status{1}. The taken_over frame stays in the protocol but is now
sent by nobody. This is not for wire-compat (binaries are lockstep, per
the prefix change below): the type byte is simply not reused, and the
client's handler for it is dead code that costs nothing to leave in.
- **Client-local view state is per-connection by construction** — scroll
mode lives in the client, scrollback fetches are served per-fd — and
pinned by a server test (one client pages history while the other
streams deltas).
- **Stats under broadcast:** delta_bytes/snapshot_bytes count actual
per-client sends; snapshot_equiv_bytes accrues once per update event
(unicast join snapshots also accrue). The bench (single client) is
unaffected.
- **Known costs and edges, accepted for the prototype:** blocking
per-client writes mean one stalled client can head-of-line-block all
others (fix = buffered/nonblocking writes, owed before any network
transport). **Superseded in M6:** client sends are queued per client
and flushed non-blocking, with a cap that drops a stalled peer instead
of waiting on it; the M6 section records the design. Simultaneous
connects funnel through 4 observer slots before promotion, so >4
clients connecting in the same instant see unexplained EOFs
(connect-then-confirm serializes fine in practice). A
client's corrupt-delta resync re-attaches at its own size, which — by
latest-wins — resizes the session for everyone. Scroll paging divides
daemon history rows by the local tty height, so page counts are
approximate when sizes differ.
- **Snapshot prefix widening (12→16 bytes) is a breaking wire change**;
binaries must upgrade in lockstep (daemon restart on upgrade).
Consistent with the recorded no-epoch gap.
- **Daemon lifetime — resolved, closing the M2 open item, with the
reboot half answered honestly.** Sessions survive logout; they do not
survive reboot. The user units in `contrib/` cover the logout case:
`muxd.socket` starts the daemon on first connect and survives client
exits, and `loginctl enable-linger $USER` keeps the user manager alive
across logout and starts it again at boot. But no process survives a
reboot — after one, the socket unit spawns a *fresh* daemon and a fresh
shell, and the previous session's grid, scrollback, and child processes
are gone. Carrying a session across a reboot would need
checkpoint/restore, which handoff §6 explicitly defers as a different
problem (process location, not terminal rendering).
## 2026-08-07 (M6)
- **Client writes are queued and never stall the daemon.** Each client
slot carries a pending byte buffer; sends `appendFrame` into it and
flush opportunistically with `MSG_DONTWAIT | MSG_NOSIGNAL`, with
POLLOUT requested only while the queue is non-empty. A queue past
`pending_cap` (8 MiB, a `Server` field so tests can shrink it) drops
the client rather than waiting on it: one stalled WAN peer must not
head-of-line-block the session, and unbounded buffering would trade a
stall for an OOM. Stats count bytes when *queued*, so "sent" now means
"accepted into the queue" — the cap is what bounds the lie (no client
can be more than one cap ahead of the truth). Reads stay blocking and
POLLIN-gated, deliberately: input frames are tiny and the observed
hazard is entirely write-side (snapshots). Observer replies stay on
blocking `writeFrame` — they are local one-shot tools.
- **exit_status is queued like everything else, then drained under a
250ms budget** at shutdown. It is the one frame with nothing behind it
to carry it out, so the queue alone would lose it; the bounded drain
delivers it unless the peer is fully stalled, and refuses to hang the
daemon's exit on a peer that is. A client that misses it still learns
the session ended, from EOF.
- **Session epoch: a random nonzero u64 per daemon instance**, 0
reserved for "none". It rides in attach (now 20 bytes: cols, rows,
have_seq, have_epoch) and in the snapshot prefix (now 24: seq,
history, cols, rows, epoch). The delta fast path requires
`have_epoch == self.epoch` as well as a serviceable seq, so a client
can never be served a delta belonging to a different daemon instance.
**Deltas deliberately carry no epoch**: a delta stream cannot outlive
the instance that opened it — the connection dies with the daemon, and
the proxy exits rather than reconnecting for exactly this reason — so
the only place an epoch can be checked is where a client asserts what
it already has. Distinctness is pinned by a daemon-restart test, not
by trusting the RNG in prose. Both payload widenings are breaking wire
changes; binaries upgrade in lockstep, consistent with the standing
no-negotiation policy.
- **`muxd proxy` + `mux --via CMD`: the transport is an opaque byte
pipe.** `muxd proxy` is a bidirectional pump between its stdio and the
local `muxd.sock` that imports only `std` — no protocol import, no
frame parsing, and the build graph enforces it. `mux --via CMD` spawns
`/bin/sh -c CMD` and speaks the existing framed protocol over the
child's stdin/stdout (stderr inherited, so ssh's errors reach the
user). Any command that exposes the daemon socket over stdio works;
ssh is just the first one.
### The transport verdict
**"Transport is a swap, not a redesign" — HELD, structurally and by
measurement.**
Structurally: the protocol crossed to a remote link with *zero* framing
changes. `readExact` already looped, so arbitrary chunk boundaries —
which an SSH channel produces constantly and a Unix socket almost never
did — needed nothing. Payloads stayed unstructured, so the M4 msgpack
tripwire did not trip. The only *protocol* work in M6 was the two debts
the log had already recorded as owed before any network (queued writes,
epoch).
The network did force two client-side changes, named here first because
they are the counterexamples the claim has to survive. Neither touches
the wire. (1) **Torn and EOF'd frames now report "connection to muxd
lost"** instead of surfacing an error trace: over a Unix socket a
mid-frame tear was near-unreachable, and over ssh it is the ordinary
failure — the transport dying mid-snapshot is a message, not a crash.
(2) **Alt-screen entry is deferred until the first frame arrives.**
Entering at setup would erase whatever the `--via` command wrote to its
inherited stderr — ssh reports auth and connection failures hundreds of
milliseconds after spawn — and would blank the screen for the whole of a
hang like `mux --via "sleep 30"`. Both are the client learning that its
transport can now fail slowly, visibly, and in someone else's words.
That is a UX consequence of distance, not a redesign of the protocol,
which is exactly the distinction the verdict claims.
Empirically, measured by `test/wan.sh` against a real WAN box through an
ssh jump host (figures in milliseconds, medians):
- **Link baseline** (raw byte round-trip through `ssh cat`, warm
channel): **15.7–16.5ms**.
- **Keystroke echo through the whole stack** (ssh → proxy → daemon →
pty → bash echo → engine → delta → proxy → ssh → client paint):
**baseline + ~4.2ms**. That figure is *constant and
RTT-independent* — under `netem delay 75ms loss 1%` the baseline rose
to 91.3ms and echo to 95.4ms, the same ~4ms of protocol cost on a link
nearly six times slower. The protocol adds a fixed pump tick, not a
multiple of the round trip.
- **Reattach after `kill -9`**: wall clock **56.7–59.5ms** on the fast
link, decomposing into a **35.3–37.5ms** ssh channel-open/exec floor
(measured, not assumed — `viafloor` in the harness) and a
**19.8–22.0ms protocol share** — **1.2–1.4× round-trip** on the clean
link. Under netem the same share works out to **~1.05×**, because the
protocol's cost is roughly fixed while the round trip it is measured
against grew; the clean-link ratio is the one to quote, being the
larger and therefore the harder test.
- **State correctness over the WAN**: the pre-kill marker was present in
the *first paint* of every rep, clean link and netem alike.
**Kill criterion (a), structural — PASS.** The remote path works with
the proxy containing zero protocol knowledge.
**Kill criterion (b), measured — PASS on both halves.** Echo: 4.2ms
against a budget of baseline + 120ms, margin **+116ms**. Reattach: the
protocol share, 1.2–1.4× round-trip on the clean link (~1.05× under
netem), against a 2× budget.
**The reattach ruling, recorded so the gate is never mistaken for a
softened threshold.** The plan's wording is "within ~2×RTT *of the
attach request*", so the criterion governs the cost the protocol adds.
The wall-clock reading fails on any fast link *regardless of protocol
design*: ssh's channel-open-plus-exec floor alone measured 2.2–2.4× the
round-trip on this link, past the entire budget before one protocol byte
moves. A gate that fails no matter what the code does cannot falsify
anything, so it is not the gate.
**The order this happened in, disclosed because it is the kind of thing
a reader should not have to reconstruct:** the harness gated the strict
wall-clock reading first, and those runs printed FAIL. The gate was
changed to the protocol share *after* and *because of* the `viafloor`
decomposition, which is what showed the failure to be ssh's channel
setup rather than anything the protocol did. A threshold moved after
seeing a red result deserves the suspicion it attracts; what defends
this one is that the decomposition is itself measured, the wall clock is
still printed and still reads FAIL on a fast link, and the reasoning
above holds independently of which number anyone hoped for.
Both readings stay on the record — `wan.sh` prints the wall clock and
its decomposition and gates on the protocol share — because the wall
clock is what a user actually waits through. Closing that gap means
killing the per-attach channel setup,
not the protocol: QUIC 0-RTT or a reused connection is the recorded
path, and it is transport work, which is the verdict restated.
- **Proxy write-side head-of-line blocking — known, measured,
accepted.** The proxy is a byte pump with no notion of priority and no
write buffering of its own, so a flooding session shares the pipe with
a keystroke's echo. Measured under a ~100 deltas/s flood (one
full-screen-scrolling line every 10ms): echo degraded by +0.5–5ms on
the clean link and +3–15ms under netem. It is bounded rather than
unbounded because the daemon's `pending_cap` terminates a hopeless
client honestly instead of queueing forever. The fix, if it ever
matters, is buffered non-blocking writes in the proxy — the same
change the daemon just made.
- **Ctrl-C had been dead in every script-started session since M1 —
found by the WAN run, not by any test.** A non-interactive shell sets
SIGINT/SIGQUIT to `SIG_IGN` for anything it backgrounds with `&`,
which is how every script starts the daemon (`e2e.sh`, `wan.sh`, any
deploy). `SIG_IGN` is the one disposition that survives `exec`, so it
rode through forkpty into the session shell, and a shell keeps
signals ignored-on-entry ignored for the jobs it spawns. Symptoms
looked healthy the whole way down: `isig` on, `^C` echoed, foreground
process group correct — and `sleep 300` immune. Fixed by resetting
both to `SIG_DFL` in the pty child before `exec`. The test aims the
signal at a *job of* the session shell, not at the shell itself: an
interactive shell catches SIGINT to abandon the current line, so it
abandons the marker either way and cannot discriminate — that form was
verified to pass against the unfixed code. **SIGPIPE is reset in the
same place for the same reason**, as hardening rather than a fix: the
daemon ignores SIGPIPE for its own sockets, and today only
initialization order keeps that ignore from riding into session shells
by the identical survives-exec route. Resetting it beside INT and QUIT
makes the property hold regardless of order, which is what the SIGINT
bug taught — a disposition inherited across exec is a latent defect
whether or not the current call order happens to hide it.
- **Two test limits, recorded rather than papered over.** (1) The
proxy's 300KB backpressure test pins byte-exact transfer in both
directions but *cannot* kill a partial-write mutation: blocking
stream-socket writes never return short, so the `writeAll` loop is
guarding signal interruption, not partial writes, and the mutation has
no observable behaviour to catch. (2) Zig's runtime installs a noop
SIGPIPE handler by default, so the explicit `SIG_IGN` installs are
defence in depth — the EPIPE error paths are reachable without them.
What the explicit ignore really buys is that `SIG_IGN` survives exec
and a handler does not (which is why `client.zig` installs its ignore
only *after* spawning the transport child). `std.options.keep_sigpipe`
staying at its default is a dependency nothing pins.
- **The WAN box is ephemeral and lives entirely in the environment**
(`MUX_WAN_SSH`, `MUX_WAN_SCP`, `MUX_WAN_HOST`, `MUX_WAN_NETEM`), never
hardcoded in a committed file. `wan.sh` refuses with a usage message
when they are unset.
- **Banked cleanup (post-M6):** proxy write-side head-of-line blocking
(buffered proxy writes); no test covers the `--via` child's lifecycle
(kill/wait on exit, ssh dying mid-session); client reconnect-with-
epoch logic — `session_epoch` is parsed from the snapshot prefix and
then unused, so the epoch machinery is only half-exercised by the
shipped client; `std.net.Address.initUnix` is bounded by `sun_path`
(108 bytes) regardless of the 280-byte buffers around it, so the test
suite is unrunnable from a checkout nested deeper than that allows —
this bit two independent reviewers, which makes it a real defect and
not a footnote; `wan.sh`'s netem stanza adds delay on egress only, so
`delay 75ms` is ~75ms of added round trip and not ~150ms (the script
says so; the plan's parenthetical said otherwise).
- **`muxd run` refuses a live socket rather than stealing it (post-M6
field fix).** The incident: three daemons were started against one
path, each unlinking it and binding fresh. None of them died — the
older two kept running with their sessions and shells intact but
permanently unreachable, and the user's two terminals were attached to
two *different* sessions, which made every cross-client behaviour
(latest-wins, shared output) look broken. Init now probes the path
first: something answers → `error.DaemonAlreadyRunning` and no unlink;
nothing there → bind; a socket file nobody answers on → a dead daemon's
leftover, unlink and rebind. The rule is **unlink only what answers
ECONNREFUSED *and* stats as a socket** — necessary because Linux
answers ECONNREFUSED for a *regular file* at the path exactly as it
does for a dead socket, so the connect result alone would have made
`muxd run --sock notes.txt` delete notes.txt. The probe is skipped
under systemd socket activation, where the path is systemd's to manage,
and it runs before the pty is spawned so a refusal costs no fork. This
is the complementary half of deinit's existing unlink-only-if-still-ours
check: one guards the path on the way in, the other on the way out.
- **`AddressInUse` gets the same one-liner as a live socket.** Two daemons
starting at once can both see an empty path and both try to bind; the
loser has simply lost a dead heat, and by the time it reads "a daemon is
already running" one is. Knowingly, that message is also what a
*dangling symlink* at the socket path produces — connect through it
gets ENOENT so the probe reads the path as free, then bind gets
EADDRINUSE off the directory entry the symlink occupies. That is the
deterministic construction the test uses, and the wrong-ish message is
an accepted trade: distinguishing the two would need a
SYMLINK_NOFOLLOW re-stat purely to reword a rare operator mistake, and
the advice ("that path is not yours to take") is right either way.
- **Banked cleanup (post-M6, from the socket-claim review):** the systemd
`LISTEN_FDS` path has no automated test proving the probe is skipped
(verified by hand only). `claimSockPath` still stack-traces on
AccessDenied / NameTooLong / an absent parent directory — cosmetic, the
same one-line treatment `DaemonAlreadyRunning` gets would do. The
stat→unlink window is a TOCTOU race, accepted as non-adversarial: a
socket path a hostile process can swap under us is already a directory
we do not control, and closing it properly means a tmux-style lockfile
beside the socket, not a cleverer stat. That `fstatat` FileNotFound arm
is consequently unreachable from any test — it exists for that window
alone.
- **Owed before this is more than a prototype, in order:** TLS or QUIC
for any deployment that is not tunnelled through SSH — `--via` borrows
ssh's authentication and encryption entirely, and has none of its own;
**reconnect-with-epoch** in the client, which is what turns the epoch
from a correctness fence into a feature (resume the stream instead of
re-attaching from zero) and is also what would let a transport survive
a blip; and **prediction / local echo**, deferred since the handoff and
now holding a measured baseline to be judged against — handoff §5 lists
it as post-prototype and conservative by default (predict in line mode,
fall back to server-confirmed inside TUIs), and the ~4ms of protocol
cost measured here is what a predictor must beat to be worth its
complexity on a link this good.
## 2026-08-07 (M7)
- **Kill criterion: PASSED on the real WAN box.** 10 consecutive transport
kills against a live session over ssh (~16.5ms round trip, via a jump
host): 10 of 10 resumed with zero manual action, and the remote daemon's
`snapshots` counter did not move across any of them (11 → 11) — every
resume was delta-served. First frame after a tear: min 254.0ms, med
254.5ms, max 255.3ms (n=10). The M6 criteria still pass in the same run
(echo med 20.9ms against a 136.5ms budget; reattach protocol share
22.0ms against 33.0ms). Re-confirmed on a second run after the
convergence clause below was narrowed and partly closed: 10/10 again,
snapshots 5 → 5, first frame med 256.4ms, state-survival check passed.
- **The criterion was NARROWED, and this is the record of it.** The plan
named four elements; the fourth — "replica converged after the last tear
(client render matches `muxd dump`)" — is **not** what the harness
checks, and the first version of this record restated the criterion
without saying so. What was substituted: each tear round-trips a marker
out to the remote shell and back through the rebuilt transport, and
(added when this was caught) after the last tear the **first** tear's
marker must still be in the daemon's grid — accumulated state survived
all ten resumes. Why: the client emits a stream of paints, not a grid, so
comparing its render to a dump means running a second VT parser inside
the measurement harness, whose own bugs would be indistinguishable from
the ones it exists to catch. That comparison IS made against a real
replica Engine in src/server.zig's tests — over a socket, not over the
WAN. The substitute is weaker than the plan's wording. **Full
convergence-over-the-WAN stays banked**, and the honest one-line summary
of M7 is "done, with the convergence clause narrowed on the record".
- **That 254ms is nearly all our own first backoff.** It decomposes as
~200ms of client backoff + 36.4ms of ssh channel open (`viafloor`,
measured in the same run) + ~18ms of protocol. The wall clock is printed
and not gated for the same reason the M6 reattach ruling gives — gating
it would rule on ssh, not on this design — but the split is worth
keeping in view: **a shorter first retry would cut resume time roughly
fourfold on this link**, and nothing measured here argues against trying
it. That multiple is link-specific and must not be quoted as universal —
it is the ratio of a fixed 200ms backoff to a fast link's costs, so on
the +75ms netem profile the same change is worth roughly 2x, and on a
slow enough link it rounds to nothing. Banked rather than tuned, because
200ms was chosen for a link that might be flapping and this run cannot
tell us how it behaves on one that is.
- **Transport death is a non-event; transport *absence* is not.** The
client keeps its replica, rebuilds the transport from the recipe it was
launched with, and re-attaches quoting `(have_seq, have_epoch)`; the
daemon's existing `sendResync` decides delta-or-snapshot. Retry is
uniform and generous with no cap, split on one thing only: **whether a
session was ever established** (`session_epoch != 0`). A transport that
dies before the first snapshot carried no session — a typo'd hostname, a
remote muxd that is not there — and exits immediately with the old
message. Retry policy is deliberately NOT split on transport type: a
dropped ssh link and a killed local daemon are the same event to a
client, and the abort key plus the indicator answer both.
- **Policy decisions, recorded because they are choices and not
consequences:** input typed while disconnected is **dropped**, not
queued (replaying a burst of stale keystrokes into a shell is worse than
losing them; prediction is the principled fix, and it is a later
milestone). A reconnect landing on a **different epoch repaints silently**
— new epoch means new session, and a fresh shell is self-evident without
a modal "session lost" state. An attach **refusal during a reconnect** is
retried for ~5s (the daemon may not have reaped our own dead
predecessor's slot yet); a refusal on first attach still exits loudly.
Ctrl-\ during a reconnect exits 0 with "detached while reconnecting" —
the user chose to leave and the session really is still running.
- **`session_epoch` graduated from parsed-but-unused**, closing the M6
banked item: it is now the value that lets a reconnect resume a stream
instead of re-attaching from zero, and the value that makes "never
established" expressible at all.
- **`mux HOST` is sugar for `--via "ssh HOST muxd proxy"`** — the VM-hop
use case in one word. Nothing parses inside HOST, so ssh's own config
(aliases, ports, ProxyJump) keeps working. **It presumes a compatible
muxd on the remote PATH and there is no version negotiation anywhere in
this protocol: mismatched binaries are undefined behaviour, not a
handled error.**
### Process rules this milestone paid for
- **build.zig test discovery is a LIVE hazard.** Tests only run for modules
listed in the `test` step's loop. `mux_main.zig` was not in it, so its
five new parser tests compiled and silently never ran — the suite
reported 79/79 green with them absent, and 84/84 once the module was
added. **`src/main.zig` (muxd's entry point) is still not in that list**,
so the first test anyone adds there will silently not run. Adding a test
file is not enough; check the module is in the loop, and check the test
count moved.
- **Counters are asserted because markers are blind to *how* a resume was
served.** A snapshot-served resume renders identically to a delta-served
one, so `snapshots` is the only witness — this is now proven three times
by mutation (a client reconnecting with `have_seq=0` passes every marker
assertion and fails only the counter). The restart scenario's mirror
assertion (`snapshots >= 1` on the *new* daemon) needs **both** the epoch
fence and the seq-range check removed before it fires, because
`buildDeltaSince` has no out-of-range guard of its own: a stale seq
yields a silent zero-row delta that renders fine. It is deliberate
defence in depth against those two guards ever being collapsed into one,
and it is recorded here so the next reviewer does not repeat the
"can't mutate it, is it dead?" cycle.
- **Never select processes by command-line pattern — anywhere in this
project, scripts or ad-hoc operations alike.** Not a test-suite
convention: a standing rule, promoted after a fourth sighting. The reason
is structural rather than incidental — this project's command lines
*contain each other*. `mux --via "ssh host muxd proxy"` carries the proxy's
command line inside its own argv; an ssh invocation carries the entire
remote script, including whatever pattern that script is about to match.
So a pattern that describes the intended victim also describes the process
doing the killing, or the one under test. The four sightings:
1. e2e Scenario A — `pkill -f "muxd proxy"` would have matched the client
under test; caught while writing it, and the reason `proxy_pid` selects
by `comm` + argv.
2. M7 manual smoke test — `pkill -f "muxd proxy --sock …"` SIGTERM'd the
very client being tested (exit 143), which briefly looked like a
product bug.
3. M8 spike cleanup on the box — `pkill -f "quicecho-$TAG server"` matched
the ssh command line carrying that same text and killed the cleanup
script before it ran `tc qdisc del`, leaving netem installed on a
shared machine until it was noticed.
4. Independently hit in review.
The replacement is always the same: select by `comm` plus a pid, or by
parent pid. The remote-side equivalent is `wan.sh`'s bracketed-pattern
trick, which works for the same reason — it breaks the self-match.
- **The never-established gate has e2e-only coverage.** The property is
process-level (a client that must exit rather than loop, with no tty to
rescue it), so it cannot be unit-tested honestly. `make e2e` is therefore
a **permanently required gate**, not an optional one: with unit tests
alone, a regression to the infinite-retry behaviour ships green.
- **Mutation testing here has a stale-binary trap.** `make test` does not
rebuild the `mux`/`muxd` binaries, and a failed build leaves the previous
ones in place, so a mutation run can report a confident false green
against code that was never compiled. Always assert the build's exit
status and the binary's freshness before believing a mutation result.
- **Banked (M7 additions, carrying the M6 list forward intact):** a shorter
first reconnect backoff (see above); repeated `--via` is untested while
repeated `--sock` is covered — an asymmetry in the parser tests, not in
the parser; no version negotiation between `mux` and a remote `muxd`
(see above); systemd socket-activation packaging for the VMs (the unit
files exist in `contrib/`, an install doc does not); the reconnect loop
has no upper bound on total retry time by design, so a client left
attached to a permanently dead daemon retries until the user aborts.
- **Out of scope, in order:** QUIC/TLS transport (M8, done), then
prediction / local echo — which now has this milestone's numbers to beat
as well as M6's ~4ms protocol cost.
## 2026-08-08 (M8)
QUIC as a third transport arm: `muxd run --quic HOST:PORT --key FILE` and
`mux quic://HOST:PORT --key FILE`. ssh stays the default; QUIC is opt-in per
invocation and `mux HOST` still means ssh.
### The four driving decisions
- **PSK, not certificates.** The deploy model is "scp a static binary to a
VM you own", so scp a 32-byte key beside it. TLS 1.3 external PSK gives
mutual authentication by possession with no CA, no TOFU, and no expiry to
manage. Certificates and a TOFU store stay banked.
- **ssh stays the default.** QUIC is faster to set up and it is one more
thing to configure; making it opt-in costs nothing and means a broken
QUIC build can never break `mux HOST`.
- **Zero protocol changes.** Frames, attach/prefix layouts and seq/epoch
semantics are untouched. `git diff b646277..HEAD -- src/protocol.zig` is
empty and is leg 4 of the kill criterion — the swap thesis, third pass.
- **Spike-gated.** No mature pure-Zig QUIC exists, so a C stack had to
prove it builds under the pinned Zig 0.15.2 for native *and* static musl
before anything else was planned.
### The spike, and what it settled
ngtcp2 1.25.0 + wolfSSL 5.9.2 build under `zig cc` for both targets and add
~1.68MB to a static binary. Two findings outlived the spike:
- **External PSK is already 1-RTT.** The handshake completes in one round
trip without a session ticket, so resumption buys nothing over a fresh
PSK connection. That left 0-RTT as the only remaining reconnect win —
and 0-RTT turned out to be unavailable (below).
- **The keylog ban is a standing build rule.** Built with
`WOLFSSL_KEYLOG_EXPORT` on, the binaries wrote every handshake's secrets
to `./sslkeylog.log` with nothing asking them to. It is off in
`deps/quic/build-deps.sh` permanently; turn it on only in a local
throwaway build, and never commit that.
### 0-RTT: dropped on evidence, not on effort
The plan named 0-RTT as the reconnect win. It cannot be had from where we
are, and the evidence is worth keeping because the next person will ask:
- It is not compiled in. The built `options.h` has no `WOLFSSL_EARLY_DATA`
and `nm -g libwolfssl.a` finds zero early-data symbols.
- It cannot be enabled through cmake. wolfSSL 5.9.2's CMakeLists exposes
118 `WOLFSSL_*` options and mentions EARLY zero times; early data exists
only as the autotools flag `--enable-earlydata`, default off.
- Forcing the define would be worse than not having it. It is not a pure
feature gate: it adds `maxEarlyDataSz` to WOLFSSL_SESSION, a
`clientInEarlyData` bitfield to Options, a member to the ticket struct
and a TLSX enum value. Compiling the library with it while the installed
`options.h` lacks it is a silent ABI mismatch — memory corruption, not a
link error. A latency optimisation is never worth that.
- And external PSK is the wrong door anyway: the client-side gate reads
`session->maxEarlyDataSz`, populated from a session ticket's early_data
extension. An external PSK carries no ticket, so there is nothing to
read. Getting 0-RTT means adopting ticket-based resumption on top of the
PSK — a different design, not a flag.
ngtcp2 exports four `early_data` entry points; it is TLS-agnostic and has
no backend here to serve them. Banked: autotools wolfSSL with
`--enable-earlydata` plus ticket resumption.
### The bugs this milestone paid for
- **ngtcp2 does not copy stream payload, and we assumed it did.**
`ngtcp2_conn_writev_stream` stores the *vector* it is handed —
`ngtcp2_vec_copy` is a memcpy of base+len, not of the bytes — and
re-reads those bytes to retransmit. The first egress buffer reallocated
on append and cleared as soon as the last byte had been handed over, so
ngtcp2 was left holding freed or recycled memory and died inside
`ngtcp2_pkt_encode_stream_frame`. It fired about three runs in seventy,
only in the one test big enough to overflow a socket buffer.
**The fix is an invariant, not a patch:** a byte handed to ngtcp2 does
not move or get overwritten until the peer acknowledges it. `Egress` is a
fixed ring; `head` advances only from `acked_stream_data_offset`.
**The lesson:** a C library's ownership rules are part of its API, and
"it copied it" is the assumption to check first.
**Not an invariant, and do not treat it as one:** the ring does not lap
its in-flight region on loopback only because the 256KB ring happens to
be at least the 256KB stream window the peer advertises. That is
configuration. Raise the advertised window without raising the ring and
laps become reachable again — the lifetime rule is what stops that being
a use-after-free, not the arithmetic.
- **Closing a connection from inside a receive callback is a
use-after-free, and only a real network shows it.** ngtcp2 calls our
receive callback and then goes on using `conn`: after
`conn_call_recv_stream_data` it calls `conn_emit_pending_stream_data`,
which dereferences the connection again. The daemon's `.detach` handling
reaches `closeConn` from inside that callback, which freed the Conn
underneath ngtcp2. The LAN box crashed the daemon on the first QUIC
attach of the first run. Teardown is now deferred: `in_ngtcp2` marks the
window, `closeConn`/`kill` mark rather than free, and `feed` reaps on the
way out.
**It was never loopback-only, and the first version of this note said it
was.** `conn_emit_pending_stream_data` is called unconditionally
(ngtcp2_conn.c:7646), and its first statement dereferences the
connection — `conn_is_tls_handshake_completed(conn)` at :7220 — *before*
the `if (!strm->rx.rob)` early-out two lines below it. The freed
connection was therefore read on every receive, on every path, from the
moment the bug existed. What a reordering network changed was not
whether the read happened but whether it landed on memory the allocator
had already reclaimed, and so whether it faulted.
**The lesson is sharper than "loopback is not a network": a
use-after-free that reads successfully is exactly the kind that ships.**
Every test we had was exercising it and reporting green. The real
network did not introduce the bug, it withdrew the luck.
- **Migration was defeated, which is most of why QUIC is here.** `route()`
matched the destination CID against the one a connection was created
with, but `get_new_connection_id2` advertises more, and a client that
changes network switches to one of the others (RFC 9000 §9.5). Those
packets fell through to `accept()`, which has no token for them and drops
them: the migration presented as a dead connection. Connections now match
any CID they have advertised and not retired. The test tells a routing
hit from a miss by what a *miss* does — fall through to `accept()`, which
answers a token-less Initial with a Retry — so a probe socket that hears
nothing was delivered, and the same probe hearing a Retry for an unknown
CID is what proves the packet was well-formed enough to have been
answered.
- **One port, one daemon.** The listener set SO_REUSEADDR, which on UDP
lets a second daemon bind the same address while the kernel hands each
datagram to one of them — two sessions silently splitting a port. This is
the socket-steal incident from M7 in its QUIC edition, and it gets the
same answer: fail the bind loudly. The bind also moved ahead of the
session socket, so a refused port costs no shell and leaves no socket.
- **Every QUIC attach was snapshot-served.** The daemon's client-slot
`.attach` arm snapshotted unconditionally, on reasoning that held while
every client began life as an observer and did its first attach from
there. A QUIC connection is promoted to a client slot when its handshake
completes, so its *first* attach lands on that arm — and no reconnecting
QUIC client could ever resume from a delta. That is kill-criterion leg 1,
and **the counter was the only witness**: a snapshot and a delta render
identically. Fourth time a counter assertion has caught something markers
could not. Fixing it also improved socket double-attach into parity with
the promotion path.
- **The socket unlink guard compared inodes across filesystems.** `fstat`
on a bound unix socket's descriptor answers with the sockfs inode;
`fstatat` on the path answers with the filesystem inode. They can never
be equal, so the guard was false every time and the daemon never unlinked
its socket on a clean exit — masked since 45ab077 by the stale-socket
recovery cleaning up on the next start. **The lesson: a guard whose false
branch is indistinguishable from success needs a test that observes the
EFFECT.** "Refused to delete someone else's socket" and "failed to delete
its own" leave identical evidence.
### Things that are true and were not written down
- `closeConn` closes one connection and never the shared socket; it does
not call back into `onClose` (a close the owner asked for needs no
callback telling it so); and it sends no CONNECTION_CLOSE — the peer
learns via idle timeout. That last one is a real cost, accepted for v1
rather than unnoticed.
- `onOpen` and `onClose` do not pair. A handshake that never completes
produces neither: the connection is torn down having never been
announced.
- Flow control is extended before the owner consumes, which is sound only
because the owner's queue is bounded — it was not, until the egress ring.
- `max_conns` (16) exceeds `max_clients` (8) on purpose: a connection
exists from handshake completion and only then asks for a slot.
### `idle_ms` is doing three jobs
One knob currently serves death detection (wants seconds), the handshake
timeout (wants a small multiple of RTT), and reaping a connection that
handshook and never attached (wants to be long). That is why the abort key
mattered so much: the handshake bound is far longer than a handshake needs
to be, and `waitReady` polled only the transport, so for its whole length
nothing watched for Ctrl-\ — measured at 14.6s of deafness on the default.
It now watches stdin too and answers in ~200ms. Splitting the three is
banked.
Related policy, decided rather than defaulted: bytes typed during a
handshake are **carried** on a first attach (they are the user's opening
command, and dropping them silently eats the first line of any piped
session) and **dropped** during a reconnect, matching the long-standing
rule that input typed while disconnected is not replayed.
### Process rules this milestone paid for
- **The build.zig test-discovery hazard is CLOSED.** `src/main.zig`'s
module is in the test loop; removing it again drops the count from 100 to
94. Every module with tests is now in that list.
- **A test that hangs on regression is worth much less than one that
fails.** Restoring SO_REUSEADDR made the double-bind scenario's second
daemon bind and run forever; the invocation had no `timeout`, so the
suite hung for ten minutes instead of failing. Every refusal path now
runs under `timeout`, and the same mutation reports "exit 124, want 1" in
ten seconds. CI would have read the old behaviour as an infrastructure
timeout rather than a bug.
- **`sun_path` is retired as a hazard.** `std.testing.tmpDir` lives under
`.zig-cache`, so a socket path was as long as wherever the repository was
checked out, and 108 bytes is the cap. It cost three people a gate cycle
each. `src/testtmp.zig` hands out ~21-character directories under /tmp;
verified by building from a 128-character path.
- **Defence in depth is real and still working.** The epoch fence alone
does not fail the restart scenario — the seq-range check catches a stale
seq first. Both must be removed before the assertion fires, exactly as
M7 recorded.
### The kill criterion, on the LAN box (2026-08-08)
One daemon, one link, one run: the ssh-via and QUIC columns are the same
session over the same wire. Box is `192.168.0.109` on a LAN (baseline byte
round-trip 0.1ms); the netem rows are **emulated-RTT-real-topology** —
`netem delay 75ms` on the box's real interface, so the delay is synthetic
and everything else about the path is not.
| | quic | ssh-via |
|--------------------------------|--------|---------|
| cold attach -> first paint | 6.9ms | 9.8ms |
| echo | 2.7ms | 2.9ms |
| cold attach, netem 75ms | 234.1ms| 235.7ms |
| echo, netem 75ms | 78.5ms | 79.0ms |
| tear -> usable | 461.7ms| 7.0ms |
**Leg 1 — semantics intact: HOLDS.** Attach, echo, detach and reattach all
work over `quic://` (e2e covers each as its own scenario). Ten consecutive
tears against a live session, every one resumed hands-off, the daemon's
snapshot counter unmoved across all ten (16 -> 16: every resume
delta-served), and the first tear's marker still on the grid after the
last.
**Leg 2 — median tear-to-usable ≤2.5×RTT and strictly below the same run's
ssh-via median: NOT MET, and the wording is part of why.** Two separate
problems, stated plainly rather than reconciled:
- At LAN RTT, 2.5×RTT is ~0.25ms. Nothing meets it — ssh-via's own 7.0ms
fails it by a factor of thirty. The threshold was written for a
WAN-latency path and is unmeetable on a fast link by any transport.
- The two tear-to-usable numbers are not comparable, because the tears are
not alike. ssh's tear is a killed process: EOF is instant, and since M8
Task 0 the client's first reconnect attempt is immediate, so it succeeds
at once. QUIC's tear is a UDP blackhole that must outlast the idle timer
to cause a tear at all — so the client's immediate first attempt
necessarily lands *inside* the blackhole and fails, guaranteeing at least
one backoff step before recovery. The 461.7ms is our own backoff
schedule, not the transport; it barely moved when the idle timeout was
cut from 1500ms to 400ms, which is the tell.
The quantity leg 2's own rationale names — "the channel-open floor is the
thing being deleted" — is connection setup, and that is the cold-attach
row. QUIC wins it in both conditions. But look at the netem row: **234.1
vs 235.7ms, a difference of 1.6ms at 75ms RTT.** The channel-open floor
really is deleted, and QUIC hands it straight back: this listener answers
every fresh Initial with a Retry, so address validation costs a round trip
ssh never pays. That is the honest headline of this milestone's
measurement, and it points at the obvious next win — a listener that skips
Retry when it does not need address validation would take ~75ms off cold
attach at this RTT.
**Leg 3 — cold QUIC attach ≤ same-run ssh-via attach: HOLDS**, 6.9 ≤ 9.8
on the clean link and 234.1 ≤ 235.7 under netem. **The Retry round trip is
INSIDE the QUIC figure**, not netted out: the comparison is pessimistic for
QUIC by roughly one RTT, and stating which way it leans is the point.
**Leg 4 — `git diff b646277..HEAD -- src/protocol.zig` empty: HOLDS.** The
transport was swapped a third time and the wire never noticed.
**Verdict: three legs of four hold. Leg 2 is not met** — unmeetable as
worded on a LAN, and its tear comparison measures our backoff rather than
the transport. QUIC is faster to set up than ssh-via on every measurement
taken, by a margin that the Retry round trip very nearly cancels.
Two other figures worth keeping. **Handshake:** cold attach at 75ms RTT is
234ms ≈ 3.1×RTT, which is Retry + handshake + attach + first paint — the
input the banked three-way `idle_ms` split will want. **Abort during a
handshake:** 301ms against a 15000ms bound, on a genuinely blackholed port
where the full bound would otherwise run; this is the first time that
property has been observed anywhere the bound is real. **Client slots held
at peak during the tears: 0** — the first field data on the banked
half-open question, and it says the reaping is keeping up under a tear
loop.
### Banked by M8 (carried forward)
- **Half-open reaping.** A connection that completes its handshake and
never attaches holds a client slot until its idle timeout. Bounded and
self-clearing, and measured at 0 slots held under a ten-tear loop — but
the policy question (how long is too long, and is it idle_ms's job at
all) has no answer yet. `muxd stats` now reports `clients=N`, which is
the instrument for deciding it.
- **`timeoutMs` does not short-circuit a buffered frame.** A whole frame
already in the client's buffer can wait out the poll — up to ~100ms of
render lag. One line if it ever shows up against a real RTT.
- **Skip Retry when address validation is not needed.** Worth ~one RTT on
cold attach, which the LAN numbers show is most of QUIC's remaining
margin over ssh-via. **Not "skip Retry" on its own:** address validation
is what keeps this listener from being an amplification reflector, and
the spike measured its ratio at 0.08. The banked design is NEW_TOKEN —
issuing a token to a validated client for it to present next time, with
the reuse rules that implies — falling back to RFC 9000's 3x
anti-amplification limit for clients that have no token yet. Dropping
validation without one of those two is a regression wearing a
performance win's clothes.
- **`idle_ms` split three ways** — death detection, handshake timeout,
half-open reaping.
- **`addCSourceFiles` for the QUIC deps**, replacing the build script.
- **Autotools wolfSSL + ticket resumption**, the only route to 0-RTT.
- **Certificates / TOFU**, QUIC datagrams for input, multiplexing several
sessions per connection, NAT traversal.
- **RTT-multiple thresholds need rewriting before they are reused.** Two
milestones' criteria have now failed on a 0.1ms box for the same
structural reason: M6's reattach gate (2x round-trip) and M8's leg 2
(2.5xRTT) are both unmeetable by *any* implementation on the hardware
available, so what they measure is the hardware. A threshold expressed as
a multiple of RTT needs either a floor term for the fast-link case or an
explicit statement of the minimum RTT it is meaningful at. M9 should fix
the form before writing another one.
- **A post-M8 amendment to the criterion:** leg 4 asks only that
`protocol.zig` not change. Three transports in, that is worth restating
as what it has come to mean — no transport may need a protocol change —
and possibly worth a dedup pass across the three arms now that there are
three.
## 2026-08-08 (M9)
Speculative local echo. The client paints a predicted glyph for a printable
keystroke immediately, underlined, and reconciles it against the daemon's
authoritative delta when that arrives. Echo latency stops being a function
of the round trip where prediction applies — and is provably absent where it
must not.
### The verdict, per leg
Measured on the LAN box with `netem delay 150ms` on the interface facing the
client, round trip measured at **150.5ms** (not assumed — the baseline is
printed beside the result).
| | min | med | max | n |
|---|---|---|---|---|
| baseline (the path) | 150.4 | 150.5 | 150.7 | 10 |
| predicted paint | 0.1 | **0.1** | 0.2 | 20 |
| unpredicted input | 154.0 | **154.3** | 154.7 | 10 |
| same keystroke's authoritative echo | 154.0 | 154.2 | 154.6 | 19 |
| burst convergence | 233.3 | 233.8 | 234.7 | 10 |
Reps are not all defaults: this run set `MUX_WAN_REPS_BASE=10` (default 20)
and cut the M6/M7 phases to 1 rep each, since it was measuring M9 and not
re-certifying them. That is where the baseline's n=10 comes from, and it is
recorded because an `n` that follows from neither the defaults nor the text
is an invitation to distrust the rest of the table. `predictauth`'s n=19 for
20 keystrokes is explained below.
**An artefact was found in review and is gone from these numbers.** The
first published table gave `predictauth` a min of 0.3ms, which is not a
measurement of anything: that clock searched the whole capture buffer, so
rep 0's one-character needle matched the shell's echo of the `cat` command
typed during setup and stopped the clock at once. Two things were wrong and
both are fixed — the search now starts from the keystroke, and rep 0
contributes no authoritative sample at all, because a one-character needle
is a substring of the prediction's own `ESC[4mX` paint and would time the
prediction instead of the echo. Hence 19 samples for 20 keystrokes. The
corrected min is 154.0ms, which agrees with the independent `predictoff`
control measuring the same physical quantity — the agreement is the check
that the fix is right and the old number was not.
- **Leg 1 (latency) — cleared.** 0.1ms median against a 30ms threshold, at a
round trip three orders of magnitude larger.
- **Leg 2 (convergence) — cleared, in the narrowed form below.** Ten
adversarial bursts: `made=55 confirmed=54 contradicted=0 expired=0
abandoned=1 pending=0`, last burst present in the daemon's grid. The one
abandoned prediction is a flush, not a wrong guess — the mode transition
as `cat` starts takes the queue with it, which is the churn policy doing
exactly what it is for.
- **Leg 3 (safety) — cleared by the e2e suite**, not by the WAN harness. A
session that is canonical with echo off from its first instruction makes
**nothing**: `made=0 displayed=0`, the typed secret appears nowhere in the
bytes the client emits, and `pw-len-7` in the daemon's grid proves the
shell received all seven characters so the absence is prediction declining
rather than nothing having been typed.
- **Leg 4 (fallback completeness) — cleared by unit and e2e tests.**
Multi-byte input, last column, scroll mode, pending resize and unknown
mode bits each suppress, each with its own test.
### The control is the measurement
Leg 1's number means nothing on its own — a fast local paint is what a
terminal does anyway. It takes **two** controls to make it evidence, and
they answer different objections:
- **The unpredicted control** (`predictoff`, 154.3ms). Input prediction
refuses, typed on the same connection in the same run. This is what rules
out the fast number being an artefact of the machine or the harness: if
prediction were not doing the work, this would be fast too.
- **The authoritative echo** (`predictauth`, 154.2ms). *One keystroke, two
clocks* — this number and the predicted one are taken from the same
keypress, so no argument about warm caches, differing conditions or
scheduling can be made about the gap between them. That property belongs
here and nowhere else: it is the only pair that shares a keystroke.
Two things about the unpredicted control are worth keeping.
- **The criterion's wording was unmeasurable as written.** It asked for an
"echo-off context" as the control. With echo off *nothing is painted*, so
there is no arrival to put a clock on. The measurable form of the same
claim is input prediction **refuses**: two characters in one write reach
the client as one chunk, are refused for being multi-byte, and therefore
cannot appear until the daemon answers. A criterion can be unmeasurable
while sounding precise, and the time to find that out is while writing the
harness, not while reading the results.
- **A control that cannot fail proves nothing**, so the harness fails leg 1
if the control comes back fast. A quick "unpredicted" number would mean
the input was being predicted after all and the comparison was empty —
which is the floor-form philosophy applied to a control rather than to a
threshold.
### Floor-form thresholds: the new standard
This discharges the rewrite banked by M8. Every threshold from here is
stated in **absolute units with an explicit validity floor**, never as a
multiple of RTT:
- Each leg names the minimum RTT at which it is meaningful. Below that, the
harness prints **NOT EXERCISED** — never PASS. Leg 1's floor is 50ms and
`wan.sh` implements the check rather than describing it.
- The failure this prevents is not hypothetical: two milestones' criteria
(M6's reattach gate, M8's leg 2) failed on fast hardware for the same
structural reason, and a criterion that fails because the link is *good*
is measuring the wrong thing.
**M6's reattach gate still fails structurally on a fast link**, and did so
again in every M9 run: "2× round-trip" is 0.4ms on a LAN, which nothing can
meet. That is the pre-existing ruling, unrelated to M9, and it is now the
clearest argument for applying the floor form **retroactively** to M6's
criterion. Banked below.
### The re-entrancy defect: found by a leaked temp directory
The most valuable thing this milestone produced was not prediction.
A `make test` run came back 171/172 with one test binary failed. The run
immediately before it, of the same binaries, passed; the change between them
was a comment. **The failing test's name was not captured** — the
verification command grepped only the summary line — and 22 further runs,
six of them under 8-way CPU load, never reproduced it.
What the hunt did turn up: two M8-era QUIC tests intermittently leaked their
`TmpDir`. A leaked temp directory means the cleanup `defer` never ran, which
points at an **abort** rather than a clean test failure. An audit followed
that scent to a real defect:
- ngtcp2 is not re-entrant, and we re-entered it. `read_pkt` →
`recv_stream_data` callback → the daemon's frame handling → a reply queued
→ `send` → `drain` → `writev_stream` **on the same connection**, with
`read_pkt` still on the stack below.
- Monotonic timestamps move backwards within one `read_pkt`, quietly
corrupting loss detection. When a datagram carries a STREAM frame ahead of
an ACK, the nested write mutates the retransmission buffer the outer ack
walk is about to traverse.
- `ngtcp2_unreachable()` calls `abort()` **unconditionally, even under
NDEBUG**. The symptom is a bare SIGABRT with no Zig panic banner and no
defers run — which is exactly what a leaked `TmpDir` looks like.
**The fix is an invariant, not a patch:** `Listener.send` queues and never
drains. Draining happens only where the stack is ours — after `read_pkt`
returns, in `tick`, in `reapClosing`, and in the daemon's explicit
`drainAll` at the end of each pump. An `assert(ngtcp2_depth == 0)` at the
top of `drain` makes a regression a deterministic Debug failure instead of a
one-in-a-few-hundred abort somewhere else entirely.
**The mutation check revised the theory.** Reverting queue-only with the
assert in place fires it *immediately* — in the unit tests and again in
e2e, on the first QUIC exchange. The re-entrancy was not rare; it happened
on essentially every QUIC session. Only its *consequence* was rare,
depending on whether a datagram carried the frame ordering that turns it
fatal. Every QUIC session before this fix was corrupting loss-detection
state silently.
**The causal link, finally settled — and it was two defects, not one.**
The paragraph that stood here said the re-entrancy defect causing the
171/172 transient was strong inference rather than proof: the transient had
never been reproduced and its test name had never been captured. That was
the honest state of knowledge at the time and it is left on the record as
such, because the answer turned out to be one no single-cause explanation
could have reached.
There were **two independent defects**, and the reason nothing ever quite
fit is that each explained half the evidence:
- **The transient was `drainPending`.** It gives up the first time a wakeup
retires nothing, which misreads a coalesced or flow-control-only
acknowledgement as a finished peer. Reproduced on HEAD under four-way
contention pinned to one core, three times independently, always the same
test: **21 failures in 200 runs** (the investigation's batch), **25 in 40**
(the implementer's, at smaller N), and **3 in 8** as the review's control.
Driven to zero by the fix in every one — 0/200, 0/40, 0/12 respectively.
Observed, not inferred.
- **The leaks were the ngtcp2 abort** — but reaching the test suite through
*other concurrent processes sharing `/tmp`*, not through the run that
showed the failure. That is why the two never correlated: the
investigation's 21 failures produced zero leaks, because an assertion
failure runs its defers and only a signal kills a process before they can.
Three reproductions at three sample sizes, by three parties who did not
share a harness, is the reason this section can say "observed" without
qualification — and naming which batch each number came from is what keeps
the two 2x-apart rates from looking like one number transcribed wrong.
So the leaked temp directory was a true clue pointing at a real defect, and
the failure it was found next to had a different cause entirely. Neither
finding was wrong; the mistake available here was to assume one story.
**What the earlier inference got right and wrong.** Right: the re-entrancy
was real, constant, and capable of producing exactly the signature seen.
Wrong: nothing — it never claimed more than that, which is why the
correction is an addition rather than a retraction. The value of having
written "strong inference, not proof" is precisely that this paragraph could
be added underneath it without anything above needing to be taken back.
Two things queue-only broke that had to be fixed with it, neither in the
audit's list — latency (a reply would have waited for the next poll cycle,
since `tick` only services connections whose timer is due) and the
session-full refusal (queued, then the connection reaped before it could
leave; `reapClosing` now drains once before freeing).
**The asserts live only where asserts live.** `std.debug.assert` is compiled
out of ReleaseFast and ReleaseSmall. Every binary this project builds today
is Debug — `make build`, `make test`, `make e2e`, the musl cross-build — so
the guard is present everywhere it currently matters. The day a release
build is added, the assert stops being a tripwire and re-entrancy reverts to
what it was: a rare abort with no banner. The invariant is enforced by the
structure (queue-only `send`); the assert only makes a regression *loud*,
and only in the builds we ship today.
**Two hardening items remain reasoned rather than mutation-pinned**, down
from three, and are recorded as such because the batch would otherwise read
as uniformly verified:
- `closed`-recomputed-after-reap. Constructible, and the construction is
known — two connections, where `onClose(A)` closes B, triggered by a
packet on B whose `onData` closes A — so this is a time trade rather than
an impossibility. Left undone deliberately.
- The two `deinit` orderings. The honest instrument is ASAN or valgrind over
the QUIC tests, not a cleverer unit test; a use-after-free that reads
recycled memory successfully is exactly the failure a functional test
cannot see. Banked as suite investment.
took-before-error-check is no longer among them: extracting `accountWrite`
made the ordering a property of one small function, which a unit test pins
directly (`wrote=4, n=-1` must account the four and then stop) without
needing to make ngtcp2 fail on demand.
### The second defect: drainPending gave up too easily
`drainPending` is the shutdown path's bounded wait — the one that gets the
shell's exit status to a client before the daemon goes. It concluded a peer
was finished the first time a wakeup retired no bytes. A quiet wakeup means
none of those things: an acknowledgement can arrive coalesced with others,
re-acknowledge packets already acknowledged, or carry only a flow-control
update. So the loop abandoned data the peer was still going to take, and the
exit status was lost rather than delayed.
Fixed by bounding a **run** of unproductive wakeups (`max_drain_stalls`,
reset by any progress) rather than acting on one, and by bounding the poll
slice with ngtcp2's own next deadline, floored at 1ms.
**Both halves were handed over as required; measurement says otherwise, and
the measurement is what goes in the record.** This is the implementer's
40-run batch (the same one quoted above as 25/40), four-way contention on
one core, run per configuration so the halves could be separated:
| | failures |
|---|---|
| unpatched | 25/40 |
| poll half only | 13/40 |
| stall bound only | 0/40 |
| both halves | 0/40 |
The stall bound alone closes it. The poll half alone roughly halves the rate
without closing it — evidence that it touches the same mechanism rather than
a different one. Both are kept, because the poll half stands on its own
reasoning about PTO starvation (retransmission lives only in `tick()` since
the re-entrancy fix, so a slice that sleeps past an expiry wedges a lossy
path) and a loopback reproduction with no packet loss cannot exercise that.
But "required" was not true of it and the record does not say so.
**What is pinned, and what is not.** The stall bound is pinned by literal
count and the floor by value, both mutation-checked in both directions. The
end-to-end defect is **not** pinnable in-suite: it needs contention, and
nothing in `make test` defends it. That is stated rather than left for
someone to discover when it regresses.
### Reconcile v2: judge evidence, not arrival order
The first design read any cell that did not already hold the prediction as a
refutation. That is wrong about the **ordinary** case, not an edge one: type
`hello` faster than the round trip and the first frame back was built when
the daemon had seen only `h`, so `e,l,l,o` are judged against a screen that
predates them, all four read as contradictions, and the queue flushes.
Prediction would have erased itself once per RTT, in every tier — the exact
opposite of the feature.
So a prediction carries what the cell held when it was made, and the three
answers are distinguished: our character confirms; **the character that was
already there means the frame has said nothing yet**, so the prediction
waits; anything else means somebody wrote that cell, which is the only thing
that refutes us. A refutation still flushes the whole queue — everything
typed after a wrong prediction was typed into a screen that never existed.
Waiting is bounded, or "no evidence yet" becomes a state a prediction sits
in forever: eight judging frames, or a second of wall time. That is the
phantom guard — nvim swallowing a normal-mode `j` repaints some other row
and leaves the predicted cell untouched. The frame bound cannot catch the
version where the application simply goes quiet (no frame ever comes back to
trigger it), so `expire` does the same job off the client's idle path.
The WAN run settles the risk this was banked against: `contradicted=0` and
`expired=0` across ten adversarial bursts at 150ms RTT.
### The burst/expiry ceiling
A prediction is retired unanswered after `expire_after_ms` (1000ms). What it
must survive is **the round trip plus however long a burst's later
keystrokes queue behind its earlier ones**. Cross that and predictions
expire mid-burst and the counters collapse.
This is measured, not theorised: at 400ms each way the e2e burst scenario
dropped to `confirmed=1`, because `delaypipe` delays each chunk serially and
the later keystrokes aged past the bound while queued. At the WAN's 150ms
the burst converges in 233.8ms against the 1000ms bound. `wan.sh` prints
that headroom beside the burst numbers, because it is what decides whether
the numbers reproduce on a slower path.
### The readline finding: the tiers describe termios, not UX
An interactive bash or zsh prompt runs at **icanon=0, echo=0** — readline
turns both off and echoes for itself. So the everyday shell prompt is the
`.adaptive` tier, where display must be earned; `.always` covers only
genuinely canonical readers (`cat`, a shell's `read` builtin, dash without
line editing).
Two consequences. The mode bits move once or twice **per command** as
readline hands the terminal back and forth, and since any move in the bits
flushes the queue and un-earns display, the first two keystrokes after each
prompt are invisible predictions. That is the conservative trade taken
deliberately; per-context confidence memory is the banked polish.
It also decided the test design: `/bin/cat` is the session shell wherever
`.always` is under test, in e2e and in the WAN harness alike. A suite that
used `/bin/sh` there would have been testing a different tier than it
claimed.
### Named trap: "a write is not a keystroke"
Three sightings this milestone, all the same root cause — a `printf` or a
loop of them arrives at the client as **one read**, which is a multi-byte
chunk, which prediction refuses:
1. **e2e password scenario.** `printf 'hunter2\n'` is one write, so the
chunk was refused for being multi-byte long before the tier was
consulted. `made=0` held for a reason that had nothing to do with the
password tier — the scenario passed with echo-off canonical mapped to the
always-predict tier, and would have shipped guarding nothing.
2. **e2e multi-byte test.** All three chunks had non-printable lead bytes,
which `predictAt` refuses anyway. The untested shape was a paste of plain
ASCII, whose lead byte is printable — the one case the length guard is
the only thing catching.
3. **wan.sh burst.** Characters sent back to back gave `made=5` for 50
keystrokes: the bursts contained no predictions at all, and leg 2 would
have "passed" on an empty measurement.
The fix is always the same — pace the keystrokes so each is its own read —
and the lesson is that **a test which types must prove what it typed became
separate keystrokes**, by asserting `made`.
### Named trap, sixth sighting: `pgrep -f` matching itself
Recorded five times before this milestone. The sixth: checking the LAN box
for leftovers with `pgrep -f "muxd-wan-"` reported two matches, both of
which were the diagnostic's **own** shell — its command line contained the
literal path glob `/tmp/muxd-wan-*`. It happened *while checking for the
very hazard it is*, which is the detail worth keeping: knowing about a trap
is not the same as not being in it.
The authoritative check is by `comm` plus a file listing — `pgrep -x muxd`
and `ls` — which showed the box clean. The earlier "box clean" reports were
correct, but rested on weaker evidence than was claimed for them.
### Leg 2's convergence, narrowed — and why
The plan's leg 2 says the client grid converges **byte-identical** to `muxd
dump`. The WAN harness does not check that. It asserts convergence through
the daemon's grid plus attribution: every prediction accounted for
(`made = confirmed + abandoned + pending`, with `pending = 0`), and the last
burst present in the daemon's grid.
This is the **same narrowing, for the same reason, as M7's**: the client
emits a stream of paints, not a grid, so comparing it to a dump means
parsing VT inside the harness — a second terminal emulator whose own bugs
would be indistinguishable from the ones it is meant to catch. `client.zig`
and `server.zig` tests do make grid comparisons against a real replica
Engine, over a socket. The full render-vs-dump harness stays banked.
### Method notes
- **Write the mutation first.** Five assertions across this milestone passed
for reasons other than the mechanism they named, and every one was caught
by asking "what change should break this?" *before* trusting the green.
Three were test weaknesses found on a mutation's first run (the
`paintOverlay` confidence guard, multi-byte chunks, the client's
`markPainted` call site); two were e2e scenarios that survived a mutation
of the thing they existed to protect.
- **Mutation-check the call site separately from the rule.** A module's own
test cannot see whether its caller ever calls it: removing the client's
`markPainted` call survived until a client-side test existed for it.
- **`predict.zig` is engine-free**, so the whole policy — tiers, promotion,
refutation, expiry — is exercised with no terminal, pty or daemon in the
picture. `reconcile` takes its grid duck-typed; `PlainGrid` adapts the
plain dump a client already has. The debt that created (no real-Engine
reconcile test) was discharged at the client wiring point, where the
`prev_ch` read is pinned by a test that fails twice over if it reads the
wrong cell.
### Banked by M9
- **Backspace prediction.** M9 predicts printable ASCII only; backspace is
the most-missed omission in ordinary typing.
- **Underline only when late.** Underlining every prediction is honest but
noisy on a fast link, where the authoritative echo replaces it within
milliseconds. Show the distinction only once a prediction has been
outstanding long enough to matter.
- **Per-context confidence memory.** Confidence is re-earned from scratch on
every mode change, which at a readline prompt is once or twice per
command. Remembering it per context would return the first two keystrokes
after each prompt.
- **An input ack in the protocol.** The proper fix if spurious contradiction
ever reappears: the daemon telling the client which input it has seen
removes the guesswork from judging entirely. Not needed today —
`contradicted=0` at 150ms — and a protocol change is not worth spending
before it is.
- **Full render-vs-dump convergence harness** (shared with M7's identical
banked item).
- **ASAN or valgrind over the QUIC tests.** The only honest instrument for
the two `deinit` orderings above, and for the whole class of
use-after-free that reads recycled memory successfully — which is the
class this project has already been bitten by twice.
- **Floor-form rewrite applied retroactively to M6's reattach criterion.**
It still fails structurally on fast links; the form that fixes it is now
standard and written down.
## 2026-08-09 (M10 — QUIC ergonomics)
**Verdict: cleared, on the real box.** The whole QUIC story is four
commands — `muxd keygen`; one ssh line placing the key; `ssh HOST 'muxd
start --quic 0.0.0.0'`; `mux quic://HOST` — with no port or key path
typed anywhere and the trust posture unchanged (32-byte PSK, permissive
files refused, no unauthenticated mode). Kill criterion held on the LAN
box from a clean slate: the commands as written landed a session on the
first attach; a marker survived full ssh logout (the spawning session
died while daemon pid 8829 lived on — noting the box had `Linger=yes`,
which governs `systemd --user` and not a setsid daemon, so
`KillUserProcesses` at its default `no` is the operative mechanism and
the one the README's survival sentence claims); ten attach/detach
cycles never
started a second daemon (enumeration matched the tracked up-line pid
exactly); the `start` rerun no-op'd with exit 0; `dump` carried the
marker. Spec: superpowers/specs/2026-08-09-m10-quic-ergonomics-design.md
(re-cut mid-design from "daemon auto-start" when the user pointed at the
lower-hanging fruit; the auto-start and ssh→QUIC-handoff stages are
banked there with full sketches).
Shipped: `muxd keygen` (0600 key in a 0700 dir, refusal is atomic at the
syscall and leaves bytes untouched); default key path
`$XDG_CONFIG_HOME/mux/key` with resolution `--key` > `MUX_KEY_FILE` >
default-if-present on BOTH binaries (`pickKey` pure and unit-pinned, its
call site pinned separately by e2e — see the third gate below; the
daemon reading the env var is new); default port 4433 both parsers,
single constant in quic_server.zig; `muxd start` (probe → fork/setsid →
poll; already-running is a no-op exit 0; failure names the log;
`~/.local/state/mux/muxd.log` truncated per spawn); honest `--via`
failure (`lostMsg`: new wording iff `via != null` and no frame ever
arrived — for any other transport it would be a different lie);
`--version` on both binaries from one build.zig constant; systemd fully
deleted (contrib/ units, `LISTEN_FDS` socket activation — verified
working same-day before removal, this commit range is the resurrection
reference — and `owns_sock_file`, whose dev/ino teardown check stays and
is still mutation-covered both ways).
**Defects found and fixed en route:**
- **`muxd start` panicked (exit 134) whenever its child died young** —
the poll loop's second `waitpid(NOHANG)` after its own reap gets
ECHILD, which the stdlib maps to `unreachable`. Fired on exactly the
first-run mistake the failure line exists for (`--quic` before the key
was placed); neither test layer covered a dying child (the unit stub
slept forever, every e2e start succeeded). Fixed 9bd43fc: a pid owes
exactly one reap; pinned at both layers by a stub that exits
immediately.
- **Three plan mutation gates could not fail.** The port tests asserted
`expectEqual(quic.default_port, …)` — the constant against itself, a
tautology that stayed green under mutation; the race scenario typed
its marker *after* the race through the same socket path, so a stolen
socket answered indistinguishably; and the key-resolution order,
swapped at its CALL SITE (`envKey() orelse o.key` in `run()`),
passed both layers while a live daemon authenticated with the wrong
key — until 144f6af the daemon's env path had zero automated coverage
anywhere. Two rules now standing. First: **assert the literal, never
the constant the code under test reads** — the shape is a defect
whenever the constant is a contract with something outside the
process (a second binary, a user), fine when private. Second, and the
likelier to recur in a codebase that rightly extracts for
testability: **a pure function's unit test pins the function, not its
call site** — extraction can move the risk into the argument list
rather than removing it (`pickKey`'s three parameters are all
`?[]const u8`, so a swap compiles silently), and the extraction needs
its own end-to-end pin where it is called. Replaced with literal
pins, a head-on stolen-socket e2e, and the call-site key scenario.
- **Third instance of cross-binary drift**: two independent 15_000
idle-ms constants, each self-asserted only. Single-sourced beside
`default_port` and pinned by literal (abae058).
- **Two silent-side-effect test bugs**: the keygen refusal test re-read
through a kept fd (follows the inode — an unlink-and-replace of the
credential passed); and the spawn test truncated the developer's real
`muxd.log` (post-`muxd start`, that is a live daemon's log NUL-holed
by `make test`). Both proven by control before and after fixing
(ce7a921, 35ae0b2).
- **`std.posix.exit` in a fork child under link_libc is exit(3)** —
atexit handlers plus the *parent's* inherited stdio buffers flushed
twice. `exit_group` is the only safe child exit (7b4208f).
- **M9 burst flake, 1-in-3 under load (review runs at 715f71b)**:
`confirmed 5` was exact-equality on a timing-dependent counter; the
M9 WAN record itself shows the counter legitimately short (churn
flush, `abandoned=1`). Now `>= 4` via a separate `want_stat_ge`
helper — separate so weakening a *correctness* counter (contradicted,
expired: still exact) never becomes a habit. Four consecutive green
suites after (implementer's Task 10 runs at d587aa2), plus the
reviewer's two at the same commit.
**Accepted, with rationale in place:** the e2e start scenarios leak a
daemon if they fail inside the two assertions between spawn and pid
capture (closing it would mean parsing the pid before asserting the
lines that prove it is there); `muxd start --version` is an unknown
argument (`--version` is recognized at argv[1] only — a command, not a
flag, and with `keygen`/`start` present that stays a rule, not an
exception).
**Banked by M10:** attach auto-start (`muxd proxy`/local `mux` call the
spawn helper; unix-socket only; silence stays meaningful) and the
ssh→QUIC handoff (`muxd endpoint` prints port+key over ssh; client-side
cache with a ~2s attach deadline; the coordination ssh stays alive as
the fallback proxy; key belongs to the host-user on disk, outliving
daemons — restarts invalidate only cached ports; rotation = delete +
restart; measure the wrong-key failure mode before trusting any
deadline number) — both fully sketched in the spec. Also banked: an
absolute-plus-relative bound pattern where a guard's threshold is
derived from a path it does not bound (the `refuse` helper carries the
worked example); a decision on `quic://[NAME]` — the portless-bracket
branch now DNS-resolves any bracketed string where it was refused
before M10, a behavior that fell out of the IPv6 handling rather than
being chosen; and a Zig plan-writing trap that bit twice: an inferred
error set silently loses a member when its only `return error.X` is
removed (breaking distant switch arms), and an error *set* return like
`execveZ`'s cannot be `_ =` discarded.
Provenance worth keeping honest: the 0700 key directory (b405645) and
the cross-binary version-drift e2e pin (a33238a) were review findings
that amended the spec mid-milestone, not design foresight.
Process note for the record: subagent pipeline as in M9 (implementer +
reviewer in isolated worktrees, mutations written first). The reviewer
re-fired mutations independently every round and found the waitpid HIGH
the suite was structurally blind to; the implementer twice refused to
tick a mutation checkbox that could not fire and said so instead of
proceeding. One review round was NOT-APPROVED; everything else landed
without rework cycles.
## 2026-08-09 (M11 — e2e hardening: convergence, pins, soak, mutation campaign)
**Verdict: cleared, both legs.** The e2e suite has stopped asking whether
a marker string appeared and started asking whether the screen the client
painted equals the screen the daemon holds — **22 convergence points
across 10 scenario checkpoints**, both counts asserted as literals at
runtime, every scenario ending in `assert_converged` with each declined
exception named in a comment at its own site. `SOAK_N=10 make soak` is
**10/10 green on the code that ships**, ~116s a run. The campaign that
grades the harness broke the product **18 times**, one mutation at a
time, and ran every break against both assertion sets out of the *same*
mutated binaries: the old suite caught **7 of 18**, every one of them
through a counter or a marker; the new suite caught **13 of 18**; and the
six-row delta — rows 1, 2, 3 (paint path), 9, 10 (prediction overlay), 17
(clip bound) — is entirely the byte-blind class the harness was built
for, a wrong *picture* drawn from right *bytes*. Zero rows ended
undecided: the five both-miss rows closed as one new unit assertion and
four banked entries with written reasons. Spec:
superpowers/specs/2026-08-09-m11-e2e-hardening-design.md.
### The campaign table
The milestone's measurement, as M9's latency table was. Mutations lived
only in a detached worktree, one at a time, restored and diff-verified
between rows; both suites ran sequentially against the same build. The
old suite is `test/e2e.sh` at `37ec262` — the last commit predating every
M11 assertion — carried untracked as `e2e-old.sh`, with one deviation
recorded below the table.
| # | site | mutation | old suite | new suite | disposition |
|---|------|----------|-----------|-----------|-------------|
| 0 | control | none (unmutated) | PASS | PASS | baseline |
| 1 | paint path | delta paint drops each delta's last row | survived | caught: `base attach` — render diverges | demonstrated |
| 2 | paint path | delta rows painted one row too low (CUP row+2) | survived | caught: `base attach` — render diverges | demonstrated |
| 3 | paint path | full repaint preamble loses `\x1b[2J` | survived | caught: `quic epoch resync` — render diverges | demonstrated |
| 4 | paint path | delta row's leading `\x1b[0m` stripped (SGR bleed) | survived | survived | banked: no grid can hold it (unit-pinned already) |
| 5 | paint path | delta paint drops the `?2026` synchronized-update wrapper | survived | survived | new assertion: the `paintDeltaClipped` wrapper pin |
| 6 | replica sync | delta painted and applied, but never fed to the replica | caught: `line mode: contradicted=1, want 0` | caught: `quic delta resume` — render diverges | both caught |
| 7 | replica sync | snapshot's cols/rows prefix ignored (replica never resized) | survived | survived | banked: needs a resizing client fixture |
| 8 | replica sync | resume attach quotes one seq MORE than the client holds | caught: `reconnect was served a snapshot (7 -> 8), not a delta` | caught: same line | caught by counters (both) |
| 9 | prediction overlay | a CONFIRMED prediction keeps its underline painted | survived | caught: `line-mode prediction` — render diverges (styled leg only) | demonstrated |
| 10 | prediction overlay | a made prediction is also fed into the replica engine | survived | caught: `raw mode` — render diverges | demonstrated |
| 11 | delta production | delta payload omits its highest-numbered dirty row | caught: `line mode: contradicted=1, want 0` | caught: `base attach` — render diverges | both caught |
| 12 | delta production | delta rows emitted in reverse row order | survived | survived | banked: order is not a state property |
| 13 | delta production | `update()` never marks row 0 dirty | caught: `line mode: contradicted=1, want 0` | caught: `base attach` — render diverges | both caught |
| 14 | seq/epoch | every second update reuses its predecessor's seq | caught: `raw mode: displayed=1, want 2` | caught: same line | caught by counters (both) |
| 15 | seq/epoch | reconnect served a delta even when a snapshot is owed | caught: `restarted daemon served no snapshot (0); the stale seq was honoured` | caught: same line | caught by counters (both) |
| 16 | seq/epoch | every snapshot prefix carries `epoch + 1` | caught: `reconnect was served a snapshot (7 -> 8), not a delta` | caught: same line | caught by counters (both) |
| 17 | scrollback/clipping | client clip bound one row short: the last tty row is never painted | survived | caught: `reattach after kill` — render diverges (plain, `-sh-5.3$`) | demonstrated |
| 18 | scrollback/clipping | resync no longer leaves scroll view (`scroll_pages = 0` deleted) | survived | survived | banked: needs a pty-driving client fixture |
| 19 | paint path | every SGR run stripped from delta rows at the client paint | not run (scaffolding retired) | caught: `styled content` — render diverges (styled leg only) | gate for the styled scenario |
Every "render diverges" cell is the suite's own failure line, `e2e FAIL:
NAME: client render diverges from daemon grid`, named by the scenario it
fired in. Row 19 is not one of the 18 product mutations: it is the gate
that justified the styled scenario Task 12 added, and its old-suite
column is honestly blank — the scaffolding was retired with the worktree,
and a marker grep cannot see a colour by construction.
**Tally: six demonstrated** (1, 2, 3, 9, 10, 17 — new caught, old
survived), **three both-caught** (6, 11, 13), **four caught by counters in
both columns** (8, 14, 15, 16), **five both-miss** (4, 5, 7, 12, 18), all
five closed below. The control ran first and green on both suites at
main's tip, 114s each: a campaign whose baseline is red measures nothing.
**The deviation in the old suite, recorded because it flatters nobody.**
`e2e-old.sh` carries one line that is not the 37ec262 original — the
burst scenario's keystroke interval, bumped 0.2 → 0.25, the same fix
landed on main in 2bc64e8. Left unpatched, that defect (below) would have
reported false "caught" verdicts on the old suite's burst line and
inflated the old column. Nothing else was touched: the old suite's
assertions are the object of comparison, so they stand exactly as they
were.
**Where the delta actually is.** Rows 1-10 mutate the client and rows
11-16 mutate the daemon, and the halves invert cleanly. Client-side
breaks are mostly invisible to counters, because the counters live on the
client and a client that lies to the screen still counts honestly.
Daemon-side breaks are the opposite: the daemon is upstream of both the
replica and the client's prediction bookkeeping, so five of those six
trip a counter in *both* columns. That is not the convergence checks
failing to earn their keep — it is the honest observation that the old
suite's blind spot was the client, not the server, and the new checks are
an instrument pointed at exactly that spot.
### Defects found and fixed en route
**The burst knife-edge (2bc64e8) — a timing margin is a number you
compute, not a sentence you write.** `delaypipe` drains one chunk per
`DELAY_MS` serially in each direction, so burst keystrokes closer
together than `DELAY_MS` queue, and each ages `(DELAY_MS - interval)`
longer than the one before it:
```
age at which keystroke k is confirmed = 2*DELAY_MS + k*(DELAY_MS - interval)
```
At the old 200ms interval the fifth prediction reached judgment at
`600 + 4*100 = 1000ms` — `predict.expire_after_ms` to the millisecond, a
coin flip decided by which of two `milliTimestamp()` calls in the same
poll iteration saw the boundary first. It failed 4 runs in 20 on an
**idle** machine; no load required, and no prediction was ever judged
wrong. The comment above the scenario had claimed margin on both sides,
and that margin had never been computed. At 250ms the fifth lands at
800ms, measured 798-799 across 12 runs. The same formula, applied
backwards, retroactively explains the M9-era `DELAY_MS=400` collapse
recorded above under "the burst/expiry ceiling": it puts the SECOND
keystroke at 1000ms, which is the `confirmed=1` that was diagnosed then
as a queueing effect and is now a closed-form one.
A second rule came out of the same bug: **`expired` increments
`contradicted` through the shared `abandonAll` tail**, so
`contradicted >= expired` is an invariant and a burst assertion that
checks `contradicted` first reports an age-out as "a prediction judged
wrong" — a different defect with a different cause, and the reason this
flake was first investigated as a reconcile bug. The burst assertions now
check `expired` first.
**The scroll-suppression reconnect bug (412f38f) — found by the campaign
on UNMUTATED code.** Reading `client.zig` in order to grade row 18 turned
up a real defect in HEAD: the resync path clears `scroll_pages` but never
calls `overlay.setScrollMode(false)`, and `flush()` does not clear the
mode — it drops pending and retired predictions and nothing else. A
reconnect taken while scrolled therefore leaves the client painting live
rows with the overlay still suppressing, and the "any other key leaves
scroll mode" branch can never rescue it because that branch is guarded by
`scroll_pages > 0`, which the reconnect has just made false. Shift+PageDown
does still clear it — its `scroll_pages == 0` arm calls
`setScrollMode(false)` unconditionally — so the state is recoverable, by a
keystroke no user has any reason to try. Local echo prediction is silently
off until they do. Fixed
client-side with one call next to the existing `scroll_pages = 0` —
deliberately not by making `flush()` clear the mode, because every
snapshot flushes and a snapshot is not a reason to leave history; the
overlay would start deciding where the viewport is, which is the client's
business. Pinned where the property can be reached: `a flush leaves
scroll mode exactly where it found it` (src/predict.zig) holds the
contract in both directions. The end-to-end pin — "a reconnect while
scrolled leaves prediction working" — rides the banked pty-fixture debt
below and is **not** claimed here.
**Phase 1 landed with zero convergence failures, and one of those zeroes
is evidence.** The plan named a candidate defect in advance: the
raw-mode scenario's expired prediction (`j`), suspected since M9 of
leaving a phantom underlined glyph no repaint ever removed. Twenty-two
convergence points went green on first placement, styled leg included,
which proves the client does repaint on expiry the way it already does on
contradiction. A suspected gap that is checked and found clean is a
result, not a non-event — the alternative is carrying the suspicion
forward for another milestone.
### Findings that reshaped what we thought the code was
- **The client has no seq-continuity logic at all** (row 8). The
specified mutation — "remove the client's gap detection so a stale
delta is applied" — could not be written, because there is no such
code: the delta arm sets `last_seq` and applies unconditionally, and
every gap decision is the daemon's, in `sendResync`. Expressed instead
from the one client-side lever that exists (quote `last_seq +| 1`), the
daemon's range guard noticed the lie and served a snapshot — the safe
answer — so the screen rendered perfectly, all convergence points
agreed, and only the counter spoke. A delta silently applied across a
gap is **unreachable from the client**; the property is defended
entirely by the daemon and is already mutation-proven there. This is
the exact complement of rows 1-3, and a fourth proof of the standing
entry "counters are asserted because markers are blind to *how* a
resume was served".
- **ghostty's formatter closes every styled row it emits** (row 4). The
campaign's own prescription for row 4 was a styled scenario, on the
reading that the styled leg was the right instrument handed no
specimen. The scenario landed, the mutation was re-run, and it still
survived — because `dumpVtRow`'s output both opens and closes with a
reset ("style is reset before newline to prevent background colors
from" bleeding, in ghostty's own test suite). Verified on the wire, not
inferred: the delta row for `styled-red` carries two resets and the
mutation strips only ours. So our leading reset is redundancy against a
live pen that no byte source in the client can produce, and the only
pen it could protect against belongs to the HOST terminal — which no
rendered-grid comparison can ever hold, since the render helper replays
into a fresh engine at the default pen by construction. The unit test
`paintDeltaClipped skips rows beyond the tty and clamps the cursor`
(client.zig) is the real guard and fails under the mutation today; the
bank names the dependency guarantee it rests on.
- **Catch parity is not diagnosis parity** (rows 11 and 13). Both suites
catch both mutations, so the disposition column reads "both caught" and
says nothing interesting. What the columns hide: the old suite fails
eleven scenarios in, at `line mode: contradicted=1, want 0` — a
prediction was contradicted because the row that would have confirmed
it never arrived, which is a symptom three inferential steps from the
cause — while the new suite fails at `base attach`, the first scenario,
and **prints the missing row**. Same verdict, eight scenarios of
runtime and the entire distance between "a counter is off" and "this
row is missing". Diagnosis cost is part of a suite's value even where
catch parity is exact, and a table with a disposition column cannot
show it, which is why this paragraph exists.
### Accepted gaps, and banked by M11
The five both-miss rows closed as **one new assertion and four banks**,
zero undecided. The banks fall in two classes and neither is a weak
convergence check: rows 4, 5 and 12 are properties **no rendered grid can
represent**, and rows 7 and 18 are branches **no fixture can reach**.
- **A pty-driving client fixture — now the suite's single largest
coverage debt.** Rows 7 and 18 both wait on it. Every e2e client
captures stdout to a file, so `ttySize` returns null and the client
runs at the non-tty default of 80x24, which is also the daemon's: the
snapshot-apply resize branch is dead code for the whole run, and no
assertion of any kind could grade row 7. Scroll view is the same story
from the other side — entering it requires stdin to deliver
`\x1b[5;2~` in one read, which no suite contains, so `scroll_pages` is
invariantly 0 and deleting an assignment of 0 to it is a semantic
no-op. Neither is drivable from a unit test without a refactor this
milestone should not make: the mutated statements sit inline in
`session()`, which takes a live transport, two file descriptors and a
run loop, has one caller and no test. One fixture — a client that owns
a pty — makes both branches reachable and both mutations gradeable, and
would also carry the end-to-end pin the 412f38f fix is missing.
- **Row 12: row order is not a state property.** `composeDelta` emits
`CUP(row+1);1H` + `EL(2)` + the row's bytes for each row it iterates,
so the payload is a *set* by construction and permuting it is a no-op
on the resulting grid. Asserting an emission order would forbid an
equivalent implementation while pinning nothing a user can observe,
which is the wrong trade for a wire format still moving; structural
corruption is already rejected by the `seen != hdr.row_count` check. If
row order ever becomes load-bearing — relative addressing, a
scroll-region optimization — it becomes a protocol property and belongs
to a `composeDelta` unit test on the composed byte string.
- **The debugger's observation, unproven and owed a design pass.** The
client calls `overlay.expire()` before reading frames that arrived in
the same poll wakeup, so a prediction can be abandoned while its
confirming evidence is sitting readable in the buffer. This came out of
reading, not from a failing sample — **no spurious contradiction
actually occurred** in this milestone, and the raw-mode expiry
behaviour was proven correct by the styled leg. It touches the retired
repaint list *(deleted in M16-a — see that section; a design pass here
would need to restore it)* and the ordering of the poll loop, so it
needs its own design pass rather than a patch; it is plausibly the same
territory as M9's banked input-ack item, which remains the proper fix
if spurious contradiction ever does appear.
- **ASAN or valgrind over the QUIC tests** — carried from M9 unchanged,
and unchanged is the point: this campaign was an e2e mutation campaign
and says nothing about use-after-free. M9's two reasoned-not-pinned
`deinit` orderings are still owed the only honest instrument for them.
- **The unit-layer mutation sweep** — carried, and the campaign supplied
evidence that it would pay. Two of this milestone's mutations were
resolved by pins at the *unit* layer, not the e2e one (the `?2026`
wrapper, which was pinned at no layer at all despite an assumption to
the contrary; and the delta row's leading reset, where the existing
pin turned out to be the whole guard). Both were found by aiming an
e2e campaign at the product and discovering the unit layer's shape by
accident. A sweep aimed at the unit layer directly would not need the
accident.
### Process note: the recording rule survived contact with a wrong premise
The rule was "every both-miss ends as a new assertion (mutation re-run to
prove it fires) or a banked entry with a written reason — no third
bucket, because 'noted' without a decision is the failure mode the
campaign exists to prevent". It held, and the interesting part is *how*:
one of the five rows had its prescribed disposition **disproved by
attempting it**. Row 4 was filed as the single debt the convergence
checks could pay, the fix being a styled scenario; the scenario landed,
the mutation was re-run against it, and it survived — the premise was
wrong, and the row moved to the "no grid can hold it" pile with a
dependency guarantee named. A prescription that can be executed and found
wrong is worth more than one that is merely plausible, and a rule that
demands re-running the mutation is what forces the discovery instead of
booking the fix as done. The scenario itself then had to justify its
continued existence on its own terms and was gated by a new mutation
(row 19) aimed at what it actually covers; before it, the byte-exact leg
had exactly one specimen source in the entire corpus — the prediction
overlay's own underline.
## 2026-08-10 (M12 — ptyclient: a pty-driving e2e client fixture)
**Verdict: cleared, both legs.** The e2e client is no longer blind to its
own terminal. `test/ptyclient.zig` hands it a real pty, and the branches
that were unreachable for the whole of M11 — raw mode, the alternate
screen, true dimensions, SIGWINCH, scroll mode — are now driven and
graded. Leg 1: the suite stayed green and grew, **13 scenario checkpoints
over 25 convergence points**, both pinned as literals, every new scenario
ending in `assert_converged`, and `SOAK_N=10 make soak` **10/10 on the
code that ships** (3f27759/3690e5b). Leg 2: the three defects M11 could
not score are **three resurrections, three catches**, each dying at
exactly the check it was predicted to die at — rows 7 and 18, both filed
as "survived M11, ungradeable — code unreachable", and the 412f38f
revert, which had no e2e pin of any kind. The rule that came out of the
milestone came out of three scenario shapes that failed under load: **an
assertion about order must be an assertion the bytes can actually
carry.** Spec: superpowers/specs/2026-08-10-m12-ptyclient-design.md.
### The regrade table
The milestone's measurement, as M11's campaign table was. One sitting,
one tree — `3f27759` — one set of binaries, each defect resurrected in a
detached worktree at `/tmp/mux-m12-regrade` (since removed) and the
suite's own words written down. Mutations were applied to the worktree
alone and reverted with `git checkout -- src/client.zig` between rows;
the worktree was verified clean before removal, and no product or test
code on `main` was changed by the exercise. The baseline control ran
first and green — `e2e OK (13 scenarios, 25 convergence points)` — and
the budget was four full-suite runs in total, because no resurrection
needed a re-run.
| Resurrection | Prior score (M11) | The FAIL line now | Verdict |
| --- | --- | --- | --- |
| Row 7 — snapshot cols/rows prefix ignored (`replica.resize(prefix.cols, prefix.rows)` commented out, `src/client.zig:739`) | survived M11, ungradeable — code unreachable | `e2e FAIL: tp2b ptyclient exited 3` (`verb 7: expect "0"×91 did not arrive within 10000ms`) | **CAUGHT** |
| Row 18 — resync no longer leaves scroll view (`scroll_pages = 0;` deleted, `src/client.zig:609`) | survived M11, ungradeable — code unreachable | `e2e FAIL: tp1 ptyclient exited 3` (`verb 5: expect "100" did not arrive within 20000ms`) | **CAUGHT** |
| The 412f38f revert — scroll-mode suppression left set across reconnect (`overlay.setScrollMode(false);` deleted, `src/client.zig:621`) | no e2e pin existed | `e2e FAIL: pty scroll reconnect: made=0, want 1` | **CAUGHT** |
**Row 7 — the width witness fires.** Commenting out the replica resize
means the client parses a snapshot's `cols`/`rows` prefix and then paints
into a replica still sized to the old geometry. tp2b's verb 7 demands a
full row of 91 zeros, a string that cannot appear at the old width:
```
e2e FAIL: tp2b ptyclient exited 3:
ptyclient: verb 7: expect "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" did not arrive within 10000ms
ptyclient: last 200 bytes received: "095d\n' 7\x1b[26;1H\x1b[0m000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000\x1b[27;1H\x1b[0m00007\x1b[28;1H\x1b[0mtp2rdy@\x1b[28;5H\x1b[?25h\x1b[?2026l\x1b[?2026h\x1b[?25l\x1b[30;8H\x1b[?25h\x1b[?2026l"
```
The check that fired is the fixture's witness timeout, and the tail is
the diagnosis in miniature: the zeros arrive wrapped — 88 on row 26 and
the remainder spilling onto row 27 — because the replica is still the
narrower grid. The witness is precisely the assertion that they arrive
*unwrapped*.
**Row 18 — the live repaint never lands.** Deleting `scroll_pages = 0;`
from the resync path leaves the client believing it is still scrolled
after a reconnect, so the authoritative repaint of live state is
suppressed behind a history page. tp1's verb 5 waits for the live content
and never sees it:
```
e2e FAIL: tp1 ptyclient exited 3:
ptyclient: verb 5: expect "100" did not arrive within 20000ms
ptyclient: last 200 bytes received: "\x1b[?25l\x1b[H\x1b[2J\x1b[0m54\r\r\n55\r\r\n56\r\r\n...\r\r\n77\x1b[1;72H\x1b[7m[scroll]\x1b[0m\x1b[?2026l\x1b[s\x1b[1;66H\x1b[7m[reconnecting]\x1b[0m\x1b[u"
```
The tail shows the screen frozen on the history page (rows 54–77) with
the `[scroll]` badge still lit and `[reconnecting]` painted over it: the
client sat in scroll mode through the reconnect, exactly as the deleted
line was there to prevent.
**The 412f38f revert — the exact counter.** Deleting only
`overlay.setScrollMode(false);`, with `scroll_pages = 0;` left in place
so this is 412f38f's defect and not row 18's, leaves the overlay
suppressing predictions with no page left to suppress for. The screen
looks right; prediction is silently dead. The scenario catches it on
counters, not on content:
```
e2e FAIL: pty scroll reconnect: made=0, want 1
predict made=0 displayed=0 confirmed=0 contradicted=0 expired=0 abandoned=0 suppressed=1
```
The check that fired is the exact counter assertion — `want 1`, not
`want >= 1` — and the stats line names the mechanism outright:
`suppressed=1` with `made=0`, the keystroke swallowed by a scroll mode
that outlived the scroll. This is the fix that was previously guarded
only by reading and by a unit-level contract test.
**Independent corroboration.** None of the three rests on the single
sitting. tp2b's width witness was seen red 10/10 by the implementer and
independently confirmed 4/4 on the full suite and 10/10 targeted by the
reviewer; tp1's two were each seen red during development, and the spec
reviewer independently re-ran the 412f38f revert. Three for three: two
rows M11 had to bank as "needs a pty-driving client fixture" are graded
CAUGHT by that fixture, and the 412f38f fix has an end-to-end pin for the
first time.
### What shipped
`Pty.spawnArgv` (`src/pty.zig`) — one child-setup path for both the
daemon's shell spawn and the fixture's argv spawn, with an optional
stderr redirect off the pty and parent-side `EmptyArgv` validation. It
also completed 7b4208f's `exit_group`-only sweep, which had missed
`pty.zig` entirely; two further instances were caught in review.
`test/ptyclient.zig` (~490 lines) — a script engine over that pty:
`send`, `expect`, `resize`, `settle`, `waitexit`, an `Expecter` whose
cursor *consumes* the match rather than the buffer, distinct exit codes
(2 usage, 3 timeout, 4 child died), escaped-tail diagnostics on failure,
and a bounded final drain. Five fixture controls guard the fixture
itself: the cat roundtrip, an impossible expect that must fail at exit 3
*exactly*, and the stderr-split leg proving predict stats land in the
`.err` sibling and never in the capture the convergence machinery diffs.
Three scenarios: tp2a (first attach at a real 100x30), tp2b (resize
mid-session — the row-7 catcher), tp1 (reconnect while scrolled — row 18
and the 412f38f pin). `converged_quiet`/`assert_converged` grew optional
size arguments so a non-80x24 grid can be compared against the size it
actually is.
### The finding that reshaped the scenarios: delta rows never touch the replica
Load-bearing, and it inverted the plan. Delta rows do **not** pass
through the replica on their way to the tty — `paintDeltaClipped` paints
them straight out, absolutely addressed, clipped to the tty. The replica
is read back only at a **full repaint** (`renderClipped`, clipped to
`min(replica, tty)`). The consequence is that a stale replica is
*invisible* on the delta path: it is observable only when damage is on
screen at a full repaint **and** outside the reach of the stale geometry.
Three things follow, and all three are now in the file at their sites.
The plan's original resize-DOWN scenario could never have caught row 7,
because a too-wide replica clips identically to a correct one. tp2b
therefore resizes **up**, with the 95-wide row printed *before* the
winch, so the stale 90x28 replica has somewhere to show. And tp2a is
documented as **not** exercising the prefix at all: an attach applies the
attacher's size to the grid server-side before the answering resync, so
the prefix always equals the size the replica already holds and the
resize-on-prefix guard never fires on a first attach. That last one is a
negative result, verified by probe rather than by reading, and it is
recorded in the scenario's own comment so the next reader does not
mistake tp2a for coverage it does not provide.
### The choreography war, and the rule it produced
Prompt-row sentinel occurrences have **no order guarantee** relative to
output rows once a scrolling repaint splits across deltas. Two
sentinel-counting shapes of tp2b failed under a 16-way nice'd load gate —
9 failures in 150 and 15 in 150 — and the worst specimen is the one worth
remembering: a **clean** build failing with a diff byte-identical to row
7's mutation signature, missing rows and a still-wrapped wide row. A
harness that cannot tell a load flake from the bug it exists to catch is
worse than no harness.
The cures are in the file as comments and are the general form: sync only
on (a) content unique to its phase, (b) a **structural witness** — bytes
that are impossible before the event, such as 91 contiguous zeros at a
width of 90 — or (c) `settle`, silence, **after** an arrival proof and
never alone, because settle's quiet window starts at the verb and never
requires a byte to have arrived. Witness and settle each cover the
other's blind side: the witness permits a detach mid-repaint, and settle
alone passes having observed nothing at all. Hence the rule: **an
assertion about order must be an assertion the bytes can actually
carry.**
The loaded-gate scoreboard, since the numbers are the argument: sentinel
shapes 9/150 and 15/150 failures; the witness shape 150/150 pass;
witness+settle 150/150; tp1 20/20 with a byte-identical counter line
every run. And the reason the gate had to be loaded at all: idle runs
measured **0 failures in 420** on a shape that failed 6% of the time
under load. Idle statistics cannot gate this flake class.
### The prediction counters are exact, and that took exactly one keystroke
`offerKeystroke` refuses any stdin chunk that is not one byte, counting
it suppressed, so a multi-byte send predicts nothing — and consecutive
sends have no barrier between them, so the client's next read can pick up
two at once. Measured on a three-keystroke draft of tp1: `made=3` on one
run and `made=2` on the next, identical scripts. Under load a full
coalesce reads `made=0`, which is the mutation's own signature.
tp1 therefore types **one** character and asserts five counters exactly:
`made=1 displayed=1 confirmed=1 contradicted=0 suppressed=0`. That pins
all seven, because `expired <= contradicted` (they share the `abandonAll`
tail — the M11 invariant again) and `made = confirmed + abandoned +
pending` forces the remainder. The arrival needle is structural for the
same reason the witness is: in canonical mode the only thing one
keystroke produces is the line discipline's echo of the very glyph the
prediction just painted, so no *content* needle can tell the daemon's
answer from the client's guess. `\x1b[2K` can — it reaches the client's
stdout from exactly one paint path (`paintDeltaClipped`), predictions
paint with no erase, and both full repaints use `2J`.
### Smaller findings, each with its rule
- **`spawnArgv` execve's `argv[0]` with no PATH search.** An
`env VAR=x cmd` prefix therefore dies at 127 before the first verb.
Harness-side environment export is the form, and the comment says so.
- **The doctored-stream control was a no-op on pty captures.** `render`
replays only up to the last alt-screen exit, and a tty client's capture
*ends* with one, so appended bytes landed after the grid under test and
changed nothing — a control that could not fire. Fixed by dropping the
trailing 8-byte `\x1b[?1049l` first; that tail is now asserted by `od`
rather than assumed, because if teardown ever stops ending there the
control would silently go back to being a no-op.
- **`kill -0` cannot detect the death of the shell's own unreaped
child** — signalling a zombie succeeds. tp1's daemon guard now proves
the daemon is *serving*, via `muxd dump`, not that it exists.
- **A Zig mutation that orphans a capture fails the BUILD**, which reads
as a kill and proves nothing. Discard the capture in the same edit.
- **`waitexit` drains to a bounded quiet after observing the exit.**
Bytes arriving between the last poll and the `waitpid` were silently
lost; provable to 0-byte captures with a widened window.
### Method note
The reviews were adversarial and load-bearing, which is the only reason
several of the above are recorded as findings rather than shipped as
bugs. The Task 4 spec reviewer personally ran the row-7 resurrection and
proved the first committed version of the scenario graded nothing. The
implementer measured the doubled-sentinel cure at one red in ten before
it could land, and proposed the witness instead. Two inaccuracies in
implementer reports were caught by reviewers re-verifying the claims
rather than reading them — a "verbatim transcription" that was not one,
and a "reaps by pid" that was actually the tty's SIGHUP. One process
lesson, without naming infrastructure: instructions issued against a
moving tree can be stale by the time they arrive, so instructions should
address content, not commit SHAs.
### Banked by M12
- **The outer `timeout` on the pty scenarios sits below the sum of their
verb deadlines** — 40 on tp2 against ~65s of verbs, 60 on tp1 against
~120s. House-consistent with every other scenario's outer bound, and
noted rather than changed: the verb deadlines are the diagnosis and the
outer bound is only the backstop, but a slow machine would report the
backstop's silence instead of the verb's message.
- **tp2b's `fill-done` expect retains a survivable one-sided race
class** — survivable only while the content it precedes is what it is
today; a new kind of content typed after it would need the same
witness-or-settle treatment the post-resize verbs got.
- **Loaded gating exists only as session tooling** — a scratchpad driver
that runs the suite N times under a 16-way load, not a repo target.
Every number in the choreography section came from it, and the idle-vs-
loaded gap (0/420 against 6%) is the argument for a future `make gate`.
- The M11 banks are unchanged: prediction polish, the unit-layer mutation
sweep, ASAN/valgrind over the QUIC tests, and rows 4, 5 and 12 as
properties no rendered grid can represent.
## 2026-08-10 (M13 — trial friction: attach auto-start, `muxd stop`, error audit)
**Verdict: cleared, both legs.** The three findings a week of real use
produced are closed, and none of them by a message-only fix. `mux
user@host` and `muxd proxy` against a box with no daemon now start
one — one helper, `spawn.ensureForAttach`, at both call sites, spawning
a **bare** `run --sock` (never `--quic`), with no opt-out and a
contractually silent warm path. `muxd stop` is a protocol verb,
`stop_req = 0x07`, armed on **both** dispatches and idempotent when
nothing is listening. The audit reworded six messages and added two
refusals, the `sun_path` one turning a two-second timeout story into an
instant named no — **2.006s → 0.0014s** measured on the `mux` side. Leg
1: the suite grew to **15 scenario checkpoints over 28 convergence
points**, both pinned as literals, with `SOAK_N=10 make soak` at
**10/10 on the code that ships** (414d282). Leg 2: three resurrections,
three catches, each dying at exactly the check it was predicted to die
at and printing exactly the predicted text. The rule the milestone
produced is a grading rule: **a regrade catch must be legible** — a
bare command under `set -e` with stderr redirected is an anonymous
abort whose evidence the trap deletes, so the predicted-catch *text* is
part of the criterion, not commentary on it. Spec:
superpowers/specs/2026-08-10-m13-trial-friction-design.md.
### The regrade table
One sitting, one tree, one set of binaries; baseline green before, final
state green after, `git status` clean throughout. Each row names the
resurrection, what the milestone predicted it would print, and what it
actually printed.
| Resurrection | Predicted catch | Verdict |
| --- | --- | --- |
| The `muxd proxy` auto-start call site reverted | `e2e FAIL: proxy auto-start: attached daemon lost the marker`, with `muxd proxy: cannot connect to <sock>` in the dump | **CAUGHT** |
| Both `stop_req` arms deleted (`serviceObserver` and `handleFrame`) | unit: both Task 2 tests fail at `expect(shutdown_flag.load(.acquire))`; e2e: `e2e FAIL: stop exited 1, want 0` followed by `muxd stop: <sock> still answering after 2s (if it was started detached, its log is <path>)` | **CAUGHT** |
| `lostMsg` reverted, implementation only | unit: the old-vs-new string diff; e2e: `e2e FAIL: --via death message:` and the old string, at `test/e2e.sh:499` | **CAUGHT** |
**The proxy call site — the dump corroborates twice over.** The
predicted line and the predicted dump content both appeared, and the
capture carried two lines nobody had asked for: the probe's own
`muxd dump: nothing listening on ...`, and a trailing `mux: transport
command failed before a session started` — the audit's new wording
naming the failure from the other end of the same event. Three
independent messages, one cause, no guessing.
**Both `stop_req` arms — the deadline is the failure's own.** Both lines
appeared, in the predicted order. The cost of the failure was ~2s
(124s of wall against the baseline's 123s), and that 2s is the verb's
own deadline expiring, not a suite timeout absorbing it — which is the
difference between a scenario that reports and a scenario that hangs.
The trap reaped every daemon even though the mutant ignores the stop
verb outright: teardown does not depend on the feature under test.
**`lostMsg` — the first line of defense fired.** The e2e catch is at
`test/e2e.sh:499`; the belt-and-braces control at `:531`, which fails if
the *old* wording is still emitted anywhere in the capture, was never
reached, because the positive pin had already exited. A control that
never fires because the primary caught it first is the layering working.
### What shipped
`spawn.findInPath` (`src/spawn.zig`) — PATH search taking PATH as a
parameter, with empty segments skipped so the implicit current directory
is never searched, while an explicit relative entry is honored as the
choice somebody typed. `probe` is now public, because both the stop verb
and auto-start need to ask "is anything listening" without attaching.
`stop_req = 0x07` (`src/protocol.zig`) — armed in **both** dispatches.
The load-bearing arm is `serviceObserver`: a connection starts as an
observer and is promoted only on attach, and `muxd stop` never attaches.
The `handleFrame` arm exists so the verb means the same thing on any
connection rather than only on the one shape of connection the client
happens to use today. Safety against an old daemon rests on `MsgType`
being non-exhaustive — an unknown verb is already a defined outcome.
`muxd stop` (`src/main.zig`) — connect, send `stop_req`, then poll
probe-first/deadline-second until connect refuses, then `muxd: stopped`
and exit 0. Write errors are swallowed deliberately (a daemon that dies
mid-write did what was asked), but an `asked` bool records whether the
request was ever delivered, so the timeout message cannot claim a
request that never left. Nothing listening exits 0 — the verb is
idempotent, which is what makes it usable in a teardown. A wedged daemon
gets the still-answering line with the log clause attached only when
there is a log path to name, and exit 1.
Attach auto-start at both sites via `spawn.ensureForAttach`, one
`start_deadline_ms = 2000` for both. `muxd proxy` uses `/proc/self/exe`
and calls from `main.zig`'s dispatch arm, so `proxy.zig` stays
import-clean; local `mux` uses `findInPath`, and a missing `muxd` is
fatal only if the probe *also* fails — the message is then `mux: no
daemon on <sock> and no muxd in PATH to start one`, which names both
halves of the situation rather than the half that happened to be checked
last.
The audit: six rewords, two new refusals (`sun_path` over 107 bytes, in
both binaries), `muxd dump`/`muxd stats` now say ``nothing listening on
<sock> (`muxd start` starts a daemon)``, and `muxd start`'s
already-running hint became fully actionable — ``stop it first with
`muxd stop --sock <sock>` `` — which it could not have been before this
milestone, because the verb it names did not exist. `lostMsg` is now
`mux: transport command failed before a session started`: ssh's own
stderr has already named the cause, and the old parenthetical guessed a
different one.
e2e grew from 13 scenarios / 25 convergence points to **15 / 28**: the
proxy cold/warm arc (with the warm-silence pin, `muxd stop` as the
tested teardown, and an idempotence control), a pty leg for local-`mux`
auto-start — the first reuse of M12's fixture by a later milestone — and
the `sun_path` refusals with both a timeout-story can-fail control and a
`--version` exemption pin. A `wait_pid_gone` helper replaced the
open-coded waits.
### Findings
- **A connection is an observer until it attaches.** Any verb that must
work on a bare connection belongs in `serviceObserver`, not
`handleFrame`. Found because the spec cited a *test helper's* switch as
though it were the dispatch — the helper's shape had drifted from the
daemon's, and reading the helper would have shipped a verb that worked
only for clients that had already attached, which `muxd stop` never
does.
- **Auto-start turns "a second daemon on another socket" from an
explicit act into an ssh side effect.** That is the whole reason the
shared daemon log became a hazard: a truncating open on a log another
live daemon is appending to leaves a NUL hole in the middle of the
file. The fix is `O_APPEND` — an atomic seek-to-end per write, not a
one-time seek — and the reviewer's proposed `createFile(.truncate =
false)` would have been a second bug, front-overwriting the existing
contents from offset 0. Truncation is now reserved for `muxd start`,
where somebody asked for a fresh daemon. The cost is named and
accepted: monotonic growth for a user who never types `start`,
rotation banked against `logPathFor`. A CLOEXEC regression in the
hand-rolled open (`createFile` had been setting it for free) was caught
in review and verified by observation rather than by reading — before
the fix, `/proc/<pid>/fd` showed the user's own shell holding fd 3 on
`muxd.log`; after, zero.
- **Review prescriptions are themselves fallible, and the pipeline
caught its own.** Three in one milestone: a reviewer's blank-line fix
that did not compile (it left an unattached doc comment), the
`truncate = false` prescription above, and a line-number
cross-reference that went stale *within its own review cycle*. An
implementer rejecting a bad prescription with evidence is the process
working, in the direction it is easier to forget it runs.
- **Two defects were traced to the plan's prescribed code and amended
there first.** The `stopCmd` loop checked its deadline before the final
probe — a failure line about an interval nobody had actually checked —
and the failure path called `try xdg.logPath`, which under an absent
HOME replaces the verdict with a stack trace. Both were fixed in the
plan before being fixed in the tree, because a defect left in the plan
is a defect the next implementer re-derives.
- **A regrade catch must be legible.** A bare command under `set -e`
with stderr redirected produces an anonymous abort whose evidence the
cleanup trap then deletes. The predicted-catch text is part of the
criterion: a resurrection that kills the suite without printing its
line proves the code is load-bearing but not that the check is.
- **`TCSAFLUSH` discards input queued across the auto-start spawn
window.** Raw-mode entry flushes what the fixture typed while the
daemon was starting, so a pty scenario must open on `expect
\x1b[?1049h` — the alternate-screen enter — before its first send.
M12's rule again, in a new costume: the barrier has to be a byte the
scenario can actually observe.
### Method note
Subagent pipeline, as M11 and M12: one implementer and two reviewers per
task, quality and spec, with every new assertion written mutation-first
(write the test, break the code the prescribed way, *see* it fail,
restore, see it pass). The adversarial flow is only worth its cost if it
runs in both directions, and this milestone is the evidence that it
does: three reviewer prescriptions were rejected with evidence rather
than implemented, and two plan defects were amended at the source
document before the tree was touched. The three resurrections were run
against a green baseline in one sitting, with the tree verified clean
before and after; no product code was left mutated.
### Banked by M13
- **Log rotation via `logPathFor`.** The named, accepted cost of the
append-only decision: a user who only ever attaches never truncates
the daemon log. The seam exists; the policy does not.
- **Per-socket daemon logs.** Auto-start makes a second daemon a side
effect of an ssh command, and every daemon appends to the same file.
Per-socket paths are what make a multi-daemon box readable.
- The M11/M12 banks are unchanged: prediction polish, the unit-layer
mutation sweep, ASAN/valgrind over the QUIC tests, a `make gate` for
loaded runs, and the M12 pty-scenario timeout notes.
## 2026-08-11 (M14 — the ssh→QUIC handoff)
**Verdict: cleared, both legs.** `mux HOST` stopped being sugar for an
ssh pipe and became a transport negotiation: one ssh to `muxd endpoint`,
whose first stdout line is a mandatory announce of the daemon's QUIC port
and key, then QUIC carries the session and the coordination ssh is
reaped. The coordinates are cached, so the second attach spawns no ssh at
all. Leg 1: the suite grew to **20 scenario checkpoints over 33
convergence points**, both pinned as literals, with `SOAK_N=10 make soak`
at **10/10 on the code that ships** (d9b17d5). Leg 2: three
resurrections, three catches, each legible at the check that predicted
it. The kill criterion held on the real LAN box (`192.168.0.109`) with
`muxd` on the remote PATH as the one manual step. Spec:
superpowers/specs/2026-08-11-m14-ssh-quic-handoff-design.md.
### The regrade table
One sitting, one tree, one set of binaries; `make build && make test &&
make e2e` green before, green after, `git status` clean between every
row. Each mutant was required to **compile** — a build failure
masquerading as a kill grades nothing.
| Resurrection | Predicted catch | Verdict |
| --- | --- | --- |
| `serviceObserver`'s `endpoint_req` arm deleted | unit: `Server: endpoint_req binds a listener lazily…` fails first and cheapest; e2e (a) fails too | **CAUGHT** (20s unit, 128s e2e) |
| `endpointCmd`'s announce write deleted | e2e (a) fails at its bounded wait with the scenario's own message, the announce absent from the dump | **CAUGHT** (149s) |
| `openHandoff`'s fallback stderr line deleted | e2e (d) fails its grep while (c)'s absent-grep control stays green in the same run | **CAUGHT** (156s) |
**The observer arm — the unit layer answered first, as predicted, and
named the missing thing itself.** `make test` failed at
`src/server.zig:4634` with `return error.NoEndpointReply`, which is the
whole diagnosis in one identifier, 20 seconds after the mutation. The
e2e corroboration landed too, but **not at the assertion the prediction
named**: (a) died at `e2e FAIL: cold handoff: QUIC took over, so ssh
must be gone: pid 2931354 is still running 2s later`, not at the later
cache/QUIC assertion, because that is the earliest check in (a) the
mutation reaches. The prediction was right about the scenario and wrong
about the line — which is finding 3 below, at assertion granularity
rather than scenario granularity.
**The announce write — the failure says why, not just that.** Both lines
of the scenario's own message printed:
```
e2e FAIL: cold handoff: no marker in 25s — the attach never converged.
A client that blocks here read no announce line off the ssh pipe.
muxd endpoint: starting…
up (0.1s) pid=2940304
mux: aborted before attaching
```
The diagnostic dump is what separates this kill from Task 7's
stderr-routing kill, which produces the same first line: here `muxd
endpoint`'s own preamble is present and the `endpoint …` announce is
simply absent from it. The mutation as the plan first wrote it — moving
the write below `return proxy.run(…)` — does not compile: Zig rejects
unreachable code outright, so the sanctioned alternative was taken and
the write deleted. Recorded because "each mutant must compile" and "make
the announce unreachable" were, in this language, incompatible
instructions.
**The fallback line — the control proved it was a control.** (d) failed
with `e2e FAIL: key mismatch printed no fallback line in 15s; its
stderr was:` and an empty capture, in a run whose immediately preceding
line was `e2e OK: a stale cache self-heals: one refetch (2231ms), no
fallback line, real port cached`. (c) asserts that line is *absent* and
(d) asserts it is *present*; deleting the line flipped exactly one of
them. A pair like that is the only way an absent-grep assertion can be
shown to be load-bearing rather than vacuous.
### The kill criterion, on the LAN box (2026-08-11)
`ubuntu@192.168.0.109`, a static musl `muxd` scp'd to the box and put on
PATH — the one allowed manual step. Both ends zeroed first: remote
daemon stopped, `~/.config/mux/key` removed, local cache entry removed,
every local `mux` run under hermetic `XDG_*` so the developer's own key
and cache were never touched. All three legs then ran with **zero
further manual steps**.
| Leg | Wall | What was observed |
| --- | --- | --- |
| Cold | **244ms** attach → first marker | ssh child (`ssh ubuntu@192.168.0.109 muxd endpoint`, pid 2959493) gone by `kill -0` at the moment the marker painted, while `mux` stayed alive and a second marker sent 8s later still painted |
| Warm | **13ms** attach → marker | **no ssh child at any point** — polled across the whole session, not just the attach |
| Blocked, warm cache | **4264ms** | fallback line printed, session landed over ssh, exit 0 |
| Blocked, cold cache | **2177ms** | same line, one deadline + 177ms |
The cold leg's announce was `endpoint 56407 8560…a307`, written to the
cache verbatim; on the box `ss -uapn` showed `UNCONN 0.0.0.0:56407
users:(("muxd",pid=17958,fd=6))` and `muxd stats` reported `clients=1
deltas=3` — a session being served by deltas over a listener that did
not exist before the attach asked for it. The blocked legs dropped
inbound UDP with `iptables -A INPUT -p udp --dport 56407 -j DROP` and
both printed the exact coordinates they had tried: `mux:
quic://192.168.0.109:56407 unreachable, attaching over ssh`. The rule
was removed on the way out and the removal proved by listing the chain
(`-P INPUT ACCEPT`, nothing matching `--dport 56407`), not by trusting
`iptables -D`'s exit status. Teardown was `muxd stop` over ssh: `muxd:
stopped`, exit 0, then `pgrep -a muxd` empty, no UDP listener, socket
file gone.
**The two blocked numbers are the interesting result, and they are not
the same case.** From a **cold** cache the blackhole costs **one**
deadline, because the ssh that carried the announce is still open and
*is* the fallback transport. From a **warm** cache it costs **two**: the
cached dial spends a deadline, then the refetch over ssh returns the same
(still-blocked) port and spends another before falling back. The spec's
criterion says "within one deadline" and the cold leg meets it as
written; the warm leg is recorded at 4264ms rather than reconciled,
because it is what a user on a UDP-blocked network actually pays on
every attach after their first. It is the direct cost of refusing
negative caching, which remains the right call — network state changes,
and a cached "UDP is blocked" would strand a user on ssh after their
firewall was fixed.
**A stale daemon from M10 had to be killed by pid, not stopped.** The
box was still running the Aug-9 binary, which predates `stop_req`
entirely, so `muxd stop` printed usage and exited 2. It was killed by
its tracked pid and the absence confirmed with `pgrep`. Not part of the
criterion — state zeroing, not attaching — but the shape is worth
knowing: the stop verb is only as available as the binary on the far
side, and a box that has been up across a milestone boundary may not
have it.
### What shipped
`endpoint_req = 0x08` / `endpoint_reply = 0x89` (`src/protocol.zig`),
with an `encode`/`decode` pair and golden wire bytes pinned as literals,
following the rule M10 extracted — assert the literal, never the
constant the code under test reads.
The daemon's **lazy QUIC bind** (`src/server.zig`). `endpoint_req` is
answered by binding a listener on demand if one is not already up, and
the load-bearing arm is `serviceObserver` for M13's reason: `muxd
endpoint` never attaches, so it is an observer for its whole life. A
`quic_owned` flag records whether the listener was created by this path,
so `deinit` hands back exactly what it took and a daemon started with an
explicit `--quic` keeps ownership of its own. **Became a `union(enum)`
in M15 (Task 9)**, which makes the `(null, owned)` pair untypable rather
than merely unreachable.
`src/handoff.zig` — one grammar, one place. Cache read/write, announce
`format`/`parse`, the dial-host strip and the deadline constant all live
in a pure module with no sockets and no processes, so the entire surface
tests without a daemon. **The announce is mandatory in both
directions**: `endpoint none` is an explicit negative rather than
silence. The spec was amended (dd6c511) to say so after the alternative
was found to be unresolvable — at the reading end, "the remote has no
QUIC to offer" and "ssh is being slow" are the same zero bytes, and the
client blocks on one line either way.
`muxd endpoint` (`src/main.zig`) — `ensureForAttach`, then key
resolution matching the daemon's exactly (`MUX_KEY_FILE`, then the
default path, and only the default is ever created), then the announce
as the **first** stdout bytes, then the byte-for-byte `muxd proxy` pump.
Every unhappy path announces `none` and stays on ssh rather than
panicking: a bug in a preamble must not take down a session ssh was
about to carry perfectly well.
Client-side, the handoff sits at the **Transport** layer
(`src/client.zig`), not above it, so a reconnect re-runs it rather than
resurrecting a dead recipe. Cache first, ssh second. The fallback line
is attach-only (`report_fallback`) so a flapping link cannot turn it
into a log. `readAnnounceAbortable` has **no timeout by design** — it is
bounded by ssh itself, the same trust `--via` already extends to the
command it spawns — and it honors the carry contract, holding
non-abort bytes typed during the wait rather than dropping them.
### The deadline, pinned from measurement (Task 4, 2026-08-11)
**There is no fast-failure case.** A wrong PSK against a live listener
ran 2037ms of a 2000ms budget, 3049 of 3000, 8031 of 8000: mutual auth
means the listener does not answer a peer it cannot authenticate, so
silence is all the client ever gets. A blackholed UDP port did the same
(4001 of 4000, 8001 of 8000). So did an unbound port **on loopback**,
2051ms of 2000 — the ICMP refusal is real, but it lands on the `sendto`
inside `Client.drain`, whose `catch return` discards it, so the
`ConnectionRefused` branch in `readable` never sees it. Every failure
runs the budget out, and the deadline is the only thing that ends any of
them.
**Superseded in M15 (Task 4) for the refusal case only.** Both socket
paths now route ECONNREFUSED through one verdict, so an unbound loopback
port fails in **1ms** instead of 2051. Silence still runs the budget out
and still sets the number: re-measured 2026-08-12 at 2012ms for a wrong
PSK and 2002ms for a blackholed port, so 2000 stands. The same task gave
`quic://` its own `deadline_ms`, so a direct dial to a dead host now
gives up at 2000ms rather than at the connection's 15000ms idle timeout.
Silence spending the budget fixes the shape of the trade. Below it: a real
handshake is **4ms** local, **6.9ms** cold attach on the M8 LAN, and
**234ms** at 75ms RTT — 3.1× RTT, because every fresh Initial costs a
Retry round trip — so 2000 covers RTT to ~645ms, past any terrestrial
link. Above it: with negative caching refused, every attach on a
UDP-blocked network pays this in full, so it wants to be as low as the
floor allows. 1000 would halve the tax and also halve the RTT ceiling to
~320ms — buying latency on a degraded network at the price of abandoning
QUIC paths that would have worked. Wrong way round.
### Findings
- **A wedged `zig test` step prints nothing at all.** Not a truncated
report, not a partial list — nothing, for the whole timeout. So the
ordering rule from M13 ("a regrade catch must be legible") needs a
second clause: a legible catch must also be ordered **before** any
test the same regression can hang. The ownership pin is the case that
taught it. Left where it was, the mutation produced a 600-second empty
wedge with no output to read; moved ahead of the test that hangs, the
same mutation produces 8 named failures in 14 seconds.
- **The suite was running the developer's login shell.** Hermetic
`XDG_*` homes cannot make a suite hermetic while it still executes
arbitrary user rc files: a zsh plugin manager cloning a repository at
shell startup ate a convergence window and failed a scenario that had
nothing to do with it. `SHELL` is now pinned suite-wide. The general
form is that "hermetic" has to include the *code* the environment
causes to run, not only the paths it points at.
- **A mutation only proves the earliest scenario that catches it.**
First-failure exit means every later scenario is untested against that
break, and a campaign that reports "five kills" is silently reporting
five *first* kills. Task 7's five left scenario (e) and (c)'s
self-heal arm unproven until the accounting was audited scenario by
scenario; re-running against each in turn brought the real total to
eight. Kill counts are claims about ordering as much as about
coverage.
- **Review prescriptions remain fallible in both directions, and the
pipeline caught both.** A spec reviewer prescribed a "complementary"
post-hoc liveness check on the reaped ssh; it was **unfalsifiable** —
dead processes do not resurrect, and a reused pid presents as alive —
so it could only ever pass or lie, and an independent review proved
that and deleted it. The same review corrected the lead's own `set
+e` alignment. M13's rule holds: an implementer rejecting a bad
prescription with evidence is the process working in the direction it
is easier to forget it runs.
- **`waitReady` answers the detach chord *during* a dial.** That is
correct behavior — a user who presses Ctrl-\ while waiting asked to
stop — but it means any scenario built to spend a deadline must hold
its detach bytes out past the budget, or the client aborts early and
the scenario passes having tested nothing. A timing assertion whose
subject can be cancelled by its own fixture is not an assertion.
- **A fixed port must dodge the kernel's ephemeral range.** Read
`/proc/sys/net/ipv4/ip_local_port_range` and stay out of it: a port
chosen for being dead can come alive mid-suite when something
unrelated is handed it, and the scenario that depended on nothing
listening there then fails for a reason no one will find by reading
it.
### Errata
The commit message of bde1af5 states `kill -0`'s polarity backwards. The
code and its comments are correct; only the prose in that message is
wrong. Recorded here rather than fixed by rewriting history, which is
the standing preference — a wrong sentence with a correction beside it
is more honest than a history that never contained it.
### Process note
Subagent pipeline as M11–M13: one implementer and two reviewers per
task, quality and spec, every new assertion written mutation-first. One
new shape is worth recording. A spec reviewer was blocked by the
permission classifier on `make e2e`, three times, situationally rather
than by rule. It did not work around the block, invent an equivalent
command, or report a verdict it had not earned: it stopped, reverted its
own edit, and handed the exact change to the implementer to run. That is
the correct response to a tool refusal, and it is recorded because the
tempting alternatives all produce a green report that means nothing.
### Banked by M14
- **`drain()` swallows `ECONNREFUSED`** — the Task 4 incidental, and an
M15 candidate. Fixing it would make unreachable ports fail in ~1 RTT
instead of a full deadline. **Re-pin, do not delete:** e2e (d)'s
`elapsed >= floor` bound and `client.zig`'s dead-coordinates test
bound both currently lean on the quirk, and both would need re-timing
in the same change.
- **The transport recipe as a `union(enum)`** — from Task 6's quality
review. `Transport.open` now takes four mutually-exclusive nullables,
two of them sharing a type; `union(enum) { sock, via, quic, hand }`
makes the invariant unstatable-wrong rather than documented. M15.
- **Per-client keys, certificates and TOFU** — still parked, unchanged
since M10. The handoff hands out the *same* shared key over ssh, which
is exactly the trust model M10 chose and no worse; per-client
credentials remain the thing that would change it.
- **Negative caching of "UDP blocked"** — still refused, by design, and
the LAN box's 4264ms warm-blocked attach is now the measured price of
that refusal rather than an estimate of it.
- **The WAN-box RTT on record disagrees with itself** — ~290ms in one
place against the 15.7–16.5ms baseline at decisions.md:314. Verify
against the live box before the next measurement leans on either.
- **Scenario (e) has two unexercised assertions** — its absent-fallback
grep and its no-budget bound were never driven red. Cheap candidates
for the next regrade, and cheaper than the audit that found them.
## 2026-08-12 (M15 — refactor: typed invariants, single owners, module seams)
**Verdict: cleared.** Twelve tasks, each run as implementer + spec
reviewer + quality reviewer, landed in **37 commits without moving the
product**: the e2e suite stayed at **20 scenario checkpoints over 33
convergence points** from the first commit to the last, and `SOAK_N=10
make soak` was **10/10 on the code that ships** (445d80b). The unit layer
grew from **245 to 255 tests**, and exactly **two behaviours changed** —
both in QUIC dialling, both intended, both measured below. Spec:
superpowers/specs/2026-08-12-m15-refactor-design.md.
Four modules were carved out with their tests — `paint.zig` (295),
`quic.zig` (586), `delta.zig` (233), `sockpath.zig` (122) — and the two
files they came out of shrank where it counts: `server.zig` **4708 →
4474**, `quic_server.zig` **2412 → 1972**. `client.zig` went the other
way, **2406 → 2506**, and the split says why: its implementation half is
flat at **1640 → 1647**, and **+93 of the +100 is new test code**. That
is the coverage section below arriving in the line count, and it is the
reason a whole-file `wc -l` is a poor clause to write a refactor against
— it charges a file for the pins the refactor bought it.
### The two behaviour changes
Both live in QUIC dialling, and both were the same omission seen from
two sides: **nothing distinguished a refusal from silence.**
`drain()` had been discarding the queued ICMP refusal with a bare `catch
return` — M14 recorded this as an incidental — so the `ConnectionRefused`
branch in `readable` never saw it and every failure ran the deadline out.
Task 4 gave both call sites one verdict, `sendRecvFailed`, so a refusal
is acted on wherever it lands. Separately, `quic://` had no attach budget
of its own and gave up only at the connection's 15000ms idle timeout;
`QuicTarget.deadline_ms` split the two, so the attach budget is 2000ms
and `idle_ms` goes back to meaning what its name says.
Local, medians, same box, before and after:
| Dial | Before | After |
| --- | --- | --- |
| refused — an unbound loopback port | 15001ms | **1ms** |
| blackholed address | 15001ms | **2002ms** |
| wrong PSK — a live port answering nothing | 15009ms | **2012ms** |
**The 2000ms deadline stands, and silence is still what sets it.** Only
the refusal got fast; the two silent cases land within 12ms of the
budget, which is the deadline doing its job rather than a bound anyone
tightened.
The named cost: **`--quic-idle-ms` no longer raises the attach budget by
accident.** It used to, because the attach shared the idle timeout, and
that made it an undocumented escape hatch for a slow link. The hatch is
gone. It is irrelevant below roughly 645ms RTT, and no path mux has been
measured on comes close.
### On the LAN box, against an M14-era daemon (2026-08-12)
The box at `192.168.0.109` was left on its pre-M15 `muxd` **on purpose**.
Both behaviour changes are client-side, so measuring against the old
daemon also measures cross-version compatibility. Version strings match
(`mux 0.0.1-3` local, `muxd 0.0.1-3` remote) because the release was not
cut until M15 closed; the binaries do not — the remote one was built at
**06:27**, eight hours before M15's first commit at 14:31.
| Leg | Reps | Median |
| --- | --- | --- |
| refused — `quic://192.168.0.109:11997`, the box answers ICMP | 1 / 1 / 1 ms | **1ms** |
| blackholed — `quic://192.0.2.1:11997` | 2002 / 2002 / 2003 ms | **2002ms** |
| control — attach `quic://192.168.0.109:4433` | exit 0, marker painted, `\034` detach clean, daemon reported `clients=0` | — |
The control leg is the cross-version one: an M15 client drove a session
on an M14 daemon, painted, detached cleanly and was observed gone from
the daemon's own count. The refusal that cost ~15s before M15 now costs
1ms across a real network, and the blackhole still costs one deadline.
### The regrade table
One sitting, one tree, one set of binaries. Each mutant was required to
**compile**, `git status --porcelain` was empty and `make test` green
between every row, and every revert was a file copy — never a `git
checkout`, for the reason in the process note.
| Resurrection | Layer | Test that answered | Verdict |
| --- | --- | --- | --- |
| `drain()`'s send-error arm reverted to `catch return` (leaving `sendRecvFailed` itself) | unit | `client: handoff: dead coordinates are a fast no, and the pipe is the fallback` | **CAUGHT** |
| `Transport.open`'s `.sock` and `.via` arm bodies swapped, captures renamed so the bodies moved verbatim | unit | `client: Transport.open: a --via target yields a pipe, a --sock target an fd` | **CAUGHT** |
| `keyRefusalBody`'s `chmod 600` → `chmod 601`, implementation line only | unit, **two** | `quic: keyRefusalBody: the words four binaries print, byte for byte` **and** `client: openFailure: a quic:// target names the key or the address, and only an abort exits 0` | **CAUGHT** |
**All three answered at the unit layer; `make e2e` was never needed.**
That is not the unit-layer mutation sweep the roadmap has banked since
M11 — three aimed mutants are not a sweep — but it is the third data
point arguing for one, and the cheapest yet: seconds per row instead of
the two minutes an e2e run costs.
Each catch named the property rather than the symptom. The refusal
mutant printed its witness line:
```
error: 'client.test.handoff: dead coordinates are a fast no, and the pipe is the fallback' failed: mux: quic://127.0.0.1:1 unreachable, attaching over ssh
refused dial took 302ms of a 300ms budget: the ICMP refusal was swallowed, not acted on
```
— a number, a bound, and the diagnosis, in one sentence. (The first line
carries the client's own stderr, which is the fallback working
correctly; the assertion is the second.) The dispatch mutant printed its
named decision:
```
error: 'client.test.Transport.open: a --via target yields a pipe, a --sock target an fd' failed: a --via target must spawn a command, not connect a socket: open failed with FileNotFound
```
The literal mutant printed a byte-for-byte diff at **both** layers, the
vocabulary module's and the caller's, with `('\x30')` against `('\x31')`
under the caret — which is the one-owner refactor demonstrating that it
kept two independent pins rather than collapsing them into one.
### Findings
Seven, each stated as the rule it became.
1. **A panic prints no assertion, so pin ORDERING buys attribution.** A
mutation that aborts is charged to whichever test is running, not to
the test whose property it violated — unless the pin that states the
property runs first. Ordering is not cosmetic in a suite that grades
itself by mutation.
2. **A comptime error is a third prints-nothing route.** The module
simply vanishes from the run (227 of 251 tests, and no property
named) rather than failing loudly. Length-asserts go first and the
slices get sized at runtime, so a bad mutation fails as a test rather
than as an absence. This hazard class is named "silent-module-loss"
elsewhere in this file and in the module roots' refAllDecls comments.
3. **Flipping a conjunction's operands is not negation.** Shared
structure absorbs it — two paths on one filesystem share a `dev`, so
swapping the operands of the identity check changed nothing
observable. A mutation meant to negate must be written `!`.
4. **When a pin shares the implementation's literal, a sentence-wide
`sed` self-heals.** Mutate the implementation line and nothing else,
or the mutation edits its own detector and grades zero.
5. **A substring pin is satisfiable by incidental prose.** Anchor to the
position that carries the meaning — `"\n muxd " ++ name`, not
`name` — or the pin passes on a mention rather than a definition.
6. **Wire-format mutations wedge the suite silently.** Second data
point; a wedged step prints nothing at all, so it must be watched by
wall clock rather than by output.
7. **An e2e scenario can die UNNAMED under `set -eu`** when the failing
pipeline is the client itself: the script's own `FAIL` line never
prints. Banked as legibility debt.
### Plan amendments, all made at source
**Nine**, each written into the plan with the evidence that forced it,
rather than worked around in an implementation. The pattern worth
keeping is that the plan was wrong in ways only the code could reveal:
- Task 1's assumed frame `0x86` was `stats_reply`; `exit_status` is
`0x82`.
- `writeFrame`'s "golden pin" was a false doc claim — there was no such
pin until the task wrote one.
- `ensurePtyModeSent` was **reversed**: the name promised a skip the body
did not perform.
- The strike named the wrong target (`waitReady`'s doc, not
`default_idle_ms`).
- Task 5's specified mutation aimed at an unpinned function, so
`renderClipped`'s epilogue got its first pin as a rider.
- Task 6's regrade premise was false — there was **no** unit pin on
sock/via dispatch at all; the rider added it, and it is regrade row 2
above.
- Task 7's null-means-silent premise was false: aborts print.
- Task 10's specified mutation was operand-flipping, not negation
(finding 3).
- Task 11's `sed` note, which became finding 4.
Correction ran in every direction, which is the part that is easy to
lose. Reviews corrected implementers; **implementers corrected
reviewers** (an error-set `||` needs parens on 0.15.2; a sketch that was
described rather than quoted got rewritten from the spec); and
**reviewers corrected the lead** — the `ensurePtyModeSent` reversal,
`egress_cap`'s home, the vestigial stat-before-close ordering, and the
anchored usage pin were all lead decisions overturned with evidence.
### What a survey is good for
The pre-task surveys **located structure correctly and systematically
undercounted instances**: 17 union edit sites turned out to be 33, 5 path
literals 6, 8 message classes 12. One survey premise was simply false
(the dispatch decision was described as pinned; it was not).
**Surveys locate; they do not count.** A plan that budgets from a survey
count is budgeting from a lower bound.
### Coverage: what got its first pin
M15 was a refactor, so most of its test growth is coverage that had never
existed rather than coverage that moved. First-ever pins landed for:
- the refusal **wire bytes**, plus the proof that the QUIC and socket
producers emit the same ones;
- `writeFrame`'s golden bytes;
- `renderClipped`'s epilogue **and** its prologue position;
- the transport dispatch decision;
- `openFailure`'s **12 message classes** and its truncation policy;
- `keyRefusalBody`;
- `PathId`, including the historical sockfs bug carried as a live
assertion rather than a comment;
- `oneShotQuery`'s nobody-serving exit;
- usage-names-every-verb, anchored (finding 5).
Two pieces of coverage debt were banked rather than paid: `uses_socket`
is pinned for **1 of its 9 rows** (`dump`; `--version`'s row is
structurally dead), and a session-full refusal **over QUIC** is still
unexercised behaviourally.
### Banked by M15 (the M16 candidates)
- **The stat-after-close reorder in `deinit`** — narrows the window in
which a successor daemon's socket could be claimed by a departing
one's cleanup. The analysis is already written, in the `sockpath`
comment; only the reorder is owed.
- **`egress_cap` ↔ transport-params, one owner** — `256*1024` is spelled
three times and the tie between the copies is prose only.
- **Fold `Transport`'s `alloc`/`qout` into the `.quic` `Link` payload** —
deletes the struct's one remaining `undefined`.
- **`Pty.write` is dead in production** — adopt or delete.
- **e2e scenario-naming legibility** — finding 7: a scenario that dies
unnamed, because `set -eu` kills the script before its own `FAIL` line.
- **`predict.zig`'s retired channel** — adopt or delete.
- **`keyRefusalBody`'s e2e coverage** — the announce-none path pins it
only loosely; regrade row 3 was caught by unit pins alone.
### Process note
**The file-copy revert idiom is now standing practice.** It became one
the expensive way: a `git checkout` used as a mutation revert ate an
uncommitted test. Copy the pristine file aside, mutate, copy it back,
and verify with `git status --porcelain` plus a green `make test` before
the next row.
**Relay sketches verbatim, never described.** A described sketch was
reconstructed wrongly and had to be rewritten from the spec — one of the
nine amendments above.
Two permission-classifier blocks were handled correctly: the agent
stopped, reported the step as **unobserved** rather than assumed, and
handed it to an implementer. An unobserved step reported as observed is
the one failure mode this process has no other guard against.
**The soak's pre-flight hygiene check earned its keep at the close.** The
first `SOAK_N=10` invocation refused to start, naming one stray
`mux-e2e-*` file in `/tmp` left by an earlier run. It was real evidence:
a capture of `muxd dump --sock` printing `nothing listening on …` where
the `sun_path` scenario demands `socket path too long`. It was also
**stale** — the current binaries refuse the long path in both binaries,
and the file's timestamp put it inside Task 12's working tree, between
the commit before the subcommand spec table and the commit that landed
it. The check cannot tell a live leak from a fixed one, which is exactly
why it refuses to guess and makes a human look. Inspect, confirm against
the current binaries, then clear — never clear first.
### Errata
Recorded rather than rewritten, since the commits are published:
- Task 4's fix commit says "eight named cascade failures"; seven are
below the pin.
- Task 9's commit message claims the test edits were three and nothing
else; two comments also changed.
- "12 pins" in Task 7's report was the message-**class** count, not a
count of pins.
## Open (owed by later milestones)
- Scrollback retention *tuning*. The policy itself was decided in M1 and
is not open: retention is engine-native, a 10k-line ring
(`Engine.Options.max_scrollback`), evicting oldest-first. What remains
deferred is making the limit configurable and choosing a non-default
value — nobody has hit the ceiling in practice, and M3 already accepted
that scroll positions drift as history evicts.
## Prototype verdict (2026-08-07)
All five milestones are complete and all three kill criteria are
cleared. The two questions handoff §0 says this prototype exists to
answer are both answered yes.
*(Written at the close of M5. M6 added a sixth milestone and a fourth
criterion — the transport kill criterion, both halves cleared and
measured over a real WAN link — without disturbing anything below. See
the M6 section and its transport verdict.)*
- **M1 kill criterion — cleared.** The grid is extracted from upstream
ghostty-vt with no fork: an unmodified pinned dependency, driven
headlessly, serialized through its own formatters.
- **M3 kill criterion — cleared.** This is the one the log never recorded
a verdict for, so it is recorded here against a real run rather than an
impression. With `nvim` open full-screen in the session, the client was
killed with `kill -9`; the daemon survived, and a fresh client reattached
and painted the restored TUI screen — including buffer text typed before
the kill — **5 ms** from launch to first painted byte, from a single
snapshot. Quitting nvim then revealed the primary screen with the
pre-TUI shell line intact, exercising the M3 dual-screen snapshot.
Reattach is therefore both fast and correct for full-screen TUI
sessions, which is exactly what the criterion demanded. The e2e suite
pins the line-mode half of this (kill -9 mid-`seq`, then reattach);
the TUI half is demo-verified, not automated — see the banked list.
- **M4 kill criterion — cleared.** Delta traffic measured at 1% of the
snapshot equivalent on the typing workload (7751 vs 689537 bytes), and
~6% under adversarial full-screen scrolling where every row changes.
- **Question 1 — can libghostty serve as an authoritative, headless,
serializable grid in a daemon? Yes.** Unforked, with the caveats
already logged: the API is documented-unstable so the pin is
load-bearing, and DECOM remains a known gap.
- **Question 2 — does detach/reattach as state sync feel correct and
fast? Yes.** Reattach is a single snapshot the client rebuilds
natively rather than a replay of session history, it restores TUI and
line-mode sessions alike, and under deltas the steady-state wire cost
is 1% of snapshotting.
Everything downstream of these answers — network transport, mesh,
multiplayer, panes, checkpoint/restore — remains deliberately out of
scope, and the known gaps owed before any of it are recorded above (no
session epoch, blocking per-client writes, hand-rolled wire format).
*(M6 closed the first two of those three and put the protocol on a real
WAN link; the hand-rolled wire format stands, and its tripwire has still
not tripped. See the M6 section and its transport verdict.)*
## 2026-08-13 (M16-a — two adopt-or-delete rulings, both delete)
The first two issues filed in the new git-collab tracker, closed the
same day (a3848ce2 → 9db679f, cf18fe67 → 3d4e3f0). Both were "the code
offers something the product doesn't use," and both rulings were delete.
- **`Pty.write`** had zero production callers — the daemon writes the
master fd through `proto.writeAllFd` at both of its sites — and
survived M15's `Pty.read` deletion only because four of pty.zig's own
tests called it: an abstraction kept alive exclusively by its own test
suite. The tests now write `std.posix.write(pty.master, …)`,
byte-identical to the deleted body, and so drive the pty the way
production does.
- **`predict.zig`'s retired channel** (`retired`/`retiredCount`/
`retiredAt`) was designed as a targeted-rollback repaint list and read
by nobody: on contradiction the client full-repaints, a rollback that
is certainly right on a path M9's design works to make rare. Deleted
with its three self-tests (none of the judging surface leaned on the
recording — the STOP-and-check that established this found the brief
had undercounted the channel's pins, one vs three, without the
protective condition being realized).
**The readoption record, stated here because the design's only prose
home was the doc comment the deletion removed** (the commit body said
"survives in decisions.md M9", which the review found does not resolve —
M9 banks the input-ack trigger but never described this mechanism):
`reconcile` retired confirmed-or-contradicted predictions into a list
the caller could repaint cell-by-cell instead of in full; and `expire`'s
no-op pass deliberately did NOT clear that list, because the idle path
runs between a reconcile and the repaint its list describes — clearing
on a quiet pass would silently empty the list and leave confirmed
predictions underlined with nothing to say which cells to fix. A
readopter restores the mechanism, that rule, and the rule's test
together; the trigger remains M9's banked input-ack item.
Process note: both issues closed through `git collab` with sha-naming
comments — the tracker's first full lifecycle.
## M17 — web client (browser wall of devices)
Shipped 2026-08-13 as 04861b7..002d1f7 plus this close-out: muxweb (the
hub), replica.zig (the replay core, extracted and finally unit-tested),
keymap.zig, the wasm core (ghostty-vt on wasm32-freestanding, 345KB),
the web shell, wsclient (the scripted browser stand-in), and three e2e
scenarios (suite now 23 scenarios / 35 convergence points). Kill
criterion (the live wall of three devices) still owed — the milestone
is code-complete, not field-complete, until it runs.
**The passivity contract: wall tiles attach at 1×1.** The spec's
original "browser attaches at the tile's cols×rows" would have failed
its own kill criterion — any attach at a differing size moves the
shared grid and repaints every client. The daemon already contained the
mechanism: applySize refuses cols<2/rows<2, the refused attacher gets a
unicast snapshot carrying the true grid, its slot stays 0×0, and
claimGrid refuses it on every keystroke thereafter. Permanent
passivity, zero daemon changes, pinned by a hub-free e2e scenario so
the contract outlives the hub.
**The proxy thesis, amended for message-delimited transports.** The
hub parses the 5-byte frame header in both directions — WebSocket is
message-delimited, the daemon socket is a byte stream, and re-framing
requires knowing where frames end. It still never parses payloads.
"Frame-agnostic" was a property of byte-pipe transports, not of pumps
in general.
**std.http's WebSocket server, and the two hazards it hands you.**
respondWebSocket does the RFC 6455 accept-key work, but (1) it does
NOT check Origin — that gate is ours, before upgrade, exact origins
only — and (2) readSmallMessage BLOCKS mid-message and silently
swallows pongs and loops (std Server.zig: `.pong => continue`), so a
pump that polls must gate every read on a complete-frame check over
the reader's buffered bytes (headFrame), and must consume pongs
itself: handing a buffered pong to readSmallMessage re-enters the
blocking hazard by construction. One buffer bounds both max HTTP
header and max inbound message (64 KiB; browser pastes chunk at 32 KiB
under it, wrapped ONCE via paste_begin/end).
**headFrame's capacity check is load-bearing twice.** Beyond refusing
oversize messages, `.too_big` before the buffer fills is what keeps
fillMore off std's defaultRebase assert (`buffer.len - seek >=
capacity`), which panics on a full unreclaimable buffer. And the check
itself was the milestone's one Critical regression: `off + payload_len`
with a wire-controlled u64 overflows — 14 bytes panicked the hub in
Debug and wrapped to a permanent hang in ReleaseFast. The pin that
covers it now fails BOTH ways (panic in Debug, wrong verdict in
ReleaseFast); a Debug-only pin would have left the release wrap
unguarded. Lesson: bounds involving wire-declared lengths get pinned
at the TYPE boundary, not just the capacity edge.
**"The page never ran."** WebAssembly.instantiate is shape-polymorphic:
bytes yield {module, instance}, a compiled Module yields a bare
Instance. The shell destructured the wrong shape, every tile died on an
unread promise rejection, and verify.js — which feeds bytes — stayed
green. Found only because the fix round added the missing .catch and
then drove the real page under headless Chrome. verify.js now pins
both shapes, but the residual stands: nothing automated boots the page,
so the shell's use of the contract is still uncovered. The named
follow-up, highest value in this milestone: automate the headless
browser boot. The general lesson repeated itself precisely — defects
clustered in the one layer with no tests.
**Process record.** The execution fork could not spawn subagents, so
Tasks 1–10 shipped with NO reviews; the post-hoc pair found 1 Critical
+ 5 Important, all in the untested JS layer, and the two fix rounds
added (then caught) one Critical of their own. Review-after-the-fact
worked, but the Critical-per-round rate says reviews belong in the
loop, not after it.
**Deliberate v1 stances, recorded so they read as chosen:** bracketed
paste is unconditional (the replica cannot see DECSET 2004 — pty_mode
carries icanon/echo only; extending it is a daemon protocol change,
deferred); {"state":"gone"} is in the control vocabulary but never
emitted (the browser derives gone from ws.onclose; kept for the day
the hub gives up); a slow browser stalls only its own tile's thread;
a mid-paste socket death leaves the application in paste mode (the end
marker cannot transit a dead socket; state lives beyond the pty);
after unzoom a tile keeps the grid the zoom claimed (unzoom sends
nothing, per spec); snapshots cannot exceed the 256 KiB staging cap in
practice because Engine.dumpState is viewport-only by design.
**Banked by M17:** the headless boot automation (above); a keyboard
route out of zoom (the shade ring is mouse-only; Escape belongs to the
application); automated coverage for the 30/60/90s dead-leg timing (the
pong path is covered every e2e run — with the caveat that the pin
catches framing regressions loudly but a swallowing regression only
probabilistically; the timing itself is reasoned, not observed);
GONE_AFTER_FAILURES bounds two different counters (socket opens,
replay attempts) and the name no longer says so; per-tab daemon slots
(N tiles × M tabs = N×M attaches against max_clients=8) — half-open
pressure the M16 reaping-policy item now also covers.
### M17 field verdict (2026-08-13, same day)
The kill criterion PASSED — driven end to end under headless
Chromium/CDP (the browser extension being unavailable), screenshots
archived per stage. Chain of evidence: three tiles `up` (desktop sock,
LAN quic, WAN quic through the 192.168.0.251 UDP door); zoom → vim →
`:wq` and the file read back on the box over ssh ("web wall drives vim
over quic"); `seq 1 200` then wheel to `history -3` and back; unzoom
via the shade ring; the concurrent ptyclient CLI observer detached
CONVERGED (render of its captured stream diffed clean against `muxd
dump`); iptables DROP on udp/4433 → `reconnecting` (neighbors
untouched) → rule removed → `up` with the marker still on the grid
across the daemon epoch; page + /tiles fetched through an ssh -R
tunnel from the LAN box (-L's mirror; the box has no route back).
Two field findings, one filed (per-tile --key), one operational: the
per-box `muxd keygen` deployment habit had left THREE different keys
on desktop/LAN/WAN, and mutual auth renders that as pure silence —
the "did not answer" line cannot distinguish a wrong key from a dead
box, which is by design on the wire but undiagnosable at the fleet
level. The trial unified the boxes on a fresh key (the classifier
rightly refused to ship the user's personal key around; a fresh PSK
was the better answer anyway). Desktop's default key is still its own
— the boxes' daemons now authenticate the trial key only.
Instrument notes for the next trial: an 80×24 observer of a large
grid sees renderClipped's top-left window, so markers meant for it
must be painted at the origin (`clear` first); EOF on a piped mux
client aborts the QUIC dial mid-handshake — probes must hold stdin
open and detach with 0x1c, or rc=1 "did not answer" lies about the
network.
### M17 simplify round (2026-08-13, post-field-verdict)
Three fresh-eyes reviewers (hub / core+shell / test+build) over the
shipped milestone, then three sequential fix batches, each re-reviewed
by its finder before push. Five Importants survived three prior review
rounds, and every one clustered where the coverage wasn't: the hub's
drain gating (buffered browser bytes invisible to poll — fixed by
draining unconditionally, which also made the drain unit-testable for
the first time), the badge state machine living in the DOM (now a
status field with terminal-state precedence; revive-on-applied-frame
is what makes a restarted session reappear on the wall), macOS metaKey,
the wsclient stand-in re-attaching at the authoritative grid (the
passivity contract's own fixture violating it, latently), and marker
polls that had lost their asserts in copy-editing.
Line-count honesty, recorded because both the fixer and the reviewer
converged on it: the drainBrowser dedup was NOT a line win (+7 code
lines after paying for the seam) — it was a one-owner win whose new
seam produced the stranded-bytes pin at all. Estimates priced bodies;
seams cost signatures.
Declined simplifications, so nobody re-proposes them: sharing the
fixtures' verb loops (three shared names, zero shared actions — an
abstraction tax); merging the native/wasm build blocks (comment-per-
module is the file's organizing style; no drift existed); merging
pump/dialLoop (three concerns interleaved to save lines the two
extractions already saved); std.json for /tiles (it emits invalid-UTF8
slices as integer arrays — argv labels would change shape); the M13
stop scenarios keep their spelled-out teardowns (muxd stop is their
subject; a pin must not flow through machinery other tests can
pressure into loosening).
Process doctrine, banked from the re-review's own near-miss: `make
test 2>&1 | tail` reports TAIL's exit code — a green claim over a
failing build. Capture status before piping. The companion of
"legible catches depend on test order": a catch that prints is
worthless if the harness reads the wrong exit.
Owed cheaply later, filed as observations not blockers: a
handTarget() seam so mux_main's idle_ms threading gets a pin;
verify.js's wall-attach sequence covers the shell's call set but the
page's boot stays browser-only (the banked headless item).
## 2026-08-13 (agent surface — native LLM integration)
Twelve tasks, shipped the same day as M17: OSC 133 command boundaries in
the daemon, three new frame pairs, and `muxa`, a fourth binary that speaks
JSON to an agent's shell tool. An agent now *knows* when a command
returned, with its exit code and its output span, locally or over QUIC —
the thing `tmux send-keys` + `capture-pane` structurally cannot say. The
spec is `docs/superpowers/specs/2026-08-13-agent-surface-design.md`; what
follows is what the implementation decided, which is not always what the
spec guessed.
**Output spans are rows, not seqs — and rows are locators, not anchors.**
The tracker's `seq` is a viewport delta generation: a whole command's
output can share one, and a row loses its seq the moment it scrolls into
history. So a mark records the absolute screen row (`historyRows() +
cursor.y`) at mark time and output recovery is the existing
`fetch_scrollback`, unchanged. The honesty is on the field, not in prose:
`Engine.MarkEvent.row` says that pruning past `max_scrollback` shifts the
origin (a command longer than the scrollback can leave `end_row <
start_row`), that resize reflow renumbers history, and that alt-screen
marks live in a coordinate space where `historyRows()` is 0. Point a human
at output with it; never key durable state on it.
**An await is answered from a persisted snapshot, never from live tracker
state.** Real shell integration emits `D;code` and `A` in one burst, so
both fold in a single pty read and the phase is back to `at_prompt` before
any await ever looks. A gate on `cmd.phase == .returned` therefore loses
deterministically — it was written that way first, and the test that
caught it drives a real shell rather than a hand-built mark sequence.
`last_return` is the fix and the one owner: stamped at the `D`, carrying
the exit code, the span AND the seq that qualifies it, so the answer and
the reason it qualifies are the same record and no later reading of live
state can drift out from under it. It also answers correctly once the
*next* command is already running, which is the general rule worth
keeping: **an await asks about a past event; live state describes now.**
**OSC 133 interception wraps the dependency's handler rather than patching
it.** ghostty-vt already parses semantic prompts including the `err` code,
but its stock handler drops the code and exposes no callback. `vt.Stream(H)`
is generic over the handler, so `MuxHandler` intercepts `.semantic_prompt`
and forwards every other action verbatim: terminal state stays identical,
the pinned dep stays unmodified, and there is no byte-stream scanning
anywhere. The engine's hardcoded `vt.TerminalStream` became
`vt.Stream(MuxHandler)` and nothing else moved.
**Awaits resolve at the run loop's 100ms tick, with no deadline folding
into poll.** The spec expected settle/timeout/pgid deadlines to fold into
`wait_ms` the way QUIC's `timeoutMs` does. They do not need to: nothing in
`checkAwaits` blocks, the granularity bound is the tick the daemon already
beats at, and an agent's cheapest verb costs a round trip anyway. The pass
sits after every arm that can move the session on — so it sees this pump's
marks, pgid and silence — and *before* the QUIC `drainAll`, because a
resolved await queues a frame and `drainAll` is what puts it on the wire.
The other order costs every remote await a whole extra poll cycle.
**Session death is an answer on the lifecycle verbs and an error on the
rest.** `muxa run` and `muxa await` print
`{"reason":"session_ended","exit_code":N}` and exit 0 — a command that
killed its own shell answered the question that was asked. `status`,
`capture` and `send` have no answer to give, so they print the house's
`{"error":…,"detail":…,"exit_code":…}` shape and exit 1. **Field note that
cost a test:** `send`'s ack round-trip reliably WINS the race against a pty
death, so `send` cannot be relied on to report a session that is dying —
the next call is what sees it. That is a property of the ordering, not a
flake.
**`--timeout` is the daemon's window; the client adds bounded grace.** The
daemon's clock starts when it reads the request, so a client that waits
exactly `--timeout` always loses the race to the reply and reports a
timeout the daemon never had. The grace is `min(30s, max(2s, 4 ×
connect_ms))`: flat 2s over a unix socket, RTT-derived over QUIC from the
one measurement the client already has (its own handshake), capped so a
handshake that took a minute cannot buy a minute of grace. The cost is
named because agents budget wall clock: a `--timeout N` wait can overshoot
N by the grace window.
**Reconnect is at-most-once for the wire and exactly-never for input.**
One redial per process, on `ConnectionLost` and on nothing else (a
`SendStalled` peer is still there and has already spent the flush bound
proving it). What is re-sent is the attach and the `await_req` with the
ORIGINAL `since_seq` — idempotent by construction, and the reason a return
that landed inside the gap is answered instead of missed — and never
`run`'s input. A command line lost with the connection surfaces as an
honest timeout, not as `make deploy` running twice: **a wait may be
repeated because asking twice changes nothing; an input may not, because it
changes everything.** A redial that fails is recorded rather than
swallowed, so the verb reports both halves — `connection lost; reconnect
failed: <err>` — and a second tear after a spent redial says so in its own
words.
**`settled` means output went quiet, not that the process exited.** The
settle and pgid resolutions — and a timed-out wait, whatever its
mechanism — carry `exit_code: null` and a `phase` derived from marks, which on a markless shell reads `at_prompt` — the tracker
never saw a `C`, so it is telling the truth about what it knows. Only
`mechanism == "marks"` carries a trustworthy exit code, which is why every
reply names its mechanism. An agent that reads `exit_code` without reading
`mechanism` is reading a guess.
### Field limitations, shipped knowingly
- **zsh under the `ZDOTDIR` shim never sources `~/.zshenv`.** zsh looks for
`.zshenv` under `$ZDOTDIR`, and the shim directory has none. The `.zshrc`
is handed back (the common case); a `.zshenv` shim that restores
`ZDOTDIR` the way ghostty's does is the roadmap fix.
- **The bash shim's DEBUG trap displaces a user's own DEBUG trap** —
silently, and that is bash-preexec, atuin and iTerm2's integration. The
roadmap fix is coexistence via bash-preexec detection. Two limits of the
membership guard that keeps `PROMPT_COMMAND`'s own members from being
counted as commands: a typed command textually identical to a
`PROMPT_COMMAND` member loses its marks (degrading to pgid/settle, not to
a wrong answer), and a compound string member (`a; b`) can re-arm the
guard. Arrays — bash 5.1's default and what the shim prefers — are exact.
- **A SIGKILLed daemon orphans its `mux-shellint-{pid}-{random}`
directory.** Bounded and per-pid, cleaned on every ordinary exit; a
startup sweep of dead-pid directories is the roadmap item. `prepare()` failing is degraded
and never fatal: the session runs on pgid and settle, and says so on
stderr, because refusing to start a daemon over an optional enhancement
would invert the module's premise.
- **`muxa` has no idle-timeout flag, by design.** It takes
`quic.default_idle_ms` and lets `--timeout` be the only bound on a wait;
`muxd`, `mux` and `muxweb` keep `--quic-idle-ms`.
### Deferred, and named so the deferral reads as a choice
The spec's non-goals stand: **no MCP server** (a wrapper over `muxa` needs
no protocol change, so it can be layered whenever someone wants it), **no
read-only or capability-scoped auth** (one key is still full control, which
is the same posture every other transport has), **no input attribution**
(an attached human cannot tell which keystrokes were the agent's), **no
semantic event subscriptions** beyond the `cmd_state` push, and no
single-shot `muxa drive` (`send` + `await` + `capture` composes it). Added
by the reviews: the `.zshenv` shim, bash-preexec coexistence, and the
shim-directory startup sweep.
**`capture --diff-since` was promised and not built, and this is the
record.** The spec names it three times — the verb list, the TUI-driving
composition ("`send` + `status` + `capture --diff-since`"), and the testing
plan — and execution dropped it with no code and no deferral written
anywhere; the whole-branch review is the only reason it is written down
now. `muxa capture` ships as the whole grid, `--vt` or plain. What a future
one owes is more than the flag: `status_reply`'s `seq` carries the RETURN
WATERMARK (`last_return`'s seq, 0 when nothing has returned this session),
which is exactly what an await's `since_seq` wants and useless as the SEQ a
diff quotes — after this milestone the live stream seq is on no reply an
agent can read. So `--diff-since` needs a protocol field of its own, and
must not be built by re-reading a field that already means something else.
**Two smaller narrowings against the spec, recorded rather than left to be
rediscovered.** (1) The **fish** injection arm is unit-tested only:
`prepare` writes `fish/vendor_conf.d` and prepends `XDG_DATA_DIRS`, and
that is pinned, but no test anywhere runs fish — there is none on this box.
zsh and bash are both driven live against a real session. (2) The spec's
**version-skew test** — `muxa status` against an old-protocol daemon
reports the structured error — was not written. The behaviour is real and
not unpinned (a daemon that drops the frame and a daemon that never answers
are the same wait and the same `{"error":"status: no reply"}`, which the
dead-daemon paths do exercise), but the named fixture, an actual old
binary, does not exist.
**Paid in a separate commit, and only after the e2e pinned the spellings:**
`parseQuicAddr`/`resolveHost` now live in `quic.zig` as one owner for the
dial-address grammar, taking an allocator (`client.zig`'s copy hardcoded
`std.heap.page_allocator`, so the fold is also a fix). A human typing
`mux quic://HOST:PORT` and an agent typing `muxa --quic HOST:PORT` were
parsing the same grammar through two copies, which is how one flag becomes
two dialects.
## Hygiene kit (2026-08-14)
- **zlint: unobtainable 2026-08-14, drop-for-now.** Clone succeeded but the
build failed against both toolchains tried: the pinned 0.15.2 rejects
`build.zig` itself (`array_hash_map.zig` `put` called with 3 arguments,
expects 2) which is consistent with the checkout declaring
`minimum_zig_version = "0.16.0"`, and the one free retry against the
system `zig` (0.17.0-dev.135) also failed, first error
`src/main.zig:32:25: error: root source file struct 'heap' has no member
named 'stackFallback'` (std API drift past what zlint targets). The
compiler's native strictness (unused locals/params, shadowing) already
owns the highest-value lint classes; re-try when the pin moves to
0.16.x, the version zlint now targets.
- **valgrind first run (2026-08-14): clean.** Recipe at
tools/valgrind-quic.sh, non-gating (10-50x slowdown distorts every timing
path). Run under valgrind 3.25.1 after the interactive install:
`definitely lost: 0 bytes in 0 blocks`, `indirectly lost: 0`,
`ERROR SUMMARY: 0 errors` — a clean bill over the daemon lifecycle
including wolfSSL/ngtcp2 init, a real QUIC attach, and SIGTERM teardown.
The muxa `run` itself timed out at 60s under the slowdown (the handshake
and status frames completed — muxa got a live `at_prompt` answer over
QUIC; only the command await outlasted its window), which the recipe
absorbs by design: the summary is the product, not the probe's exit.
Two environmental findings from getting there, both now codified:
valgrind 3.25 decodes no AVX-512, so (1) the Zig side must be built
`zig build -Dcpu=x86_64_v3` (stated in the recipe's usage header), and
(2) deps/quic's zigcc-native wrapper now pins `-march=x86_64_v3`
permanently — a native-CPU build on an AVX-512 box baked EVEX into
wolfSSL's memset and SIGILL'd instantly under valgrind. v3 costs nothing
measurable at terminal bandwidth and the musl release target was
baseline all along, so nothing shipped changes.
- **The layer table is the law.** build.zig's module graph is declared data:
a table row per module (name, root, frozen stratum, production imports,
test-only imports), the wiring loop derives every grant, and a production
import that does not point strictly downward is a comptime error — as is
a table-row rename (comptime idxOf) and a wasm row importing a non-wasm
row (the wasm set is closed under production imports). Layers are
COMPUTED topological strata, frozen 2026-08-14 — hand-assignment was
tried first and misplaced engine on the first draft. Changing the
architecture now means editing the table, which is the point. Proven by
extract-and-diff: 76 edges before and after, empty diff. (Re-stratified
2026-08-20: the interact extraction added a stratum under client, so
client sits at 3 and the entrypoints at 5 now; the freeze and the
comptime law are unchanged.)
- **Two adjudicated edges.** `server -> replica` is test-only (sole use is
the applyFrame test helper) and lives in the test_imports column.
`client -> proxy` is production (ignoreSigpipe in the live attach path)
and is grandfathered with the debt comment at client.zig — relocating
ignoreSigpipe to a leaf is the recorded fix, deliberately not taken here.
- **Test-only imports are a declared column, and a mechanism.** The testtmp
pattern (production modules importing test scaffolding used only inside
`test` blocks) is stated per-row, and since the review round the column
is enforced: the eight rows with test grants get a test twin — a second
module instance carrying the test imports, used only by the test builds —
while the production instance never receives them. "Test scaffolding
never ships" is a compile error a production reference triggers, not a
convention lazy compilation happens to honor. The twins import
production dependency instances, so no compilation holds two instances
of one row. The strata computation excludes the column.
- **Every grant must be spelled.** A build-graph-time check reads each
row's root file and refuses every build — not just the gate — when a
declared import never appears as `@import("name")` there. It caught its
motivating case at once: exe→cmd, dead since its call site left
main.zig, carried faithfully by the extract-and-diff (a pure refactor
preserves mistakes too); the table stands at 75 edges. Stated limit:
text cannot tell which column a use belongs to — the twin split is what
separates the columns.
- **refAllDecls in every test-loop module root, scope stated honestly.**
Pub decls only — std.meta.declarations sees nothing private. Narrower
still than the original silent-module-loss framing: in 0.15.2 a pub decl
with an unresolvable type fails eagerly once the module is touched at
all (the planned probe proof did not reproduce; two isolated repros
confirmed), so the gap this closes is the module that is imported but
never referenced by name — plus forcing analysis where nothing
referenced the decl. quic/quic_server/quic_client use the plain variant
(the recursive walk would analyze the whole wolfSSL/ngtcp2 cImport
namespace); engine was downgraded to plain because recursion reaches
pre-existing comptime errors inside vendored ghostty-vt (re-promote
when the dep is bumped); wasm_core is excluded because it is not in
the native test loop and a block there would never run — it is the one
remaining dark root: compile coverage comes from the wasm build `check`
forces plus the ABI verify step, behavior only from the web e2e.
- **Leak verdicts print, never panic.** The three gpa-backed binaries
(muxd, mux, muxweb) check `gpa.deinit()` and print
`<binary>: LEAK: allocations outlived deinit` to stderr on `.leak` —
never an exit-code change, because muxd's exit code carries the
session shell's. e2e captures every daemon's stderr and sweeps all
captures in its EXIT trap — after the kills, before the rms — so a
failing run reports leaks too, and the trap moves the status in one
direction only (a leak promotes green to red, never masks the suite's
own code). Captures deleted mid-suite bank their verdict at the moment
of deletion (`rm_swept`), which paid off the old "uncovered $OUT.q"
debt. The nothing-to-read canary stands wherever it would guard a green
run and degrades to a note on an already-failing early exit — a
fabricated second defect would bury the real one. agent.sh gives every
daemon its own log via a per-scenario XDG_STATE_HOME and sweeps all of
them at the end behind the same vacuous-green guard, so the daemons the
per-stop leakcheck never saw (session-ended, destroyed, abandoned by a
failing scenario) get verdicts too, and a missing log is a failure.
Proven by deliberate leak at every layer: banked-at-rm, trap-on-green,
trap-on-failing, and a marker in one daemon's log among five. kill -9
paths print nothing — no false positive, no coverage, stated. muxa is
arena-over-page_allocator by construction and has no verdict to check.
- **The persistence soak phase owns the classes the Zig-side LEAK
marker (6a) never sees.** One daemon, N client lifecycles: every cycle
must prove its client attached (a vacuous cycle fails the phase); fd
count must return to baseline exactly, with one settle resample to
absorb the unacked-detach race; RSS gets a 4MB growth bound past a
3-cycle warmup (allocators retain pages; equality would flake). The
zombie /proc pitfall is guarded (`VmRSS` absent on a dead child reads
as empty, not 0). Failure evidence is swept into the faildir, never
rm'd. First real numbers (SOAK_CYCLES=8): fds 5->5 exact; RSS
9020->9128 kB (+108 kB across the 5 post-warmup cycles).
- **`zig build check` is the pre-commit gate**: fmt (including
build.zig.zon) + unit tests + `sh -n` over every script in test/,
tools/ and deps/quic (shellcheck at error severity when installed,
silently skipped when not), seconds. The scripts are file args, so
their contents hash into the graph and only edited scripts re-check.
e2e/agent/soak stay separate steps — minutes-long and process-spawning.
- **deadcode.sh is non-gating, run by hand.** A textual pub-decl
cross-reference whose output is review material, not a verdict —
refAllDecls keeps deliberately-unreferenced decls analyzable, so
"referenced nowhere outside its file" is a prompt to look, not a
failure. Matching is literal (`-F`/`-qxF`): a filename is never a
regex.
### Review round (2026-08-14)
One pragmatic review over the whole branch; every finding above Minor
was fixed the same day, each fix proven by a test that fails without it.
- **The shim directory is created exclusively and named unaimably.**
`mux-shellint-{pid}-{random}` via `xdg.makeNewPrivateDir`: one
`mkdir(0o700)` that neither follows a symlink nor adopts an existing
entry, then a no-follow open. Closes a CWE-59 window (shared /tmp
sockdir + guessable pid let a local user aim the old makePath+chmod at
a directory they chose) and, via the random half, retires the pinned
oddity where a leftover at the pid name silently cost a session its
marks. EEXIST degrades to no-marks as before. `makePrivateDir` keeps
its adopting behavior for the two callers under `~` where adopting is
correct; the two functions' doc comments now name their threat models.
- **A timed-out await clears `exit_code`.** The timeout arm now matches
pgid and settle, so "an exit code is real only under `marks`" is
uniformly true rather than true except in the one arm agents hit most.
The persisted `last_return` snapshot is untouched — only the reply.
- **muxa exit codes are enumerated and honest.** 0 answer, 1 error
object, 2 usage, 3 await timeout, and new 4: the reply object could
not be written (EPIPE/ENOSPC) — previously a silent exit 0 with no
JSON, the one shape the contract forbids. A command's exit code is
never muxa's. The post-return scrollback fetch now gets exactly its
own 2s window (`spanFetchDeadline()`, no run-deadline parameter to
misuse): under `@max` a `--timeout 0` run made the bonus fetch an
unbounded hang.
- **A port is an observation, not a derivation.** The `$$`-derived
numbers in agent.sh and valgrind-quic.sh are first candidates only:
bind-with-retry steps on the daemon's own "already listening on udp"
refusal (any other failure reports immediately), and the port written
back is the one the up-line named, so every dial targets what was
bound. Two suites congruent mod 900, or a tuned ephemeral range, no
longer collide.
## 2026-08-15 (M18 — the multi-session daemon)
One daemon, N sessions, one socket path / QUIC port / key. The design was
locked in conversation on 2026-08-13 and executed as eight tasks; what
follows is the locked set, then the places execution disagreed with it.
- **A connection IS a session.** The attach payload grew an optional
UTF-8 name tail after its fixed 20 bytes, and *nothing after attach
changed*: no frame tagging, no per-frame session id, no QUIC stream
surgery. Same-host tiles are separate connections to the same port.
This is the decision every other one falls out of, and it is why the
diff is a name tail plus a `Session` struct rather than a protocol.
- **Empty name means the default session, and that is the compat
story.** `encodeAttachNamed("")` writes the same twenty bytes
`encodeAttach` always did, so an M17 client works against an M18
daemon and an M18 client with no `--session` works against an older
one. `status_req`, `await_req` and `debug_dump` took the same tail for
the same reason — one rule across every session-scoped verb.
- **Known limitation, documented rather than fixed — and it is two
different failures, not one.** The plan predicted a single symptom
("silently ignored by `decodeAttach catch return`, the client hangs");
the cross-version run measured both arms and they differ by transport,
because a pre-M18 daemon has two attach paths:
- **Unix socket, ssh and `--via`** reach `serviceObserver`, which
DROPS the connection on an unparseable attach. The client says `mux:
connection to muxd lost` and exits 1. A clean, bounded failure.
- **QUIC** reaches `handleFrame`, whose `decodeAttach catch return`
ignores the frame in silence — and there the client really does
hang, measured as an 8s timeout with no output. A plain `quic://`
attach from the same M18 client to the same old daemon works, so it
is the NAME that is fatal, not the transport.
This is the worse half landing on the deployment that most needs it
right: remote boxes are exactly the QUIC ones, so upgrading the client
or the wall ahead of the box turns `--session` into a hang with
nothing printed. Still not worth a capability dance for a flag that
did not exist when that daemon shipped, but a bounded first-snapshot
wait on a NAMED attach — "no answer; that daemon may predate
`--session`" — is a cheap mitigation and is banked as a candidate.
- **Purpose bounds the surface.** The wall showing one host twice is the
whole point, so there is no list verb (the wall names its sessions;
attach-or-create makes discovery unnecessary), no kill verb (a session
ends when its shell exits), no rename, no per-session shell or
scrollback config. `max_sessions = 4`, wall-sized, in the spirit of
`max_clients = 8`.
- **A bare attach joins the zeroth session and never creates one —
seamless handoff is the reason.** The default session is spawned in
`init` (`createSession(..., proto.default_session, ...)`), so `"0"`
exists before any client connects and an empty name always *finds* it.
The alternative — tmux's `new-session` semantics, where an unnamed
invocation makes a fresh one — would break the product's central
trick: the M14 handoff attaches over ssh, re-dials, and re-attaches
over QUIC, and every reconnect and resume path in `client.zig`
re-attaches the same way. All of those carry an empty name, so all of
them must land in the SAME shell. Under create-on-bare-attach a
handoff would be a new login and a network blip would spawn a shell.
The tmux analogy holds once mapped correctly: mux has no separate
"new" verb — `mux` IS the attach verb — so its bare form is `tmux
attach`, not `tmux new`. Naming a session that does not exist is the
explicit request, and that is the arm that creates.
- **An attach is answered, always — there is no wait state.** Both arms
refuse the same way `resolveSession` returns null: `exit_status 1`,
queued on the client path and written-then-dropped on the observer
path. Resolution is synchronous — map `""`→`"0"`, validate, find, else
create, else no. This is worth stating because the pre-M18 hang
documented above looks like a wait and is not one: that daemon's
`decodeAttach catch return` fails to PARSE a named attach and returns
before any resolve-or-refuse logic runs, so it never decides anything.
Silence is not a state this protocol has; it is what a peer that
predates the rule does.
- **Creation requires a real size.** Attach-or-create only creates when
the attach carries nonzero cols×rows. muxa attaches at 0×0 — it makes
no size claim, deliberately — so `muxa send` to a dead session is
refused with `exit_status 1` instead of silently spawning a 0×0 shell.
- **A session name means one session, whichever transport carried the
attach.** The session table belongs to the daemon, not to the listener
that accepted the connection, so `--session a` over QUIC and
`--session a` over the unix socket join the *same* shell — and two
QUIC dials to one port come out as two shells. This is the operator's
half of decision 6 and it is now pinned end to end: the QUIC block
drives two dials into two sessions, then joins one of them from the
socket and asserts both clients' markers on one grid with the session
count unmoved. QUIC is where the claim is least obvious, since a QUIC
client is promoted to a slot when its *handshake* completes — before
any attach — so its slot starts session-less and the name is the only
thing that ever binds it.
- **Command tracking is per-session**, forced by the agent-surface
merge. A tracker fed by two shells would interleave their command
lifecycles into nonsense, so `cmd` and `last_return` live on the
session; `AwaitState` stays on the client slot, because an await is
one client's question.
**Amended in execution — reality disagreeing with the plan.**
- **A `/bin/sh` session has no marks, so the planned `muxa status` phase
assertion could not hold.** The plan's e2e sketch asserted that
`status --session b` names a running command while `--session a`
reports `at_prompt`. `shellint` detects shells by basename and unknown
shells get nothing, so `cmd.phase` in a `/bin/sh` session never leaves
`at_prompt` and the assertion would have been asserting a constant
against itself. Switching the block to bash was rejected for the
reason `e2e.sh` already pins `SHELL=/bin/sh` at the top: `bash_init`
sources `$HOME/.bashrc`, which is arbitrary code on the session under
test. What the e2e asserts instead is what it can honestly observe —
that `status --session` returns a real StatusReply for a named session
and that `capture --session` returns one session's grid and not its
neighbour's. The per-session tracker itself stays pinned where marks
actually exist: `server.zig`'s unit tests, and `agent.sh`'s bash
daemons.
- **`muxa` is now an e2e.sh argument (`$9`).** The plan assumed the
suite could reach it; it could not — muxa went only to `agent.sh`. It
is asked for here rather than there because the verbs under test
address a session *without attaching*, and the sessions they address
can only be CREATED by `mux`, which `agent.sh` has no copy of. The
soak step takes the same argument, since soak *is* the e2e suite run N
times — an argument added to one and not the other makes every soak
run abort on an unbound variable before its first scenario, which is
exactly how it was caught.
- **The scenario detaches before asserting.** The plan's sketch held
both clients open across the assertions; the block as written has each
client plant its marker and leave. A session ends when its shell
exits, not when its last client leaves, so every routing assertion now
runs against sessions with nobody attached — which is also what makes
the answers impossible to explain by a live client holding something
open.
**The one real defect, and the e2e found it rather than review** (see
`fix(pty)`, f25f62c). `forkpty` returns the master without CLOEXEC, so
every session spawned later inherited every earlier session's master —
observed on a three-session daemon as fds 0,1,2,255 in the first shell,
those plus fd 3 in the second, plus fd 3 AND fd 6 in the third. A master
with a second holder never sees its last close, so `Pty.deinit`'s close
stopped hanging up the far side; the shell is interactive and therefore
ignores the SIGTERM that follows; and the blocking `waitpid` after that
never returned. `muxd stop` unlinked the socket and printed `muxd:
stopped` while the daemon sat in `do_wait` forever with every session's
shell alive — and because deinit walks sessions in slot order, wedging on
the first meant none of the others were torn down either.
Two things worth keeping from it. **A fd leak is a shutdown bug, not a
tidiness bug** — the daemon's own sockets were already CLOEXEC, and the
pty master was the single oversight in an otherwise consistent
convention. And **the bug was strictly unreachable at N=1**: nothing in
the unit layer, the review rounds, or fifteen prior milestones could have
surfaced it, because none of them ever spawned a second pty from a
process that already held one. The first test that ran two sessions on
one daemon found it on its first execution.
## 2026-08-15 (shell integration becomes opt-in)
**The target changed, and this default changed with it.** mux is being
aimed at replacing tmux as a daily driver; an agent driving a session is a
nice-to-have behind that. Under the old target the OSC 133 injection was
on by default, and the reasoning was sound for it — marks are what make an
exit code knowable, and every fallback below them is a guess. Under the
new one the same default is a tax every session pays for a feature only
`muxa` reads.
**What it costs, on the shell you live in.** Both costs were shipped
knowingly and written down at the time (roadmap, agent surface field
limitations) as items to fix later; what changed is the recognition that
they are not agent-surface limitations at all — they hit the human on
every session:
- zsh: the shim directory `ZDOTDIR` points at has no `.zshenv`, so the
user's is silently not read. `shellint.zig`'s own comment has said so
since the feature landed.
- bash: the shim's `DEBUG` trap displaces the user's — that is atuin,
bash-preexec and iTerm2 integration, all silently off.
**Inverted, not re-spelled.** `MUX_SHELL_INTEGRATION` keeps its name and
flips meaning: `=1` and nothing else turns the injection on; unset, `=0`,
`true`, `yes` are all off. Reusing the variable with the opposite sense is
safe in exactly one direction, and this is that direction — a stale `=0`
in someone's profile still reads as off. The policy is a pure function
(`shellIntegrationEnabled`) so it can be asserted without a daemon to set
an environment for, which is the same shape `announceKeyFrom` and
`pickKey` already have.
**The default moved in the library too, not just the binary.**
`Server.Options.shell_integration` is now `false`. The alternative — a
library default of `true` with the daemon overriding it — would have left
the product's real policy visible in exactly one place, and every test
inheriting the opposite of what ships. Four tests that need marks now say
`.shell_integration = true` explicitly, which is four assumptions written
down rather than inherited.
**Pinned at both layers, and the pin was watched failing first.**
`shellIntegrationEnabled` has the policy table (unset → off; `1` → on;
`0`/`""`/`true`/`yes` → off). `Server: the injection is off unless the
caller asks for it` builds a real server on `/bin/bash` — a shell
`shellint` HAS scripts for, because one it has none for would pass this
whichever way the default points — and asserts no shim directory exists.
All three assertions were run against the old default first and failed
there; the server one caught a second test (`the shim directory is
private`) that had been relying on the default without saying so.
**Verified on a real bash session, not only in tests.** A daemon started
with no `MUX_SHELL_INTEGRATION` at all, `--shell /bin/bash`, asked what its
shell actually holds: `TRAP=[]` (no DEBUG trap), `ZDOT=[]`, `ORIG=[]`, and
a `PROMPT_COMMAND` carrying only the box's own xterm-title setter. No
`mux-shellint-*` directory beside the socket. Suites: units 417 pass, e2e
26 scenarios / 35 convergence points, `make agent` 10/10 — the agent suite
now exports `MUX_SHELL_INTEGRATION=1` once at the top, which is the
requirement made visible rather than assumed. Its two degraded-path
scenarios (settle, quiet2) run `/bin/sh` and are untouched by that export,
so they still pin the fallbacks they were written for.
**The cost, named:** `muxa run` against a daemon nobody opted in for now
answers from `pgid`/`settle` and carries no exit code. That is legible
rather than silent — every reply already names its `mechanism`, and
`marks` simply stops being the one you get by default.
## 2026-08-15 (copy/paste discovery — what the instrument showed)
Discovery only: no product code changed, three issues filed (`ee062dd9`
mode mirror, `7c777ec6` OSC 52, `063cec67` scrollback copy). The point was
to stop reasoning about copy/paste from the source and measure it.
**The rig.** `muxd --shell /bin/sh`, a real `mux` client on a pty via
`test/ptyclient --out`, and the capture file read as "what the host
terminal would have received". `--out` already existed for convergence;
pointed at this question it is a passthrough oscilloscope.
**Outbound: nothing gets through.** The session emitted `ESC[?2004h`,
`ESC[?1000h`, `ESC[?1006h` and an OSC 52 clipboard set, then a visible
marker. In the host capture: marker present (1), `ESC[?1049h` present (1),
`ESC[?2026h` present (1) — so the capture was live and the session really
ran — and `?2004h`, `?1000h`, `?1006h`, `]52;` all **zero**. The marker was
also in `muxd dump`, so the daemon's engine had every one of them and the
client simply never forwards any.
**Inbound: already transparent, and this was the surprise.** Same rig,
session running `cat -v`, client sent a literal `ESC[200~pasted line
oneESC[201~`. `cat -v` echoed `^[[200~pasted line one^[[201~`. Paste
brackets reach the session's pty byte for byte. **The inbound half needs no
work at all** — the defect is entirely that the host terminal is never told
to *produce* them.
That is the finding that reorders the work. "Bracketed paste is broken"
sounded like a paste-path feature; it is one direction of one missing
mirror, and the same mirror is the whole of mouse support. Two symptoms,
one mechanism, and the daemon already holds the state (ghostty-vt tracks
`bracketed_paste`, `mouse_event_*`, `mouse_format_*`; `term.modes.get`
reads them).
**Three rules extracted.**
**Measure the direction, not the feature.** Reading the code would have
said "no bracketed paste" and a fix would have been designed for both
halves. One `cat -v` said half of it already works.
**A repainting client is not a passthrough client, and the difference is
exactly the side channels.** Modes and OSC escapes are what a two-engine
design silently drops, because neither is grid state and neither survives
"paint what the daemon has". That class — not this instance — is what to
audit next: title (already carried), bell, hyperlinks, colour queries.
**The mode mirror trades a working path for an unbuilt one.** No mouse
reporting is why your terminal's own selection works over a mux screen
today. Mirroring the session's mouse modes takes that away the moment an
app asks for the mouse — correct, and what every terminal does under tmux,
but it means mux needs a copy mechanism of its own landing with, or before,
the mouse half. Bracketed paste carries no such coupling and can ship
alone. The claim that native selection works is reasoned plus confirmed in
the negative (the escapes are absent); nobody has dragged a mouse across a
real mux screen to check wide chars and trailing spaces, and that is named
as unverified in `063cec67` rather than assumed.
## 2026-08-16 (side-channel passthrough close-out)
**Classification came before implementation.** Four symptoms reduce to two
delivery mechanisms, which is why this landed as one coherent change rather
than four special cases. Sampled state — bracketed-paste mode and title — is
read after the engine feed, sent on change and on attach, then undone when
the client detaches. Events — clipboard writes and bells — are intercepted
at the engine boundary, drained by the daemon and replayed by the client.
That split gives a new client the state it needs on attach without turning a
bell into state or losing the occurrence of a clipboard write.
**Replay follows the grid's own resync verdict.** Every event is stamped
with the grid sequence current when it was observed. A reconnect receiving a
delta gets the last pending event of each kind whose watermark it can serve;
a reconnect receiving a snapshot gets none. Pending storage is one slot per
kind, so a reconnect can deliver the current clipboard write and one bell,
not a stale history or an unbounded queue. An event whose watermark becomes
unservable is dropped rather than waiting indefinitely.
**Clipboard is write-only by design.** OSC 52 set is forwarded only for the
allowed clipboard target and bounded base64 payload; its query/read form is
refused. Replying to that query would make any program in any session able
to read the host's last copied text, which is a boundary mux must not cross.
**The measurements inverted.** The discovery capture counted zero OSC 52
writes at the host; the e2e capture now counts one. The real-editor pin
measured the nvim autoindent staircase changing from eight spaces to the
four spaces the pasted block contains. The xterm title-stack probe was run
in Alacritty and restored the original title, so mux pushes on terminal
setup and pops on teardown rather than leaving the session title behind.
**The gates were made able to disagree.** Removing the clipboard
interception makes precisely the new OSC 52 e2e fail. Neutering the
bracketed-paste mirror makes the nvim scenario fail on the four-space
assertion. Cross-version runs against the real v0.0.1-5 build passed on
socket and QUIC; replacing the old client with the new one failed both legs,
naming the clipboard, title and bracketed-paste frames it rendered. Bells
coalesce to one replay per drain, so a binary full of BEL bytes cannot turn
into one host-terminal write per byte.
## 2026-08-16 (shared client semantics and muxweb copy/paste)
**Protocol meaning has one platform-neutral boundary.** `client_core.zig`
decodes sampled terminal state and one-shot host effects; both the CLI and
WASM/browser adapters consume its validated `term_modes` and `term_event`
results. Browser paste therefore follows the sampled DEC mode 2004 state
rather than maintaining a web-only interpretation of the protocol.
**Browser clipboard authority belongs to the zoomed tile.** Accepted OSC 52
payloads that decode as valid UTF-8 text make a serialized, coalesced automatic
Clipboard API attempt. If that fails, one latest-wins explicit Copy action is
exposed; its accessible retry path restores terminal focus and IME handling
after the attempt. Unzoomed tiles neither write the clipboard nor offer the
retry.
**Selection remains a second slice.** Mouse-drag selection and scrollback
copy will use the correlated Reply family and the shared selection
architecture. They will not add a web-only protocol parser alongside the
shared client semantics.
## 2026-08-16 (muxweb mouse selection and scrollback copy)
**The daemon owns selection truth; the browser owns the interaction.** A
`selection_req` carries two screen-row coordinates and a client-local request
ID. The daemon extracts against its authoritative terminal screen and sends
the `selection_reply` only to the requesting client, so another attached
client cannot observe either the reply or its text. Successful replies are
valid UTF-8 and at most 1 MiB; extraction is all-or-nothing, with explicit
`invalid`, `too_large`, and `unavailable` statuses rather than truncated text.
Ghostty's selection and plain screen-formatter machinery decides soft-wrap,
hard-break, wide-cell, history, and trimming semantics instead of duplicating
terminal rules in JavaScript.
**The Reply lane keeps CLI and web architecture aligned.** The portable
`ClientCore` owns the one-pending-request correlation rule and rejects stale
or malformed replies. The native client and WASM adapter consume the same
semantic result family; the CLI currently has no selection requester, while
the WASM ABI exposes only the correlated result to muxweb. That is a framework
for future request/reply actions without growing a second browser protocol
parser, not a claim that native keyboard copy mode or terminal mouse-mode
mirroring shipped here.
**Visible selection is optimistic; copyability is authoritative.** Pointer
dragging immediately paints a canvas highlight in retained screen-row
coordinates, but copy remains unavailable until the matching daemon reply has
been decoded and copied out of the borrowed WASM buffer. Edge dragging uses
one scrollback request in flight at a time. Its endpoint advances only through
coordinates belonging to the viewport actually painted after a matching
history response; failure restores the last painted view, preventing a fast
pointer from selecting rows the user never saw.
**Copy chords are selection-sensitive.** With an authoritative retained
selection, `Ctrl+C`, `Ctrl+Shift+C`, and macOS `Cmd+C` are consumed for an
explicit clipboard write and send no PTY bytes. Without one, `Ctrl+C` falls
through to the terminal keymap and `Ctrl+Shift+C` is not prevented, preserving
Firefox's Web Inspector shortcut. Input, paste, composition, plain click, and
unzoom invalidate the retained selection; a copy attempt alone does not.
**Clipboard completion has one accepted browser-owned edge.** The UI is
versioned latest-wins, so stale completions cannot restore stale buttons or
selection state. An already-issued Clipboard API promise cannot be cancelled,
however: if explicit selection copy overlaps an OSC 52 automatic write that
has already entered the browser, the browser owns the external clipboard's
completion ordering. The UI still converges on the latest action, but the
external clipboard ordering needs real-browser validation and is not claimed
from the deterministic harness.
**Validation completed 2026-08-18.** The rerun that `b0d6972`'s hardening
left owed has now happened at `8164ce9`: `make build` and `make test` both
exit 0, and `web/verify.js` reports 671/671 against `zig-out/bin/mux_core.wasm`.
The verifier was run directly as well as through the build graph, because
`build.zig` skips it silently when `node` is absent — a skipped check reads
identically to a passing one.
The Firefox matrix was then exercised by hand against a live `muxd` and
`muxweb`, on a session seeded with 120 scrollback lines, a wide-cell CJK line,
and a 300-column soft-wrapped line. All nine cases passed: drag selection with
reply-gated copyability; soft-wrap joining; hard-break preservation; wide-cell
integrity; edge auto-scroll advancing only through painted rows; the
selection-sensitive `Ctrl+C` / `Ctrl+Shift+C` split with Web Inspector still
reachable; OSC 52 clipboard write; DEC 2004-sensitive paste inside and outside
`vim`; and zoom-gated clipboard authority. That is human observation of the
real browser, not a harness result, and it is the evidence this feature's
acceptance always depended on.
**Tracker mapping, without tracker mutation.** `063cec67` now has concrete
implementation evidence for muxweb mouse-drag selection and scrollback copy,
while its native CLI copy-mode scope remains open and separate. `7c777ec6`
maps to the OSC 52 browser clipboard path, and `ee062dd9` to the shared mode
semantics and DEC 2004 browser paste path; the mouse-mirroring half of the
latter remains separate. No issue is closed by this record.
## 2026-08-18 (selection review: totality, the history watermark, intent)
**A request the daemon could decode leaves with an answer.** The selection
lane is request/reply, and the only thing a requester can do with silence is
wait out its timeout. So the decode now happens BEFORE the session lookup,
and the two orderings mean different things: a malformed payload carries no
id to correlate, and silence is the only honest response to it — but a
well-formed request from a slot with no session (promoted by a QUIC
handshake, never attached) does carry one, and answering `unavailable` costs
a frame where dropping it cost the client five seconds. The same rule
governs the encode: `unavailable` with no text is the one reply that cannot
fail validation, so a failed encode of anything else falls back to it, and
the capacity that fallback needs is reserved before the attempt that can
fail. Totality is a property of the lane, not a property of the happy path.
**Absolute screen rows are not a name for a line, so the reply carries the
watermark that says so.** Row zero is the OLDEST RETAINED row, not the
oldest row the session ever had. When the page list evicts a page every
absolute row shifts underneath a request already in flight, and the reply
that comes back is `ok`, valid UTF-8, and text the user never highlighted —
the worst shape a bug can take, because nothing about it looks wrong.
`selection_reply` therefore carries the daemon's `history_rows` sampled at
extraction time, and the requester compares it with the reading it took when
its own request became authoritative.
The comparison is deliberately `lower means discard`, not `different means
discard`, and that is the whole subtlety. Ordinary output RAISES the retained
count without renaming a single row, because new rows are appended below the
ones already there; demanding equality would make every selection taken
during output unusable, which is most of the selections anyone wants. Only
eviction lowers it. Measured on a five-column grid: the count climbed to
10000, dropped to 5214 in one step, and absolute row zero changed identity at
exactly that drop — page-sized jumps, never a slow drift.
What this buys is a watermark, not a lease, and the difference is the point:
the daemon holds no per-client selection state and gains none here. The
residual hole is that an eviction followed by a whole page of fresh output
INSIDE one round trip would read as a rise. That needs thousands of rows
between the client's last applied frame and the extraction, and closing it
would mean the daemon remembering who asked for what — a trade this feature
has refused from the start.
**User intent beats arrival order on the clipboard.** An explicit copy is the
newest statement of what the human wants, so it retires the automatic OSC 52
lane rather than racing it. A write already handed to the browser genuinely
cannot be cancelled and that limit still stands — but the re-dispatch such a
write performs when it settles is ours to start, and starting it put terminal
text on the system clipboard after the gesture that asked for something else,
with the UI still reading "Copied" for the selection. Retiring the version and
the queued value together is what suppresses it. The rule is about which
writes we choose to begin, and it does not pretend to own the ones already
in the browser's hands.
**The browser honours one OSC 52 target.** The protocol still accepts every
Pc xterm defines, because the native client can route them: `p` is PRIMARY,
`s` the selection, `0`-`7` the cut buffers. A browser tile has one
destination and no way to reach the others, and writing them all to the
system clipboard meant that an ordinary mouse drag inside the session — which
is exactly what emits `ESC]52;p;…` in many X11 applications — silently
clobbered whatever the human last copied on their own machine. Narrowing the
web path to `c` is a loss of nothing anybody asked for. Non-`c` targets
remain valid on the wire and are simply not this tile's business.
**Validation run for this round.** `make build` and `make test` both exit 0,
and `web/verify.js` reports 704/704 against `zig-out/bin/mux_core.wasm`, run
directly as well as through the build graph. Every new assertion was watched
failing first, against the mutation or the bug it exists for.
The cross-version gate ran too, containerised, with this tree against
v0.0.1-5 in both directions: 9 passed, 0 failed, 1 skipped, the skip being
the agent surface the old ref predates. What it measures about the wire
change is the general property and not the specific frame — an old client
ignores daemon-to-client frames it has no arm for and keeps parsing past
them, because they are length-prefixed and its dispatch ends in
`else => {}`. It could not be made to measure `selection_reply` itself,
and that is a fact about the protocol rather than a gap in the rig: nothing
can make a daemon send one to a client that never asks.
## 2026-08-18 (the wall becomes runtime state)
**argv seeds, the file restores.** `muxweb` with targets means those targets:
argv replaces the wall and is saved as it. `muxweb` with none restores the
last run's wall from `$XDG_STATE_HOME/mux/wall`, one spelling per line, order
being wall order. The standing non-goal — no config file — survives intact,
because that file is written by the program and never by hand; what changed is
that the wall is now something a session PRODUCES rather than something the
command line has to re-type. A restore deliberately does not re-save what it
just read: a read that failed would otherwise overwrite the wall with the
nothing it managed to parse.
**Ids are per-run; the file holds spellings.** A tile's id is the hub's own
handle for it — what `/ws/<id>` names and what the reorder CSV addresses — and
it is minted at birth, so ids are birth order within one run and are
reassigned 0..n-1 in wall order at the next start. Persisting them would have
been persisting a coincidence. The spelling is the durable thing: it is the
label on screen, the line in the file, and the resolver's input, all the same
string.
**Mutating verbs are Origin-gated; GET is not.** A `text/plain` POST is a CORS
"simple request", which means any page in the browser can fire one at
`127.0.0.1` with no preflight to warn us. So POST, DELETE and PUT on `/tiles`
check Origin before they touch anything, and refuse with 403 — the same gate,
in the same place, as the WebSocket upgrade's. GET stays open on the opposite
argument: we send no CORS headers, so a hostile page can make the request and
can never read the answer.
**Remove is detach.** Deleting a tile takes it off the wall and shuts down its
pump; the daemon keeps the session, every process in it, and its scrollback.
The button is on a browser page in a room somebody walked out of — the
destructive reading of it would be the wrong default even if it were the
convenient one, and `muxd stop` remains the way to end things.
**`--sock ` joined the spelling grammar.** A tile is now ONE string —
`HOST[#SESSION]`, `quic://HOST[:PORT][#SESSION]`, or `--sock PATH[#SESSION]` —
because the page has one text box and the file has one line, neither of which
can hold a flag and its value as two argv words. `wall.zig` owns that grammar
for both binaries, so what the page POSTs and what the command line takes are
the same language, and a refusal (bad session name, empty target, a socket
path past `sun_path`) happens at ADD time rather than surfacing later as a
tile that will never attach.
**A dead id is answered in HTTP, not in WebSocket.** `/ws/<unknown>` is a 404
before any upgrade, which took moving the tile checkout ahead of the 101. The
alternative — upgrade, then close — tells a page that its request succeeded
and then leaves it to infer why the socket went away, which is exactly the
shape the reconnect loop cannot distinguish from a tear.
**A tile added by the box zooms itself.** Not decoration: a wall tile attaches
at 1×1, and `resolveSession` refuses to CREATE a session below the minimum
size (the passivity contract that keeps a browser wall from resizing a human's
grid). So `HOST#newname` for a name that does not exist yet sits at `session
full` until something claims a real size for it — zooming the tile the user
just asked for is what makes the add mean what it looks like it means. A tile
restored from the file does not auto-zoom; only the one typed into the box.
**The add box grants nothing the page did not already have.** A `HOST` spelling
typed into the browser reaches `ssh <host> …` through a shell, exactly as one
typed on the command line always has, and the page holding that box is the same
page holding a live terminal on the other side of the wall — so the box hands an
allowed origin no authority its own keystrokes do not already hand it by design,
and the Origin gate, not the grammar, is where the boundary actually sits.
Proper per-arg quoting in the handoff recipe is follow-up material, worth doing
for the spelling that contains a space rather than for the attacker who is
already typing into a shell.
**DNS rebinding reads the names and stops there.** An attacker who rebinds their
own hostname to `127.0.0.1` does get a same-origin `GET /tiles` it can read back
— tile labels and session names, the wall's table of contents — but cannot
mutate the wall or open a WebSocket to any of it, because `originAllowed`
compares the Origin header against the fixed strings `http://127.0.0.1:<port>`
and `http://localhost:<port>`, and every mutating verb and every upgrade is
gated on that comparison. Leaking the names is the accepted cost of leaving GET
ungated; leaking a session is not, and does not happen.
**A switch is an attach, not a mode.** `Ctrl-\ c` ends the attached session and
re-dials the same target under a new name: fresh transport, fresh replica,
snapshot from seq 0. The alternative — a `switch_session` frame the daemon
honours on a live connection — would have put the client's identity in the
daemon's hands and given every session-scoped thing on that connection (the
await watermark, the pending side events, the advertised size) a second way to
change owner. Attaching already does all of it correctly, once. The seam is
`attach()`, which loops on an `Outcome` from `session()` instead of returning
its exit code; the transport was always built there, so a switch is one more
turn around that loop rather than a new teardown path.
**The daemon names nothing; the client picks and the daemon refuses.** A new
session is named after the lowest non-negative integer not already in use, so
the series a user sees stays the one the default session `0` started, and a
closed session's number comes back. That takes a list, which is all
`sessions_req`/`sessions_reply` (0x0c/0x91) is: names '\n'-separated in slot
order, no codec, safe to split because `validSessionName` admits no whitespace.
Picking client-side means the pick can be refused — four sessions already exist
— and the refusal is `exit_status 1` before any state, exactly like every other
refused attach. That used to be fatal, and on a first attach it still is; a
switch's arrival has somewhere to go instead, so it reports `.refused` and
`attach()` returns to the session the chord was typed in with one line of
explanation.
**A step with nowhere to go stays put.** `Ctrl-\ n` / `Ctrl-\ p` walk the same
`sessions_reply` list `Ctrl-\ c` reads, stepping one place from where the client
stands and wrapping at both ends; slot order is a ring precisely because the
daemon reports it the same way every time, so `n` then `p` returns you to where
you started. When the step has no neighbour to land on, the chord does nothing
rather than switching to self. A switch is an attach (above), so a switch to the
session already on screen would tear the terminal down, re-dial, and paint the
same grid back — a visible flicker bought with a lost scroll position and a
freed-then-reclaimed daemon slot, in exchange for no change at all. Two cases
reach it: a one-session daemon, and a current name the reply does not carry,
which means the session died between the request and the reply — there being no
position to step from, any neighbour would be a guess, and a guess here moves
the user somewhere they did not ask to go. The picker returns null for both and
the caller reads null as "stay".
The three switching chords share one `SwitchIntent` field rather than a flag
each. A client can have exactly one question outstanding on the wire, and an
enum makes that a fact the reply arm can rely on instead of a rule every future
chord has to remember; the arm picks a name by intent and everything after the
pick — write `.detach`, return `.switch_to` — is one path. The intent is spent
on arrival whichever name it picks, which is what keeps an unasked-for reply
harmless.
**The wall reached from inside a session is a CHILD PROCESS, not a call.**
`Ctrl-\ w` asks the daemon the same `sessions_req` question the switching
chords ask, then spawns `/proc/self/exe wall SPELLING...` with stdio inherited
and waits for it. The reason is wallview's teardown: it detaches its per-tile
pump threads and calls `exit(0)` rather than joining them, because a tile
blocked in `readFrame` is not joinable and the window between the last paint
and process death must contain no free. That is correct for a process whose
whole job is the wall, and unusable inside a client that has to survive the
wall and go back to a session — in-process, leaving the wall would end the
session with it. As a child, the existing teardown is right by construction and
none of it had to change. What the child paints over is repaired by the
re-attach that follows, which asks for a fresh snapshot the way every attach
does; the child also restores the termios and the alternate screen it found,
and the re-attach re-establishes both regardless.
Two consequences fall out of that shape. The reply arm now reads an optional
ACTION out of `sessions_reply` rather than an optional NAME — a wall ends the
run without naming a session to go to — and the whole payload has to travel by
value, like `SessionName` and for the same reason: the frame is freed before
the child is spawned. The carrier is sized from this client's own bound on how
many stripes it will show, deliberately not from the daemon's `max_sessions`,
which the client cannot watch change; a reply too long for it shows no wall
rather than a wall that silently omits sessions.
**`--via` has no wall.** The wall grammar spells a socket, a host, and a
`quic://` endpoint, and has no form at all for "an arbitrary command's stdio" —
so for a `--via` target there is no spelling to write down and nothing to hand
the child. That is refused where it is discovered, with one line, and the
client re-attaches to the session the chord was typed in: the same shape a
refused switch already had, because the fact is the same one — nothing failed,
and there is somewhere to go back to.
**A chord waits 2s, because an old daemon answers by saying nothing.** A daemon
that predates `sessions_req` (0x0c) does not refuse the frame — its dispatch
ends in `else => {}`, so the question is dropped and the connection stays
perfectly healthy. An intent with no deadline then outlives the universe: it
stays armed for the rest of the session, every later `Ctrl-\ c/n/p/w` is a
silent no-op, and nothing in the picture suggests the daemon rather than the
keyboard. So the intent carries an expiry, which is the whole reason it is a
struct and not the bare enum. 2s is far longer than any round trip a switch is
usable over and short enough that the keystroke is still in the user's head
when the answer arrives; the poll's 100ms cap is what makes the deadline
reachable at all, since the case being caught is exactly a daemon gone quiet.
Expiry is self-clearing, so one silence is reported once rather than on every
poll for the rest of the session.
It is reported as a corner banner, `[no session list: upgrade muxd]`, not on
stderr: the terminal is in raw mode on the alternate screen, and a print there
lands mid-grid with no newline discipline. `[reconnecting]` already sits in
that corner and is read the same way — it survives until the next full repaint
paints over it, which is the right lifetime for a marker the user has to
actually notice. ASCII only, because the banner is placed by byte length.
A reconnect CLEARS the pending chord instead of expiring it. The two are
different facts: an expiry means the daemon heard nothing and the user is owed
an explanation, while a reconnect means the daemon that was asked no longer
has the question — the user presses the chord again rather than being switched
by a reply that outlived its connection. And the silence already has an
explanation they have been shown, since the reconnect painted its own banner.
Reporting both would put two markers in one corner about one event.
**A client refuses the session it is standing in, because it can no longer be
rescued.** `mux --sock S` run from a shell inside session 0 of the daemon on S
attached that session to itself. The inner client painted into the grid it was
reading, so paint became a delta became a repaint; it took the alternate screen
and, being the terminal's foreground reader, it swallowed every keystroke. That
was survivable while `Ctrl-\` was a single detach key — one chord went to the
inner client and it left. It is not survivable now: `Ctrl-\` is a prefix, both
clients filter it, and the outer keyboard has no spelling that reaches past the
inner one. There is no escape chord left to document, so the loop is prevented
rather than escaped, and the choice between "refuse" and "pass a chord through"
was settled by that fact and not by taste.
Prevention needs the client to know where it is standing, which nothing on the
wire could tell it: an attach describes a destination, not an origin. So the
daemon plants the origin in the only channel a shell carries into every process
it starts — its environment. `MUX_SOCK` is the path the daemon bound and
`MUX_SESSION` is the session's RESOLVED name, `0` for the default rather than
the empty spelling the wire uses, because the name a client compares against is
the one it resolved too. `MUX_SOCK` rides the shared spawn plan, which is built
once for every session this daemon will ever spawn; `MUX_SESSION` cannot, so
`createSession` appends it per session. Both are planted after `extra_env`,
inverting that field's usual last-word rule: a caller that could overwrite them
could hand a shell a lie about where it is, and this is identity, not option.
The refusal is the exact self-pair and nothing wider. Same socket AND same
session refuses; a different session of the same daemon is allowed, because it
is useful and only the self-pair feeds its own paint back. A host or `quic://`
target is never refused — a remote daemon is a different daemon however its
sessions are named — and `--via` has nothing to compare. The socket comparison
is string equality against the canonical path the daemon planted, so a
symlinked or relative `--sock` spelling of the same socket evades it; that is
accepted rather than fixed with a stat on every attach, because this guards
against the mistake people make (typing `mux` in a mux shell), not against
someone trying to get around it. An emptied variable counts as unset, so
`MUX_SESSION=` overrides in a shell that cannot unset an exported name — the
message names unsetting as the way out, and both spellings of that have to work.
Where the check sits is what keeps the chords working. It is in `mux_main`'s
attach arm, after the default socket path is resolved — so a bare `mux`, the
actual incident, is caught — and before the PATH search, the auto-start and the
dial, none of which change the answer. `Ctrl-\ c/n/p` re-attach from inside
`client.attach` and never pass back through that arm, so switching from session
0 to session 1 is untouched by construction rather than by an exception.
`mux wall` refuses the WHOLE wall when a tile is the session it was launched
from, naming the offending spelling. A tile is read-only, but it is still an
attach, and a stripe of the session the wall lives in paints into the grid it
is reading — the same loop with a nicer name. Dropping just that tile would be
cheaper and worse: a wall that silently omits a session you asked for is a wall
that lies, which is the objection already recorded against a truncated
`sessions_reply` showing a partial wall.
The e2e leg for this deliberately has no `ptyclient`. A real client on a real
pty is what the refusal PREVENTS, so the test injects one into the session
shell with `muxa send` and reads the result out of a capture — no expect timing
at all. Its money assertion is `"alt_screen":false` AFTER the injection, not
the refusal message: before this milestone that field read true, because an
inner client really had taken the screen, and a message grep alone would still
pass for a client that printed the line and attached anyway. The daemon runs
200 columns wide there, because every assertion greps a MESSAGE out of a grid
dump and a grid wraps — at 80 the wall refusal, whose length follows `$TMPDIR`,
split mid-sentence and a correct refusal read as a missing one.
**The wall's zoom is a real attach, and that is the whole design.**
SUPERSEDED by the 2026-08-19 phase-1 entry below (zoom is a lens: the child
spawn is gone, and `Enter` promotes the connection the tile already has).
The argument recorded here is not wrong; its premise stopped holding. It
assumed a typing tile types AT 0×0, and a promote resizes the attach to the
terminal before the first keystroke, so the client that types claims the
grid the legitimate way and the rule keeps no exception after all. Kept
verbatim because the reasoning is the one a reader needs in order to
understand why the promote must resize FIRST. `Enter` on
the selected stripe spawns an ordinary `mux` on this terminal, waits for it, and
takes the terminal back when it detaches. The obvious alternative — forward
keystrokes down the selected tile's transport — forks the latest-wins rule at
its root: a tile attaches 0×0 precisely so it can never claim the grid, and a
client that types is exactly a client that must claim it, so a typing tile is
either a 0×0 client claiming a grid (a rule with an exception is a rule nobody
can reason about) or a session being typed at by a client whose size the daemon
was told to ignore. Spawning keeps both halves honest: the wall stays an
observer, and interaction claims the grid the legitimate way. It is `Ctrl-\ w`
read backwards — that spawns a wall from a client, this spawns a client from a
wall — and the same argument settles it, that a process boundary is what makes
"the terminal belongs to exactly one of you" true by construction.
The selection is under the paint mutex, not atomic. It is read while a label bar
is being drawn, and the bars are drawn by the pump threads: an atomic read would
let a bar paint the marker the selection had when its frame arrived, so a wall
could show two markers, or none, for as long as nothing else painted. One
keystroke moves it and repaints both bars under one hold, so the marker is
single-valued in every frame the terminal ever shows.
Coming back needs a lever, because a correct replica is not a painted screen.
Holding `paint_mu` for the child's whole life does more than stop the stripes
painting over it: a pump blocks ON the mutex at the first frame it would paint
and stops reading its socket there, so the pumps do NOT keep replicating behind
the zoom — each one stalls a frame in and the wall is as far behind as the zoom
was long. So the zoom bumps a generation counter on the way out and each pump
repaints when its own last-painted generation lags — checked on the poll
timeout, so a stripe is back within ~100ms whether or not its session ever
speaks again. Between the two, the wall returns current: the backlog every pump
drains once it is released, and the generation bump for the sessions that said
nothing at all. A tile still inside its reconnect backoff is the one gap: it
repaints when it next narrates a state, which is the same moment it would have
had anything true to say.
The tail of blocked pumps is a drop, and it is accepted rather than fixed. A
tile that stops reading accumulates queue on the daemon side, and past
`pending_cap` (8 MiB) the daemon drops that client — so a chatty session under
a long zoom loses its tile, which reconnects and re-attaches from a snapshot,
re-running the whole ssh→QUIC handshake for a `.hand` or `.quic` target. The
alternative is a pump that keeps reading and applying while it may not paint,
which is a second buffering path through the replica for the sake of a case
that already self-heals; and the cap itself is not negotiable, since it is the
only thing bounding what one wedged client can cost the daemon in memory. The
visible cost is a stripe reading `[reconnecting]` for a moment after a long
zoom, which is what a stripe that fell behind should say.
A failed zoom is said on the label bar, not on stderr — the alternate screen the
wall re-enters would eat the line — and never propagated: a spawn that did not
work out is not a reason to end the wall the user is still standing in. The
`--via` case is `unreachable` rather than an error, because the wall grammar has
no `--via` spelling to parse: no tile can hold that target, and pretending
otherwise would add an error path nothing can reach.
**Superseded in the wall-home-screen phase 1 (2026-08-19).** The zoom is no
longer an attach at all; the whole child-spawn path above — `zoomArgv`, the
spawn, the wait, the failure notice, the `paint_mu` held for a child's whole
life — is deleted. What survives verbatim is the generation counter and the
selection-under-the-mutex argument, which turned out to be about a terminal
changing hands and not about a child process. The argument that a typing tile
forks latest-wins had a hidden premise: that the tile types AT 0×0. Promote
resizes before the first keystroke can be sent, so the client that types is a
full-size one claiming the grid the legitimate way, and the rule keeps no
exception. See the phase-1 entry below.
## 2026-08-19 (the wheel, and who owns the mouse)
The complaint: spinning the wheel in a mux session walked the shell's history
instead of the view. Nothing in mux was reading the wheel — that was the bug.
A terminal that has been asked for no mouse reporting answers the wheel on the
alternate screen by synthesising arrow keys (DEC 1007, "alternate scroll"), and
the client is always on the alternate screen, so every notch arrived as `\x1b[A`
in the stdin stream and went to the pty as input. Shift+PageUp was the only
scrollback control there was.
So the client asks its own terminal for real wheel events — 1000 (button
presses) in 1006 (SGR reports) — and spends them on the scrollback it already
has. Asking is also what stops the synthesis, so the fix removes the symptom
and the cause with the same bytes. The cheaper alternative was considered and
rejected: intercepting the synthesised arrows while already scrolled needs no
mouse reporting at all, but it does nothing for the filed complaint, which is
about the wheel at the LIVE view, where an arrow is a keystroke the shell has
every right to receive.
**An application that asks for the mouse owns it.** This is the rule the design
turns on, and it is the bracketed-paste rule applied to a device the client also
has a use for: what the session asked the terminal for, the client mirrors onto
the real terminal. `term_modes` grew eight bits, one per mouse DEC mode
(9/1000/1002/1003 tracking, 1005/1006/1015/1016 format), so the client can ask
for exactly the set vim asked for and forward every mouse byte verbatim. Not one
"wants the mouse" bit, which was the obvious cheaper shape: a report spelled in
a format the application did not ask for is garbage typed into it, and an
application that asked for drag reports and got only presses is one whose
selection silently does not work. The reserved bits were already documented as
the mouse modes' future home, so this needed no new frame type and no version
check. An old daemon sends them zero, which a new client reads as "nobody wants
the mouse" and keeps the wheel — the pre-mouse behaviour; an old client ignores
them entirely.
Clicks and drags with no application asking are DISCARDED, not forwarded.
There is nobody to send them to, and forwarding them types `[<0;40;12M` at the
user's shell — which is the failure mode the whole feature exists to stop.
There is a third case, and review caught it missing: an application on the
ALTERNATE screen that never asked for the mouse — `less`, `man`, any pager.
The alt screen has no scrollback of ours (`historyRows` is 0 there by
contract), so the notch was consumed by the filter and then dropped by
arithmetic that saturated at a history of zero: the wheel was simply dead in
a pager, which is worse than what it replaced, since the terminal's own
alternate-scroll synthesis had been switched off by our asking for reports.
So mux now does the synthesis itself, as tmux's `alternate-scroll` does: at
the live view, on the alt screen, with nobody holding the mouse, a notch
becomes `wheel_rows` arrow keys sent as input.
And the arrows have to be spelled the way the session reads them. The first
version sent `ESC [ A` and moved `less` not one line: less puts the cursor
keys in APPLICATION mode (DECCKM) and reads `ESC O A`, as every curses
program does. The failure is silent — the escape is ignored and the page
stays put — which is exactly the symptom the review had reported, so a fix
verified only by "arrows reached the pty" would have shipped still broken.
It is verified by a real pager now: 200 lines, `+G`, eight notches, view top
178 to 154. DECCKM is read off the replica's own engine, which carries it in
the snapshot's mode section — no new wire bit.
A keystroke that shares a read with a notch is a third thing again. The
"any other key returns to live" rule is about a key typed AT a history view;
one typed at the live view milliseconds before the wheel moved it was typed
at the shell, and swallowing it loses input to a view the user had not seen
yet. `was_live`, sampled before the scroll is applied, is what tells them
apart.
The filter is gated on `alt_screen` — this client having taken a terminal
over — and not on the mode bits alone. A client whose stdin is a PIPE never
wrote `client_mouse_setup` to anything, so nothing it reads can be a mouse
report, and filtering there is pure loss: measured, `printf 'hello
\x1b[<64;10;5M world\n' | mux` reached the pty with the escape deleted.
Scroll state counts rows, not pages. The two devices disagree about the unit —
the keys move a screenful, the wheel three rows, which is what every terminal's
own scrollback does per notch — and only one of them can be the state's.
`fetch_scrollback` already addressed absolute rows, so no wire changed for this;
`replica.scrollStart` lost its `view_rows` argument and the browser client, which
still scrolls by the page, multiplies.
The stdin parser holds an incomplete report across reads only from `ESC [ <`
onward, and that is a deliberate incompleteness. The complete parse would hold a
bare `ESC` waiting to see whether `[ <` follows, which strands every Escape typed
in vim until the next keystroke — an every-session cost to close a split that
needs a terminal to write a mouse report in two writes and the pty to deliver
under 3 bytes in a 16 KiB read. `ESC [ <` is three bytes no keyboard produces,
so the hold is unambiguous where it exists.
Measured while wiring the e2e: the client's own `\x1b[?1000h` puts the substring
`100` into every capture, and tp1's `expect 100` — the attach snapshot's last row
— matched THAT instead, sending the scroll key before the snapshot had landed.
The needle is now `\x1b[0m100`, anchored on the SGR reset the paint writes in
front of a row. The no-counting rule in that suite assumes needles stay unique to
their phase, and a client that writes new bytes to the terminal can break that
assumption from outside the scenario it breaks.
The cost this pays is the one the roadmap banked the item for: a terminal that
is reporting the mouse stops selecting text with it, so a drag inside mux no
longer marks a native selection — and the CLI still has no copy mode of its own
(the browser client does). Every terminal in use here bypasses reporting while
Shift is held (xterm, ghostty, alacritty, kitty, gnome-terminal all do), so
Shift+drag still selects, and that is the mitigation rather than a fix. It is
taken on purpose: the wheel is used constantly and the drag has a documented
key to fall back on, where the wheel had nothing. A native copy mode is what
would settle it properly, and it stays on the roadmap.
## 2026-08-19 (the wall is the home screen, phase 1: zoom is a lens)
Spec: `docs/superpowers/specs/2026-08-19-wall-home-screen-design.md`. Phase 1
of three: in-place zoom inside `wallview`. Phase 2 is attach-adds-a-tile and
`x`-forgets; phase 3 converges `client.zig`'s session loop with the wall.
**An unzoomed tile claims nothing; zooming promotes the tile's existing
connection, unzooming demotes it.** That one sentence replaces "tiles are
passive, and interaction spawns a client". Promote = resize the tile's attach
from 0×0 to the terminal's size, then forward keystrokes down it. Latest-wins
already makes a resize the legitimate claim on a grid, and the resize goes out
before the first keystroke can, so the client that types is a full-size one and
the passivity rule keeps no exception. The daemon changes not at all — not one
line of `server.zig`, no new frame, no new verb.
**Demote is client-local and sends NOTHING.** The tempting symmetry — hand the
grid back by resizing to 0×0 — does not exist: `applySize` refuses sub-minimum
resizes, so a 0×0 "relinquish" would be silently ignored, and a slot that kept
its promoted size claims nothing anyway under latest-wins, because latest-wins
follows the most recently ACTIVE client and a demoted tile is never active
again — except for bytes typed BEFORE the demote, which are deliberately still
delivered (they were typed at that session; losing them to a race would be the
worse bug). `.input` calls `claimGrid`, so such a tile can re-assert the size
it already holds, once, on its way out. Re-claiming a size it was already at
moves nothing. So the slot keeps the size it claimed, the session keeps its grid (no
resize storm every time the eye moves), and the stripe crops it — which is what
stripes have always done. "Claims nothing" is enforced by the keyboard never
queueing a byte for a tile it is not zoomed into: `sendKeys` has exactly one
caller, inside the `zoomed` branch of the input loop, and that is the whole
mechanism.
**Moving the zoom is a repaint, not a dial.** Every tile's pump keeps running
whether or not it is the zoom, so every replica is hot — `Ctrl-\ n`/`p`/`l`
change which one is painted full-screen and which one the keyboard feeds. The
old design could not offer this at any price: skipping between two sessions
meant a detach, a dial and a snapshot round trip each way. Measured in e2e as
the absence of a second attach: `muxd stats`'s per-session `clients=` gauge,
sampled every 200ms for the whole life of the wall, peaks at 1 on both tiles
across a zoom, a skip to the other tile, a skip back, and an unzoom.
**The chord table is shared, not copied.** `client.PrefixFilter` is now `pub`
and wallview reads its zoomed chords out of it; `zoomChord` maps actions to
wall meanings and is the only new table. A twin chord table is the failure mode
this repo has already paid for once. `l` joined the shared table as
`.last_session`, which the client swallows — it has no last session to skip to
until phase 3, and a shared table naming a key one caller has no meaning for is
cheaper than two tables.
`d` and `w` BOTH unzoom, deliberately. `d` is the muscle memory the child-spawn
era left behind (that child was a real client, and `Ctrl-\ d` detached it) and
`w` is where the model is going. During phases 1-2 a plain client's `Ctrl-\ w`
still spawns a wall while a zoomed tile's `w` leaves one; the two meanings
coexist on purpose and phase 3 collapses them.
**Prediction is the client's, not a second overlay.** `offerKeystroke`,
`reconcileOverlay` and `paintOverlay` became `pub` rather than being reproduced
— the overlay is the single place where "a prediction never enters the replica"
is kept, and a second implementation would be a second place to break it. One
consequence worth naming: a tile feeds `.pty_mode` frames to its overlay even
while it is a stripe, because the pty's line discipline is the entire gate on
speculation (a password prompt must never be predicted) and a promote that
started at `.never` would take a round trip to learn what it already knew.
**Input leaves on the pump's thread.** The keyboard runs on the main thread and
hands bytes to the tile through a fixed mailbox plus a non-blocking doorbell
pipe. A `Transport` has exactly one owning thread — QUIC's `service()` and
`writeFrame` share state, and a reconnect swaps the whole struct out from under
the pump — so a keyboard that wrote frames itself would be a data race on every
`.quic` tile. The doorbell is what keeps a promote and a keystroke immediate
rather than one poll timeout (100ms) away.
**Mouse reporting is deliberately not wired up.** The wheel/scrollback
machinery landed in `client.zig` days earlier, but the wall never asks its
terminal for mouse reports, so nothing arriving at a zoomed tile can be one and
filtering there would be pure loss. Phase 3 brings it for free when the two
input loops become one; until then a zoomed tile is keyboard-only.
**A gauge could not prove it; a counter can.** The first witness was
`clients=`, sampled every 200ms and asserted on its peak, and review found two
ways past it: a connection that closes as another opens never shows two at
once, and one that lives under a sample interval is invisible. So `muxd stats`
grew `attaches=`, a cumulative count of accepted attaches, and the assertion is
now that the delta across a whole wall session is exactly the tile count — two
tiles, two attaches, however far the zoom moved. Measured, not argued: a
mutation that makes the zoom close its connection and re-dial leaves the gauge
peaking at 1 on BOTH sessions (it passes) while the counter reports 3 and the
leg fails. The gauge is kept anyway, because it localises — it names which
session grew a second watcher — but it is no longer the proof.
**Only pumps answer `repaint_gen`, and a dead tile has no pump.** Observed in a
live capture during review, two symptoms of one hole: zooming a tile whose pump
had ended painted an entirely blank terminal (no cursor, no label, no way out —
indistinguishable from a hung multiplexer), and after any zoom a dead tile's
stripe never came back at all. Survivable when the screen was cleared once per
zoom; phase 1 clears it on every `n`/`p`/`l`. `Tile.alive` is set false by the
first `defer` in `pumpTile`, so every exit path sets it, and the keyboard — the
only thread left — paints for the dead: one line naming the tile, its state and
the chord out, and its bar redrawn on the way back to the wall. Deliberately
ONLY for dead pumps: a `[reconnecting]` tile still has a thread that will
repaint its hot replica within a poll timeout, and drawing over that would
replace something true with something stale.
**A mailbox that overflows drops the chunk WHOLE.** The first version copied
what fit, which splices: the head of one read lands, the middle of the stream is
lost, and the next read appends to the head — so a shell can be handed a command
nobody typed. Losing a whole read is a keystroke that did not arrive, which
users understand and retry. The tile's bar narrates it (`[up, input dropped]`)
until input moves again, because input vanishing silently is the worst version
of this.
Three mutations, to check the legs have teeth. Making a demoted wall forward
input to the selected tile fails the zero-input leg; making a promote dial a
second connection and attach full-size on it — the child-spawn design's exact
observable — fails the flat-attach leg at `the daemon accepted 3 attaches (want
2)`; making a zoom move close and re-dial fails the same leg while the gauge
alone would have passed. The first mutation also exposed a dead assertion: the
negative marker was spelled `wallnope`, whose `n` and `p` the unzoomed wall
consumes as selection keys, so what a leak would actually deliver is `wallne`
and the grep could never have matched. Renamed to `wallmute`, every letter of
which the wall swallows, and re-checked under the mutation before being
believed.
## 2026-08-20 (the wall is the home screen, phase 2: attach adds, `x` forgets)
Spec: `docs/superpowers/specs/2026-08-19-wall-home-screen-design.md`. Phase 2 of
three: the wall file becomes attach history. Phase 3 converges `client.zig`'s
session loop with the wall.
**The write seam is inside `client.attach`'s loop, not in `mux_main`.** The
spec's rule is mechanical — any `mux` attach that CLAIMS the grid (attaches at
nonzero size) writes its tile — and the argv layer cannot honour it, because
`Ctrl-\ c`/`n`/`p` re-dial full-size from inside `attach`'s own `while (true)`
and never come back through `main`. A seam up there would record the session
the user started at and none of the ones they actually visited, which is the
opposite of "the wall grows by the truth". So `client.zig` grew a `wall`
import (layer 2 ← layer 1) and one call per turn of the loop.
**The rule is enforced by WHERE the call sits, not by a size check.** There is
no `if (cols > 0)` and deliberately no tty check. `client.attach` is the only
grid-claiming attach path in the tree — it attaches at the tty's size, or at
80×24 when stdin is a pipe, never at zero — while the two passive attachers
never reach it: `muxa` does not link the module at all, and `wallview`'s tile
pumps build their own `Transport`. A tty check would additionally have excluded
the `ptyclient` fixture, which the spec explicitly wants included (tests run
under an isolated `XDG_STATE_HOME`). "muxa attaches at 0×0" stops being a
sentence in CLAUDE.md and becomes an e2e leg: `muxa send` drives the same
session and the wall file's sha256 does not move, anchored on a marker the
agent actually landed so a muxa that did nothing cannot pass by doing nothing.
**Recorded when the first STATE arrives, not when the dial succeeds.** The
first version wrote the tile straight after `Transport.open` and argued that a
transport coming up is the strongest "this attach is real" signal available
before the first snapshot. It is not a signal at all: the daemon can still
refuse (its session table is full at four), and the case analysis that followed
from the wrong seam was wrong in the way case analyses are — it enumerated the
refusal it could see. A SWITCH's refusal returns `.refused`, which has somewhere
to fall back to and so a place to undo the write; a FIRST attach refused
identically returns `.{ .exit = 1 }` and left the line stranded, a tile naming a
session that never existed. Review reproduced it: a daemon at `max_sessions`,
`mux --session PHANTOM`, rc 1 and a `#PHANTOM` line on the wall. The
`session(...) catch |err|` path had the same hole.
So the seam moved inside `session()`, keyed on `rep.state_since_attach` — the
daemon's own answer to "did this attach land", and the very flag BOTH refusal
paths read to decide there was none. A refusal now cannot record, on any path,
present or future, without anyone having to keep an enumeration exhaustive; and
with nothing recorded there is nothing to take back, so `unrecordTile` is
deleted rather than fixed. The reason the old seam existed survives untouched:
the snapshot is milliseconds behind the dial, so a `mux wall` in another
terminal still shows the session you are attached to WHILE you are attached to
it. A reconnect clears the flag and it turns true again, so the write is latched
per `session()` run — dedup would make the second one a no-op, but a resync
should not pay a read-modify-write to find that out.
The general lesson, third time in this file: **when a write has a "was this
real" question attached to it, key it on the flag the code already uses to
answer that question, rather than on a proxy plus a list of exceptions.** The
proxy is what needs the list.
**Dedup is byte-exact on the spelling, never on identity.** The same session
reached as `HOST#S` and as `quic://…#S` is two tiles, deliberately: identity
dedup would need an endpoint handshake the wall does not have and the spec does
not want. The default session records as `#0`, the RESOLVED name, not the empty
wire name the attach frame carries — a wall line must be a spelling a user
could type back.
**Best effort, one warning, never a failure.** An unwritable wall file (a
directory in its place, a read-only state home) costs one line on stderr and
nothing else: the attach happens, the session is usable, the exit code is
untouched. The warning is latched per `attach()` call rather than per write,
because a chord loop visits N sessions and a wall file that cannot be written
cannot be written for any of them — repeating the sentence would scroll a
working session for a record nobody is reading. `--via` records nothing at all
and says nothing: the wall grammar has no form for "an arbitrary command's
stdio", the same refusal `Ctrl-\ w` already makes.
**`muxweb TILE...` adds; it no longer overwrites.** Argv used to be saved with
`wall.save`, replacing the file with whatever this run was told to show — which
was defensible while the file meant "the last wall muxweb was given". It stopped
meaning that the moment attaches started writing to it: one `muxweb HOST` would
have silently erased every tile every `mux` had recorded, and the erasure would
look like a feature ("argv overrides"). Argv still overrides the VIEW — the run
shows the tiles it named and only those — but each is now `wall.record`ed,
deduped, and nothing is removed. Forgetting stays explicit: the page's `×`, the
wall's `x`, `mux wall rm`. The e2e assertion INVERTED rather than moving: it
used to fail on `argv appended to the wall instead of replacing it`, and now
fails if the pre-existing line does not survive. Both halves are asserted,
because the "argv reached the file" grep alone passes on an overwrite.
**Removal reads the file leniently; growing it does not.** `load` refuses a wall
holding a line that no longer parses, which is right — silently dropping a tile
the user wrote down is worse than making them fix it — but when every path went
through `load`, one hand-edited line made the file unrepairable by the tool that
owns it, including the command whose entire job is removing a line. So `forget`
(and therefore `mux wall rm` and `x`) reads with `loadLines`, which applies no
grammar: it can delete the broken line, and it writes every line it did not
match back byte for byte. `record` and `Wall.add` keep refusing — growing a wall
whose existing content is not understood would re-save garbage as though it had
been read.
**`mux wall add`/`rm` build the whole edit in memory and save once.** The doc
comment promised validate-all-before-write, and the per-spelling `record`/`forget`
loop made that a half-truth: a bad line was caught before anything moved, but an
IO error on the third of four arguments left the first two applied. One
in-memory pass, one atomic rename. `rm` still reports each absent spelling and
exits non-zero while removing the ones that were there — the report is about the
argument, not a reason to abandon the others.
**Atomicity is writer-vs-reader, and the staleness gap is left open.** Every
mutation goes through `wall.save`'s temp-file-plus-rename in the same directory,
so the browser hub reading concurrently never sees half a file. Two writers
resolve as last-rename-wins, which is what a single user's state file deserves.
What is NOT solved, deliberately: a `mux wall` or `muxweb` that is ALREADY
RUNNING reads the file once at startup and does not notice a line another
process adds. The file is state, not a channel; a change feed (inotify, a hub
verb, a poll) is real work for a case the model does not need yet — you see the
new tile the next time you open the wall.
**`x` is an ordered removal that never speaks to the daemon.** The line leaves
the file (`orderedRemove`, so the survivors keep the positions `1`-`9` jump to),
the tile's pump returns — which closes its transport and frees the daemon slot —
and the stripes are re-cut over what is left. Nothing else is sent: "remove is
detach", the dynamic-wall doctrine, now with a leg that proves it (`muxa
status` still answers for the forgotten session, and its marker is still in the
grid). A tile that came from `mux wall`'s own command line and was never in the
file is forgotten from the VIEW just the same, silently — the file had nothing
to remove and the screen is the answer either way.
**Forgotten tiles become holes; nothing is compacted.** The pump threads hold
`*Tile` pointers for the wall's whole life, so the array cannot shrink. The two
pure chord tables therefore take a `present: []const bool` instead of a count:
`j`/`k` step over holes, the digits renumber with the BARS the eye can actually
see, and `Ctrl-\ l` aimed at a forgotten tile unzooms — which is exactly the
fallback phase 1's comment predicted for "a tile that has since been forgotten".
The `gone` flag is also read inside `paintModeLocked`, under `paint_mu`: without
it a pump already past its own check could paint a stale stripe onto rows that
have just changed owner.
**An empty wall says so.** Forget the last tile and the terminal would
otherwise be blank with no cursor, which reads as hung — phase 1's dead-zoom
lesson, second occurrence. One line, cursor shown, naming the state and the way
out.
**`mux wall add`/`rm` are file operations only.** They do not dial, resolve a
key or spawn ssh, which is what makes them safe in a script. `add` validates
every spelling through the one grammar BEFORE writing any of them — a refused
line should not leave the earlier ones half-applied — and adds the one refusal
that belongs to the transport rather than the grammar (a `sun_path` too long to
bind), because add time is the only moment the user is still looking at what
they typed. `rm` of a spelling that is not on the wall says so and exits
non-zero: a script that thinks it cleaned up a tile should learn it was spelled
differently. The price of the subcommand is that a tile spelled literally `add`
can no longer be `mux wall`'s first argument.
**Two independent pumps, two consecutive expects, one coin flip.** The `x` leg
first expected BOTH tiles' markers in a row. Tiles are threads; `expect` consumes
forward; whichever stripe painted second left the other's marker behind the
cursor, and verb 2 spent its whole 20s budget waiting for bytes that had already
gone past. It passed twice here and failed twice on the reviewer's box, which is
the only interesting fact about it — a green run proved the scheduler, not the
code. Every other two-tile ptyclient leg in this suite expects exactly ONE
marker and then `settle`s, and that is not a style: it is the only pattern that
is a fact about the program rather than about thread order. Fixed to match.
Three mutations, to check the new legs have teeth. Making `wall.record` always
append fails the dedup leg at `a second attach to the same session made 2
lines`. Putting the seam back at dial time fails the phantom-tile leg, which is
the regression test for the case review found. Making `muxa`'s `attachZero`
record a tile like a human attach fails the byte-identical leg at `muxa attached
at 0x0 and still wrote a tile`. That one is worth a note on how it was read: on
its first run the same mutation's extra file I/O also pushed the CLI-wall
block's 5s injection expect past its budget, and the suite stopped THERE, before
reaching the leg under test — a kill that proves the mutation is detectable
without proving which assertion detects it. Re-run after the review fixes it
died on its own leg. A mutation that fails the suite somewhere is not a mutation
that fails the leg you aimed it at; when the two differ, say so and go get the
targeted answer (here, `muxa send` against a hashed wall file, standalone). No
mutation was needed for `x`:
the leg asserts a line count, a surviving spelling and a session that still
answers, and no two of those can be satisfied by the same accident.
## 2026-08-20 (the wall is the home screen, phase 3c: convergence)
Spec: `docs/superpowers/specs/2026-08-19-wall-home-screen-design.md`. The last
of three. `mux [TARGET]` is now a wall of one tile, entered zoomed; `mux wall`
is the same program entered on the wall. There is one interaction loop in the
tree, and it is a tile pump.
**The seam is the DIAL, not the loop.** `wallview.runAttach` opens the
transport on the main thread — with the tty, before any of the wall exists —
and hands it to the tile's pump, which `adopt`s it. Everything that made the
old `client.attach` worth keeping lives on that side of the seam: a first
contact reaches ssh through a shell and can want the terminal for a hostkey or
a password (the `b1abaa3` fix); a bare `Ctrl-\` has to abort the wait; and a
dial that never came up owes the user `openFailure`'s sentence and its exit
code, not a wall with a tile stuck on `[connecting]`. Nothing above the dial
needed a second copy, so nothing above it has one.
`Transport.adopt` exists for one fact: a `Transport`'s QUIC out-queue holds an
allocator, the dialling thread's allocator is not the pump's, and two threads
allocating from one non-thread-safe arena for a connection's whole life is a
bug nobody would find twice. Legal only while the queue is empty, which is
exactly the moment between `open` and the first frame — asserted, because
there is no other such moment on any path.
**A session that ends under the zoom ends MUX when it was the wall's only
tile, and drops to the wall when it was not.** This is the rule the
convergence had to invent, and both halves are load-bearing. `mux` is what
people put in scripts, so the single-tile case must propagate the shell's exit
code exactly as the plain client did — a wall that always dropped to itself
would hang a pipeline forever. And a wall with other tiles on it has somewhere
to go and something to say when it gets there: phase 1 already built the
dead-tile narration, so the drop lands on `[exited]` beside the survivors.
A session ending while UNZOOMED is not this rule at all: its stripe narrates
and the wall goes on, which is what a wall is for.
Review found the rule was stated with one half missing, and reproduced the
cost: **a wall nobody can reach the keyboard of is not somewhere to drop a
user.** A piped `mux` with two tiles whose zoomed session ends AFTER stdin
has closed dropped to a wall it could never leave and sat in `poll(-1)`
forever with the exit code in hand — RC=124, measured twice (and RC=7 with
the fix, in 8s). The first cut enforced "nobody left to steer" only at the
read that discovers EOF, so it caught the case where stdin closes on an
unzoomed wall and missed the case where the wall arrives afterwards. The fix
is where the fix belongs: `stdin_open` is an ARGUMENT to `endAction`, so the
whole rule lives in the one pure function that states it and can be tabled in
a unit test. That test now fails on the old behaviour, which is the point —
`endAction` had none, and it is the function deciding whether `mux` returns a
shell's exit code or does not return at all.
The same shape answers a refused attach. A tile a chord created and the daemon
would not have (`Ctrl-\ c` on a full table) leaves the wall without a trace —
it recorded nothing, because a refusal is precisely "no state since attach" —
and the zoom goes back to the tile the chord was typed at. That is the plain
client's `.refused` recovery, kept because the M2 leg is the only thing in the
suite that reaches it.
**`d` and `w` stopped being the same key, and that is a deliberate break.**
While the zoom was the child-spawn's replacement, `d` meant "come back",
because the child was a real client and `Ctrl-\ d` detached it. Now that `mux`
IS the wall, `d` has to keep the meaning a user's fingers already have —
detach the session, leave mux — or the convergence breaks the muscle memory it
is judged by. `w` is the model's key and the only way out of a zoom that stays
inside mux. The in-place-zoom e2e leg moved from `d` to `w`; nothing else did.
**`n`/`p`/`c` are the DAEMON's ring, not the wall's tiles.** The spec's rule,
and the reason the client's `sessions_req`/`ringNeighbour`/`PendingSwitch`
machinery was reused rather than deleted: which sessions exist is the daemon's
to say, so all three chords become one question asked over the zoomed tile's
own link. A sibling with a tile is an instant zoom move at zero round trips; a
sibling without one gets a tile, because visiting it is an attach and attach
adds. The question is asked by the PUMP (a Transport has one owning thread)
and answered to the KEYBOARD (only it may move a zoom or grow the wall), which
is what the `ask`/`ans` pair on a tile is for. The `[no session list: upgrade
muxd]` banner survives intact — same `PendingSwitch`, same deadline, painted
by the pump that owns the terminal.
**Hydration is deferred to the first unzoom, and that is a decision.** `Ctrl-\
w` shows THE wall — the saved file's tiles plus the one you are standing on —
but a bare `mux` that dialled a long attach history before showing the session
would have converged the code and broken the feel. Nothing is dialled until
somebody asks to see the wall, deduped by spelling (the file's own rule), and
a wall with no room for another stripe simply stops rather than growing a tile
pointing at rows that belong to somebody else. Known sharp edge, recorded
rather than fixed: a saved wall full of dead sockets makes the first `Ctrl-\ w`
a screenful of `[connecting]`. `x` is the answer, and the model is telling the
truth about what it was asked to remember.
**Frames before keys, and drain the whole burst that has arrived.** The
convergence's own bug, found by the suite two runs in three. `Core.forward`
decides what a wheel notch MEANS from the session's terminal modes, and those
arrive at the END of a resync burst — snapshot, pty mode, title, modes. A
single-threaded client hid the hazard by being busy painting the snapshot when
the notch arrived; a keyboard on its own thread notices immediately, so the
pump judged a notch against a session it had not finished listening to and an
application holding the mouse lost it to this client's scrollback. Reading one
frame per pass was never "one event, one frame" — it was one frame per event —
so a pass now polls with a zero timeout and keeps reading while bytes are
already there.
**Five smaller promises the old loop kept by where its code sat.** Each was
invisible until the suite asked. A pump's end must ring the keyboard AFTER
`alive` clears, or the wake finds nothing and a piped `mux` whose shell exited
hangs. A tile promoted before its first snapshot must not repaint, or every
`mux` opens by painting a blank grid the plain client never drew.
`[reconnecting]` is a banner as well as a label, because a zoomed tile has no
bar on screen. SIGWINCH has to be armed by the wall (no tile calls
`takeTerminal`) and answered by the promoted pump, because a zoomed tile is a
full-screen client and `mux` resizing with its terminal is part of "unchanged
in feel"; a tile promoted after a resize adopts the size it missed.
And `MUX_PREDICT_STATS` moved out of `Core.deinit`: a wall tile's Core is on a
detached thread the process exit kills where it stands, so the counters are
published to the driver that owns the exit and printed once, after the
terminal is back — same line, same screen, same order.
**A terminal that cannot be MEASURED is still a terminal.** `Core.init` always
answered those two questions apart — `is_tty` off stdin, the size off an ioctl
that gives up below two columns — and the wall's first cut conflated them. A
1×1 pty is a real terminal reporting a size nothing can paint at; the plain
client took it over and painted 80×24 into it, and the conflation put that
client on no alternate screen at all. The wall's terminal writes are gated on
`is_tty` alone, which is also what lets a scripted `mux` on a pipe be a wall of
one zoomed tile that writes exactly the bytes the old client wrote: no
alternate screen, no raw mode, no clears, nothing to undo.
**The abort key inherited a rule from a client that had nowhere to go.** "While
dialling or reconnecting there is no session to command, so a bare Ctrl-\
aborts" was true of `drainStdinForQuit` because a plain client's only answer to
a dead link was to leave. On a wall it is not: `Ctrl-\ w` while a tile is
reconnecting means "show me the other tile", and the first cut scanned the raw
chunk for `0x1c` and exited instead. The two readings of that byte are told
apart by the filter that already exists — a `\x1c` still HOLDING OUT for its
command key (`PrefixFilter.pending`, with no action produced) is the abort,
while `\x1c w` is a chord — so the abort is judged after `feed`, not before it.
Secondary, from the same review: the sentence was wrong for half the states it
covered. Nothing has been "detached while reconnecting" from a first attach
that never came up, so `.connecting` says `aborted before attaching` instead —
which is also what `openFailure` says for an abort one layer down. — and both had to be re-aimed,
which is the finding worth recording rather than the kills.
Making a ring step re-dial a sibling that already has a tile kills the SUITE at
`zoom skip`, four legs before `ring grow` ever runs: the same flat-count
assertion, on the phase-1 path, and it gets there first. Run against the
`ring grow` leg standalone the mutant gives `delta=3 want=2`, which is the
assertion that leg was built for. Making a single-tile session end drop to the
wall instead of exiting is worse: it wedges an early piped leg and the suite
stops having proved nothing about the rule at all. Standalone, `exit 7` under
the mutant comes back `0` rather than `7` — not the timeout's 124 that the
obvious reading predicts, because the wall it dropped to reads the script's
closed stdin as a wall nobody is left to steer and leaves. Zero is not seven,
so the leg fires either way, but the failure text had been written for the 124
and now names both shapes.
The rule this cost re-learning is the one already in these notes: a mutation
that fails the suite somewhere is not a mutation that fails the leg you aimed
it at. When the two differ, go get the targeted answer, and quote THAT.
## 2026-08-20 (the comment-discipline gate: prose became a check)
CLAUDE.md has said "comments say *why*, not *how*" and "code, comments, docs
drift" since the first commit. In one week the tree accumulated about fifty
findings' worth of exactly that drift, and a manual sweep cleaned it. This
entry is the *next* move: text in CLAUDE.md is instruction, and only a check
that RUNS is codification — the same reasoning that put the layer laws in
build.zig's comptime block, where violations are impossible rather than
detected.
`tools/docscheck.zig`, wired into `zig build check`. Three tiers, two of them
gates.
### The ast-grep spike, and why it was rejected
Timeboxed, and run before any line of the fallback was written, because
structural comment-to-decl pairing would have been strictly better than line
arithmetic if it worked. Every question got a measured answer:
Installs pinned? YES. ast-grep 0.39.6, prebuilt
app-x86_64-unknown-linux-gnu.zip, 7.4MB
compressed / 47MB binary. No build needed.
Zig grammar exists? YES. tree-sitter-grammars/tree-sitter-zig v1.1.2
(2025-09-10), pregenerated parser.c, no external
scanner. `cc -shared -fPIC -O2` produced a 713KB
zig.so exporting tree_sitter_zig, first try, and
ast-grep accepted it as a customLanguage.
Parses Zig 0.15 cleanly? ESSENTIALLY. One ERROR node across all 32 modules
and 37,756 lines: `await` used as an enum field
name in muxa.zig:37 (`verb: enum { …, await }`),
a word the grammar still reserves and Zig 0.15
does not. The error span is 5 bytes on one line —
tree-sitter recovered, and 47 function_declaration
nodes were still found in that file.
So the tooling works. It was rejected anyway, on the one question that
actually decides it: **`comment` is a tree-sitter *extra*.** It floats; it is
not a child of the declaration it documents. The grammar therefore does not
offer comment-to-decl attachment at all — pairing a doc block with its decl is
positional line arithmetic under ast-grep exactly as it is without it. Nor
does the grammar distinguish `///` from `//`, so telling a doc comment from an
ordinary one means re-reading the raw text either way. What ast-grep would
have contributed is decl ranges, in exchange for a 47MB pinned binary, a
vendored grammar, a compile step and a custom-language config inside a
pre-commit gate.
The fallback is also *better* here, not merely cheaper, and for a reason
specific to this repo: `zig fmt --check` is already a gate. Indentation is
canonical, so a decl's closing brace sits at the decl's own indent and nowhere
else, and the line arithmetic is exact rather than approximate.
### Tier 1 — cited symbols must resolve
Two halves, both tuned against the corpus rather than guessed.
`.zig` file references, backticked or bare: measured 10 backticked against 99
bare, and the bare ones are overwhelmingly this repo's own modules — so
backticking is NOT the discriminator, and both are checked. What discriminates
is a directory prefix: `osc/parsers/clipboard_operation.zig` and
`lib/types.zig` name foreign trees (ghostty-vt, the Zig stdlib) and are
skipped. The repo already wrote references that way; the gate turns the
convention into a rule, which is what makes the prefix load-bearing — it is
how a reader, and now the tool, tell "not in this repo" from "renamed last
week".
Symbol citations: **only `module.symbol` rooted at one of this repo's own
modules is checked** — 68 citations, 41 distinct, `client.recordOnState`,
`proto.MsgType`, `quic.default_port`. Both halves are knowable there: the root
is a src/*.zig the tool was handed, the tail must appear in code.
Bare identifiers are NOT checked, and that decision is the measured one. Of
1138 bare citations, 1125 resolved and 13 did not — and all 13 were names the
tool cannot possibly verify:
ngtcp2_conn_writev_stream, ngtcp2_vec_copy, ngtcp2's C API
ngtcp2_pkt_encode_stream_frame, writev_stream, ndatalen
max_title_len ghostty
sockaddr_un, isig POSIX
keep_sigpipe, NameTooLong Zig stdlib
scroll_pages a field described in the
past tense, deliberately
unrecordTile cited by client.zig
precisely to say it does
NOT exist
This repo wraps three foreign libraries and names their symbols constantly. A
bare identifier gives the tool nothing to tell "renamed last week" from
"belongs to libc", and false positives are the death of a gate. Widening the
corpus to ghostty, ngtcp2 and the Zig stdlib would fix all eleven foreign
cases and was rejected for a reason already in these notes: it puts
machine-specific, sometimes-absent paths inside a pre-commit gate, which is
the skip hazard wearing a new hat. The gap accepted in exchange is that
renaming a MODULE silently un-checks every citation rooted at its old name;
the `.zig` file rule covers module renames from the other side, which is why
that half is not narrowed the same way.
Every skip class — phrase, flag, wire, enumlit, numeric, short, keyword,
placeholder, bare — is named in the tool's own header with its reason. A skip
nobody wrote down is indistinguishable from a bug.
### Tier 2 — no project-history codenames in src/*.zig comments
"M18" and "Phase 3c" name nothing a reader can look up from the code. The
EVENT survives: "which is why the multi-session daemon is where it surfaced"
still explains itself in a year. Scope is src/*.zig comment lines only —
decisions.md, roadmap.md, handoff.md, test/, web/ and commit messages are
dated journals, and history is what they are for.
The patterns were measured before being turned on, and one of them nearly went
in wrong. `M[0-9]{1,2}`: 13 occurrences in src/, every one a milestone, zero
collisions — safe. But `phase` is *live domain vocabulary* here: `cmd.phase`
is a real field with ten comment mentions ("phase is already back to
`at_prompt`"). A pattern on the bare word would have fired on all of them. The
digit is what makes it a codename, so the rule is `Phase N` / `Task N`, and
`Tasks 6-7` is caught by the plural.
The brief said one known leftover, in replica.zig. The gate found **24 sites
across 12 of the 32 modules** — including the one named. That gap is the
entry's real content: a sweep that had just been through this tree by hand saw
one of twenty-four.
### Tier 3 — comment weight, and deliberately not a gate
Per doc block: block lines against the decl's own line span. 430 blocks in
src/ outweigh what they document. That number is exactly why this is a report:
long comments are earned here often enough that a gate on it would be
retrained-around within a day. `zig build check` prints the count; `zig build
doc-report` lists them; neither can fail on it.
### The red tests, and the one that found a bug
Each tier got a deliberate violation, watched, and reverted. The tier 1 red
test failed to go red — and the reason was a real defect: the word corpus was
built from whole files, comments included, so a citation resolved against **the
very comment that made it**. Every invented name passed. The corpus is now
built from code lines only, and re-running on the swept tree immediately
surfaced the 13 foreign-symbol findings above, which is what forced tier 1's
scope decision. A red test that stays green is not a formality.
### Wiring
`stdio = .inherit` on both runs. Inherit carries its own term check, so a
non-zero exit fails the step without `expectExitCode` — and adding one would
have silently switched the step back to captured stdio, hiding the violations
inside a dump. Inherit also makes the run unconditional: no cache hit can
stand in for a check that did not happen. The tool is built from source in
this repo, so "not installed" is a compile error rather than a green tree; the
file lists are globbed with a fatal on an empty directory; and the tool
refuses an empty `--check` or `--index` group. Four separate ways for this
gate to fail loud, because the one failure mode a green tree cannot show is a
check that never ran.
`tools/` joined `zig fmt --check`'s paths, which is how the tool's own
formatting is held to the same standard as the code it inspects.
## 2026-08-20 (the whole-terminal path goes)
`interact.Core` carried two ways to take a terminal. The whole-terminal one —
`init` (measure the tty and size the Engine from it), `takeTerminal` (raw mode
and the SIGWINCH handler), `ownTerminal` (the alternate screen, entered on the
first frame), `readTyped` (read stdin, split the chord off the front) and the
`Claim.whole` arm that told the exit teardown which bytes to write — has had
zero production callers since the convergence made every session a wall tile.
Reachable only from `refAllDeclsRecursive` and from tests of itself.
`src/interact.zig` goes 2893 → 2789 lines: 206 deleted against 102 rewritten
in place, and only 47 of the deletions were code. The rest was prose. **Why
now:** the
comment-discipline sweep the day before kept finding drift *inside* this path —
a doc paragraph had already been added saying "No driver takes this path
today", and the essays underneath it described a client that no longer exists
(a process that owns a screen for its whole run, a reconnect loop that
re-enters with `claim` already set). Dead code is a drift magnet: it costs
nothing to run and everything to keep honest, and every future comment sweep
pays for it again. The docs gate cannot help here — a citation of a live symbol
from a dead function resolves perfectly.
**What stayed, and why.** The wall derives its terminal lifecycle from the same
building blocks, so tracing each one mattered more than the deletion:
- `terminal_frame_setup` — the SCREEN half. `wall_setup` is built from it.
- `session_claim` / `session_release` — the SESSION half. `claimTerminal` and
every demote path.
- `terminal_teardown` — `wall_teardown` *is* it. Still the only `?2004l` mux is
certain to write, and still the string that must end in `?1049l` for e2e's
`tp1` doctored control.
- `client_mouse_capture` / `client_mouse_setup` / `inClientCapture` /
`mouse_teardown` — the comptime-generated mouse pairing, unchanged.
- `ttySize` / `watchWinch` — public, and the wall calls both directly.
`terminal_setup` (`terminal_frame_setup ++ session_claim`) went with the path:
`ownTerminal` was its only writer. So did three Core fields nothing else
touched — `prefix` and `in_buf` (16 KiB per Core, `readTyped`'s alone) and
`orig_termios` with the `deinit` restore reading it, since only `takeTerminal`
ever set it and the wall owns raw mode now.
**The essays moved rather than died,** where the argument still defends live
code. `ownTerminal`'s claim-is-a-flag-not-an-ordering argument (a `?2004h`
written outside a claim is one nothing will ever undo; it was a race before it
was a flag) is now on `Claim` itself, which is the flag. Its title-stack
argument (an unmatched pop restores a stranger's title, so the push must be
exactly one deep) is now on `wall_setup`, which is what pushes `22;0t` today.
`takeTerminal`'s ISIG/IXON argument moved to the wall's raw-mode block, which
sets those bits. The DEC 1007 alternate-scroll argument was already stated at
`claimTerminal` and needed only its second copy dropped. What died with the
code: the "enter the alt screen late so `--via`'s stderr survives" argument
(the wall enters at startup, before it dials — a different design, not this
one's) and the "call this ahead of routing every frame" discipline, which was
about a call nobody makes.
**Tests.** The four-way pairing test lost its one assertion about
`terminal_setup` and kept the rest; the teardown pin retargeted its push
half from `terminal_setup` to `wall_setup`, which is the same six bytes of
title-and-alt-screen a wall writes. The promote/demote pipe test and the
mouse-mode `inline for` drift tests are untouched.
**No behavior change**, and the suite was the referee: `zig build check`,
`make test` and `make e2e` all green with the e2e pin still at 55 scenarios /
35 convergence points. Recovery, if a whole-terminal client is ever wanted
again: git remembers it, at the commit this entry ships in.
## 2026-08-20 (ssh agent forwarding: the daemon owns the socket)
`mux HOST` uses ssh as a bootstrap only — after the QUIC upgrade there is no
ssh process left, so `ssh -A` has nothing to forward through in steady state
and `git push` on the remote fails. That was the last named blocker to
replacing tmux as the daily driver. tmux has the dual disease: the socket
exists but goes stale on every reconnect, which is what `update-environment`
and the symlink hacks are patching over. Both failures have one root — the
process that owns the agent socket dies with the connection.
**The decision: muxd owns a stable agent socket per session, and clients that
opted in answer for it.** The socket outlives every connection, so the
`SSH_AUTH_SOCK` a shell was born with is valid for that shell's whole life
and the staleness disease cannot be expressed. Who answers changes with
attachment, invisibly to the shell. Four consequences, each argued below: the
bytes ride **frames, not transport**; routing is **latest-active offerer**;
the daemon **pumps blind**; and it is **opt-in**, because the threat model is
`ssh -A`'s threat model exactly.
**Frames, not transport** (`agent_offer` 0x0d, `agent_data` 0x0e,
`agent_close` 0x0f client-side, `agent_open` 0x92 daemon-side). Only the
daemon opens channels, so ids need no opener scoping; `agent_data` and
`agent_close` are bidirectional. This keeps the invariant that `proxy.zig`
and the QUIC modules carry opaque bytes — forwarding works over ssh, QUIC and
`--sock` alike because none of them knows it happened. `agent_data` is a u32
LE id followed by at most `agent_data_max` = 4096 opaque bytes; a daemon
holds at most `max_agent_chans` = 8 live channels, a tile the same.
**The daemon pumps blind.** It parses the id prefix and nothing else — never
the agent protocol. That is what keeps policy where policy belongs: `ssh-add
-c` confirmations still fire on the machine holding the key, and mux has no
opinion of its own to drift out of date. Agent-protocol filtering and
key-confirmation prompts were considered and refused for the same reason.
**Latest-active offerer, decided once per connection.** `Server.agentAnswerer`
ranks the session's `-A` clients by the same `activity` counter the grid
follows (bumped by attach, input and resize), so the signer is whoever typed
last — the person driving. It is called once per *accepted connection*, not
per frame: an in-flight exchange that swapped identities under ssh would fail
the signature rather than move it. With no offerer the daemon accepts the
dial and closes it immediately, which ssh reports as a refusal instead of a
hang.
**Opt-in, per attach, per tile.** `-A` is a flag on the attach form only —
`mux wall`'s grammar reads a bare `-A` as a hostname, and the wall is a view,
not an attach. Sibling tiles grown by chord (`Ctrl-\ c`/`n`/`p`) inherit the
flag: same target, so nothing is exposed that the user has not already
exposed to that host, and a chord-made tile has no command line to spell the
flag on — not inheriting would end forwarding silently at the first session
switch. Tiles hydrated from the wall file never inherit: the user never named
them in this session, and a stranger's host must not be handed keys by
history. `muxa` has no `-A` at all — agents do not wield the user's keys, the
same posture as the OSC 52 clipboard-read refusal. `muxweb -A` (the hub
offering its own `$SSH_AUTH_SOCK`) is phase 2, not built.
**What the build changed about the design.** The spec put
`agent-<session>.sock` at 0600 directly beside the control socket. Shipped
instead: a 0700 `mux-agent-<pid>-<rand>` directory holding
`agent-<name>.sock` per session, for the reasons `shellint.install` already
wrote down — that parent is a shared `/tmp` whenever `$XDG_RUNTIME_DIR` is
unset, a pid alone is guessable, and an entry pre-created there by another
user as a symlink would put this daemon's sockets somewhere it does not own.
Failure to make the directory degrades to null and says so on stderr rather
than failing the daemon: a session with no forwarded agent is a working
session, and from inside the shell an absent `SSH_AUTH_SOCK` looks exactly
like a client that never asked to forward one.
**Compatibility.** The offer is re-armed on *every* attach through the single
`sendAttach` funnel in `wallview.zig`, because a redial lands on a fresh
daemon-side slot that remembers no offer. The payload is empty and the frame
type is new, so a daemon too old to know it skips it: a new client with `-A`
against an old daemon attaches normally and forwards nothing.
That was witnessed, not argued — but **not by `test/xversion.sh`**, and the
distinction is worth writing down. The rig's legs never pass `-A`, so what
they witness is "a new client drives an old daemon" (both transports, green
against the previous release) and not "a new client that *sends*
`agent_offer` drives a daemon with no arm for it". The missing half was run
directly, new client against a v0.0.1-11-era daemon, `ptyclient` on a real
pty: attach works, the session is interactive afterwards (two typed markers
echo back), `SSH_AUTH_SOCK` is empty, and no `mux-agent-*` directory is
created. Bracketed both ways — the same script without `-A` behaves
identically, so the flag changes nothing on that pairing, and the same script
against a *new* daemon prints
`AUTHSOCK=[…/mux-agent-<pid>-<rand>/agent-0.sock]`, which is what makes the
empty answer an assertion rather than a script that never ran.
**Measurements.** The e2e suite goes 55 → 58 scenarios (35 convergence points
unchanged), across three legs: a real `ssh-agent` and a generated key
crossing the wire to a real `ssh-add -l`; a keyless session refusing; and the
signer flipping to whoever typed last. The refusal leg is the one with a wall
clock on it — attach, two 400 ms settles, the `agtsock` round trip, shell
startup and the refusal measured **816–837 ms** across three grading runs,
held under a 10 s ceiling (~12x headroom, matching the number the
key-mismatch leg picked for the same job). The ceiling's honest band is
narrow and the comment says so: a refusal that *hung* is caught by `waitexit`
as an exit code, so what the clock owns is only a refusal that came back
after seconds of retrying. Refusing, `ssh-add` returned **1** (`error
fetching identities: communication with agent failed`) — the answer a socket
that accepts and closes gives. Not pinned, openssh's internals; but the
distinction is load-bearing, because a session with no `SSH_AUTH_SOCK` at all
returns 2 (`Could not open a connection to your authentication agent`)
instead, and for a while that 2 was what this leg was measuring: the positive
leg's `exit` takes its daemon down with its last session, so the refusal
leg's client was auto-starting an *installed* `muxd` off `$PATH` — v0.0.1-10,
no agent code in it — and passing vacuously against it. The leg now starts
its own daemon and pins `agtsock=present` before typing `ssh-add`, which is
the needle only a daemon that bound an agent socket can answer, and the suite
puts the build's own bin directory first on `$PATH` so no future accidental
auto-start can grade a release again. Mutation testing on `agentAnswerer` caught both attempts
to break routing: pinning to the first offerer, and deciding once per daemon.
`test/xversion.sh` itself does not reach rc=0 for this delivery, and the
reason is the rig's, not the feature's — the limitation already filed as
"xversion: epoch identity pins refuse any post-M18 'old' side". With
old = the previous release, five identity probes fail by construction (they
pin the old side by the *absence* of `sessions=`, `term_modes`, `term_title`
and `term_event`) while all five real compat legs pass; building the same
old side into both halves of the rig reproduces those five failures
byte-for-byte with this branch nowhere in the picture. With a genuinely
pre-M18 old side (v0.0.1-5, built from a `git archive` export) the rig goes
9/10, and the one failure — new client to old daemon over a socket, "connection
to muxd lost" — reproduces identically with `main` as the new side. So:
no cross-version regression is attributable to agent forwarding, and the
gate's own vintage pins are what stand between it and a green run.
**Known limitations, recorded rather than fixed.** A SIGKILLed daemon leaks
its `mux-agent-*` directory — the same lifecycle as `mux-shellint-*`, and the
same trade. The redial re-offer is structural (one funnel, so it cannot be
forgotten) but has no dedicated e2e leg. The reverse pin — a channel already
open staying with the client it opened on when activity flips — is asserted
in unit tests only; end to end it would need a signing operation slow enough
to flip activity underneath it. And the `-A`-against-an-old-daemon check
above was run by hand: codifying it needs the rig to take client flags per
leg, which is the same change the filed epoch-pin issue wants.
**The generalization we did not build.** Port forwarding (`-L`/`-R`) is the
known next step and this frame vocabulary is its embryo: the same
open/data/close shape with a target in the open payload, never an ssh side
channel (steady state has no ssh, and pure `quic://` targets never had one).
What it adds that agent traffic does not need is opener-scoped channel ids —
a `-L` client opens channels, whereas today only the daemon does, and SSH's
each-side-names-its-own model retrofits onto these frames without breaking
them — plus per-channel flow control, so a bulk transfer cannot
head-of-line-block a repaint. Nothing shipped here assumes their absence.
**Review round: what the review found and what it changed.** Five follow-ups
and four nits, all taken. Two were behaviour, and both had the same shape —
a rule that only one side of the wire was keeping.
`agent_data_max` had been a send-side buffer size and nothing more: the
receive path checked no length, so a peer could frame 16 MiB (the generic
`max_payload`) and both ends would hand the lot to a blocking `writeAllFd` —
the daemon stalling every session, the client freezing the tile. The
blocking write's own rationale, that agent traffic is small, is a property
of the senders we ship rather than of anyone who might dial. `proto.agentDataOversize`
is now checked at both ends and answered by closing the channel: dropping
the frame would leave the agent stream short of bytes its far end is still
waiting on, and a truncated signature request is worse than a hangup. This
matters twice over because port forwarding inherits the receive path.
`SSH_AUTH_SOCK` was set only when there WAS a socket, so every failure path
of `makeAgentDir`/`bindAgentSock` fell through to the daemon's own inherited
agent — the exact state the arm's comment forbids, reached where nobody
looks. `Pty.EnvPair.value` went optional so "no socket" can mean unset
rather than empty (an empty value is still a socket to ssh, which then fails
on it instead of falling through). The test gives the daemon an agent of its
own first: without one it passes whatever the code does, which is the same
vacuous-leg trap this branch already hit once in the e2e.
The rest were visibility and coverage. The per-tile consent gate had no test
— the refusal leg's client has no agent, so the channel is refused at the
dial whatever the gate says, and both gates could be deleted under a green
suite; `openAgentChan` came out of the pump to get a seam a test can reach,
and the test binds a real listening agent socket so `offered` is the only
variable. Daemon-side refusals were invisible: eight long-lived channels can
turn forwarding off for every session with nothing anywhere to say why, so
both reasons are counted and a full table says so once per episode rather
than once per dial. The channel cap now lives in `protocol.zig` beside
`agent_data_max` — a client table bigger than the daemon's has slots nothing
can fill, smaller refuses channels the daemon believes it opened. And
`mux wall -A host` is a usage error instead of a tile for a host named `-A`.
The user-visible cost, which the README had not stated: a locally
auto-started muxd used to pass its own inherited agent through, so `git push`
in a local session worked with no flag. The daemon's socket now overwrites
it, and a local session needs `-A` where it previously needed nothing. The
overwrite is the point — otherwise every client shares one identity — but it
is a regression for one workflow and is now documented as one.
**Releases are tarred, and `make release` is why there is a rule.** From
v0.0.1-4 through v0.0.1-10 a release was one `mux-vX-x86_64-linux-musl.tar.gz`;
v0.0.1-11 published the four binaries bare, with no decision recorded anywhere
and nothing in the repo consuming either shape. The bare form costs the exec
bit — a downloaded binary arrives 0644, so `mux HOST` on a freshly installed
box dies with `Permission denied` from the remote `muxd` and reports it as
`no endpoint announce (UnterminatedLine)`, which names neither the file nor
the mode. Observed on a LAN box the day v0.0.1-12-pre1 went out.
So: tar, flat, one `tar xzf - -C ~/.local/bin` over ssh to install, and the
recipe lives in the Makefile rather than in a memory of how the last one was
built. The target asks the STRIPPED artifacts their `--version` and refuses
to ship if it disagrees with `build.zig` — a stale stage directory under a
bumped number is the failure a hand-run recipe actually has. The binaries are
reproducible (a rebuild matched the published sha256s byte for byte); the
gzip wrapper is not, since it embeds an mtime.
## 2026-08-21 (the `-A` preflight asks instead of dialling)
**A connect proves a socket exists, not that an agent is behind it.** The
preflight shipped as `connect()` + `close()`, which is the right question
everywhere except the one place `-A` is easiest to get wrong: inside a mux
session, where `SSH_AUTH_SOCK` names the daemon's own per-session socket.
The daemon accepts every connection and only afterwards looks for an offerer
to route it to — deliberately, so that a session nobody has offered an agent
to refuses fast instead of making ssh wait out a timeout. The dial therefore
succeeded, the preflight passed, and the nested client attached as an
offerer that could answer nothing. It could also out-rank a working `-A`
client on the inner session, since `agentAnswerer` picks the latest-active
OFFERER and offering is a declaration, not a capability.
So the probe became a round trip: `SSH_AGENTC_REQUEST_IDENTITIES` — the
five bytes `ssh-add -l` sends — and a wait for a reply header. This also
keeps the legitimate nesting working, which no environment-based check
could: with an `-A` client on the outer session the request is forwarded
out, the real agent answers, and the nested `-A` is allowed. The alternative
considered and rejected was noticing `proto.session_env` is set and
`SSH_AUTH_SOCK` is that session's socket, the signal the self-attach refusal
reads. It cannot distinguish the two cases at all; it would refuse the chain
that works.
**Fail open on silence, closed on a hangup.** The refusal is a close on an
already-accepted connection and arrives in microseconds, so slowness is not
the discriminator: a poll timeout PASSES. That keeps a hardware token or a
cold-starting gpg-agent — both of which can take a while to answer a first
request — from being refused by a check that only meant to catch an absence.
The 500ms bound is not asked to separate refused from slow; it only has to
outlast a real agent's round trip, including one forwarded back out of an
outer session over a link with an RTT.
**The layering.** The daemon still pumps blind and `proxy.zig`,
`protocol.zig` and the QUIC modules still carry opaque bytes. What knows
five bytes of ssh-agent framing is the CLIENT, which is the agent's own peer
and the process that made the `-A` promise — a different thing from a
transport that would be parsing someone else's exchange. `send` with
`MSG_NOSIGNAL`, not `write`: the peer may already be gone and the preflight
runs before the client installs any signal handling.
The unit test's stand-in compares against a spelled-out `{0,0,0,1,11}`
rather than against the constant under test — mutating the constant survived
until it did, which is the "checks that fail green" hazard in its smallest
form. e2e scenario 58 runs the nested case for real: inside a session nobody
offered an agent to, a nested `mux -A` must exit 2 AND say so with the
preflight's message, because the self-attach refusal standing behind it also
exits 2 and the code alone cannot tell which one spoke.
## 2026-08-21 (a detached session stops rendering, and what that moved)
Every number below is one box: idle 16-core x86_64, ReleaseSafe, `make
throughput`'s own harness. They are quoted so a later reading has something
to disagree with, not because they transfer.
**`make install` shipped Debug for the whole prototype's life.** `install`
depended on `build`, which is `zig build` with no `-Doptimize`, so
`~/.local/bin` got a Debug tree — and ghostty gates its page-integrity check
on ITS OWN optimize mode, rebuilding a hash map over every cell and standing
up a fresh DebugAllocator per page mutation. 20k lines of `yes` at 80×24, no
client: **3713ms installed, 6ms ReleaseSafe, 6ms tmux, 24ms a bare pty.**
perf put 96% of cycles in `verifyIntegrity`. The dev tree stays Debug on
purpose — the assertions are worth having while working on the engine — so
`install` and `throughput` each build ReleaseSafe into their own prefix
rather than flipping the default. The rule that follows: **no speed number
measured in the dev tree means anything**, and `make test`/`e2e`/`soak` all
inherit Debug.
**A user-space profile misranked the thing it was used to rank.** The
detached daemon's cost looked like the formatter (57% in
`PageFormatter.writeCodepointWithReplacement`) with the allocator a ~5%
footnote. The real split was `utime 24.1s / stime 106.5s` — **81% in the
kernel**, from `dumpVtRow` taking a fresh GPA buffer per row, fifty per pty
read, churning mmap/munmap and page faults. `perf record -p` cannot see it:
those frames come back as unresolved `[unknown]` addresses. Removing the
loop cut instructions 5.9× and wall time 27× (64784ms → 2361ms on 374MB of
full-width repaint at 200×50); the missing 4.7× was system time. **Check the
user/system split before ranking work off a profile.**
**The blind path owes three things `update()` used to provide.** `seq` must
advance — it is stamped into `last_return.seq` and two commands returning
during one blind stretch must not share one, or an await cannot tell which
it was told about. Every `row_seq` must be stamped, so a reattach gets a
delta carrying the whole grid rather than one omitting what moved unseen.
And the hashes must be marked stale: blind output moves a row A→B, the
reattach delta carries B, but the stored hash still says A — so a later B→A
reads as "unchanged", nothing is sent, and the client shows B for the rest
of the session. Only a change-and-change-back triggers it and nothing else
in the tracker would catch it.
**It moved side-channel replay, and that was not intended.** `noteBlind` has
no `.none` case, so a gap advances seq on every pty chunk. Events are
stamped at `tracker.seq` and replayed strictly above the reattaching
client's watermark, so a bare BEL or an OSC 52 with no redraw behind it —
previously stamped at the seq the gap began on and therefore dropped — now
survives a delta reattach. Better behaviour, arrived at sideways. Pinned in
delta.zig rather than in the gap fixture, which prints a visible marker
first precisely so it never depended on the seq rule.
**`hasClientsIn` counts a 0×0 attach, and must.** `muxa run`/`await` hold a
client slot for the whole verb, so the win does not reach the agent
workload — those sessions take the rendering path. The tempting narrowing,
treating `cols == 0` as "not watching", is **unsafe**: the webhub stand-in
attaches at 0×0 and reads real content back over its WebSocket — the e2e
scenario whose comment reads "A POSTed tile is a REAL tile". 0×0 means
"claims no grid", never "wants no bytes". Anchored on that sentence rather
than a line number, which drifts.
**The throughput gate's bounds are calibrated, not principled.** Best-of-5
per leg, because noise only ever adds time and a regression moves the
minimum as much as the mean. Solo 6ms/ceiling 10; client 45-60ms with a 60ms
comfort target that only reports and a 75ms ceiling — a hard 60 flaked 1 run
in 10 with nothing wrong, and a gate pinned to the noise floor gets disabled
by its third false alarm. Repaint 87ms/ceiling 200, set near the good number
rather than under the bad one: at 400 a machine twice as fast runs the
BROKEN build in 240ms and the gate says nothing. All bounds are env
overridable so a slower box can relax them without deleting the target.
`yes` cannot see any of this — one-character rows are nearly free to render,
which is why the regression this branch fixed was invisible to the first two
legs (5ms vs 6ms) and obvious to the third (87ms vs 479ms).
## 2026-08-22 (an offer nobody can answer is hung up on)
The `-A` preflight (2026-08-21) fixed "offering is a declaration, not a
capability" at one call site, client-side. The daemon still took every
client's word for it, and what that produced was the exact failure the
"refuse fast" rule exists to prevent. Measured, ssh against a unix socket
that accepts and never replies vs. one that accepts and closes:
silent agent ssh-add -l blocked past an 8s timeout (exit 124)
accept+close "communication with agent failed" in 2ms (exit 1)
So a bogus offerer that out-ranks a working `-A` client does not degrade
forwarding, it wedges it — `git push` on the remote hangs instead of falling
through to the next auth method — and eight such dials fill
`agent_chans_max`, at which point the real offerer is refused too.
**The courteous path already existed, and is the reason the fix is small.**
An honest client answers `agent_open` it cannot serve — no `SSH_AUTH_SOCK`,
a stale path, a full local table — with `agent_close`, the daemon closes
the fd, and ssh reads the refusal in microseconds. What remained was the
peer that does not speak at all: a wedged client, a third-party one, or a
browser tab that sent `agent_offer` through a hub that transits any type.
**The clock runs from the first forwarded request to the first reply, and
never again.** Two anchors were rejected, each pinned by a test watched to
fail under the mutation:
- Not from the open. The daemon announces `agent_open` the moment ssh
connects, but until ssh writes a request the client has nothing to
answer. A clock on the open hangs up on a working client for ssh's pause.
- Not per request. One reply proves the peer speaks for an agent; a later
SIGN may wait on a hardware token for as long as the human takes to
touch it. The preflight's rule stands: the bound must never separate
slow from refused.
5s, not the preflight's 500ms: that bound sat on a client-local socket, this
one crosses the link. Still well short of the 8s wedge, and the close the
daemon makes is the same `close(fd)` the courteous path ends in, so ssh
sees no difference. Expiry also clears the client's `agent_offer` — without
that the next dial routes to the same mute peer and every ssh pays the
bound. A field on Server, not a const, so the unit tests set it to 150ms
rather than wait; not env-overridable, because it is a judgement about the
product rather than a gate calibrated to a box, and nothing in `make ci`
depends on it.
**Deliberately unfixed: a peer that answers once and then goes mute.** It
wedges every later SIGN. Clocking later requests is exactly the
hardware-token case above, and the issue scopes this as robustness, not an
escalation — a peer in a position to out-rank the real offerer has already
passed the Origin gate and can inject keystrokes into every tile, which is
strictly more. Leave it. Likewise a reattach restores the offer: `agent_offer`
is re-sent after every attach by design, and the arm that lands it does not
care whether the sweep took one away — a reattach may bring a working agent,
and a mute one now costs one bound per attach rather than per dial. And only a
REPLY proves: bytes the client sends before any request was forwarded leave
the clock armed, or a mute peer could clear it by talking first.
**The hub keeps the three agent frames for itself.** `muxweb -A` (next, the
issue after this one) offers the agent on the HUB's machine; the browser
never holds a key, so every agent frame on the daemon leg is the hub's own.
`parseFrameMessage` now refuses `agent_offer`/`agent_data`/`agent_close`
from a tab as one named exception, which leaves "unknown types transit"
exactly as wide as it was. Before `muxweb -A` this closes the silent-offerer
hole at its source; after it, it is what stops a tab injecting `agent_data`
into a channel the hub owns.
## 2026-08-22 — in-band size reports (mode 2048): the daemon speaks them
nvim 0.11+ sends DECRQM 2048 on start; ghostty-vt's stock handler answers
"recognised" (`?2048;2$y`) because ghostty proper supports it — in its APP
layer (`Termio.sizeReportLocked`), which mux does not use. nvim then set the
mode and, per the protocol, ignored SIGWINCH from then on, so a resized
session stayed painted at its old size (a drag-resize: two screens overlaid;
reattach could not clear it because the daemon's grid held that picture).
A shell and Claude Code never opt in, which is why only nvim showed it.
Decision: implement the protocol rather than deny it. The engine queues
`CSI 48;rows;cols;0;0 t` when the mode is set and after every resize;
`applySize` flushes it, the one engine event that answers the app without
a feed. Pixels are 0 (headless, allowed). Denying (`;0$y`) was one line but
would have left nvim on the SIGWINCH race during drags. Measured: nvim
`&lines x &columns` 26x80 → 60x180 after a 10-step grow burst; the e2e
witness is `printf '\e[?2048h'; cat -v`, no nvim dependency. Fix is in
muxd: every daemon (the LAN box included) needs the new build.
## 2026-08-23 — the doc gate grows teeth, and indicts its own author
Tier 3 was a printed count nobody owned: 498 flagged blocks, report-only by
design, unchanged week to week. A warn that cannot be discharged is a warn
everyone learns to read past, so it became a gate.
Ruler first, because 498 was mostly noise. 336 of them sat on a decl with no
body — a field, an enum tag, a bare const — where `block > span` compares
prose against a line that is pure declaration. Exempting bodyless decls left
162. Weight then moved from lines to prose bytes: a comment line runs ~66
bytes and a code line ~20, so counting lines rewarded a 100-column comment
over a wrapped one, and any rewrap changed the verdict. Bytes took it to 334.
The median flagged block is 2.17x its decl, p90 is 7.3x — after the noise
class is gone the density is real, not measurement error, so no threshold
above 1.0 was chosen to make the number look better.
`//!` headers were skipped entirely (they answer to no decl) and are now held
to an absolute 1024-byte cap. The cap is not taste: server.zig is the largest
module in this tree and states its contract in 532 bytes. Zig's own stdlib is
the outside oracle — 150 of 540 files carry a `//!` at all, median 170 bytes,
and the largest header in the whole library is 4262 bytes (os/linux/seccomp.zig,
documenting a kernel ABI). Eight files here are over the cap. Two of them,
wallview.zig at 5310 and this tool at 5889, were larger than anything in std.
The gate is a per-file byte figure in docscheck.budget, met EXACTLY. The first
version budgeted flag COUNTS and a mutation killed it: growing cmd.zig's
`marksOpen` block — already that file's only flag — moved no count and passed
green. A count lets flagged prose grow forever, which is the same warn-and-
ignore failure one level down. Exact match rather than `<=` because slack left
in a file is room to regrow for free; every change to the number is now a diff
line someone signed. Starting debt: 34 files, 191,592 bytes.
tools/ went into the corpus at the same time. A gate its author is exempt from
is an argument, not a rule, and the fold immediately found three violations in
this tool (two illustrative `foo.zig` citations that resolved to nothing, three
codenames spelled out as examples of what tier 2 forbids) plus a message bug —
tier 1 and 2 errors were formatted with a hardcoded `src/` prefix and reported
`src/docscheck.zig` for a file in tools/.
### What moved out of docscheck.zig's header
Its header was 5889 bytes, most of it arguing for its own existence and
restating `test "a CLI, wire or enum spelling is not a citation; a bare name
is"` in prose — ~1400 bytes describing eight skip classes that the test
already pins executably. Deleted. The measurements behind the skips are the
part worth keeping, and they belong here:
- Skip classes were read off the corpus, not guessed: 1138 of 2049
backtick-quoted tokens in src/ survive the filters.
- Bare identifiers are eligible by class but dropped by the `module.symbol`
root filter, and the reason is measured. Of 1138 bare citations in src/,
1125 resolved and 13 did not — and every one of the 13 was a name this tool
cannot verify: ngtcp2's C API (`ngtcp2_vec_copy`,
`ngtcp2_conn_writev_stream`), ghostty's `max_title_len`, POSIX's
`sockaddr_un` and `isig`, Zig std's `keep_sigpipe` and `NameTooLong`, plus
`unrecordTile`, cited by client.zig precisely to say it does NOT exist. This
repo wraps three foreign libraries and names their symbols constantly.
Widening the corpus to ghostty, ngtcp2 and the Zig stdlib would fix it and
was rejected: it puts machine-specific, sometimes-absent paths inside a
pre-commit gate.
- `.zig` file references are checked backticked or not — 10 backticked against
99 bare, so backticking is not the discriminator. A reference carrying a
directory prefix that is not src or test names a foreign tree and is skipped;
that prefix is load-bearing documentation.
- Trailing comments are not inspected: the 219 candidate lines in src/ are
overwhelmingly `quic://` inside a string literal. They are still stripped
from the corpus, which matters more — see `codeOf`.
- The gap tier 1 leaves: renaming a MODULE silently un-checks every citation
rooted at its old name. The file-reference rule covers module renames from
the other side, which is why it is not narrowed the same way.
Doctrine the session settled on, and the reason none of this was written as a
comment beside the code it justifies: a justification is a test. A comment
arguing that behavior is correct should be a `test` whose NAME is the claim —
same file, executable, and it fails when the claim stops being true. Writing
text to justify text (an inline "reviewed" marker, an accept-list of forgiven
blocks) was considered and rejected: its existence should be justification
enough.
## 2026-08-23 — what the client's doc blocks were carrying
The tier-3 burn-down of `client.zig` moved these here rather than deleting
them. Each was a paragraph in a doc block, and each is a fact about a past
run or a past compile, which is this file's job and not the source's.
- **`quicTransport` is type-load-bearing on `error.UserAbort`.** It is the
only source of that error either switch in `openHandoff` can see —
`readAnnounceAbortable` is a second source on the handoff path overall, but
its abort arrives before these are reached. Deleting the call narrows the
inferred error set and the compiler refuses `error.UserAbort` as "not a
member of destination error set". Verified by mutation, not assumed.
- **Why the handshake wait lives inside `quicTransport` and not its caller.**
`connect` only creates state; the first flight has not been answered. A
transport returned before that makes "open succeeded" mean something weaker
for QUIC than for every other transport, and the reconnect loop believes
it — counting a dead attempt as live, firing an attach into a connection
that never completes, and repeating on the next pass. Against a daemon that
was merely paused this left a trail of half-open connections and duplicate
attaches.
- **`waitReady`'s bound is the only thing that ends a blackholed dial.** A
refused port now ends the wait early: quic_client surfaces the ICMP refusal
from whichever syscall the kernel hands it to. A blackholed one produces no
error at all.
- **`recordTile` has no tty check, deliberately.** A test fixture on a real
pty is a human attach by the mechanical rule ("attaches at nonzero size"),
which is fine because the suite runs under an isolated `XDG_STATE_HOME`.
- **`recordOnState` replaced a "the dial succeeded" seam.** A first attach
the daemon refused — its session table is full at four — exited 1 with the
line stranded, naming a session that never existed; so did an error out of
the pump before any state. Only a SWITCH's refusal had a fallback and so a
place to undo the write. See also the `state_since_attach` entry above.
## 2026-08-23 — what the wall's doc blocks were carrying
Same burn-down, `wallview.zig`. Each of these was a paragraph in a doc block
and is a rule the tests already assert or a fact about a past run.
- **`wallMouse`: a plain click is `MouseDown1Pane -> select-pane`,**
deliberately — it is what the user already has in their hands from tmux. A
DRAG highlights the lines it crossed, inside the stripe it started in, and
the highlight stands when the button comes up. Left button only: middle is
the terminal's own paste and right its menu, and stealing either would be a
surprise the wall has no answer for. The button word arrives from the filter
verbatim, motion and modifier bits included, so the low two bits name the
button. The keyboard never PAINTS the highlight — it writes the drag under
`paint_mu` and rings, and the pumps draw it inside the stripe paints they
were doing anyway; anything drawn from the key loop would be overwritten by
the next frame from the session.
- **`tilePaintBegin` is the only enforcement point, and used to be four.**
The promote repaint, the frame repaint, the expiry repaint and the
speculation each spelled the zoom test themselves, and any one of them
forgetting it was a prediction glyph painted onto whichever session the
zoom had moved to. A demoted tile's Core now cannot write a grid byte at
all: no rows, no overlay, nothing.
- **`endWith` ringing was measured, not reasoned about.** It hung a piped
`mux` whose shell had exited: the keyboard's test for a finished tile is
`!alive`, so a ring before that store wakes the keyboard, which finds the
tile still alive, drains the bell and sleeps again — and the news never
arrives.
- **`sendKeys`: a mouse report CUT by a read boundary is the one uncovered
corner.** It is the same split class `interact.MouseFilter` admits to at its
own edge. The wall holds the head; `setZoom` drops it (`WallInput.reset`),
since the bytes describe a screen that has changed hands; the next read
arrives zoomed and its tail reaches `sendKeys` as keystrokes, so a shell can
be handed `0;3;2M`. It needs a terminal that splits one report across two
writes AND a zoom landing in the gap. Named rather than defended against:
the alternative is holding a partial report across the transition, which is
state about a screen nobody is looking at any more.
- **`zoomChord`: `w` unzooms, `d` detaches, and they used to be one key.**
While the zoom was still a child-spawn's replacement, `d` meant "come back
from this tile". Now that `mux TARGET` is itself a wall entered zoomed, `d`
has to keep meaning what it means to every user's fingers — detach and
leave mux — or the muscle memory the convergence is judged by is the first
thing it breaks. `n`/`p`/`c` are NOT tile motion: they move the zoom around
the DAEMON's session ring, the plain client's discipline moved a layer out,
while the wall's selector (`j`/`k`/digits) still walks tiles. `Ctrl-\ l`
with nowhere to go back to unzooms rather than guessing, which is also the
answer a tile that has since been FORGOTTEN gets — hence `present` is
consulted and not just the length — and the answer `l` aimed at the tile
already zoomed gets, as tmux's `prefix-l` does: "go where I was" cannot mean
"stay here".
- **`forgetTile` on a tile that is not in the wall file** — a spelling named
on `mux wall`'s own command line — forgets it from the VIEW just the same,
silently: the file had nothing to remove and the screen is the answer either
way. A file error is remembered rather than printed, because this terminal
is on the alternate screen and a stray line would corrupt the paint, and is
said once on the way out.
- **`showsSelf` drops a self-tile only because the list was auto-built** —
the saved wall, restored on an unzoom nobody spelled out — so a name the
user never typed can leave without contradicting anything they said. `mux
wall HOST#a` refuses the WHOLE wall instead (`mux_main`), for the opposite
reason: those tiles are what was asked for, and silently omitting one would
be a wall lying about what it shows. The predicate is the true
socket-and-session pair, unix sockets only, and an emptied variable counts
as unset — `MUX_SESSION=` is the documented way to override the refusal.
- **`endAction`'s first half:** a session that ends under the zoom ends MUX
when it was the wall's only tile — which is every plain `mux TARGET`, so the
shell's exit code propagates as it always did — and drops to the wall when
it was not, because there is still something to look at.
- **`awaitingSession` was `state != .up` once.** That cost a zoom into a
refused tile its only way out: the `w` of `\x1cw` never arrived, because the
`\x1c` had already been spent quitting.
- **`runAttach`: a first contact with a host reaches ssh through a shell,**
which can want the tty for a hostkey prompt or a password; the abort key has
to work while it waits; and a dial that never came up owes the user the
sentence and exit code the plain client always gave (`client.openFailure`).
None of that is a pump's to do.
- **`relayout` failing keeps the old geometry.** Forgetting a tile can only
give the survivors MORE rows, so `TooSmall` there is unreachable except for
the empty wall, which is handled before it.
## 2026-08-23 — the tree-wide tier-3 burn-down
Eight agents took the remaining 279 flagged doc blocks to zero across thirty
files, on the doctrine the two headers established earlier today: a comment
states the WHY a reader at the decl cannot get from the code, a justification
is a test, and history stays here.
What follows is what came OUT of those comments — past compiles, past hangs,
timing numbers, mutation ledgers, and rules the tests already assert. It is
grouped by the file it was cut from and names the decl each fact belonged to.
Nothing here was deleted; it moved, because this file is grepped and the
source is re-read every turn.
### server.zig — facts moved out of doc comments
History, measurements, past bugs and mutation-testing ledgers cut from
`src/server.zig` doc blocks during the tier-3 burn-down. Each bullet names the
decl it was attached to.
#### Transport / QUIC
- **`drainWaitMs`** — the floor and the ceiling each fix a different wedge.
Since commit `207ebbb`, ngtcp2 retransmission happens ONLY in `tick()`: sleep
past an expiry and a lost packet is never resent, the peer has nothing to
acknowledge, the socket never becomes readable, and the connection is wedged
for the whole budget.
- **`boundUdpPort`** — placement rationale: it lives on the daemon rather than
on `quic_server.Listener` beside `pollFd` because the daemon and server.zig's
own quic test helper are the only things outside `quic_server.zig` that have
ever needed to ask the socket for its port, and two callers were not enough to
widen that type's surface.
- **`Sink.close`** — the close-the-connection-not-the-socket rule is pinned by a
test in this file.
- **`Server.quic`** (the ownership union) — replaced a `?*Listener` + `bool`
pair that could type the unrepresentable `(null, owned)` state.
- **`endpointPort`** — the reason a 0 answer goes to stderr and not into the
frame: the asker can do nothing with it but relay, and `muxd endpoint`'s
announce-none already tells the client everything it can act on.
- **`endpointPortFrom`** — key resolution order is `MUX_KEY_FILE` then the
default path. There is deliberately no `--key` half: a daemon being asked
lazily is one that was never handed a flag.
- **`refusalFrame`** — kept parameterless because a builder taking a payload
would need a copy loop no caller would ever exercise.
#### Sessions, sizing, agent forwarding
- **`resolveSession`** — the bug the real threshold prevents. Gating creation on
merely "nonzero" forked a shell whose engine, pty and winsize were all 1×1;
`applySize` then refused to move it, `recordSize` was skipped, the slot stayed
0×0, and `claimGrid` could never claim — so the client that caused the size
could never fix it. A session nobody can use is worse than a refusal nobody
can miss.
- **`makeAgentDir`** — the random half of the directory name also settles the
mundane collision: a SIGKILLed predecessor whose pid we redraw. The 0700 +
exclusive-create posture, and the degrade-to-null posture, are the ones
`shellint.install` wrote down at length.
- **`agentAnswerer`** — a slot that has never attached has no session and no
activity, so it is excluded by the session test before the ranking sees it.
- **`bumpActivity`** — which frames count as `.input` (and why paging scrollback
or asking for stats is not one of them) is argued in the `.input` arm's own
inline comment.
#### Pending side events (clipboard / bell)
- **`pendingSlots`** — the three consumers (replay, expiry, teardown) each used
to hand-write `{ &pending_clipboard, &pending_bell }`, so a third
`SideEvent.Kind` would have compiled clean and been recorded, never replayed,
never expired, and leaked on teardown: the privacy contract would have
silently stopped applying to it.
- **`recordPending`** — a failed dupe drops the event silently, exactly as a
failed encode in `drainSideEvents` does and for the same reason: there is
nowhere to say so, and what is lost is one replay of one event.
- **`rebuildTracker`** — where `rebuild` fails is what matters (delta.zig). It
fails at two POINTS, not in two ways (both are allocation failures): a failure
in the resize block backs out before touching any field, so the drop correctly
retains everything; a failure inside the dump loop happens after `reset_seq`
has already advanced and leaves `rows` at 0, which makes `canServe` false for
every seq in existence — payloads permanently undeliverable and still
resident. The unconditional `defer` is safe because the drop is predicated on
`canServe`, not on which branch got there. Both halves of that predicate are
pinned by direct call in the expiry tests, since neither failure point is
reachable from a test.
- **`replayPending`** — the snapshot branch is guarded twice over, found by
mutation rather than designed: every snapshot path goes through a rebuild, the
rebuild moves `reset_seq` past every recorded seq, and `rebuildTracker` drops
what it has just put out of reach, so a call added AFTER a `snapshotTo` finds
both slots empty. The guards are not redundant (a failed rebuild leaves only
the branch holding the line) but no test can tell that mutation from the real
thing.
- **`replayPending`** — `> have_seq` used to cost an invisible chunk its replay:
`update()` answers `.none` when no cell moved, so a bare BEL or an OSC 52 with
no redraw behind it stayed stamped at the seq the gap began on and the loop
refused it. It no longer does, and not by design — a gap is by definition
unattached, `noteBlind` has no `.none` case, so every pty chunk during one
advances seq. Pinned in delta.zig ("a blind chunk that changed nothing still
advances seq"), because nothing on the server side would notice it going away.
- **`replayPending`** — deferring the replay alongside modes and title is not
even available: defers unwind last-registered-first, so anything registered
there would run BEFORE the sampled-state block at the top.
- **`resyncSnapshot`** — the "still do the bookkeeping, then bail on the
sending" shape used to be shared with `drainSideEvents`, which cited it. The
divergence is deliberate: the pending slots gave that drain something to fold
into session state on the way past, so it now encodes for nobody on purpose.
#### Command state / awaits
- **`sendCmdStateTo`** — closes the same gap `sendPtyModeTo` does, one level up:
`cmd_state` is only pushed on a transition, so a client attaching between two
commands would know nothing until the next one. `muxa status` is the way to
ask about a session that has never spoken marks; it reports the regime rather
than claiming a transition. The ordering is the opposite of `sendPtyModeTo`'s
for the same underlying reason: mode bits describe how to read bytes that have
not arrived yet, rows describe bytes that have.
- **`fallbackState`** — one owner for the overrides because the three fallback
arms in `checkAwaits` differ only in which overrides they take, and
hand-patching the struct at each site made a set of deliberate differences
look like three drifting copies. `phase` and `clear_exit_code` default to
leaving what `cmdState` built: a mechanism overrides only what it can claim to
know. The seq override orders the reply against the grid content the client
has, which is what a fallback answer is about; whether these arms should move
the watermark at all is an open question recorded on `proto.CmdState.seq`.
- **`queueSelectionReply`** — encoding into isolated scratch means a failure has
not touched the client's pending bytes, so it costs this reply only;
`queueFrame` retains its own rule for failures after the complete payload
reaches the real queue. `.unavailable` with no text is the one reply that
cannot fail validation, leaving allocation as its only remaining way to be
lost.
#### Sampling, stats, tuning
- **`sampleTermModes`** — sampled rather than intercepted because a mode has no
history worth keeping: a reattaching client needs the current value.
- **`sampleTermTitle`** — same sampled-state discipline and same early return as
`sampleTermModes`: a title changes when you cd or start an editor, not per
chunk.
- **`accrueSnapshotEquiv`** — pays a full snapshot serialization per update
purely to measure the saving. Put it behind an option if it ever costs
anything.
- **`liveClients`** — slot occupancy was previously unobservable from outside: a
QUIC connection that completes its handshake and never attaches holds a slot
until its idle timeout, and there was no way to see that happening, or to see
it clear, without attaching a debugger.
- **`statsText`** — the old leading `seq=` field was ONE session's tracker,
which had no honest answer once there could be more than one, so it moved off
the main line entirely. The fields that were always parsed by name
(`snapshots=`, `clients=`) keep meaning what they always meant. The bench
measures a live single client whose queue drains every pump, so accruing byte
counters at queue-accept time leaves the ratio it reports unaffected.
- **`shrinkSendBuf`** — the tests depend only on the result being small, never
on its value (Linux doubles the request and clamps up to `SOCK_MIN_SNDBUF`).
#### Test fixtures
- **`awaitFrame`** — `iters` is roughly 6ms of wall clock each (a 5ms pump plus
a 1ms poll), so 200 is about a second and a quarter. Bounded so a regression
fails there instead of hanging the suite. The blocking read is safe only
because nothing in the test path can split a frame across two writes; see
`shrinkSendBuf` for the deliberate work it takes to make the daemon's socket
buffer too small to swallow a frame whole.
- **`expectInitRefused`** — a Server built when it should not have been owns a
live shell on a pty; letting the test discard it leaves that shell holding the
test runner's stdout, so the build never sees EOF and hangs for hours. That is
exactly how the daemon-stealing bug hid in the first place.
- **`bash_rc_with_prompt_member`** — Arch's `/etc/bash.bashrc` appends a
`PROMPT_COMMAND` member under any `xterm*` TERM, and muxd sets exactly that;
the fixture plants the same shape rather than relying on the system file.
- **`zsh_rc_with_precmd_hook`** — zsh hands each precmd hook the original
command's status rather than the previous hook's, which is what makes mux's
reading safe. Measured on this box before it was asserted.
- **`writeGapShell`** — the mutation ledger for the fixture's four gap events:
- `after-osc`, the SECOND clipboard set and the BEL: remove any and a test
goes red.
- the FIRST clipboard set: removing it leaves the suite green. It holds up a
MUTATION — it is the only reason the CLIPBOARD slot is written twice, so
without it `recordPending` keeping the first occupant instead of the last is
indistinguishable from correct. Specifically that set, not the replacement
free next to it: the bell tests write the bell slot repeatedly by
themselves, so with the set deleted AND the free deleted a bell test still
reports the leak. (Both checked by mutation; the first draft of this bullet
named the free and was wrong.)
- `gap-open`: removing it leaves the suite green and uncovers no mutation. It
removes a DEPENDENCY instead — without it these tests pass because the pty
echoes `go`, so they would keep passing until they ran somewhere with ECHO
off and then fail describing the replay rule rather than the terminal
setting. It was load-bearing back when only a changed cell, a moved cursor
or history growth advanced `tracker.seq`.
- the BEL also matters because every comment on the bell replay path is
clipboard-flavoured, which is how a well-meant tightening ("the privacy rule
is about the clipboard") could quietly drop it.
- **`writeDyingGapShell`** — a session dies through `reapSessions`, not through
`Server.deinit`, and those are the two places the teardown half of the pending
contract is honoured. Every gap test above holds its shell open and so
exercises only the deinit site; nothing reached the reap site at all, and a
shell that exits between the copy and the next attach is the ordinary case.
- **`modesWithResync`** — separate from `awaitFrame` because the question is
about a PAIR (which branch ran, and what it said about the modes) and
`awaitFrame` drops everything that is not what it was asked for, including the
frame that names the branch.
### interact.zig — facts moved out of the source
History, measurements, past bugs and rationale trimmed from `src/interact.zig`
doc blocks. Each bullet names the symbol it belonged to.
#### `appendTermTitle`
- OSC 0 is written rather than OSC 2 so the icon name moves with the title:
OSC 0 is what a session's own applications write, and both forms reach the
engine as one window-title operation. Mirroring what the session wrote is
the point.
- Restoring the user's ORIGINAL title on exit is not this function's job and
is not left undone: the host terminal's own title stack carries it, via the
`22;0t` that leads the alt-screen entry and the `23;0t` that closes
`terminal_teardown`.
- mux cannot read a title back, and the engine cannot help: ghostty's terminal
handler ignores `title_push`/`title_pop` outright, so the SESSION's title
stack does not exist to be mirrored. This is why the host terminal's own
stack is the only mechanism available.
- The control-byte refusal (anything below 0x20, plus DEL 0x7f) exists because
such a byte terminates the OSC early — BEL is the terminator itself, ESC
begins the other one — and everything after it lands on the user's screen as
text they have to clear. Re-checked client-side because the daemon's own
check (`sampleTermTitle`) is on the other side of a wire whose peer need not
be this version of muxd.
#### `writeSideChannel`
- **The bug this gating closed** (the title found it): `is_tty` is `isatty` of
STDIN — it gates raw mode and the alt-screen entry, both about input — while
the side-channel writes go to STDOUT. With stdin redirected and stdout still
a terminal (`echo x | mux`, `mux < /dev/null` typed at a prompt) the claim
never left `.none`, so mux would set the user's title and never pop it, and
turn bracketed paste on and never turn it off. Every side channel had the
same shape; only the title made it a broken promise, because the title is the
one mux justified by saying it could put things back.
- **The accepted cost:** in that mode the session's title, clipboard and bell
go nowhere even though a terminal is attached to stdout and would have shown
them. That matches what mux already does there (no raw mode, no alternate
screen, no hidden cursor); the alternative is a client changing terminal
state it has arranged no way to change back. Gated in one place rather than
at the three call sites so a fourth channel cannot arrive without it.
- Side channels are written OUTSIDE the paint's synchronized-update bracket:
they are messages TO the terminal, not part of the picture, and a sync
bracket around one would hold it until the next frame.
- `append` is a declared function type rather than `anytype` because `anytype`
accepts a builder that fails some way other than allocation, and that error
propagates out of `Core.frame` into the driver's pump, which abandons the
rest of the read (`wallview.pumpTile`'s `break :frames`). A stray clipboard
byte does not get to do that. `Value` is comptime for the same reason: it
names the contract and lets each caller's value coerce to the type its
builder declares.
#### `appendHostEffect`
- The clipboard target and base64 alphabet are re-checked here rather than
trusted from the effect because `ClipboardSet` is a plain struct — Zig cannot
make the validating decoder its only constructor — and one caller already
builds an unvalidated one: `wasm_core.zig` default-initialises its borrowed
clipboard slot to `.{ .target = 0, .base64 = &.{} }`, which `validClipboard`
refuses. A linear scan over at most 64 KiB is free next to the write it
guards; the alternative is `ESC]52;<NUL>;BEL` on a real tty.
- Every builder in this file is built whole before the first byte is appended:
a rejection must not leave half an escape behind for a caller that reuses one
buffer across events.
#### `writeSelectionCopy`
- Routed through `appendHostEffect` rather than a second OSC 52 of its own: the
target check, the alphabet check and the all-or-nothing shape belong to that
function, and a hand-rolled write would be a second place to get them wrong.
- Target `c` is the clipboard proper — what tmux's `set-clipboard external`
sets, and what a paste reads back.
- `owns_terminal` is true unconditionally because the claim answers whether a
SESSION's clipboard event may reach a terminal this tile does not hold; this
text is the user's own drag on the screen in front of them, and at the wall
the tile answering it is a demoted stripe every time. The caller gates on
`is_tty` instead — a piped `mux` claims no mouse modes and can have no drag.
#### `appendMouseModes`
- An application that asked for the mouse gets EXACTLY the modes it asked for
and every mouse byte verbatim (see `MouseFilter`'s call site); otherwise the
client keeps its own capture set and spends the wheel on scrollback.
- A full level-set of all eight modes every time, not a diff, because
`term_modes` is sampled state that repeats on every attach and reconnect and
the terminal on the other end may be one this process never configured (a
reconnect, a `--via` that reconnected under us). Level-setting is idempotent,
so the repeats cost bytes and nothing else.
#### `mouse_teardown`
- Built from the wire table rather than typed out because the set it has to
undo is exactly the set the daemon can ask it to mirror. A forgotten mode is
a terminal left reporting clicks into the user's shell as escape sequences,
long after mux exited.
#### `altScrollSeq` — measurement
- Measured on `less +G`: with DECCKM set (APPLICATION cursor keys, which every
curses program sets), `ESC [ A` scrolled nothing at all — `less` reads
`ESC O A` and ignores the normal spelling as an escape it does not know.
#### `sendAltScroll`
- Sent as input rather than predicted partly because `offerKeystroke` refuses
anything that is not a single byte anyway.
- The batch loop is what bounds the buffer, not the burst; `alt_scroll_batch`
is twenty-one notches' worth, which no hand produces in one read.
#### `watchWinch`
- Public because the driver that owns the terminal is not always a Core: the
wall puts its own terminal into raw mode and has no Core at all, while the
zoomed tile still has to follow the tty. The wall arms the signal, the
promoted pump answers it.
#### `dumpPredictStats`
- Public because a Core is not always the thing that says goodbye: a wall
tile's Core lives on a DETACHED pump thread that process exit kills where it
stands — no return, no deinit — so wallview's `run` prints the line after it
has put the terminal back.
#### `Sink`
- Two function pointers rather than a driver interface because there is exactly
one question — "may I write to `out_fd` now, and will that stay true until I
say I am done" — and the wall already had the answer (`paint_mu` plus its
zoom check) before this Core existed.
- A plain client never asks: its terminal is its own for the whole run, so the
default answers yes and holds nothing.
- It gates paints and not side channels because a mode or a title is a write
whose meaning does not depend on where the cursor is.
#### `initSized`
- The wall measures the terminal at startup and re-reads it only through the
promoted tile that answers the SIGWINCH (`setWallSize`), so a tile clips to
the size its stripe was cut from rather than to whatever a second ioctl says
after the user dragged a corner.
#### `adoptSize`
- A tile's Core is born at the terminal's size and only `winch` changes it;
`winch` reads a PROCESS-wide flag, so only the tile holding the terminal can
answer one. A tile promoted after somebody resized would otherwise clip its
paints to a screen that no longer exists. The resize frame that follows a
promote tells the daemon; `adoptSize` is the local half.
#### `claimTerminal`
- Writes `session_claim` once per PROMOTE; the wall wrote the screen half
(`wall_setup`) for its own lifetime. What a session needs is exactly the
mouse modes: without them a host terminal answers the wheel by synthesising
arrow keys (DEC 1007) that land in the session as input — which is what a
zoomed tile did before the Core existed, and the whole reason a tile's input
is routed through one.
- A claim taken outside the sink can be preempted between reading the zoom
store and writing its modes, land AFTER the release meant to precede it, and
leave a terminal in modes nothing is arranged to undo — because the demote
deliberately writes nothing.
- An attaching client is told the modes moments later (`sendResync` ends with
`term_modes`), but a promote's resize is answered by `resyncSnapshot`, which
carries no modes deliberately. Without the level-set at promote, every wheel
report would be eaten as this client's scrollback instead of reaching the
application it belongs to.
- The Core knows the modes without asking: `semantic` has tracked every mode
sample since the tile was born, because only the WRITE was ever gated on the
claim. Level-setting is idempotent, so a session with nothing to say pays a
few bytes.
#### `dropScrollView`
- A resync repaints live state, so a history page left up would be silently
replaced a moment later — and the scroll banner would sit over stale rows
until the user happened to leave scroll mode.
- The overlay's scroll-mode bit is recoverable rather than terminal: the "any
other key" exit in `forward` cannot clear it (that branch is guarded by
`scroll_rows > 0`, which `dropScrollView` has just made false), but
Shift+PageDown's `scroll_rows == 0` arm clears it unconditionally. Only by a
keystroke the user has no reason to guess, so prediction is silently off
until they hit it.
#### `reconcileOverlay` / `paintOverlay` / `offerKeystroke`
- Kept private together: the wall's pump drives a `Core` rather than
hand-rolling the client's input path, so nothing outside the file reaches the
overlay. The overlay is where "prediction never enters the replica" is kept,
so the fewer doors the better — a driver needing one of these back is a
second implementation announcing itself.
#### `replicaCellChar`
- Reading the replica's own cursor instead of the predicted one hands reconcile
a `prev_ch` belonging to somebody else's cell, which turns "the frame has not
answered yet" into "we were contradicted" and flushes the queue. The
real-Engine test in this file exists to catch exactly that.
#### `grid`
- The wall's DEMOTED tile is the only driver that paints its own view of a
session: a stripe is a crop of the same grid, painted by the wall at the
wall's rows, while the Core paints nothing because it holds no claim. One
replica per tile, one applier for it (`replica.zig`), two ways of looking.
#### `dragReports`
- The button word arrives from the filter verbatim, motion bit and modifiers
included, so it is the low two bits that name the button.
- At the wall a click moves the selection between stripes; a zoomed tile is the
only session on its screen, which is why a click is a no-op here.
- The copy is `forward`'s to send because the transport is.
#### `replyBytes` (test fixture)
- The hand-written layout is id, status, watermark, then text — the shape
`protocol.encodeSelectionReply` produces.
#### `releaseTerminal`
- "Nothing goes on the wire" is the whole of the CLAUDE.md invariant that an
unzoomed tile claims nothing.
- The demoted tile's highlight is dropped because an inversion kept across a
demote would be invisible until the next zoom and then reappear over rows the
user chose in another session's lifetime — and a reply still in flight would
copy text for them. (Unchanged — it was already an inline comment on the
`self.drag.clear()` statement, and stays there.)
### Moved out of src/muxa.zig and src/main.zig (tier-3 burn-down)
Facts, measurements and rationale removed from doc blocks. They belong in
`docs/decisions.md`, not in the source.
#### src/muxa.zig
### `Conn.link`
- The verbs above the union are written once and know nothing of the
transport. That is the claim `--quic` makes, and the union is where it is
kept.
### `Conn.openQuic`
- `connect` only creates state — the first flight has not been answered.
- `client.zig`'s `quicTransport` waits for the handshake for the same reason.
### `Conn.graceMs`
- The unix arm keeps the flat 2s (`await_grace_ms`). The QUIC arm adds
nothing until four round trips of its own handshake exceed that: never on
a LAN or loopback, most of a second on a 200ms link.
- Four round trips, not two, because the request and the reply are not the
only flights in the trip — the daemon may be settling a command when the
timeout fires.
- The cap exists because `connect_ms` is bounded only by the handshake wait:
a connection that took fifteen seconds to come up would otherwise buy a
minute of grace. Past the cap we are no longer waiting for the daemon's
answer but for a network that has already shown it cannot carry one.
### `Conn.sendFrame`
- `deadline_ms` is the caller's own bound — the same one it will wait for
the answer under.
### `Conn.sendFrameQuic`
- QUIC `send` takes what fits and reports how much (a bounded ring; the
caller holds the backlog), so a short take is not a failure and not
ignorable either — the tail is offered again once acks have made room.
- muxa's frames are a handful of bytes against a 256KB ring, so the loop is
expected never to turn twice.
### `Conn.awaitFrame`
- The returned frame is allocated from the Conn's own allocator, so
`frame.deinit` takes that one. Every caller was already passing it (one
allocator in this process); asking for it made the pairing look like a
choice.
### `Conn.awaitFrameFd`
- Debt deliberately retained on the socket arm: only the wait is
deadline-bounded, not the read. Once poll says a frame has begun,
`readFrame`'s `readExact` blocks until the whole payload lands, so a peer
that stalls mid-frame outlives the deadline.
- Harmless over a local socket: the daemon writes whole frames at once, and
a stall means a daemon that has stopped running rather than a path that
has stopped delivering. Buying it off would mean a second partial-frame
buffer for a case that cannot happen there.
### `Conn.awaitFrameQuic`
- Nothing here blocks on the transport: a datagram carries whatever arrived,
whole frames or a third of one, so frames are delimited out of the
client's inbound buffer and a partial tail simply stays there until the
rest lands. A daemon that stops mid-frame costs this loop the deadline it
was given and not a second more.
- A datagram routinely carries several frames and the reply may be the
second, which is why every buffered frame is taken before the next poll.
### `Conn.reconnect`
- Nothing reads `graceMs` after the reconnect: the await it belongs to
already has its deadline, and the re-issue continues that same deadline.
### `frameFrom`
- The arithmetic is `proto.delimitFrame`'s — the same walk the daemon does
over the same wire from the other end. What `frameFrom` adds is the copy.
### `emit`
- A closed pipe (the agent's harness stopped reading) or a full filesystem
is enough to produce exit 0 with nothing on stdout. Swallowing the write
error turned both into a silent success — the one shape the contract says
cannot happen, and the worst to hand an agent: its shell tool checks the
status first, sees success, then has no object to parse.
### `failAs`
- Produces exactly the `"<verb>: <what>"` the two verbs printed when they
were written out separately.
### `deadlineFor`
- The alternative reading of `--timeout 0` — a deadline already in the past
— would make it fail instantly instead of waiting forever.
### `failSessionEnded`
- JSON like every other outcome, but on the failure path.
### `writeCmdFields`
- The five fields are published in the one order both verbs have always
used. Each verb keeps its own envelope; what they stopped keeping is a
second spelling of the fields inside it.
### `attachZero`
- `name` is joins-only by construction, not by a separate check: this binary
never spawns a shell by asking about one.
### `spanFetchDeadline`
- Widening the fetch deadline to the larger of the two bounds handed a
`--timeout 0` run an unbounded fetch — muxa hanging forever on a daemon
that went quiet, long after the answer the agent asked for was in hand.
- Narrowing to the smaller would cut off exactly the transcript
`span_fetch_ms` exists to rescue: the one belonging to a command that
returned in the last millisecond of the window.
- Pinned by the test "the span fetch is bounded even when the run it follows
was not".
### `awaitReissuing`
- A wait is the only round trip long enough for a network to die under: a
status round trip is over in a millisecond, a `run` on a build is not, and
losing it costs an agent the whole command it was watching. This is the
one place a transport failure is retried rather than reported.
- What makes the retry safe rather than a second command is `since_seq`: the
request is a question about a watermark ("tell me about a return newer
than this"), so re-asking it after a reconnect is the SAME question and
the daemon answers it identically whether or not it saw the first one.
The server's own tests pin that idempotency.
- At-most-once, not at-least-once: what is re-sent is the attach and the
`await_req` and nothing else — never `run`'s input. If the command line
was lost with the connection, the re-issued await finds no return and the
agent is told `timeout`, which is true and checkable, instead of the shell
running `make deploy` a second time because a client decided to be
helpful. A wait may be repeated because asking twice changes nothing; an
input may not, because it changes everything.
- Three things are deliberately not reset: the deadline (a redial that ate
four seconds has spent four seconds of the wait, not bought a fresh one);
`since_seq`; and the attach, which IS re-sent, at 0x0 like every other
attach this binary makes.
- `ConnectionLost` is the only error that reconnects. `SendStalled` does not:
a stall means the peer is still there but has stopped acknowledging a
quarter-megabyte of backlog — it has already spent the flush bound proving
that, and at muxa's frame sizes it is very nearly unreachable.
- A redial that fails does not swallow the reason: it is recorded on the
`Conn` and `ConnectionLost` is re-raised, so the verb reports what went
wrong FIRST (the path tore) and second (the redial), not only the second.
### `fetchSpan`
- `end_row` is the row the D mark landed on (the prompt redraw), so the span
is [start_row, end_row) and an end at or before the start is no output.
- Rows are absolute screen rows and best-effort by construction (see
`MarkEvent.row`): the alt screen and scrollback pruning can invalidate
them between the reply and the fetch. The exit code is the answer; the
transcript is the bonus.
### `printAwaitReply`
- One JSON object: what ended the wait, the session's command state when it
ended, and how long we waited.
### `reportAwait`
- `returned` and `settled` are both answers, including a command that
returned nonzero.
### `awaitVerb`
- `await` and `run` are one pipeline: attach claiming no grid, read the
watermark, wait for the session to come to rest, report. `run` is that
pipeline with a command line put in — sent between the watermark and the
wait, with the marks span fetched at the end — so the two are written once
rather than twice with the middle diverging.
#### src/main.zig
### `pickKey`
- An order that quietly inverted would otherwise only show up as a daemon
authenticating with the wrong key.
### `envKey`
- `Key.load` would blame a confusing `""`.
### `specForName`
- Exact match on the whole word, first row wins.
### `specForCmd`
- The first-wins scan is unambiguous only because the same comptime block
refuses a second row for one `Cmd`.
### `shellIntegrationEnabled`
- Shell integration was an OPT-OUT through the agent surface and the
multi-session daemon, on the reasoning that marks are what make an exit
code knowable. The daily-driver reading is the opposite: the shim costs a
zsh user their `~/.zshenv` and displaces a bash user's DEBUG trap (atuin,
bash-preexec) on every session, while only `muxa` reads what it buys.
- Inverting the sense rather than adding a second spelling means a stale
`MUX_SHELL_INTEGRATION=0` still reads as off.
### `oneShotQuery`
- `dump` and `stats` are the same round trip and differed only in the verb
they name, the frame they send and the frame they wait for.
- Frames of other types are skipped rather than refused: the reply is the
answer to THIS request, and a daemon is free to have said something else
on the way to it.
- `askEndpointPort` may be asked by a binary from before `endpoint_req`
existed, which is why it waits under a deadline; `oneShotQuery` has no
such case (a daemon that understands the socket understands both verbs).
### `logHint`
- The clause was written twice, with a comment saying so, before it became
this function. Its two users are `stopCmd` and `reportNoListener`, both
about a daemon not doing what was asked while the reader is elsewhere.
- The finding that must not be replaced by an error trace: "the daemon did
not stop", "the daemon has no listener".
### `reportNoListener`
- This is the likeliest way the announce goes negative in production, and it
must not be silent.
- The log clause is `logHint`'s, conditions and hedge included — the same
clause `stopCmd` ends with, for the same reasons.
### `reportKeyRefusal`
- "no usable key" alone would cost a reader on another box the trip to find
out which of the three refusals it was.
- The words are `quic.keyRefusalBody`'s and this half owns only the prefix
and the `; staying on ssh` that says what the refusal cost. Word for word
what `run` prints for the same refusals, path in the same position.
- Every error is handed over, catch-all included: `announceKeyFrom` only
reaches `load_failed` with what the load returned, so an unclassified
error is still a key that would not read — which is what the body's fourth
sentence says. `run`'s arm routes only the three for the opposite reason:
it can afford to let the rest propagate.
### `parse`
- `parseArgs` takes what `argsAlloc` produces, so the tests must speak the
same type.
### Moved out of src/protocol.zig and src/engine.zig (tier-3 burn-down)
#### protocol.zig
- **`delimitFrame`** — Kept pure and taking a plain slice rather than a
connection so both ends of the wire delimit with the same arithmetic and the
function can be exercised against a canned buffer.
- **`delimitFrame`** — The type byte is read through a non-exhaustive enum on
purpose: an unknown message type is the peer's business to have sent and the
caller's to ignore, not a reason to refuse the stream.
- **`appendFrame`** — Exists because daemons buffer frames per client and flush
opportunistically instead of blocking on a slow peer. A golden test pins that
its bytes are identical to `writeFrame`'s.
- **`agentDataOversize`** — The channel, not the frame, is the unit of refusal:
dropping the oversize frame instead would leave the agent stream short of
bytes its far end is still waiting on. Both ends hand the payload straight to
a blocking `writeAllFd`, so a frame claiming more than one bite is refused
rather than pumped.
- **`SelectionPoint`** — The protocol preserves both endpoints exactly and
leaves ordering/normalization to the component that owns the terminal grid.
- **`encodeSelectionReply`** — Allocation errors retain normal ArrayList
semantics (only `error.BadPayload` is guaranteed to leave `out` untouched).
- **`encodeAwaitReq`** — History: this decl used to carry a comment admitting
that a caller who set `.name` and called this instead of `encodeAwaitReqNamed`
would find the name silently dropped — a known way to send the wrong bytes,
written down and left live. Dropping `AwaitReq.name`'s default was considered
and does not fix it: it forces every literal to say `.name = ""` and still
lets `.name = "b"` reach the function. The assert replaced the comment because
it fires at the call site, in the build modes the tests and the daemon run
under.
- **`StatusReply`** — The fields chosen are what a driving agent needs before
deciding how to interact: size, cursor, whether a TUI holds the screen, who
echoes keystrokes, and the command state.
- **`PtyModeFlags`** — Read off the session's pty by the daemon and shipped
verbatim: the client is told what the terminal IS, never what to do about it.
The six reserved bits are pinned to zero by a test rather than left to
whatever the encoder happened to have on the stack.
- **`TermModes`** (unflagged, context for the above) — its reserved bits work
the opposite way deliberately: an unknown bit there is ignored, because those
modes are independent host settings rather than one prediction verdict.
- **`TermEvent.Clipboard`** — The payload stays base64 the whole way: ghostty
hands the OSC 52 payload over undecoded, and every transform is a chance to
corrupt bytes neither end ever needs to read. The daemon caps length on the
way in and the client re-validates before it lands in an `ESC]52;…BEL`
written to a real tty; this codec only carries the bytes between them.
- **`encodeBellEvent`** — A fixed-array return would just move the append into
the caller's `switch (ev.kind)` arm, which is why both event encoders share
the append shape.
- **`resolveName`** — Spelled in one function because "empty means default" was
otherwise a convention five daemon call sites happened to remember in the
same way, and five copies is five chances for one to be updated alone.
- **`encodeDebugDumpNamed`** — Added when `muxd dump` and `muxa capture` were
found to hold byte-identical hand-assembled copies of the payload; wire
layout belongs to the wire module. An empty name writes exactly the one-byte
payload from before named sessions.
- **`SnapshotPrefix`** — `epoch` identifies the daemon instance that produced
`seq`; a client echoes it back on reattach so the daemon can tell "you are
current" from "you are current in a session that no longer exists". (The
borrowing rule now lives at `AttachReq`.)
#### engine.zig
- **`MuxHandler.onClipboard`** — Answering the OSC 52 `?` QUERY form would let
any program in any session — including one on a box reached over QUIC,
including one an agent is driving — read whatever the human last copied.
xterm ships the query disabled; Alacritty defaults to OnlyCopy.
- **`Engine.writeSelection`** — A null selection must never fall back to
formatting the whole PageList, which would drag scrollback into the dump.
- **`Engine.snapWide`** — The bug that found this: dragging across 漢字. Each
of `dumpVtRowSpan`'s three pieces resolves its own edge independently, so a
character straddling a cut was emitted by BOTH pieces, the row landed one
column wider than the grid, and every glyph right of the pointer shifted as
the highlight moved. Mechanism in ghostty's formatter: it reaches back a
column when a selection STARTS on a spacer tail, and a selection ENDING on a
wide cell still emits both of that cell's columns.
- **`Engine.cursorKeys`** — `less` and every curses program set DECCKM, which
is why alternate scroll cannot get away with the normal arrow spelling.
- **`Engine.title`** — Nothing carried the title before it was added, which is
why a host terminal's title used to stay wrong under mux. Empty and "never
set" are the same answer; `sampleTermTitle` in server.zig has why mux
declines to forward either.
- **`Engine.reset`** — Discarding queued side_events is data loss, not terminal
state: an undrained OSC 52 copy is thrown away rather than replayed after the
reconstructed state lands. Correct for today's callers (replica.zig,
wasm_core.zig), which reset without ever draining; a session-restart or
resync path that drains must drain first.
- **`Engine.onDeviceAttributes`** — The defaults are VT220 conformance + ANSI
colour (`CSI ? 62;22 c`), the same modest identity xterm ships; nothing there
claims sixel or windowing the replica cannot honor. ghostty-vt's stock
handler answers DA1 only when the embedder supplies one, and the unanswered
query cost a flat second off every nvim start and quit before this existed.
### QUIC-layer comment burn-down — facts moved out of the source
Extracted from `src/quic_server.zig`, `src/quic_client.zig`, `src/quic.zig`,
`src/proxy.zig`. These belong in `docs/decisions.md`, not beside the decl.
#### `proxy.ignoreSigpipe`
- Defence in depth, not a fix. Zig's `std/start.zig` already installs a noop
SIGPIPE handler, so `proxy.pump`'s write-error returns are reachable without
this call. What the explicit `SIG_IGN` pins is that the reachability belongs
to mux's own code rather than to a std default (`std.options.keep_sigpipe`)
that another module could flip out from under it.
- It is exported so that every process which writes to a pipe it does not own
installs the *identical* ignore rather than its own copy. No protocol
knowledge crosses that boundary, which is the only thing `proxy.zig`'s import
list forbids.
- Consequences of the survives-exec rule, previously enumerated at the decl:
`wallview.zig` installs its ignore only after opening the transport;
`muxd endpoint` calls it after its auto-start; order is irrelevant in the
proxy itself because the proxy spawns nothing.
#### `proxy.shrinkBufs`
- The tests depend only on the buffers being *small*, not on any particular
size — which is why a floor request that Linux doubles and clamps is good
enough.
#### `quic.keyRefusalBody`
- The one-owner refactor: four literal copies of the refusal text lived in four
binaries, held in sync only by prose comments, and their catch-all arms had
already drifted apart before the function was extracted.
- Classification is by error *value*, and identical at every caller, because a
key is rejected for the same reasons whichever binary read it.
- Truncating rather than failing follows `failedMsg`'s policy in `client.zig`,
for its reason: the line is the user's only account of the refusal, so a
clipped one beats none.
#### `quic.keepAliveNs`
- A third of the idle timeout, so two keepalives can go unanswered before the
connection is called dead. (Asserted by the test
"keepAlive: a third of the idle timeout, and never disabled".)
- ngtcp2 reads UINT64_MAX as "disabled" too, so a small `idle_ms` rounding down
to zero would silently restore exactly the behaviour the keepalive exists to
prevent.
#### `quic.WriteAction` / `quic.accountWrite`
- The ordering is why this is a function rather than three copies of two ifs.
ngtcp2 can commit `ndatalen` — advancing the stream offset it will retransmit
from — and *still* return an error afterwards (NOMEM out of rtb_add, say).
#### `quic_server.Listener.send` — the re-entrancy abort
The full reproduction, moved out wholesale:
- QUEUE ONLY is the fix for a real defect, not a stylistic preference.
- ngtcp2 is not re-entrant, and draining inside `send` re-entered it. The chain
was synchronous and entirely ordinary: `read_pkt` -> `recv_stream_data`
callback -> the daemon's frame handling -> a reply queued -> `send` ->
`drain` -> `writev_stream` on the SAME connection while `read_pkt` was still
on the stack below.
- Two failures follow. (1) Monotonic timestamps move backwards within one
`read_pkt`, quietly corrupting loss detection. (2) When a datagram carries a
STREAM frame ahead of an ACK, the nested write mutates the retransmission
buffer that the outer ack walk is about to traverse.
- `ngtcp2_unreachable()` aborts unconditionally even under NDEBUG, so the
symptom is a bare SIGABRT with no panic banner and no defers run. It depends
on traffic shape, which is why it presented as a test failing once in
hundreds.
- Draining now happens only where the stack is ours: after `read_pkt` returns,
in `tick`, and in the daemon's explicit `drainAll`.
- Earlier defect on the same decl: while `send` accepted everything
unconditionally, a peer that stopped reading grew an unbounded buffer inside
the listener, where nothing watches it, instead of tripping the daemon's
`pending_cap` where something does. The short-return contract now matches the
socket sink's, so one rule bounds both kinds of client.
#### `quic_server.Listener.closeConn`
- Not sending CONNECTION_CLOSE is a real, accepted cost: a client learns of a
deliberate close no faster than it learns of a crash, because it waits for
its own idle timer. A graceful close belongs with the client transport, which
is the side that would act on it.
- Not invoking `onClose` is deliberate: a close the owner asked for needs no
callback telling the owner what it just did. Only `kill` — the listener
deciding a connection is finished — calls back.
#### `quic_server.Listener.bind`
- A listener returned from `bind` drops everything that arrives until
`setHandler` is called; nothing polls it before then.
#### `quic_client.Client.sendRecvFailed`
- ECONNREFUSED on a *connected* UDP socket is an ICMP unreachable — a dead
transport, not a blip. `recv` sees it after a failed flight; `send` sees it
when the queued error is delivered on the NEXT syscall, which on a quiet
connection is `drain`'s send. Both must agree, because the error goes to
whichever syscall runs first after it is queued and is *cleared* by it: if
that path does not act, nothing else ever sees it.
- The parameter is the union of the two call sites' error sets rather than
`anyerror`, so a misspelled prong is a compile error instead of an arm that
silently never matches.
#### `quic_client.Client.send`
- The short-return contract is the same one the daemon's socket sink obeys, for
the same reason: the ring is bounded, so somebody has to hold the backlog and
it should be somebody who can see how big it is.
#### `quic_server.zig` / `quic_client.zig` module headers
- The transport thesis, restated in three headers and now pointed at instead:
if an opaque byte pipe suffices to carry the protocol, transport is a swap,
not a redesign. Recorded as an invariant in `CLAUDE.md`.
- `quic_client.zig` imports the key, the wire constants, the `Egress` ring, the
write accounting and the clock from `quic.zig` so there is only one copy of
each to drift.
### Extracted from the web layer (webhub, wasm_core, replica, webhub_main)
Facts moved out of source comments during the tier-3 burn-down. Each bullet
names the symbol it came from.
#### webhub.zig
- **`originAllowed`** — the Origin check is a spec requirement, not a
nicety: exactly our own two spellings (`http://127.0.0.1:PORT`,
`http://localhost:PORT`) pass, and a request with no Origin header is
refused.
- **`Hub.checkoutTarget`** — a browser can always dial a tile another
device just removed, so `UnknownId` is a normal outcome, not a bug. It
stays distinct from `OutOfMemory` so a memory failure is never folded
into "no such tile" (404 vs 500).
- **`Hub.checkoutTarget`** — fd tracking is deliberately one fd per tile,
best-effort. Two browsers on one wall run two pumps per tile; FIRST
registration wins, the second pump serves its browser untracked and ends
on its own WS read. A full fd list per tile is deferred until
two-browser removal latency is shown to matter.
- **`drainBrowser`** — both loops share the drain because the five
decisions are the same five in the same order (fill, incomplete,
too_big, pong, ready); the only difference is whether a data frame has a
transport to go to. The cost of draining unconditionally is one
`headFrame` call over an empty buffer per idle pass.
- **`pumpTile`** — a slow browser stalls only its own tile. The daemon side
is protected by its own 8 MiB pending cap; the hub's upstream reads just
stall. Blocking per-tile reads (fatal to a multiplexing hub) are correct
here only because the Transport is private to the thread.
- **`pumpTile`** — the reconnection sequence: on transport death narrate
`reconnecting`, re-dial on the CLI's own backoff schedule
(`client.nextBackoffMs`, no retry cap, deliberately), then narrate `up`.
The browser's replica quotes have_seq/have_epoch in a fresh attach and
the snapshot-vs-delta resolution does the rest.
- **`pumpTile`** — the dead-leg bound is not a bound: an ssh-fallback tile
whose peer goes quiet tears on the transport's own terms, not on the
nominal 90 seconds (3 × 30 s ping interval).
- **`dialLoop`** — the backoff doubles as the WS liveness window; it caps
at 2 s against a 30 s ping interval, so the liveness tick is never more
than one backoff late.
- **`appendJsonString`** — deliberately NOT `muxa`'s `jsonEscape`, though
the two look alike. This one sends every control byte to `\u00XX` (one
rule, no table to get wrong); `muxa` spells the short forms `\n`, `\r`,
`\t`. Both are valid JSON and parse identically, but the bytes differ
and each is pinned by its own test, so sharing one implementation would
rewrite one side's output for no gain.
- **`Hub.json`** — the `/tiles` response shape: one object per tile in
index order, `{"label":…,"session":…}`.
#### wasm_core.zig
- **`panic`** — the intended later use of this hook is to surface panic
text to JS through a host-imported log before the trap.
- **`std_options`** — the no-op `logFn` is load-bearing because ghostty-vt
logs warnings on some unsupported sequences; the failure was found as
obstacle 4 of the browser-client feasibility spike.
- **`mux_viewport_ptr`** — the flat viewport ABI (the per-cell layout is
now stated beside the packing code in `paintRow`): `[0]` codepoint
(0 = empty), `[1]` fg `kind << 24 | value` (kind 0 none, 1 palette,
2 rgb; value = palette index or 0xRRGGBB), `[2]` bg same packing,
`[3]` flags.
- **`mux_paste_begin`** — history: the browser shell used to wrap EACH
chunk of a large paste, which put a paste-END in the middle of the
pasted text. vim left paste mode 32 KiB in and re-indented the rest.
The fix made the code match `keymap.pasteInto`'s existing contract,
"the wrap and nothing else" around the WHOLE paste.
- **`mux_text_encode`** — an IME's `compositionend` hands over finished
text, and finished text is TYPING. Bracketing it would tell the
application a human did not write it: vim would skip paste mode's
indentation and a shell's bracketed-paste guard would refuse to run it.
#### replica.zig
- **`Replica.scrollStart`** — the returned row saturates at the oldest
retained row; the view begins `rows_up` above the live viewport top,
which is row index `history_rows`.
#### webhub_main.zig
- **`addSpelling`** — the usage message names the tile because with
several targets on the command line, a bare `usage` would not say which
spelling was rejected.
### Render path — facts moved out of source comments
Extracted while burning down tier-3 flags in `src/predict.zig`, `src/paint.zig`,
`src/select.zig`, `src/delta.zig`, `src/keymap.zig`. Each bullet names the symbol
the fact belonged to.
#### predict.zig (module header)
- **Past bug — "not yet" read as "wrong".** An earlier version of `reconcile`
treated an unchanged predicted cell as a contradiction. Consequence: a typing
burst refuted itself once per round trip. Typing `hello`, `h` confirms while
`e,l,l,o` are judged by a frame that was built before they were typed; all four
read as contradictions and the queue flushes. Prediction that erases itself
every RTT is the opposite of the feature. (The surviving comment states the
evidence rule; the worked example is the history.)
- **Comparability claim.** The overlay's separation from the replica is what
keeps `muxd dump` and the client grid comparable byte for byte at every moment.
- **Consequence of the TERMIOS tiers (`.adaptive` vs `.always`).** The mode bits
move once or twice per command at a normal prompt as readline hands the
terminal back and forth to run each command. Since every move re-earns display
from scratch, the first couple of keystrokes after each prompt are invisible
predictions. This is a deliberate conservative trade; per-context confidence
memory is the banked polish that would remove it.
- **Memory shape.** The queue is read back by index rather than handed out as a
slice; a slice would go stale on the next append — the shape decisions.md's
egress records call the UAF-that-never-crashes. (Kept in source in one line;
the cross-reference to the egress records is the part dropped.)
- **Bounded pending** (`expire_after_frames` / `expire_after_ms`) was stated in
the header as well as at both constants' own decls. The header copy is gone;
the decls carry the phantom-glyph rationale in full.
#### predict.zig (decls)
- `noteSeq` — `reconcile` calls it itself; the client calls it directly only on
paths that apply a frame without judging anything.
- `recordSuppressed` — the counter counts DECISIONS, not which side of the
interface made them. Before this entry point existed the counter silently
disagreed with the behaviour: nothing was predicted and nothing said so. The
forcing case is a plain-ASCII paste, whose printable lead byte would otherwise
have been predicted as the paste's first character.
- `markPainted` — `displayed` is defined as "predictions that ever reached the
screen".
- `flush` — counting a snapshot/resize/scroll/reconnect as a contradiction would
fire the demotion machinery on a window resize, costing the next
`promote_after` keystrokes their visibility.
#### paint.zig
- `Highlight` — `cols` is handed DOWN rather than remembered by the caller: the
width the highlight is clamped to must be the width this paint is about to
use. Enforced by an assert in `Engine.dumpVtRowSpan`, so the comment was
re-arguing something the code already checks.
- `renderStripe` — the bottom-crop first version of the stripe window showed 14
empty rows of every fresh session. That measurement is why the window is
cursor-anchored.
- `renderStripe` — full-width rows only because an interior column would need
VT-safe truncation this module does not do. (Kept as the "so the wall stacks
stripes" clause; the truncation detail is here.)
- `renderStripe` doc block had drifted onto `stripeWinStart` (no blank line
between the block and an inserted decl), so the stripe prose documented the
wrong function. Moved back onto `renderStripe`; the per-statement parts
(cursor anchoring, no `\x1b[2J`, the unpaired sync close) are now inline
beside the statements they explain.
- `stripeWinStart` — a second copy of this arithmetic that drifted would land
clicks on a different line than the one the user pointed at, and nothing on
screen would say so.
#### select.zig
- `Range.span` — an out-of-range column is what makes the daemon answer
`.invalid` at the other end. (Retained in source.)
- `Drag.motion` — a wall is panes; a selection dragged out and back is a
selection of one cell, not a click.
- `Drag.clear` — the callers, and why each invalidates coordinates: a relayout
or a forget re-cuts the stripes under the anchor; a zoom transition hands the
screen to somebody else; a resync renames the absolute row space outright. A
highlight kept across one of those is a highlight over rows nobody selected.
(The 46-byte budget could not hold this list.)
- `Drag.on` — a caller dropping a tile's coordinates has to drop a drag anchored
on that tile too, even one that has not moved yet.
- module header — a column layout, added later, would change the driver's
hit-test and nothing in `select.zig`.
#### delta.zig
- `noteBlind` — **measurement:** rendering every row in order to hash it is
~60% of the daemon's cycles on a full-width repaint. With no client, none of
those bytes has anywhere to go. This number is the whole justification for the
no-render path.
- `noteBlind` — the reason the signature takes no allocator is defensive: a
signature that can allocate is a signature that can render, and the next
reader will put the loop back.
- `noteBlind` — `seq` is the session's watermark, stamped into
`last_return.seq`; two commands returning during one blind stretch sharing a
seq means an await cannot tell which of them it was told about.
- `noteBlind` — geometry and screen changes still have to become a
discontinuity: `noteBlind` cannot describe them, and `resyncSnapshot`'s
rebuild is owed to a detached session too. (The three `return .discontinuity`
guards at the top of the body are the code that does this.)
- `canServe` — the tracker does not know which daemon instance it belongs to,
which is why the epoch check is the caller's. A tracker never built, or whose
rebuild failed partway, can describe nothing.
#### keymap.zig
- `pasteInto` — filtering a pasted ESC was explicitly deferred out of v1; the
brackets wrap the paste verbatim.
### Fragments moved out of the small modules
History, incidents and measurements lifted from doc blocks during the tier-3
burn-down. Each bullet names the symbol it belonged to.
#### `sockpath.defaultSockPath` — why there is no `/tmp` fallback
A guessed `/tmp/muxd-<uid>.sock` used to stand in when `$XDG_RUNTIME_DIR` was
unset. It broke the property the default exists for — two binaries started with
no `--sock` landing on the SAME daemon. A tmux server started before logind
exported the variable hands every pane an environment without it, so panes went
to `/tmp` while the daemon a pane had started owned the runtime directory: one
uid, one box, two daemons, and `muxd stop` reporting nothing there. A default
that cannot make two binaries agree is not a default, so the guess was removed
and the caller now names the path.
#### `sockpath.PathId` — the guard that was unconditionally false
`PathId` records the PATH's dev+ino, not the listening descriptor's. A bound
unix socket's descriptor lives in sockfs (dev 10 on this box) while the path
resolves to an ordinary filesystem inode (dev 38), so comparing the two could
never be equal. The teardown guard read as careful and was always false, which
meant the daemon never unlinked its socket on a clean exit at all — masked ever
since by the stale-socket recovery in `claim` cleaning up on the next start.
#### `mux_main.wallEdit` — why validation is a separate pass
Per-spelling saves made the "validated before anything is written" promise a
half-truth: a bad line was caught before anything moved, but an IO error on the
third of four left the first two applied and the rest not — exactly the partial
state the validation pass exists to prevent. The edit is now built in memory and
saved in one atomic rename.
#### `wall.loadLines` — the incident that created it
When EVERY path went through `wall.load`, one hand-edited line made the wall
file unrepairable by the tool that owns it, including the command whose entire
job is removing a line. `loadLines` was added so removal can delete the broken
line and write every line it did not touch back byte for byte.
#### `shellint.install` — the two collisions the random suffix closes
- Symlink aim: `parent_dir` is the socket's directory, a shared `/tmp` when
`$XDG_RUNTIME_DIR` is unset, and a pid is guessable. Another user could
pre-create the exact `mux-shellint-<pid>` name as a symlink to a directory of
ours, and the shim files the session shell then SOURCES would land through it.
- Mundane: a predecessor SIGKILLed before teardown left `mux-shellint-<its pid>`
behind, and a later daemon drawing that pid used to lose its marks to the
leftover.
#### `shellint.zsh_zshrc` — the known cost of the ZDOTDIR shim
(Left in source; recorded here because it is a roadmap item, not a rule.)
Pointing ZDOTDIR at the shim silently costs the user their `~/.zshenv`: zsh looks
for `.zshenv` under `$ZDOTDIR` and the shim directory has none, so a config kept
there stops being read for the session. A `.zshenv` shim that restores ZDOTDIR
the way ghostty's does is the fix.
#### `client_core.validTarget` — the case that cannot reach it
Multi-character OSC 52 targets cannot reach `validTarget` at all: ghostty rejects
the whole OSC when `data[1] != ';'`, so `ESC]52;pc;...` yields no event to
forward. There is nothing to widen the list for. (Single stray bytes DO reach it
— ghostty sets `kind = data[0]` with no validation of its own, in
`osc/parsers/clipboard_operation.zig`, so `ESC]52;X;...` arrives as target 0x58.)
#### `mux_main.insideThisSession` — why there is no escape chord
An inner client repaints its own grid (paint -> delta -> repaint) and takes the
alt screen and every keystroke with it. Once Ctrl-\ became a prefix, the OUTER
keyboard could no longer steer it back out, so there is no chord to offer as an
alternative to refusing the attach.
#### `spawn.ensureForAttach` — how it differs from `muxd start`
`muxd start` deliberately does not go through `ensureForAttach`: it forwards the
user's own flags rather than a fixed `run --sock <path>`, it truncates the log
rather than appending, and it REPORTS already-running where `ensureForAttach`
stays silent. Someone who typed `muxd start` asked about a daemon and is owed a
verdict on one; someone who typed `mux` asked for a session and is about to get
it.
## 2026-08-25 — max_sessions 4 → 32
The 4 was "purpose bounds the surface" from the multi-session design: a number
picked to keep the new thing small, never a measured limit. Multipane then grew
the wall to `wallview.max_tiles = 32` and the daemon cap did not move with it,
so a wall could name more sessions than the daemon would ever hold. It
surfaced as a silently refused `Ctrl-\ c` — the client does say "cannot create a
new session", but the relayout repaint that follows eats the notice, which is a
separate follow-up.
What the 32 buys is nameable sessions, not visible ones. Every tile is its own
attach and therefore its own client slot, and `max_clients` is still 8, so a
32-tile wall against one daemon still refuses tiles nine and up. Cycling 32
sessions through eight tiles is the shape this enables.
Nothing about the 4 was performance. A session costs one engine, one pty and one
shell, slots are filled lazily, and an empty slot polls fd -1. The two places
the count does scale are worst cases that need a misbehaving peer to bite: the
per-death drain is `max_sessions × 250ms` only when the whole table dies into
stalled clients in one pass, and `pty.term_grace_ms` is paid only by a child
that survives the master's close and then ignores SIGTERM.
`stats_text_len` is derived from `max_sessions` rather than a literal, so no
future bump to the table can truncate a stats reply. (Corrected 2026-08-26: the
per-line widths inside that derivation were still hand-counted and both
undercounted — 256 for a main line 375 wide, 44 for a session segment 63 wide.
The bound is now read off the format strings themselves.)
Filling a 32-slot table from a shell script found one thing worth writing down:
`acceptConn` parks every new connection in an OBSERVER slot and promotes it to a
client only when its attach frame arrives, so simultaneous dials contend for
`max_observers` (4), not `max_clients` (8) — 7 at once lost 11 of 31 fills to a
closed connection, 3 at once lost none. `muxd stop` on a full table of `/bin/sh`
took 51ms, which is the grace-cost claim above, measured.
## 2026-08-25 — a saved local line attaches-or-creates
`mux wall` with no argv reads `$XDG_STATE_HOME/mux/wall` and puts up a tile per
line. Until now every one of those tiles attached at 0×0, which the daemon reads
as JOIN — so the wall could only ever show sessions that already existed. The
daemon that held them dies on every reboot, and the wall that came back was a
grid of `[refused]` stripes the user had to forget by hand, one `Ctrl-\ x` at a
time. The marker fix (bf18c8d) is what made that visible rather than merely
true: the focus marker now moves across tiles with no pump, so a wall of dead
tiles is something you sit in and step around instead of something you leave.
Ruled: a wall-file line whose target is the LOCAL daemon (`--sock PATH[#NAME]`)
attaches-or-creates; a remote line (`HOST`, `quic://…`) still joins only.
Recreating a shell on your own box is cheap and expected — it is the workspace
the file is a record of. Spawning one on another host out of a file the user
last edited by attaching once is not: the saved line is history, not standing
permission to run something over there.
One consequence worth stating, because nothing else says it: a wall tile
redials for as long as the wall is up, and a redial carries the tile's
`creates`. So a restored local tile whose daemon is restarted UNDER a running
wall comes back as a fresh session on its next redial rather than landing
`[refused]` — the same ruling, one step further along. It does not auto-start a
daemon: a pump only dials, and auto-start lives in `mux TARGET`'s path alone, so
a saved line whose socket is gone still sits reconnecting exactly as before.
Two exceptions stay as they were. `mux wall SPELLING...` is a view rather than
an attach, so an argv wall joins only whatever it names — the same reason argv
walls record nothing and save no layout sidecar. And the `Ctrl-\ w` fold reads
the same file through the same rule, because a fold IS the saved wall arriving
late.
The wire needed nothing. Create-vs-join was already encoded in the attach's size
claim (`server.resolveSession`), so the change is one predicate on the client
side — `wallview.hydratedCreates`, used by the fold row of the birth table and
by the startup tile loop — deciding whether `sendAttach` puts the tile's rect or
0×0 on the frame. A tile that flips to creating and finds the session ALREADY
there is unaffected either way: it joins, and its rect arrives in the attach
instead of in the resize doorbell one frame later. Nothing about a cross-version
attach changes: an old daemon reads the same size claim.
## 2026-08-25 — the bar shows the chord digit
`Ctrl-\ 1`-`9` has always focused tile N by POSITION: the handler indexes
`tiles[n - 1]` and checks `present[n - 1]`. The label bar, meanwhile, named the
SESSION — `--sock /run/user/1000/muxd.sock#2`. Two series, and nothing on screen
said which was which. They line up only on a wall nobody has edited: forget a
tile with `Ctrl-\ x`, or attach sessions out of numeric order, and the tile
labelled `#2` is no longer the one `Ctrl-\ 2` reaches. The user counted stripes
to type a chord.
Ruled: the bar opens with the tile's chord digit, `"{d}> "` when focused and
`"{d} "` when not. The digit is `idx + 1` — the tile's slot in `tiles[]`, which
is the order the tiles joined the wall, not the session's name and not where the
tile sits on screen. `birthTile` appends at `live.*` while placing the new leaf
beside the FOCUS, and hydration remaps saved leaves onto wall indices, so a wall
that has been split or restored can read `1 4 2 3` left to right. That is the
right trade: the digit must follow the chord, because that is what the chord
indexes, and the multipane chunk-2 ruling already fixed `Ctrl-\ 1`-`9` as
"focus N in the current layout". Naming the session on the bar and indexing the
slot with the chord is the divergence, not the cure.
The digit is stable for a tile's whole life because `forgetTile` never compacts:
it clears `present[sel]` and sets `gone`, leaving a hole. Pumps hold pointers
into the tile array, so compaction was never on the table — which is what makes
the digit safe to print. A forgotten tile's number vanishes with its bar and its
neighbours keep theirs.
Tiles 10 to `max_tiles` (32) print their number too, two digits wide, though no
chord reaches past 9. The number still answers "which chord reaches me" — with
"none, this far out" — and the chord's reach is the chord's limit rather than
the bar's; suppressing it would leave a bar saying nothing about its tile in
order to protect a fact about the keyboard.
Cost is one column per bar. `labelText` already sizes the label cut by
`marker.len`, so the digit is subtracted from the label and from nothing else —
the state word still survives truncation at every width, which is the property
that test pins. On a bar too narrow for `marker + " [state]"` the status is what
the final `text[0..cols]` cut loses, exactly as before the digit; a two-digit
tile reaches that width one column sooner.
## 2026-08-26 — the browser wall restores a saved local line too
The 2026-08-25 ruling above ("a saved local line attaches-or-creates") was a CLI
ruling only. `muxweb` reads the same file and put up the same tiles, and every
one of them attached at 0×0 — the passivity contract, so that a browser can
never move a grid a human is looking at. The daemon reads 0×0 as JOIN, so after
a restart the browser wall was exactly the thing the CLI had just stopped being:
a grid of dead local tiles.
Ruled: the same predicate, both doors. It moved out of `wallview` and down into
`client.hydratedCreates` — what the entry above calls `wallview.hydratedCreates`
lives there now. Not into `wall.zig` with the spelling grammar, though the
spelling is where the distinction is visible: `wall.Spec` is resolved away in
both doors (`wallview.resolveSpelling`, `webhub.resolveTile`) long before either
decides, and a predicate neither caller could reach without carrying a derived
flag alongside the target is two owners wearing one name. `client.Target` is
what both hold at the moment of the decision, so that is where the rule lives.
The hub does NOT get there the way the CLI does. A wall stripe can simply put
its rect on the attach, because a stripe owns a rect; a browser tile owns
nothing and must not start. So the hub reads the refusal instead — `exit_status`
as the FIRST daemon frame after the browser's attach, which is mux.js's own
discriminator mirrored one layer down — and calls `client.birthSession` on a
connection of its own: one sized attach at the daemon's own default 80×24, wait
for the snapshot, detach. Then it tears the tile's connection and re-dials
exactly as a network tear does. The browser sees `reconnecting` → `up`, which it
has always handled, re-attaches at 0×0 on its own, and is never told a refusal
happened. Nothing in the page decides anything about creation, which is the
point: the page is a stand-in for native UIs to come, and a rule living in it
would have to be written again in each of them.
The side connection is not incidental. A tile that claimed a size to get its
session made would go on claiming it, and the shared grid would then follow
whichever browser reloaded last — the passivity contract broken to fix a
symptom of it. Paying one attach and leaving hands the session back at a size
nobody is bound to.
The bound is NOT per connection, and the first cut of this got that wrong. Every
refusal closes the hub's connection — `serviceObserver` answers an unseated
attach with `exit_status` and then `dropObserver`, which closes the fd — so a
flag cleared on the tear bounds nothing: refuse, redial, re-attach, refuse
again, and the hub would birth once per turn of a spin it cannot end. The flags
that matter therefore outlive the dial. `birth_tried` is cleared by a GRID
arriving, so one birth per healthy period: a daemon that refuses the birth too
(a full table, a name it will not make) is answered once and then believed,
while a second daemon restart still heals because the tile was healthy in
between.
The other half of that is `ended`, and it is the same bug wearing the user's
clothes. A shell the user ends with `exit` reaches the hub as exactly the
restore case: the daemon reaps the session and drops its clients, the hub
redials, mux.js re-attaches on `up`, and the attach is refused because the
session is gone. Nothing in those frames distinguishes "this session never came
back from a reboot" from "I just closed this". So the pump remembers that it
watched this session die and stops creating — `wallview`'s ruling, which ENDS
its pump on an exit_status and leaves the tile dead, reached by a pump that
cannot end because it still serves a browser. Without it, `exit` hands the user
a new shell, and a short-lived one forks a process per turn.
The name the hub creates is the TILE's, off the wall spelling, not one read out
of the browser's attach frame — a page cannot talk the hub into making a session
it was not already standing in front of.
A remote tile is untouched throughout: no birth, refusal forwarded, and it reads
`refused` for the same reason the CLI stripe reads `[refused]`.
That word is new. The page said `session full`, which was a guess at WHY, and
the daemon never says why: a missing session and a full table refuse with the
same `exit_status 1`. Most of the time the guess was wrong. It says `refused`
now — the CLI's word, for the CLI's reason.
## 2026-08-26 — a zoomed browser tile was resurrecting the shell you exited
Found in Chrome against a live rig, not by any test here. Type `exit` in a
ZOOMED browser tile and the session comes back: a new shell, in the tile you
just closed.
The chain has nothing to do with the hub's restore rule, which is what made it
hard to see. The shell exits behind a live grid, so `exit_status` reads as a
shell exiting and the badge goes `exited` — TERMINAL, and `setStatus` will not
leave it. The daemon reaps the session and closes the hub's connection; the hub
redials and sends `up`; mux.js's `up` arm re-attaches, because the browser owns
its own re-attach and that call was gated only on `dead`/`replayDead`. A zoomed
tile's attach carries `zoomCols()×zoomRows()` rather than 0×0 — the passivity
contract exempts the zoom, deliberately, because a zoomed tile IS the client
looking at the session. And `server.resolveSession` is attach-or-create for any
attach with a usable size. So the page created it. The hub's `ended` latch never
came into it: nothing was refused, and no birth was asked for.
Ruled: the page declines to re-attach a tile whose status is `exited`. That is
`wallview`'s ending — its pump ends on an `exit_status` and leaves the tile dead
— reached by a client that cannot end, because it still has a socket and a
canvas. `exited` exactly, not the TERMINAL set: `refused` must keep re-attaching
or the hub's restore heal has no second half (refused → birth → re-dial → `up` →
attach), and `stuck` already refuses inside `sendAttach` via `replayDead`. The
latch lives in `sendAttach` rather than at the `up` arm that found the bug,
because there are three callers and the dangerous one is the least obvious:
`replayFailed`'s retry is a bare `setTimeout` that stores no handle and nothing
cancels, so a replay failure followed within its wait by the shell exiting lands
a sized attach on a tile already badged `exited` — the same resurrection by
another road. It sits beside the `dead` gate rather than beside `replayDead`, so
the selection and scroll clears below it are skipped: a dead tile's last screen
is still worth copying text off, and a hub redialing every few seconds would
otherwise wipe the selection each time. The latch clears through `ws.onopen`'s
`revive`, so a reload or a hub restart is a new epoch and may create again — a person who reloads the page is asking for the
wall they saved, which is the restore rule doing its job.
This predates the browser restore work entirely: any zoomed tile whose shell
exited has resurrected it since sized attaches existed. It survived because
nothing in `test/` can see it. `wsclient` stands in for the browser on the wire
but attaches `0 0` and never executes `mux.js`, so every rule that lives in the
page — this one, the `up` re-attach, the badge vocabulary — is invisible to the
e2e suite by construction. Faking a leg with `wsclient` would assert the
fixture's behavior, not the page's.
The gate for those rules is `web/verify.js`, which runs `mux.js` under
`zig build test` (gated on `node`): it constructs a real `Tile`, feeds it wire
bytes, and calls `sendAttach`. State rules belong there — `verifyStatusShell`
pins this latch, the refusal that must keep re-attaching, `setStatus`'s
precedence and both badge words, and each check was watched failing against the
mutation that removes its rule. A real-browser pass stays the gate for what only
a browser has: rendering, input, and the platform APIs the shell talks to.
## 2026-08-26 — `muxd upgrade`: the pid stays, only the binary changes
Rolling a new muxd was `stop` + `run`, which kills every session's shell. The
2026-08-19 design handed the pty masters and listeners to a NEW process over
`SCM_RIGHTS`. An inventory of the daemon as it stands found that every hard
problem in that design is caused by the PROCESS changing, not by the binary
changing:
- A shell whose pty crossed is still the old daemon's child. The new daemon's
`checkExited` calls `waitpid`, which answers `ECHILD`, which std marks
`unreachable` — a panic on the first pump. (Not a guess: mutating the
adoption's `Pty.adopt(rec.pty_fd, rec.child_pid)` to pass `0` reproduces it
exactly, `posix.zig:4427 .CHILD => unreachable`.) Rebuilding exit detection
on master-EOF loses the shell's real exit code forever after.
- The shell-integration shim dir (`ZDOTDIR`, `--init-file`) and the agent dir
(`SSH_AUTH_SOCK`) are named after the daemon's PID and are held open by every
live shell. `Server.deinit` `deleteTree`s both, unlinks the socket path and
every agent socket, and signals every shell.
- Zig 0.15.2 has no `recvmsg` wrapper, no `cmsghdr`, no `SCM_RIGHTS`, no
`CMSG_*`. All of it would be hand-rolled for this one feature.
`execve` keeps the pid, the children, the cwd, the environment and every fd not
marked close-on-exec. It changes only the binary. So the daemon writes what only
it knows into an anonymous memfd, clears `FD_CLOEXEC` on exactly the fds it
keeps — unix listener, QUIC UDP socket, each pty master, each agent listener,
the memfd — and execs the candidate as `muxd run --resume-fd N`. Nothing in
`Server.deinit` runs on that path: no unlink, no `deleteTree`, no signal. The
process that owns those names never exits.
**What deliberately does not cross.** Scrollback: `Engine.dumpState` is
viewport-only by construction, so a client observes `history_rows` going to 0
and its absolute row space renumbering. Clients: their fds are close-on-exec
and the exec is their EOF; they redial as they do for any tear. Delta trackers
and the `*_sent` latches: rebuilt fresh over the replayed engine, safe only
because the epoch is re-minted, and honest because no client survived to have
been told anything. Agent channels and pending awaits: bound to clients that
are gone. Per-connection QUIC/ngtcp2/TLS state: C state, dropped — which is why
the exec sends CONNECTION_CLOSE first (below).
**The watermark bug, and what it teaches.** The manifest carries `last_return`,
whose `seq` is the RETURN watermark `muxa await --since` compares against. The
adopted session mints a fresh delta tracker at seq 0, so a carried watermark is
a watermark from the future that no post-upgrade return can exceed. Measured on
a live daemon: the FIRST `muxa run` after every upgrade timed out at 30s while
the shell had already answered, and only the second worked. The verdict crosses;
its watermark is re-stamped into the new seq space. The lesson is the one
`awaits-answer-from-snapshots` already paid for, in a new place: a number is
only meaningful inside the space that minted it, and carrying a verdict is not
the same as carrying the coordinate it was recorded at. It is also why the e2e
leg asserts the FIRST await after the exec rather than "an await" — a leg that
ran two would have gone green on the bug.
**The refusals are safety, not authority.** `upgrade_req` execs a path of the
requester's choosing as the daemon's user, which is the same power `stop_req`
already grants anyone who can open the socket, and the socket is the user's own.
The version rule, the absolute-and-executable path check, the candidate's own
`--version`, and the candidate's `run --resume-fd N --check` exist against
mistakes — wrong path, wrong arch, half-copied binary — not against an
adversary who is already inside.
**Rollback is another exec.** If adoption fails before the pump starts, the new
image execs the OLD binary, whose path the manifest carries, with the same
`--resume-fd`. The loop guard is `MUX_UPGRADE_ROLLBACK=1` in the env rather than
a flag: two binaries that both refuse one manifest would exec each other
forever, sessions alive and unreachable, and an unknown VARIABLE is ignored by
any muxd where an unknown FLAG is fatal usage to the older one being rolled back
to. The same reasoning picks `MUX_RESUME_FAIL_AT` as the e2e's handle on the
abort: `execUpgrade` builds a fixed argv, so nothing a leg types can put
`--resume-fail-at` in front of the candidate, but the daemon's environment
crosses the exec untouched.
A rollback is invisible to the operator: `muxd upgrade` prints its verdict the
moment the daemon accepts, and if adoption then fails, the image that would
correct it is gone. The e2e leg reads the daemon's own log for the abort line,
because without it every other assertion in that leg passes just as well on an
upgrade that simply worked. A version field in `stats_reply` for `confirmServing`
to compare is the cheap fix if this ever matters to a human; it is not in v1.
**The QUIC goodbye is worth a number.** Unix clients need nothing. A QUIC peer
cannot tell an exec from a bad network, so every connection gets CONNECTION_CLOSE
before the exec (`quic_server.closeAll`, which drains once — `closeConn` is
deliberately quiet). Measured with the leg's own stopwatch: ~60ms back in
service with the goodbye, 19948ms without it against `--quic-idle-ms 15000`.
That gap is the whole reason `closeAll` exists.
**The xversion gap.** A cross-version handover leg needs an old side that
already carries this feature, so the first release with it is the floor. Until
then the same-binary leg (`--allow-same-version`, a flag whose help text says
it exists for the e2e) is the v1 gate, and the cross-version leg is a documented
gap rather than a test that quietly does not run.
## 2026-08-26 — hygiene: server.zig split, coverage that finishes
**server.zig is three files and a test tree.** 12,003 lines → 3,464: the 132
tests moved to nine `src/server_test_*.zig` siblings (bodies byte-identical,
821 unit tests before and after, a build.zig gate on the import list because
a dropped `_ = @import` is a domain of tests that passes by not running); the
agent relay (seven fields touched by thirteen functions and almost nothing
else) and the session table became types in `server_agent.zig` (360) and
`server_sessions.zig` (298), each reaching the daemon through named things
only; `handleFrame`'s 16 arms became methods. The pump stayed: `pumpOnce`,
`serviceObserver`, `drainPending`, `checkAwaits` are the product.
**`make coverage` had never finished the suite.** Six stops, one disease: a
pid the suite holds that under kcov is the tracer's. `endpoint` now runs bare
like `start` (the handoff client kill()s and waits on an ssh child that is
kcov); `softkill` is hardkill's TERM twin (ptrace drops a signal sent to the
tracer); the upgrade candidate is the ELF behind the shim (`MUXD_ELF`);
`real_pid` reads /proc off the tracee; the stop-gone leg names the pid `stop`
waited on. First full map, ReleaseSafe so lines merge: TOTAL 77% of 3,197
instrumented lines, server.zig 80% of 931, quic_server 92%, webhub 88%.
`initFromManifest` reads 0/58 because kcov does not instrument the post-exec
image; its unit tests own it. The map is a worklist, not a score.
**The tracer found a product race.** `muxa send` on a refused attach wrote
input blind after the attach; when the daemon's close beat the write, EPIPE
was reported instead of the refusal. Rare untraced, certain under ptrace
(scenario 75 stopped there every run). Fixed at one owner: a peer-closed
send drains the read side through `awaitFrame`, so the snapshot rule that
tells a refusal from an ending is not copied.
**e2e.sh is a runner, a library and fourteen group files.** 9,678 lines → a
179-line `test/e2e.sh` (arguments, tool checks, group order, `E2E_ONLY`, the
count pin), `test/e2e_lib.sh`, and `test/e2e_NN_<group>.sh` — flat, because
the shell gate globs `test/` non-recursively and a subdirectory would need a
build.zig edit the gate's own comment forbids. The trap reads a register
(`defer_kill`/`defer_sock`/`defer_rm`, `start_daemon` registers what it
makes) instead of four hand-kept lists; captures are NOT registered — every
one is spelled from `$OUT` and the sweep already walks that set by glob, and
registering ~350 of them by hand would have rebuilt the 102-file escape. The
trap kills with `softkill`, not `hardkill`: a SIGKILLed daemon writes no leak
verdict. `E2E_ONLY=<group>` runs one file; two groups (03_side, 11_select)
share a daemon with an earlier group and are refused by name. Running groups
alone found two defects the full suite could not: the leak sweep's
vacuous-green guard was keyed to daemon one's capture, and two socket names
were declared in the wrong group.
## 2026-08-27 — the daemon reads a frame in one read, and seals what an upgrade adopted
**One peer that stopped mid-frame parked the whole daemon.** Found on a live
wall: `muxd stats` came back rc=124, and the daemon's stack said
`unix_stream_read_generic` — one client had written part of a frame and gone
quiet, and `proto.readFrame`'s loop was still waiting for the rest, with every
other session's output, every await and every attach behind it. Killing that
one client freed everything instantly. One byte on a fresh daemon reproduces
it. The daemon side now does exactly one `read()` per readiness into a
per-connection buffer and peels whole frames off it with `protocol.delimitFrame`
through a single `Server.takeFrame`; a short read is just a buffer that is not
a frame yet. Observers gained the same buffer, and promotion moves it with the
fd, because the write that carries an attach usually carries the next frame
behind it.
**The client keeps its blocking read on purpose.** There, one thread owns one
transport and has nothing else to do while a frame is in flight, so blocking
until the frame is whole is the correct shape — the entry on `Conn.awaitFrameFd`
already argues that half, and this changes nothing about it. The rule is about
who is multiplexed, not about which read is safer.
**What is still blocking on the daemon, and why it is filed rather than fixed.**
Observer replies go out through `writeFrame` on the raw fd. Only
`debug_dump --vt` can plausibly exceed the ~200 KiB socket send buffer, and only
for a local peer that asks and then stops reading. Real, small, and not the
thing that was wedging walls.
**An upgrade adopted fds and never put the flag back.** `execUpgrade` clears
`FD_CLOEXEC` on the listener, the QUIC socket, every pty master, every agent
listener and the manifest memfd so they survive the exec. Nothing re-set it, so
every shell spawned after an upgrade inherited the lot: the listener on fd 3,
an agent socket on 5, a pty master, and one `memfd:mux-upgrade` per survived
upgrade — and that memfd carries the QUIC key bytes. `Server.sealAdoptedFds`
now re-seals them, and the memfd is closed rather than sealed, having been read.
**The seal is placed after the last rollback exit, not in `initFromManifest`.**
Rollback re-execs the OLD binary with those same fds, and a sealed fd does not
cross that exec; the first placement broke the rollback e2e leg with a panic.
So the call lives in `resumeRun`, past the point where rollback can still fire.
The witness asks the OS, not the daemon: `/proc` for the post-exec shell's fd
table (`socket:`, `memfd:`, `/dev/ptmx`) and the daemon's own for
`memfd:mux-upgrade`. The QUIC socket's seal is unpinned — no e2e daemon in that
leg runs `--quic`; filed.
**Review follow-ups, same day.** Three things the branch review asked for,
taken rather than filed. (1) The session child runs `close_range(3, ~0)`
before its exec: `execUpgrade`'s clear list and `sealAdoptedFds` are two
hand-kept lists that must agree, and the barrier that needs no list is the
child closing everything above stderr — the seal is now hygiene, and a pipe
opened without CLOEXEC is the pin (`pty.zig`). (2) Observers carry an idle
deadline (`observer_idle_ms_default`, 10 s without a whole frame): the
one-byte peer no longer parks the daemon, but four of them held every
observer slot and closed each later attach at accept with nothing said.
The pump's 100 ms poll is what makes the reap timely. (3) The half-frame
leg's `timeout` scales with `E2E_TIME_SCALE`, against the lib's own
"budgets only" wording: a wedged daemon never answers, so waiting longer
changes nothing about the fact demanded, and a kcov-slowed answer must not
read as the wedge.
## 2026-08-28 — the wall lists daemons; tiles are their live sessions
**There were two lists of "my sessions" and they disagreed.** The daemon's
list is live and is what `Ctrl-\ n`/`p` walked (`sessions_req`); the wall
file's list was attach history and is what `mux wall`, the digits and
`Ctrl-\ w` showed. A session born by a split, by `muxa` or by another
client was on one and not the other, so `n` landed the user in shells they
did not know existed. The other direction was worse: a wall line for a
session the daemon no longer had was re-created on restore
(`client.hydratedCreates`), so the file resurrected shells — and bare `mux`
had a third path of its own, attach-or-create `#0` re-recorded on every
run, which meant `Ctrl-\ x` on it was undone by the next `mux`. One list
was the fix, and the only list that cannot lie is the daemons'.
**So the file records HOSTS and nothing else.** `$XDG_STATE_HOME/mux/hosts`,
one spelling per line — `--sock PATH` | `HOST` | `quic://HOST[:PORT]` — and
`#SESSION` is a refusal with the rule in the message. That is the whole
guarantee: a line that names no session is a line nothing can resurrect
from, so "a session the user ENDED stays ended" stopped needing a latch to
enforce it and became a property of the grammar. Tiles are each host's live
sessions, in host order then the daemon's own slot order; births appear,
exits disappear, and `n`/`p`/digits walk the one list. The old `wall` file
is not read by the CLI and not migrated: its lines are attach history of
sessions, and the model no longer has that.
**Poll, not push, and a fresh connection per poll.** Each host gets one
`sessions_req` a second on a side connection that is opened, asked, read and
closed. A subscription would be a new daemon concept — state per watcher,
an invalidation path, a reconnect story — for a frame that is forty bytes
over a link already carrying deltas. The measurement that would justify
changing it is the one to take first: `muxd stats` delta_bytes against
poll bytes on a wall of N hosts over an hour. If polling is ever the cost,
`sessions_changed` is the push and this is the line that said so. A fresh
connection each time is the same argument one level down — a held observer
is a slot the daemon must reap, an idle deadline to tune, and a thing to
re-establish across an upgrade, and the daemon already accepts a connection
per `muxd stats`.
**A host is one stripe only when it has no tiles.** The naive rule — down
host, dead tiles — turns a thirty-second network blip into a wall of
`[refused]` and then a re-attach storm. Tiles ride out a blip on their own
pumps (`reconnecting`, unchanged); the stripe exists for the case where
there is nothing else to draw, and it keeps redialling. The one exception
is the user's own machine: a listed local `--sock <default>` that nothing is
serving is auto-started, because it dies on every reboot while its line
lives on, and a wall that paints your own box `[unreachable]` until you find
some other shell to start a daemon in is the empty-file case with one line
in front of it.
**`[no sessions]` is a word the local path cannot reach.** `muxd` exits when
its last session ends and `muxd run` creates `0` at start, so every daemon
that answers at all contributes at least one tile. The word is kept because
a remote daemon held open by something else could reach it, but phase 1
pins `[unreachable]` only. Whether an empty daemon should STAY alive is a
spec question for the user — it changes daemon lifetime, and it is the
difference between "the host is gone" and "the host is idle" being
distinguishable on the wall. Filed, not decided.
**`Ctrl-\ x` ends the session, and the DAEMON owns the two-step.** New
observer verbs `end_req 0x11` / `end_reply 0x94`; nothing existing changed
shape. The daemon refuses a first `x` on a session other clients hold and
answers with the count, the client arms three seconds for the forcing second
press, and the rail says `[1 other attached - x again to end]`. The count
had to come from the daemon rather than from the client's own view: a client
knows how many clients it can see, which is one, and a client-guessed
"nobody else is here" is exactly the wrong answer to give about interrupting
someone. An accepted end is BOUNDED — SIGHUP, SIGTERM, then SIGKILL past
`Pty.term_grace_ms` (500 ms) — because a shell that traps both would
otherwise be a session with no master that nothing can end, which is the bug
the user asked to fix. The tile leaves when the daemon's list no longer has
the session, not when the key is pressed, so the screen never claims
something the daemon has not done.
**An upgrade is refused while any session is ending.** The 500 ms hangup
window has a session whose pty master is already closed, and `execUpgrade`
writes that master's fd into the manifest. Rather than teach the manifest a
"masterless" case, `muxd upgrade` answers `session ending, retry` and the
operator retries half a second later. Bounded refusal beats an unbounded
number of adopt-time special cases.
**A vanished tile's wake pipe is never closed.** Pollers now open a socket a
second, so fd recycling is real: closing a vanished tile's wake fd and
having the number handed straight back to a live socket makes a stray
doorbell byte land in someone's session. Up to `max_tiles` fds held for a
wall's life is the cheaper side of that trade.
**A `mux` with no terminal is a wall of ONE.** The standing invariant is
that a scripted `mux TARGET` writes byte-identically to what the plain
client wrote, and there is nowhere to paint a second tile on a pipe. So a
headless run stops at the entry host: no other hosts resolved, no pollers,
no sidecar. This was found the hard way — removing the old `hydrated` flag
made every wall save and restore the layout sidecar, piped ones included, a
restored focus CLAIMED, the claim forced a redundant resize, and
`01_boot`'s "quic delta resume" leg started wanting three snapshots where it
wants one. Bisected to `2782c6b`, not guessed — the commit that made a
host's live sessions its tiles, and removed `Entry.hydrated` on the way
past. A wall of MANY does need a terminal and says `mux: wall needs a
terminal`.
**The sidecar is saved by every wall left on a terminal, including `mux
TARGET`.** Under the old model `mux TARGET` was an argv wall and argv walls
were views, so they never wrote. Under this one `mux TARGET` IS the wall,
zoomed on one tile, and its layout is as much the user's as any other's.
The sidecar stays derived convenience: restore is verbatim, healing is per
leaf against the hosts' live lists, and every failure degrades silently to
the default cut. Restore moved to "the first list arrives" rather than
"the wall opens", because at open time the wall has one tile and the tree
it is being asked to restore describes several.
**Births from one list keep the list's order.** Each birth anchored beside
the FOCUS, so a wall entered on `a` that learns `{b, c}` in one answer laid
them out `a, c, b`. Each birth now anchors beside the previously born tile
of that same apply, the first beside the focus. A wall's reading order is
the daemon's order, which is the only order the user has been given a reason
to expect.
**Measured, and filed: a TTY wall pays one snapshot per tile per DISTINCT
relayout.** A three-tile wall costs six snapshots to open, with or without a
sidecar. The cause is not the sidecar: `relayout` flags `resize_pending` on
every present tile, the pump turns that into a `.resize`, and
`Server.onResize` calls `resyncSnapshot` even when `applySize` returned
false — a tile whose rect did not change still buys a repaint. Two candidate
fixes, both one-line and both outside this spec's contract: flag
`resize_pending` only on a rect change, or skip the resync when `applySize`
says nothing moved. Follow-up, with the sites named here so the next person
does not re-derive them.
**Two defects named rather than papered over, in `e2e_12`'s own comment.**
A heal that collapses a container loses that container's weight and
orientation, so a restored wall can come back correctly cut but wrongly
proportioned; and the focus after a healed restore lands on the newcomer
rather than on the tile the sidecar named. The healing leg pins what it can
actually hold — no vertical rail at 80x24 after restore — instead of a
geometry assertion that would pass with the heal reverted. An assertion
weakened without saying why is worse than an absent one.
**A daemon that predates this change reads as `[unreachable]`, and that is
the cross-version cost.** `sessions_req` existed, but only on the attached
client's path; answering it on a bare observer connection is new here. So a
v0.0.1-15 daemon is up, `mux --sock PATH` attaches to it, and the wall still
paints it a stripe that never heals. Measured side by side: one hosts file
naming a tree-built daemon and an installed v0.0.1-15 daemon, both alive on
their own sockets, both answering `muxd stats` with `sessions=1` — `mux
hosts` printed `1` for the first and `[unreachable]` for the second, after
the full two-second wait. The xversion gate pins both halves of the story
(the stripe, and `Ctrl-\ x`'s `[daemon too old to end a session]`).
**What to DO about that is parked, not decided.** Two options, and the
choice is the user's. Accept it: every daemon on the wall must be upgraded,
which this release arguably demands anyway since it adds wire verbs. Or
distinguish the two silences — `[upgrade muxd]` when the dial succeeded and
only the list timed out, `[unreachable]` when nothing answered — which costs
one word, one branch in `hostsList` and the poller, and turns "your box is
gone" into "your box is old". Nothing is built for the second: it was not
asked for, and building an unasked-for mitigation into a release note is how
a stopgap becomes the design. Recorded here so the choice is visible rather
than made by default. Until it is made, the README says the first is what
happens and names the second as open.
**What went, and the legs that went with it.** `wall.zig`'s `#SESSION`
split and attach-history semantics; `client.recordOnState`; the CLI's use of
`client.hydratedCreates` and the saved-local-line attach-or-create rule with
its dead-tile marker; `mux wall`, `mux wall add`, `mux wall rm`;
`Entry.hydrate`/`hydrated`/`record0`; the `Ctrl-\ w` fold, now a plain
unzoom because there is nothing left to fold in; the daemon session ring
behind `n`/`p` and `client.ringNeighbour` with it. Their e2e legs were
deleted with the rules they pinned and the suite's count pin moved 83 → 81,
with a new group `09_hosts` carrying the model's own legs. No compatibility
spelling for `mux wall` — the project is too early for one (2026-08-20), and
`mux wall` now parses as a HOST called `wall`, which ssh says out loud. A
machine literally named `wall` or `hosts` needs a `--via` or `quic://`
spelling.
**The hub is phase 2 and was left strictly alone.** `muxweb` still reads and
writes the old `wall` file, still lists sessions rather than hosts, and its
e2e legs pass untouched — which is the evidence that this change is
contained. The cost is a real divergence for a real user: nothing on the CLI
writes that file any more, so a fresh machine's hub wall is empty until
someone names tiles on its command line or in the page. Stated in the README
rather than smoothed over, because a hub that quietly shows a different wall
from the terminal is the two-lists bug again, one layer out.
## 2026-08-28 — a daemon lives until `muxd stop`
**`x` ends a session, never a box.** `reap` returned the last dead shell's
code and `pumpOnce` handed it to `run`, so a daemon left with its last
session. Invisible while the wall listed daemons from a file; with the wall
listing live sessions and a picker offering hosts to be born into, it means
emptying a host destroys it and the picker offers what is not there.
So `reap` and `pumpOnce` answer nothing. A shell's code goes to that shell's
own clients (`exit_status`) and nowhere else; an emptied table is a daemon
with nothing on it. The picker says `no sessions`, a birth brings it back,
and one idle process per machine is the accepted cost — one daemon per
machine is the real use.
`muxd run` exits 0 on every shutdown, `muxd stop` and SIGTERM alike: a
supervisor reads a nonzero exit on a clean stop as a crash. Nonzero is a boot
failure only, stated in `main.run`'s doc block. The 130 that imitated SIGINT
went with the shell's code.
## 2026-08-28 — a popup, because a wall of sessions is a bad place for a host
A stripe was a host pretending to be a tile and every reflex was wrong on it:
`x` did nothing, the cursor stayed in the previous tile, and a daemon's last
session ending turned that host into one. The wall shows live sessions only
and the hosts moved into a popup, `Ctrl-\ s`.
A MODE (`interact.PrefixFilter.picking`, the `:` prompt's shape), so every
byte is the popup's and the key table is testable without a terminal. `s` and
`a` do not end the read as one-shot chords do — the bytes behind them were
typed at the mode they opened. `x` and Enter close it: each re-shapes the
wall, and the user is owed the wall that made.
Tiles pause under it (`Shared.picker_open`, read under `paint_mu`); replicas
stay hot, the close bumps `repaint_gen`, and the frame is hashed so the
per-host-per-second repaint cannot leave a terminal that never goes quiet.
A birth the daemon REFUSES, from Enter on an empty terminal wall, leaves the
wall standing with the sentence on its one line. `born_from` cannot say that
— the ENTRY tile of `mux TARGET` carries the same null and its refusal really
is mux's, because a script reads that code — so `Tile.keeps_wall` does, set
by `pickBirth` and read by `endAction` before the exit arm. A piped `mux` has
no wall to leave standing and still exits with the code.
Nothing is held across reads, so a SPLIT arrow closes the popup: an Esc that
arrives alone is a bare Escape, and the `[A` behind it lands in the next read
with `picking` already false and goes to the focused session. Holding the Esc
back would make every real Escape wait for the next keystroke, which is worse
— terminals write an arrow as one three-byte write, and the `:` prompt has
taken the same trade since it existed. A hold buffer is the fix if a terminal
that splits is ever found.
## 2026-08-28 — a new tile takes the lowest free digit
Reuse, not renumbering. `Ctrl-\ 1-9` addresses digits, so a digit an ended
session kept forever pushes later births out of the keyboard's reach, while
renumbering the survivors moves a digit the user had learned. The daemon
already reuses names, so digit and name come back together.
A slot is free only once its pump has RETURNED (`Tile.pump_done`): `present`
alone hands a `*Tile` a returning thread is still reading to a new tile. Its
doorbell pipe stays with it — never closed, so a fresh one per reuse leaks an
fd pair — and `live` becomes the high-water mark, growth counted in `present`.
## 2026-08-28 — a daemon starts when somebody asks, never from a read
`muxd endpoint` auto-started one before answering, and the wall runs it over
ssh once a second per host: a listed box got a daemon and a shell in session
0 from a READ, and a `muxd stop` typed there was undone a second later.
`muxd proxy` had the same call. Both now refuse an empty box.
The ASK moved to the client — `mux HOST`'s entry, the picker's Enter — which
runs `ssh HOST 'muxd start'` from the same `handoff.recipeFor`, only when the
remote exited 1 (ssh's own 255 is not an empty box), and reads the announce
ONCE more. `HandoffTarget.report_fallback` became `asked` and now gates the
start as well as that line; `wallview.pumpTile` spends it on the dial that got
the link, so no reconnect can restart a box someone stopped. The local
`spawn.ensureForAttach` is unchanged.
The break: an OLD client on a new remote gets no daemon from a cold `mux
HOST` — `ssh HOST 'muxd start'`, as README said. No xversion probe covers it.
## 2026-08-28 — `mux hosts` costs the SUM of its hosts' timeouts
`hostsList` dials each line in a plain sequential loop with a 2000 ms budget
(`hosts_list_ms`), and the budget starts AFTER `Transport.open` returns, so
it bounds the daemon's reply and not the dial. Measured, Debug build, an
isolated `XDG_STATE_HOME` and a python listener that accepts and then never
answers the `sessions_req`: one silent daemon 2002/2002/2001 ms, two
4002/4001/4002 ms — the sum, not the slowest. A `--sock` path with nothing
bound is refused by `connect(2)`: 1 ms for one and 1 ms for four, so absent
sockets are free at any count. What this does NOT bound is a blackholed
HOST, where ssh sits in the kernel's TCP retry schedule; `handoff.sshLine`'s
batch recipe carries `ConnectTimeout=5` for that.
## 2026-08-28 — four binaries became one `mux`, and the start execs itself
`muxd`, `mux`, `muxa` and `muxweb` are one executable, `mux`, whose first
word picks a mode: `d` the daemon, `a` the agent surface, `web` the hub, no
letter the client. The four mains stay four files under `src/cli/` and became
modules of one root (`src/cli/mux.zig`); each is handed the argv slice it
already read (`args[1..]`, whose `[0]` is the mode word), so no parser
changed. One alias exists, `mux run …` → `mux d run …`, because a daemon of
v0.0.1-15 or older execs its upgrade candidate as `<binary> run --resume-fd N`.
The local auto-start no longer walks PATH. `spawn.ensureDaemon` execs
`spawn.self_exe` — `/proc/self/exe`, the running image by its own kernel
link — with argv `mux d run …`. `findInPath` is gone. That closes the
ambient-PATH trap by construction rather than by a `PATH=` prefix in the e2e
runner: there is no name left in the tree to resolve, so none to resolve
wrong.
Running the daemon IN the fork instead of exec'ing was tried first and
abandoned, deterministically and for a reason worth writing down.
`std.debug.MemoryAccessor` caches the pid it reads memory through (a private
`cached_pid`), so a forked child's first DebugAllocator stack trace calls
`process_vm_readv` on the PARENT and gets ESRCH — whose arm is
`unreachable, // own pid is always valid`. The child died the first time a
client asked it for a snapshot, minutes after `up (0.1s)` had been printed;
the core trace reads `collectStackTrace <- DebugAllocator.alloc <-
DeltaTracker.rebuild <- Server.pumpOnce`. Nothing about it is reachable from
here: the cache is private and its SRCH arm is not an error to catch. The
general rule it leaves behind is bigger than this repo — fork-without-exec
inherits process-global caches keyed on the pid, and a single-thread
precondition does not cover them.
The pin is `e2e_03_side`: after a client auto-start, `readlink /proc/PID/exe`
must be the binary the suite was handed. The kernel answering about the
kernel, and the LINK rather than the argv, because argv is what a wrong
spawn still gets right. Shown to discriminate by starting the same binary
from a copy elsewhere, where the link names the copy. `spawn.zig`'s own test
asks a stub for `$0` and `$*` — the file the kernel exec'd, and the argv it
was handed — and nothing on this box resolves BY NAME to a file in a fresh
tmp dir, so a spawn that searched PATH cannot pass it.
`test/e2e_lib.sh`'s `proxy_pid` had to change anyway: one binary means the
CLIENT is a `mux` too, and its argv carries the whole `--via '… d proxy …'`
string, so the match is positional (`$4 == d && $5 == proxy`) rather than
anywhere in the line.
One cost, and it is a break:
A daemon of v0.0.1-15 or older cannot be upgraded in place into this binary.
Its `checkVersionOutput` runs `<candidate> --version` and demands exactly
`muxd <version>`; this binary answers `mux <version>`, and there is no argv
that reaches the candidate to say otherwise — the probe is a bare
`--version`. `mux d upgrade` translates that daemon's
`version: output mismatch` into the one-time instruction (`mux d stop`,
`mux d start`). The `run` alias still earns its place: it is what the OLD
daemon's exec spells, and what the rollback exec spells back, since the
binary rolled back TO is by definition the older one.
Remote is unchanged in mechanism and changed in spelling: `handoff` builds
`ssh HOST 'mux d endpoint'` and `'mux d start'`, so a box still running a
≤v0.0.1-15 install reads `[unreachable]` on the wall until it is upgraded.
## 2026-08-29 — the bare `mux run` alias is removed; it was unreachable
The entry above kept `mux run …` as the bridge for a daemon of v0.0.1-15 or
older, whose upgrade execs `<candidate> run --resume-fd N`. That exec never
happens: the same daemon runs `<candidate> --version` FIRST and demands
`muxd <version>`, which this binary does not print, so it refuses the
candidate before anything is exec'd. An alias reachable only through a door
that is already shut is a second grammar for nothing, so the rule is now
without exception — a word that is not `d`, `a` or `web` is a transport, and
`mux run` names a host called "run" (`mux.zig`'s `modeOf`, and the refusal a
user actually sees pinned in `mux_main.zig`).
`main.zig`'s rollback exec went with it: it spelled the bare `run` on the
argument that the binary being exec'd BACK is older. It is older only by
what this repo has shipped since, never by the rename, because a ≤v15 daemon
can never have been the writer of a manifest this binary is resuming. It
spells `d run` now, like every other exec here.
`mux d upgrade` still translates a `version: output mismatch` from that
daemon into the one-time `mux d stop` / `mux d start`.
## 2026-08-29 — the browser hub reads the hosts file; the `wall` file is retired
Two answers to "what is on the wall" — a CLI list of DAEMONS and a browser
list of `TARGET#SESSION` spellings the page authored — meant the two fronts
could disagree about a machine. `mux web [HOST ...]` now records its argv
into `$XDG_STATE_HOME/mux/hosts` and serves that file: one
`client.SessionPoll` per daemon, the same loop `wall_host.pollHost` runs, and
a tile is a live session on a listed daemon. `wall.zig` is gone; its grammar,
its argv collector and the one atomic writer are `hosts.zig`'s.
What the browser LOSES, deliberately: authoring. No add box, no `×`, no
drag-to-reorder, no `#SESSION`; POST /tiles, PUT and DELETE answer 405, and
daemons are managed by `mux hosts add|rm` or the picker. `+` still births on
that tile's daemon. Two consecutive missing lists drop a tile; an
unreachable answer drops nothing and clears nothing.
Found by the rewritten e2e leg: a `POST` carrying no body and no
`content-length` — what `curl -X POST` sends — reached std's `discardBody`
assert and ABORTED the hub, every tile with it. Any answer to a
body-capable request that declared no length now closes its connection.
## 2026-08-29 — the cold handoff is one ssh run: `mux d endpoint --start`
- **The three runs.** A cold `mux HOST` cost three ssh logins because the
CLIENT decided to start: `mux d endpoint` exited 1 having written nothing,
the client reaped the child to read that code, ran `ssh HOST 'mux d start'`,
then ran `mux d endpoint` again. Measured on the suite's ssh shim, one line
per run: `e2e_04_handoff.sh`'s cold leg pinned 3 and now pins 1.
- **Why it folds.** The reap existed only to tell the remote's refusal (exit
1) from ssh's own failure (255), which the announce cannot; the remote
deciding deletes the question, and `mux d start` was already idempotent, so
"ensure, then answer" has no race in it. `mosh-server new`'s shape: one ssh
run whose first stdout line is the coordinates.
- **"A read never starts a daemon" now holds by ARGV**, not by a branch a new
dial path could forget: a poll, `mux hosts` and every redial spell
`mux d endpoint` and start nothing. `--start` is `endpoint`'s alone
(`mux d stats --start` is rc 2, not a flag that vanishes) and its ensure
forwards `--sock`, so the daemon binds the path the announce then probes.
- **What ≤15 remotes read.** Measured against the released v0.0.1-15 tarball:
rc 2, a usage page on stderr, nothing on stdout — so the asked dial reads
`[unreachable]`, the same answer the rename already gave them.
## 2026-08-29 — the handoff is a step table; a warm miss pays one deadline
- **Policy out of the driver.** Which to try, in what order and when to give
up sat inline between the effects performing it, so a row was reachable
only through a daemon, an ssh shim and a blackholed port. It is
`handoff.next` now, with `Transport.openHandoff` the loop performing what
it names: fifteen mutation-proven rows, seven walks over whole traces.
- **The defect that stood since 2026-08-11.** Where the announced UDP port
cannot be reached, every attach after the first read the cache, spent the
budget on silence, refetched over ssh, was told the SAME port and key, and
spent it AGAIN before using the ssh pipe it had held open throughout:
`Blocked, warm cache 4264ms`, against a criterion of "within one
deadline". `State.failed_on` remembers what went silent, and the table
ends on the pipe: `e2e_04_handoff.sh` leg (e) measures 4068ms with the row
removed, 2137ms with it, against a ceiling of 3500.
- **Equality is port AND key.** A restart takes a fresh ephemeral port, a
re-key keeps the old one, and both are still worth dialling.
- **The `use_pipe` rows are not vestigial.** Through `ssh -J gate box` the
announced port is unreachable by construction — the jump host is the only
route and it carries TCP — so the pipe is that host's only session.
## 2026-08-29 — the handoff ssh's stderr is a pipe mux reads
- **The bug.** A hosts line naming a box that is down: the wall polls it over
ssh, ssh dies with `No route to host`, and that line landed on the wall's
ALTERNATE SCREEN, from a writer no repaint of ours can reach — the child's
stderr was inherited. `quiet` was the workaround (picker Enter only), and
`.Ignore` threw the sentence away rather than keeping it.
- **The rule.** The spawn always pipes; mux reads that fd in the announce poll
and on the `Transport` after it, where the pump and the hub poll it beside
the link. Unread it fills at 64k and ssh stops moving the session's bytes at
all — asserted with a child that floods 1 MiB before it serves.
- **`narrate` replaces `quiet`, inverted.** The entry dial alone relays, to its
own fd 2, until the wall takes the screen; silence is the default, so a path
that forgets the field cannot corrupt a paint. `--via` keeps `.Inherit`.
- **What the line buys.** `handoff.Reason` survives fragmentation, keeps
printable ASCII only (the row paints raw, and C1 has a two-byte UTF-8
spelling) and caps at 120. The row budgets it against its spare width, since
`pickerRow` cuts the spelling and never the state. It beats both old failure
lines: `UnterminatedLine` named what mux observed, this names what happened.
- **Deferred.** ssh's PROMPTS (`SSH_ASKPASS`); the dead-host poll pacing.
## 2026-08-29 — ssh's prompts get a mux-painted place: `SSH_ASKPASS` → the wall
- **The rule.** A wall dial's ssh never touches `/dev/tty`: it is spawned
with `SSH_ASKPASS_REQUIRE=force` and `SSH_ASKPASS` at this image, and
`mux askpass` carries the question over a per-client socket in
`$XDG_RUNTIME_DIR` to a popup that eats every byte. The mode is named by
`MUX_ASKPASS_SOCK` and not by a word, because ssh execs its helper with
the prompt as argv[1] and nothing else; a mode word still wins over it.
- **Two exclusions, structural.** `wall_pump.askOn` is the ONE place a
target is armed: the entry dial opens before any listener exists, and a
poll neither passes through it nor could ask under `BatchMode`.
- **A decline KILLS the dial's ssh.** A refused askpass is not a refused
login — OpenSSH turns a non-zero helper into the EMPTY password and asks
again up to `NumberOfPasswordPrompts`; measured against this box's sshd,
one Esc was three prompts. So the wall SIGTERMs that ssh (its own child,
by `dialOwner`) before releasing it, and the ring parks the tile.
- **`SSH_ASKPASS_PROMPT` decides the paint**, not a substring of the text:
`confirm` shows in the clear, `none` is a notice with no input line that
closes when ssh kills its helper, anything else is starred. Both ends fold
control bytes — a keyboard-interactive prompt is the SERVER's wording.
## 2026-08-30 — a module is an owned component, not a compilable file
- **The table halves: 41 rows to 21.** Every file that could compile on its
own had become a row, so the graph named files and the layers counted hops
between them. A row is a COMPONENT now — `term`, `daemon`, `client`,
`wall`, `agent`, `mux` — and the rest of each folder is child files its
root re-exports (`term.replica`, `client.hosts`). The strata fell from
seven to four, four being how deep the components are.
- **The compiler took the enforcement over.** A file belongs to exactly one
module, so a cross-domain grab now fails as "file exists in multiple
modules" while the build graph is built. It used to fail as a missing edge
in a table an author could simply have added the edge to.
- **`predict.zig` moved to `src/tui/`.** The overlay is the wall's, and out
of `src/engine/` the `term` component structurally cannot see it — before,
"prediction never enters the replica" rested on prose and a table edge.
- **What it cost: a per-file import BAN is now a per-component grant.**
`quic_server.zig` carrying opaque bytes and `shellint.zig` knowing nothing
of the protocol were each an empty import list the table enforced. Both are
child files of `daemon` now and may spell whatever that row grants, so those
two disciplines are prose a reviewer holds, not a refusal the build makes.
- **`webhub` stayed a row.** A module's root directory is the dirname of its
root file, so `@import("../client/webhub.zig")` from `src/cli/` is "import
of file outside module path": making the hub a child of the dispatcher
needs a file move, which this was not.
- **`daemon_main` briefly reached layer 4 and then stopped existing.** The
daemon's entrypoint is a child file of `mux`; that row lived inside the
branch and never shipped.
- **Test order puts the suites that can WEDGE last**: from `pty` on, every
row either waits on a pty or imports `testtmp` — which is what a row asks
for when its tests bind a unix socket. Ahead of that tail nothing exceeds
301ms; the tail holds both whales (`pty` 5s, `daemon` 40s, Debug). A
wedged step prints nothing at all, so the verdicts above it are the only
legible catch.
## 2026-08-31 — the prose ratchets are removed: a counter buys metaphor
`tools/docscheck.zig` had four tiers. Two asked whether a comment still
refers to something real (a cited symbol resolves; no project-history
codenames). Two asked whether it was short enough — tier 3 flagged a doc
block heavier in BYTES than the decl it documents, tier 4 counted five
consecutive comment lines as an "essay", both held to an exact, down-only
baseline in `docscheck.budget` and `docscheck.blocks`. Tiers 3 and 4 are
gone, with their baselines and the `doc-report` step; tiers 1 and 2 stay.
- **The gate worked and the result was unreadable.** Both ratchets came down
as designed over four commits. What they bought was not brevity but
compression, and compressed English is metaphor: comments arrived at a
passing byte count by replacing the explanation with a figure of speech,
which is the one thing a comment cannot afford to be.
- **The rule it produced said so out loud.** CLAUDE.md's own wording had
become "Two lines budgets the CLAIM, not the words: a comment that will
not fit plainly drops the claim, never the subject or the verb" — an
instruction to delete true information to satisfy a counter. A gate whose
documented use is "drop the claim" is measuring the wrong thing.
- **Tier 3's ruler was backwards.** Weight was prose bytes against the
DECL's bytes, so the shorter the code the less might be said about it.
The code that most needs an explanation is short and dense; the code that
can afford a long comment does not need one.
- **Tiers 1 and 2 have no opinion about length**, which is why they stay.
Tier 1 earned its keep the same week: the `src/cli/` rename in 7a530237
(`Opts` → `DaemonArguments`) orphaned a citation in `src/client/client.zig`,
a file that commit never opened. No counter was involved in catching it.
- **What is unguarded now**: nothing stops a comment growing into narration
again. That is a reviewer's job, and the trade is deliberate — a reviewer
can tell an essay from an explanation and a byte count cannot.
## 2026-08-31 — the module-architecture enforcement is removed
`build.zig` had two mechanisms enforcing SHAPE on the import graph: layers
(`ModSpec.layer`, `layerOf`, and a comptime check that every production
import point at a strictly lower stratum) and folder rules 1–3 (engine and
client name no tui/server/cli module; server names no client and no terminal;
client never names tui), with a signed-exemption table for edges the rules
forbid. All of that is gone, along with the `Folder` enum, `folderOf`,
`FolderExemption`, `folder_exemptions` and `folderExempt`;
`checkFolderRules` becomes `checkSourceBans`.
- **The trigger was a product primitive with nowhere legal to live.**
Connecting a client to a daemon is the one operation this product exists to
perform, and the daemon's own tests hand-roll it at 175 sites because no
callable primitive exists where they can reach one. Every home for a
~40-line `dial` module was refused by the build, not by the design: `term`
owns the attach encoders and is wasm-pinned, so it cannot open a socket;
layer 0 may import nothing, so `sockpath` cannot compose dial+attach; an
import must point STRICTLY downward, so a module `client` (layer 1) shares
forces renumbering the whole graph; and folder rule 2 walls the daemon's
tests off from `client.Transport.open`, the primitive the product does
have. We were costing out a 175-site test sweep to avoid a 40-line module.
- **A gate that makes the product change harder than the test rewrite is
optimizing the wrong direction.** The layers described the graph honestly
and then froze it; their failure mode is not a bad edge sneaking in but a
good edge never being drawn, which nothing reports and no diff shows.
- **The module table STAYS**, and is still where structure is written down:
it wires every module and every import, so no table-module import can exist
outside its loop. What changes is that adding an edge is now an ordinary
diff instead of a re-stratification.
- **Kept because they are not module architecture:** name-exists validation
of `imports`/`test_imports` (a stale name is still a compile error); the
`test_imports` twin mechanism, whose subject is SHIPPING — test scaffolding
cannot reach a production instance — not graph shape; the wasm-closure
check, reworded as the mechanical fact it is (a flagged row compiles
against wasm32-freestanding, so an import without a `.wasm` twin cannot
compile there, and failing at the edge beats the twin loop's later
`@panic`); and source bans 4, 5 and 6, which pin BEHAVIOR — no VT authored
outside `src/tui/`, every program exec'd as argv, one `posix.fork` site —
by reading production lines the import graph could never see. The three
keep their numbers, because files across the repo cite them in their own
`folder rule N exemption:` lines.
- **What is unguarded now**: a headless-app import (`client` reaching into
`src/tui/`) is a reviewer's catch rather than a build failure. The stated
invariants in CLAUDE.md are unchanged and `mux_core.wasm` still links
`term` + the client core with no tty anywhere, which is the check that
actually exercises the separation.
## 2026-09-01 — the label bar follows the tty, not the tile count
A one-tile wall and a fullscreened pane drew no label bar, so the very view
most sessions live in — `mux TARGET`, zoomed — was anonymous: nothing on
screen said which host or session you were typing into. The rule is now that
the bar is a property of being on a terminal: every tty tile wears one,
one tile or many, fullscreen included, and only a piped `mux` stays bare —
its byte stream is a script's input, and a bar in it would be bytes the
session never wrote.
- **The cost is one row.** A solo session on an N-row terminal runs at N-1
rows, and `wall_pump`'s `owns_screen` pass-through never engages on a tty
(painting is always clipped). The piped client keeps pass-through and full
height, so nothing script-facing changed.
- **`wallFloors` is keyed by the bar, not by `live > 1`**: on a tty every
stripe owes `min_session_rows` plus the bar row, so a 4-row terminal that
used to hold one bare pane still holds one pane — barred.
- **The TooSmall degrade keeps the bar.** The one visible pane still
deserves its name; only the rects degrade.
- **The e2e convergence check learned the bar.** A tty client's render is
the daemon grid plus a bar row, so `assert_converged_pty` proves the bar
IS on row 1 first — a drop that could cut a content row would hide the
divergence the diff exists to catch — then diffs the rows under it via
`render --drop-top 1`. `Engine.dumpVtFrom` pins the byte equivalence:
slicing the viewport formats identically to a grid that never had the
row. The convergence-point pin stays at 38.
- **Found by the change, kept as a lesson:** the thin-wall host test freed
tile labels before joining pumps (defers run last-declared-first) — a
use-after-free that was invisible while a solo pump never painted its
bar, and a segfault the moment it did.
## 2026-09-01 — gone panes: placeholders over auto-start
Measured: after a daemon restart, the seeded wall painted its saved cut at
byte 46 of the capture and collapsed it on the first poll answer (~1s) —
the flash the seed was built to remove, surviving in the one path where a
reachable host disowns a saved session. Chosen: dress those panes `gone`
(reversible, pending stays set), Enter re-arms the same tile as a creating
attach in its rect, `x` dismisses. Rejected: auto-starting shells (bends
"nothing re-creates a session" without a keypress; layers on later as
Enter-on-all), and keeping the collapse (the user's stated expectation is
the wall they left). Spec: docs/superpowers/specs/2026-09-01-gone-panes-design.md.
Landing it also fixed two recorded heal defects — a heal now keeps the
survivor's weights and the container's orientation, because nothing
collapses so nothing re-wraps (measured in the e2e heal leg's sidecar
pins).
## 2026-09-01 — remote upgrade pushes the running image; the suite gets a hermetic default socket
`mux d upgrade HOST` is the remote spelling of the local verb, and the
design leans on ONE binary: the machine you build on already holds the
image every box needs, so the push streams `/proc/self/exe` over ssh —
no release fetch, no remote network, dev builds included. Chosen shapes:
- **Preflight by line count.** One ssh runs `uname -m && command -v mux
&& mux d endpoint` (the bare read verb — a preflight that started
daemons would repeat the poll bug of 2026-08). `&&` stops at the first
answerless step, so 1 line = no mux (refused; a push replaces an
install, it does not invent one), 2 = install-only (exit 0, said so),
3 = full flow. The arch line is checked before any byte moves because
the streamed image is this machine's own.
- **Atomic rename, remote trigger.** `cat > mux.new && chmod && mv` so a
dropped stream can never truncate the installed binary; then the box's
own `mux d upgrade` runs, keeping the version rule, manifest and
serving check daemon-side rather than reimplemented client-side.
- **Rejected:** a release-tarball pull (needs remote egress, cannot ship
a dev build), an `--all` sweep (a shell loop away; per-host failure
reporting not worth designing yet), cross-arch push (refused loudly
instead).
Found by the leg's RED run, fixed in e2e_lib: the suite isolated
XDG_CONFIG/STATE/CACHE but not XDG_RUNTIME_DIR, so a leg with a bug that
dialed the DEFAULT socket reached the developer's live daemon at
/run/user/*/muxd.sock — the not-yet-wired `d upgrade HOST` sent it a real
upgrade request, and only the version rule refused the exec. The lib now
exports a scratch XDG_RUNTIME_DIR, so a stray default dial reads "nothing
listening" instead of someone's real daemon. The usage block grew a line
with `[HOST]`, so the boot leg's absolute refusal bound moved 15 -> 16,
re-derived the way its comment derives it (above complaint-plus-usage,
below the ~20-line Debug panic floor).
## 2026-09-02 — hygiene run: figures, dead pub, a socket leak filed
**What was measured.** Every figure in CLAUDE.md's reading guide was
re-run with `wc -l`: `client.zig` and `wallview.zig` had each grown a
hundred lines past their stated size, one server test file and one wall test
file sat a few lines over their "none over" bounds, and the comment ratio
read 36% of Zig bytes as whole `//` lines, not the 44% stated (the method is
now named beside the number). `tools/deadcode.sh` listed twenty `pub` decls
referenced by nothing outside their own file; every one lost its `pub` and
the compiler agreed. `client_core.zig` was the one module root without a
`//!` header (collab 2432d35e, closed). `make check` and `make ci` green
before and after: 107 e2e scenarios, 38 convergence points, agent 10/10,
throughput within its bounds.
**What was found and not fixed.** Five `mux-ask-<pid>.sock` files sat in
the real `$XDG_RUNTIME_DIR`, every pid dead: the wall unlinks its prompt
socket on the normal exit and on error returns, and nowhere else, so a
wall that dies by SIGHUP or a kill leaves one per death. Filed as collab
f97d2808 with the two fix shapes; the five were removed by hand. Patch
3262d8a5 (collectLeafIds propagates its allocation failure, the forge-3
spike's first product) cherry-picks cleanly onto main and passes
`make check`; it is verified, not merged. `.zig-cache` is 124 GB on a disk
with 272 GB free — not cleared, because a clear costs every build a cold
start and nothing was blocked on the space. Also left as found, for the
owner to decide: the `stash@{0}` of 2026-08-30, whose flagged comments in
`main.zig` no longer exist; the `origin/worktree-web-client-spec` branch
(one 2026-08-11 spec commit never merged); `wall` and `wall.bak` in the
state dir, retired with the wall file; and sixteen `/tmp/e2e*.out`
captures from 2026-09-01 hand runs.
**What `/tmp` said.** 12,008 `mux-e2e-out-<pid>.*` captures from 161 dead
runs over six days, the `.h*` suffixes of the hosts group — killed
`E2E_ONLY` loops, since a run that reaches its trap sweeps `$OUT.*` and
today's full `make ci` left none. Everything older than today was removed;
today's 1,875 stayed. And 31 `mux-agent-<pid>-<hash>` dirs from dead
daemons, four of them left by the green ci run itself, so the leak in
collab 68dc4c90 is reached by the suite's own legs and the leak sweep does
not count it; noted there, removed here.
**Both leaks fixed the same way, later the same day.** No signal handler:
the unlink half of `retire` is async-signal-safe but the mutex half is
not, and no handler runs on SIGKILL anyway. Instead `xdg.reapDeadPid`
removes, from one directory, every entry named `<prefix><pid>…` whose pid
`/proc` no longer has, and the three creators call it first — the agent
directory, the shim directory, the prompt socket — so the successor that
would otherwise sit beside the leftover is the one that asks the OS about
its owner. A live pid's entry stays even when that pid is no longer a mux.
Pinned by a unit test per site against a real dead pid (a `/bin/true`
spawned and waited for — read its pid BEFORE the wait, `Child.wait` sets
`id` to undefined, which cost one red round) and by the boot group's
restart leg, which now asserts the SIGKILLed daemon's agent directory is
there before the restart and gone after it.
**A laptop that refused its own wall.** Reported the same afternoon: `mux`
on the laptop read `refused` while a wall over ssh was fine. `mux d stats`
there said `clients=8 attaches=6 sessions=2`, with one client on each
session — six slots held by connections attached to nothing. From this
box, `ss -uanp` showed the wall's two tile connections to the laptop plus a
third that changed port every second: the session poll, which opens a QUIC
connection per tick. `quic.Client.deinit` deleted the connection and closed
the socket without CONNECTION_CLOSE, so the daemon learned of each poll's
end from its 15 s idle timer; at one poll a second the eight slots were
gone in eight seconds, and a hub polling a scratch daemon reproduced the
climb 2, 4, 6, 8 in as many seconds. The fix is `sayGoodbye` on the
client's teardown, the mirror of the listener's `closeAll`. The daemon
needed nothing: its read path already kills a connection whose peer
closed. Pinned by a listener test (slot freed within 1 s at a 5 s idle)
and a handoff-group leg sampling the daemon's gauge while a hub polls —
whose first draft passed against the bug, because `mux d stats` puts
`session 0 clients=1` on the SAME line and a greedy match read that
instead of `clients=N attaches=`. The mutant's series, 1 2 3 4 5 6 7 8,
is what the leg now prints when it fails.
## 2026-09-02 — the wall is the layout: panes are authored, the poll only grades
**The report.** "by default wall should just be the layout. I connect to 2
local and 3 remotes; every time I type `mux` I expect to view that. If I go
to a new machine and type `mux`, I expect to view only 1 local session
because I've never connected before." What the product did instead: a wall
listed every live session of every daemon in the hosts file, tiles came from
each daemon's own `sessions_req` once a second, and the layout sidecar was a
derived convenience healed against those lists. A bare `mux` on the laptop
therefore showed the two sessions this box was VIEWING, because they were
the laptop daemon's live sessions, and a session born by `mux a`, a browser
or another device turned up on every wall that listed its daemon.
**The shapes on the table.** Three. (1) Leave the live-list wall and add a
per-device hide list — rejected: it makes the default wrong and asks the
user to keep subtracting from it forever, and a new session on a shared
daemon still arrives unannounced. (2) Keep the live-list wall and add a
`mux HOST#SESSION` argv grammar for the panes you want — rejected on
ergonomics by the user: naming a session on the command line is not how
anyone finds one, and it leaves the default wall untouched anyway. (3) Flip
the layout file from derived to AUTHORED and make it the only source of
tiles. The flip won because it removes a source of tiles rather than adding
a filter over one: with the file authoritative there is nothing to subtract,
and the poll has exactly one job left, which is to grade the panes that are
already there. It also deleted the duplicated poll-planner filed as collab
68d4700e — `planHostDiff`'s `birth` arm and `applyList`'s in the hub both
went, leaving one grading rule per front instead of two birthing ones.
**The strictness reversal.** Until now `hosts` was strict (a bad line
refuses `mux` with rc 2) and the layout sidecar was lenient (every failure
degraded silently to the default cut). The reason was that a host line is
authored intent and a layout was not. The layout is authored intent now, so
it is strict too: `seedLayout` returns `.plan`, `.refused LINE` or `.none`,
and a refusal prints `mux: layout ignored (PATH): LINE` and starts as if the
file were missing. Refused: a leaf naming a host the hosts file does not
have, a leaf with no `#SESSION` or a bad session name, a repeated leaf (a
second leaf spelling the entry counts), more leaves than `layout.max_leaves`
(32, shared with `wallview.max_tiles`), or text `layout.parseReporting`
gives up on — which reports the file line it stopped at, not the header.
Whitespace-only is `.none`: nothing was authored, so there is nothing to
name in a refusal. Seating PART of a wall is the one thing that must not
happen, because a user who loses four of six panes has no undo and no line
to fix.
**The trimmed-seed no-save ruling.** A seed that could not seat everything
the file named — a terminal too small for the tree, or a leaf naming the
very session this `mux` runs inside — sets `Shared.layout_path` null for the
WHOLE run and says which on the notice line (`[layout not saved: terminal
too small for N of its panes]` / `[layout not saved: it names this shell's
own session]`). It does not write a trimmed file. `persist` serializes the
tree it HAS, so the first save of such a run would drop the leaves the wall
left out: the user's other panes gone before a key was pressed. Stopping the
whole run's saving is the conservative half of that trade — a resize you
made in a too-small window is forgotten, and the panes you authored survive.
The two sentences are counted apart (`SeedPlan.dropped` vs `dropped_self`)
because a wall that blamed the terminal's size for a shell's own stripe
would send the user resizing a window that was never the problem. In the
same spirit a save FAILURE is a notice, not a `std.debug.print`: every save
runs under the alternate screen, so a stderr line would land in the middle
of whichever pane the cursor was in and stay there until a repaint.
**A refused file is never written, and no save happens at the exit.** Two
holes in the ruling above, both found by review on 2026-09-02 and both the
same shape: a run whose tree is not what the file says, writing that tree
back. First, a REFUSED file left `Shared.layout_path` set, so a run with an
entry pane (`mux HOST`, `mux --sock S`) did `addFirst(0)` and the start-up
`persist` replaced the user's whole wall with one leaf — the printed line
telling them what to fix pointed at a file that no longer held the mistake.
`seedSidecar` returns the verdict now and `run` stops saving on `.refused`
and on the new `.self_only` (every leaf was this shell's own session),
saying so on the notice line as well as on stderr. Second, the two
exit-time saves — `.detach` and `.finish` — were unconditional, so a second
terminal opened on the same device, showing a tree from before the first
one's edits, wrote its stale copy back on the way out. They are gone: every
change already saves, and the FOCUS is the only thing they recorded that
nothing else does. Two walls open at once therefore do not merge; the last
writer wins, which is what the hosts file has always done. A
changed-underneath guard — re-read before write, refuse or merge on a
mismatch — is the real fix for concurrent walls and is deferred, because it
needs a rule for what a merge means and neither front has one.
**`# holds NAME N`, and why a `#` line.** The picker's session rows need to
say who else is on a session before you end it, and that is the only wire
addition in the change. `sessions_reply` was names one per line plus a
`# mux VERSION` meta line; it now carries one `# holds NAME N` line per
session under the SAME version gate as the meta line, so a daemon with no
version to state sends a byte-identical old payload. A `#` line rather than
a new frame or a new field because `sessionsIter` already yields only lines
that are valid session names: an old client skips the holds lines exactly as
it skips the meta line, an old daemon sends none, and `parseSessionsHolds`
reads that as unknown rather than as zero — a zero would read as "safe to
end". The count is EVERY holder, the asker included.
**The count includes this wall's own pane (ruling).** It is a number to
READ, never a verdict. Whether an end goes through is judged by the DAEMON
against the OTHERS — the holders that are not the connection asking — so a
client that decided from this number would refuse to end a session only it
was in. The visible cost: ending, from the picker, a session whose one pane
is on this very wall takes two presses, because the daemon counts that pane
as another client of that session. The alternative considered was
auto-forcing when the only other holder is this wall's own pane; it needs
the client to match a holder to a pane, which is exactly the verdict-making
the number is not for. Deferred to a collab issue as a UX change rather than
smuggled in here.
**`x` moved, and what each one means now.** `Ctrl-\ x` on the wall is
`removePane`: the notice `[pane removed - the session is still on its
daemon]`, the pump told to detach (it writes `.detach` on its way out; the
transport close is the fallback), the tile vanished, the layout persisted.
Nothing is ended, and every pane goes the same way including one that never
came up. ENDING is the picker's `x` at the session level, where the row
already shows the holder count — the question "who else is in here" is
answered before the key is pressed rather than by the refusal after it. That
end runs `client.endSession` on a SIDE connection, because the session may
have no pane on this wall to ask through, and it dials `HostSpec.poll_target`
— the poller's batch recipe, with ssh's `BatchMode=yes`, a connect timeout
and `asked` false. Never the interactive target: an end must not start a
daemon, and a password prompt going to `/dev/tty` under the popup would park
the keyboard thread in the TCP retry schedule. The press goes on the WIRE whatever the row
says: a daemon with no `end_req` arm answers an unknown frame with silence,
so `endSession`'s `error.Timeout` is what "too old" looks like and becomes
`[daemon too old to end a session]`. This was a `# holds` gate first, on the
reasoning that the counting and the arm shipped together — measured wrong on
2026-09-02: `git show v0.0.1-16:src/server/server.zig` has five `end_req`
mentions and no `sessions_holds` at all, so the released daemon answers the
end and sends no count, and the gate refused a box that works. Only
"others attached" arms the 3 s force window (`Shared.pick_end`, per host and
name); every other refusal shows the daemon's own reason and arms nothing,
since arming there would leave the next `x` forcing an end nobody said was
blocked.
**The hub reads and writes the same file.** `mux web` serves the layout of
the machine it runs on: `webhub.readLeaves`, refusing a bad file with
`mux web: layout ignored (PATH): LINE` and serving an empty wall rather than
a guess, `/tiles` in tree order carrying `id`, `label`, `session` and
`state`. The page's `+` (`POST /tiles/<id>`) checks the file for ROOM and
for a DUPLICATE under the hub mutex BEFORE it dials (`roomForLeaf`), births,
then `appendLeaf` writes the leaf beside that pane. Pre-dial because
dialling first and finding the wall full afterwards leaves a live session
nobody asked for, on no wall, that only `mux a` or a terminal could find
again. A second `+` on one tile inside a single poll interval is a 409
`duplicate`, not a repeated leaf; `WallFull` and `BadLayout` are 502. Two
hubs or walls writing in one instant lose one update — each write is a
read-modify-write over an atomic rename, which is what the hosts file has
always done, and the alternative is a lock file nobody would clean up.
Known and accepted: a zoomed `gone` pane in the browser can resurrect its
session across a page RELOAD, because the page latches `exited` only until
the reload throws that state away; the terminal wall's `gone` pane is Enter
to restart by design, and the browser reaching the same place by F5 is the
same door with no keystroke on it.
**Duplicate leaves are refused on both fronts.** The loader refuses a
repeated leaf (including a second leaf spelling the entry — only the first
can be the tile the user is typing in), and the hub refuses one before it
dials. Two panes on one session would each attach, each claim a rect, and
each grade off the same list; the first one to lose its session would take
the other's name with it.
**`keeps_wall` was NOT deleted.** The plan called for it: with the wall no
longer built from births, a refused picker birth should leave the wall as it
was without a flag saying so. It cannot go. `endAction`'s tail is reached
only when the ended tile is the ONLY present pane on a terminal with stdin
open and the reason is not a clean exit, and two panes arrive there with
byte-identical fields and opposite right answers — a picker birth onto an
empty wall must vanish and leave the wall standing, and the ENTRY tile of
`mux TARGET` must finish with rc 1 and the refusal sentence, because that
refusal is the program's. `born_from` is null for both and `creates` is true
for both. The entry tile IS also the one tile with `retry_cold` false, so
that field could distinguish them today — but it answers a different
question (is a link that died before any state arrived worth redialling),
and hanging the wall's survival off it would mean one edit to the retry rule
silently changes what a refused `mux TARGET` does. The flag stays, its doc
comment now says exactly that, and the test that would fail without it
carries the argument. Recorded follow-up, not done and wanting its own
decision: INVERTING the flag — marking the one tile whose ending is the
program's, where `retry_cold = false` is already set — carries the same
information with the safer default and fixes an edge the current polarity
gets wrong, since a SEEDED pane alone on a terminal wall, refused, has
`keeps_wall` false and so ends mux, when under this model a saved pane's
refusal is a pane's and not the program's.
**Cross-version caveat, daemon newer than client.** Stated by what the
client was BUILT from, because no version number says it: this branch stamps
0.0.1-17 itself and no v0.0.1-17 tag exists. The released v0.0.1-16 client
sizes its `sessions_reply` buffer at `sessions_text_max` — the names alone,
1056 bytes; a development build off main taken after the `# mux VERSION`
meta line and before the holds lines sizes it at 1101, names plus that one
line. Both read a longer reply as a transport error, so a NEW daemon holding
roughly fourteen or more sessions with 32-character names reads
`unreachable` on such a wall until the wall is upgraded (the payload runs
about 75 bytes per session once names, holds and the meta line are counted).
`sessions_reply_max` is now names + one holds line per session + the meta
line for exactly this reason: a receiver sized to the names alone reads a
full daemon that states its version as an unreachable box.
**A `--via` wall records nothing, found by this change's own review.**
Writing the docs turned up a live regression the nine implementation tasks
had not: `mux --via CMD` on a terminal took the entry-pane road like any
other target, so the start-up `persist` wrote the leaf `tileLabel` spells
for it — `--via CMD#NAME` — and the NEXT start's `seedLayout` found no
hosts-file row for `--via CMD` and refused the WHOLE file. One throwaway
`--via` run cost the user every pane they had authored. Reproduced on a real
pty against a state dir of its own (the layout path is opened only when
stdin is a terminal, so the suite's piped `--via` leg could never have seen
it). Closed in the family of the trimmed-seed rule rather than by teaching
the loader about `--via`: `run` nulls `Shared.layout_path` when the entry
target is `.via`, before the seed and before the start-up save, and says
`[layout not saved: a --via wall is not recorded]`. So a `--via` wall
neither reads the file nor writes one, which is the same thing it already
did with the hosts file. Pinned twice — a unit test that a via-spelled leaf
refuses the whole file, and an assertion inside the boot group's `--via`
leg that a pty run under a fresh state dir leaves no layout behind.
**The pin.** The e2e suite grades 110 scenario checkpoints and 38
convergence points, up from what the old model needed, and the leg that
states the whole change in one sentence is "two walls on the same daemons
are two layouts; neither learns of the other's panes" in the hosts group.
## 2026-09-03 — a tile's application reads mouse reports in its own coordinates
**The escape.** A tile whose application had asked for the mouse got every
SGR report exactly as the terminal wrote it, in the terminal's numbering.
`interact.Core.forward` gated on `appMouse()` and passed the bytes through,
and every pane that did not start at the screen's origin sent its
application rows and columns it did not have: on a live wall (foot,
269x76, four tiles) a press at terminal row 60 reached an nvim of 37 rows
and a fullscreen Claude Code the same way, and neither could select at
all. Found by `strace -f -e trace=read,write` on the wall — the raw
`\e[<0;90;60M` read from fd 0 was the byte string written to the tile's
daemon socket. The same wall's remote tiles selected fine, which is what
made it look like a version split: that daemon held no mouse bits for its
sessions, so mux's own selection ran there, and the alt-screen "box stays
put" behaviour was the tell.
**Why every gate was green.** Each app-mouse leg was a wall of ONE tile —
`no_saved_tree`, a single `--sock` — and every unit test of the forward
path was a pipe client or an origin Core. The mux-selection tests all sit
at `drag_row_off`/`drag_col_off`, so the hit-test had the offset right
from the start; only the hand-over path never met an offset. The working
rule that N=1 and offset=0 are extra cases, never the baseline, was
followed on one side of the `appMouse()` branch and not the other.
**The rule now.** `interact.relocateReports` is the one place a forwarded
report is spelled: the Core runs the same `MouseFilter.feed` on both sides
of the branch, and on the application's side re-emits each report with
`row_off`/`col_off` taken off, in the read's original order among the
keys. A wheel report became an event of its own kind (`Event.Kind.wheel`)
so the filter could hand it back for re-spelling; `dragReports` ignores it,
which is where the old "a wheel is never an event" guarantee lives now.
Two choices in it:
- A coordinate outside the pane CLAMPS to the pane's edge rather than
dropping the report. A real terminal reports a drag that left its window
at the edge it left by, and dropping would leave the application holding
a button the hand released over a neighbour. The edge is what the pane
shows — the smaller of the grid and the clip — like `hitTest`.
- Pixel reports (1016) pass untouched. A cell origin cannot come off a
pixel, and a wrong translation is worse than none. Translating them
needs the terminal's cell size, which the wall does not ask for; open.
The filter's hold now stays armed across the hand-over from selection to
application, so a report split across two reads is whole for whichever
side reads it; the reset survives only for the pipe client, whose reads
hold no reports at all.
**What moved.** A wall on a terminal sits under its label bar, so the two
one-tile app-mouse legs now expect row 4 for a report on terminal row 5 —
they were pinning the bug. The new leg stacks two tiles and presses in the
lower one: terminal row 16 reaches the application as row 3, and `mux a
status` says the stripe is 11 rows so the arithmetic is the layout's, not
the comment's. Graded by mutation: with the row translation capped at one
row every older leg passes and only the stacked leg fails. The suite's
scenario pin goes from 111 to 112.
## 2026-09-03 — the daemon holds 32 clients, and the listener outlives them
**The raise.** `max_clients` 8 -> 32. A slot is spent per ATTACH, not per
session: every tile on a wall is its own attach, and each QUIC host's
once-a-second poll holds one more connection. Seven or eight tiles on one box
therefore refused the next attach, which is ordinary use, not an edge. 32
matches `max_sessions` and `wallview.max_tiles`, so exactly one full wall
fits — and nothing else does at the same time, which is the accepted ceiling
rather than an oversight.
**A use-after-free the raise exposed, in the tests only.** `max_conns` (the
QUIC listener's connection table) was documented as headroom over
`max_clients`, so raising the client table meant raising it too. At 40 the
test suite segfaulted in `Listener.closeConn`, reached from `Server.deinit`
-> `teardownClient` -> `Sink.close`. Three tests in `server_test_quic.zig`
registered `defer td.deinit()` BEFORE `defer q.l.deinit()`; defers run in
reverse, so the borrowed listener was freed first and the server's teardown
then closed its QUIC client slots through freed memory.
Production was never wrong: `main.zig` registers the listener's defer at ~560
and the server's at ~600, so the server tears down first, and `Server.deinit`
already states the rule for the lazily-bound arm ("the listener has to outlive
the slots that hold it"). Only the tests had it backwards.
Size did not cause the bug, it only decided whether it was visible: at
`max_conns = 16` the freed listener's memory happened to stay readable and
the read returned garbage quietly; at 40 it faulted. The bug was live at both.
Measured by reverting the test fix: (8, 16) green, (32, 16) green, (32, 40)
segfault. With the defer order corrected, (32, 40) is green over two runs and
(32, 16) fails only on the new capacity assertion.
**Why `max_conns` was not simply left at 16.** Holding it level with or below
`max_clients` has a user-visible cost that documenting would not have fixed:
a peer that finds no free CONNECTION is dropped SILENTLY — `acceptConn` spells
it "full: drop, the peer will retry" — rather than answered and closed. Peers
17 through 32 would hang on retries while the daemon still had seats, and a
host polled over QUIC would read `unreachable` though it was running fine. So
the test order is fixed and `max_conns` is 40.
Restated in `quic_server.zig` rather than derived from `server.zig`, because a
transport does not read the daemon's tables; `server_test_quic` asserts
`max_conns > max_clients` so the restatement cannot go stale unnoticed.
**Filling a 32-slot table in the harness.** The refused-tile leg used to fill
the client table out of sessions, which worked only while 8 slots ran out
before 32 names. At 32 both tables saturate together and the picker's `c`
would be refused for want of a NAME — a different refusal reaching a different
tile state. The leg now keeps its eight-pane wall and adds 24 long-lived
holders over those same eight names: 24 + 8 = 32 slots with the session table
still at 8 of 32.
The holders taught one thing worth writing down: closing their shared fifo
does NOT release them. A wall treats stdin EOF as "the script that was typing
has gone" (`wallview.zig` sets `stdin_open = false` and continues) rather than
as a goodbye, which is why `fill_sessions` types the detach chord instead of
just closing. One shared fifo cannot carry a chord per holder, so
`release_holds` signals the pids and then waits on `mux d stats` for the slots
to come back — the daemon's gauge is the witness, never the kill.
## 2026-09-03 — the platform layer (macOS port, step 2)
Every OS-specific spelling under `src/` now lives in `src/os/`, one row per
side. `server_os` holds what only the daemon does — the pty fork, the
detached daemon fork, peer credentials, an anonymous fd for the upgrade
manifest, `exitNow`, `closeFrom`, the pty's mode and foreground pgid, the
window-size ioctl and the stale-image verdict — and `client_os` holds the
few the wall and askpass do: pid, euid, peer credentials, the parent walk,
window size and a pty pair. `spawn` moved in beside them. Each root is the
CONTRACT — a doc comment per operation says what it guarantees and which
failure it prevents — and `impl` switches on `builtin.os.tag`, so a build
for an OS with no arm is a `@compileError` at that switch rather than a
link error or a runtime surprise. The client row is deliberately SEPARATE
from the server's rather than one `os` row: the client never links a fork
or a pty, and an app that links the engine and a client must not either.
**The gate is folder rule 7.** No production line under `src/`,
`src/engine/`, `src/client/`, `src/tui/`, `src/server/` or `src/cli/` may
spell `std.os.linux`, `/proc`, `memfd`, `close_range`, `exit_group`,
`so.peercred`, `so_peercred`, `iocsptlck`, `iocgptn` or `nosignal`.
`src/os/` is absent from that folder list on purpose: its children may
spell anything, and its roots have no reason to. Comments count, as they
do for rule 4, because a comment naming a Linux mechanism is one that goes
stale the day a second arm exists. Four needles are spelled to catch a name in both the form Zig
writes it and the form C and our own prose do — `so.peercred` and
`so_peercred` for `std.posix.SO.PEERCRED` and `SO_PEERCRED`, where a bare
`peercred` would have banned `client_os.peerCred`, the very operation
callers are supposed to reach for; `iocsptlck` and `iocgptn` drop the
leading T so they catch `std.posix.T.IOCGPTN` as well as `TIOCGPTN`.
`nosignal` is the fifth and is there for a different reason: it catches
`std.posix.MSG.NOSIGNAL`, a flag Linux and the BSDs spell differently and
macOS does not have at all, so a send that must not signal goes through
`server_os.sendNoSigNoWait` or `client_os.sendNoSig`, whichever side is
asking.
**Measured before the design, on a Linux host.** zig 0.15.2 cross-compiles
a libc program using `posix_openpt`, `kqueue`, `libproc` and `dyld` to
aarch64-macos with no SDK, and links Mach-O with its own linker — LLD
refuses ("using LLD to link macho files is unsupported") — so `linkerFor`
asks for `use_lld` on every target except Darwin. ngtcp2 1.25.0 and
wolfSSL 5.9.2 cross-build with `WOLFSSL_SYS_CA_CERTS=no` (mux is PSK-only
and the system-CA path wants Security.framework) and a
`CMAKE_FIND_ROOT_PATH` fence at the target prefix, without which ngtcp2
finds the host's `libwolfssl.so`. `std.posix.socket` and `std.posix.accept`
already emulate `SOCK_CLOEXEC`, `SOCK_NONBLOCK` and `accept4` on Darwin
with a trailing `fcntl`, so no socket site needed a row at all. ghostty-vt
links three C++ libraries whose build.zig files each call ghostty's
`apple_sdk.addPaths`, which resolves the HOST libc on a Linux host and
fails the C++ compile on glibc headers — upstream code build.zig cannot
reach, the cross-compile blocker that leaves the build-host decision open,
and the reason this step stops at the platform layer and writes no
`_macos.zig` arm.
**One amendment to the spec, made while writing the row.** The spec's
`openPty` plus `becomeSession` are one `forkPty` instead: `forkpty(3)`
exists on both OSes, returning pid 0 in the child exactly as `fork` does,
so the child code stays one branch. Zig ships no `<util.h>` for Darwin,
which is a missing HEADER and not a missing symbol — that arm declares it
with one `extern "c" fn forkpty` line rather than reimplementing the pty
open as `posix_openpt` + `grantpt` + `unlockpt` + `ptsname`.
**The QUIC prefix and the linker follow the TARGET, never the host.**
`quicDeps` names one word per prefix — `native` for the host's own libc,
`musl` for the static release, and `<arch>-<os>` for any other cross target
— shared with `build-deps.sh`, `make deps`, `make clean-deps` and wan.sh's
musl cross-build, so a third OS is one more `case` arm in the script and
nothing in build.zig. A cross build that reused the host's prefix would
link x86_64 Linux archives into an aarch64 macOS binary.
**Two behaviours changed shape on Linux without changing outcome.** The
stale-image verdict is an inode compare against what the image's path held
at boot, not the kernel's ` (deleted)` suffix: `noteBootImage` stamps the
path and its inode once, and every later ask re-stats that path, so a
rename-over reads stale and a file that lands there afterwards can never
promote a stale daemon back to current. The two failures are kept apart. A
path this OS will not name at all records NOTHING and answers false —
unknown is not stale, and a wall must not dress a healthy box in a warning
over a refused readlink. A path that IS named but holds nothing records a
BORN-STALE state, the path with no ident, and answers stale for the life of
the process, because that daemon is already executing an image no path
holds. Second, `xdg.reapDeadPid` asks `kill(pid, 0)` rather than
`access(/proc/PID)`: EPERM is alive-but-not-ours and keeps the entry, and
ESRCH is the one answer that means the pid is gone.
**`sockpath.runtimeDir` owns the default socket directory.** One switch —
`$XDG_RUNTIME_DIR` on Linux, and still no fallback, because a guess cannot
make two binaries agree on one daemon — so the daemon, the client and the
askpass listener agree by construction and another OS spells its own
default in one place.
**The shell harness asks the OS through named helpers.** `test/e2e_lib.sh`
holds one name per question a pin asks about a pid, an fd table, a bound
UDP port or a file's mode: `pid_alive`, `pid_exe`, `pid_comm`, `pid_args`,
`pid_children`, `pid_fd_count`, `pid_fd_targets`, `pid_holds_unix_sock`,
`pid_rss_kb`, `udp_local_bound`, `udp_table`, `file_mode`, `file_size`,
`sha256_of`. The SPELLING of the question now lives in one place per OS and
the question itself stays in the group file, so a second OS adds a
`case "$_os"` arm here and changes no group file. `oracle_selftest` runs
once before the first group and is the helpers' own pin: a helper that
quietly stopped answering — a missing binary, a `/proc` a sandbox will not
show — would not fail a pin loudly, it would make every pin that reads it
agree with anything, and the suite would go green having tested nothing.
Its subject is off-origin on every dimension the helpers could accidentally
hold constant: a child that is not pid 1, more open fds than the three any
fixture would hold, an argv worth losing, and a child of its own. The
socket arm is asked in both directions, of a listener that holds the path
and of a shell that does not, because a `pid_holds_unix_sock` answering yes
to everything would pass the one pin that reads it just as happily as a
correct one.
**How the wire claim was graded, and what the cross-version gate could not
say.** `make ci` is green. `test/xversion.sh` is NOT, and was not green
before this branch either: its preflight demands a `muxd` in the old
prefix and it drives that side as `muxd run`, so the branch base — one
`mux` binary since v0.0.1-16 — cannot serve as its old side at all, and
its pins still assert a PRE-M18 old side ("the daemon created session zz —
this is not a pre-M18 daemon", "it decoded frames an old client has no arm
for"). Against the released v0.0.1-15 tarball it answers 5 passed, 7
failed. So the claim was graded DIFFERENTIALLY instead: the same gate, the
same v0.0.1-15 old side, run once with this branch's binary as the new
side and once with the branch base's, gives the identical verdict pin for
pin — 5 passed, 7 failed, the same seven messages. The four real
compatibility pins are among the passes in both runs: a new client driving
an old daemon's default session over a socket and over QUIC, an old client
driving the new daemon's, and an old `muxa`'s empty `status_req`. The
refactor moved no byte on the wire. The gate itself needs its old side
re-pinned to a version that still exists; that is not this branch's work.
Spec: `docs/superpowers/specs/2026-09-03-macos-port-design.md`.
## 2026-09-03 — the daemon's Darwin arm (macOS port, step 3)
`server_os_macos.zig` fills the server root's `.macos` arm. Most of the
operations are a spelling change and nothing else. FOUR are a different
mechanism, because the Linux one does not exist on Darwin, and one of the
four is a trap that a straight translation walks into. They are the four
below: `sendNoSigNoWait`, `closeFrom`, `anonFd` and `peerCred`. The
`forkDetached` pin at the end is a fifth difference of the same kind, in
the test rather than in the arm.
**`sendNoSigNoWait` sets `SO_NOSIGPIPE`, and a REFUSED set is the answer.**
Darwin has no `MSG_NOSIGNAL`: "do not raise SIGPIPE" is a property of the
socket, not of the send, so the arm sets the option per call — there is no
one place every fd this operation is handed gets created (an accepted
client, a socketpair end, an fd adopted across an upgrade), and the option
is idempotent. Measured on macOS 26 with a C program and again with a Zig
one: Darwin's `sosetopt` rejects EVERY socket option with EINVAL once a
socket is shut down in both directions, which is exactly the state a
hung-up peer leaves behind. So the one send that would raise the signal is
also the one send the flag cannot be set for, and the obvious arm — set,
ignore the result, send — signals precisely when it is supposed not to.
The first draft did exactly that and the root's own SIGPIPE test caught it
on the box: the child died of a signal instead of exiting 0. EINVAL on this
call is therefore read as the kernel saying the peer is gone (the level,
name, value and length are all fixed in the source, so nothing else about
the arguments can be invalid) and returned as `error.BrokenPipe`, which is
what `send` would have answered had it not signalled first. Every other
setsockopt failure describes a socket that cannot raise SIGPIPE either, so
those fall through and let `send` name them. The more robust design is to
arm `SO_NOSIGPIPE` once at fd BIRTH, which would need a new root operation
at every socket and accept site; it is deferred as a follow-on, and the
cost of not doing it is one misread errno if Darwin ever grows another
EINVAL path on this call.
That test now has two legs, because a socket that never carried a byte and
one that lost its peer mid-stream are different states to the kernel and
only the second is the pump's own sequence. One leg alone passes on an arm
that can never arm a live socket, the other alone passes on an arm that
only works after a successful send.
**`closeFrom` walks the fd table.** No `close_range` on Darwin, so it is
one close per slot from the floor to `getdtablesize()`, which is the soft
`RLIMIT_NOFILE` and therefore also the ceiling on any fd this process could
be holding. A few hundred cheap EBADFs once per session start, between fork
and exec, so nothing is opening fds underneath the walk. The cost scales
with that soft limit rather than with the fds actually held, so a box that
raises `ulimit -n` to a large number pays for the raise here.
**`anonFd` is an unlinked `mkstemp` file.** No `memfd_create`. A 0600 file
this uid creates and unlinks before anyone could open it by name is private
by mode where memfd is private by having no name; the window is those two
calls, on an empty file. `/tmp` rather than the runtime directory because
`src/os/` imports nothing of ours and must not learn the socket directory.
`mkstemp` is asserted to open CLOEXEC on modern Darwin and the carrier must
survive `mux d upgrade`'s exec, so the flag is cleared before the fd is
returned. That assertion was never measured and does not need to be: the
clear is harmless on an fd that never had the flag. The root's existing pin — nlink 0, not CLOEXEC, readable and
writable — passes on the box unchanged.
**`peerCred` needs two calls where Linux needs one.** Darwin's
`LOCAL_PEERCRED` answers a `struct xucred` with no pid in it, so the uid
comes from `getpeereid` and the pid from `LOCAL_PEERPID` at level
`SOL_LOCAL` (0). Both answer for a socketpair, so the root's pin holds
there too.
**The `forkDetached` pin asks `getsid(2)`, not `ps`.** macOS's `ps` has no
`sid` column at all, and its `sess` column is the kernel address of the
session, which reads 0 for anyone but root — so the natural port of the old
Linux pin compares 0 against 0 and passes whatever the child did. `getsid`
is POSIX and answers the number on both, and it makes the claim STRONGER
than it was: the returned pid names a session LEADER (its sid is its pid)
in a session that is not the caller's, where the old pin read the leader
pid off procps and could say nothing on Darwin. `getsid` needs the child
ALIVE and not merely unreaped: Darwin answers -1 for a zombie where Linux
still names its session, which is how the second draft failed on the box.
So the child's stdin is a pipe rather than /dev/null, pre-loaded with a
word it echoes back before it blocks reading a second — which holds it
still for the question and makes `forkDetached`'s `stdin_fd` argument
load-bearing, where /dev/null pinned nothing about stdin at all. The ECHO
is what pins it and not the block: a draft that asserted "still running"
with WNOHANG passed green with stdin dup2'd from the wrong fd, because a
child wired elsewhere also ends early and when it ends is a race the
parent wins most of the time. Both claims were graded by mutation on
Linux — deleting `setsid` fails the session claim, wiring stdin to the
wrong fd fails the echo — and the whole file was then run on the Mac.
**`@cImport` of `<util.h>` works after all.** Step 2 recorded that Zig
ships no `util.h` for Darwin and that the arm would need one
`extern "c" fn forkpty` line. On a box with the Command Line Tools the
macOS SDK supplies the header, and `@cImport` of `util.h`, `sys/ioctl.h`,
`sys/socket.h`, `sys/un.h`, `unistd.h` and `stdlib.h` compiles, which is
what the arm does. `forkpty` needs no `-lutil` there; it is in libSystem.
**Rule 6's `except` is a list now.** The one `posix.fork` site is one PER
OS ARM, so build.zig names each arm's file and a third file that forks is
still caught. `test/bans.sh` still plants its needle in `src/server` and
reads rule 6's own fatal.
**How it was graded.** `src/os/server_os.zig` imports nothing of ours, so
the whole arm was compiled AND RUN on the Mac ahead of the client arm, as
`zig test src/os/server_os.zig -lc` — 11 tests, all passing, including the
pty, peer-credential, fd-barrier, carrier and SIGPIPE pins. The tree itself
still cannot link there until the client arm exists; `make build` on the box
stops at the client root's `@compileError` and nothing else.
Spec: `docs/superpowers/specs/2026-09-03-macos-port-design.md`.
## 2026-09-03 — macOS port step 3: the Darwin arm
The client arm, the toolchain, the two gates and the verdict. Written on
2026-09-04, over work that ran 2026-09-03 and 2026-09-04. The daemon's own
arm is the entry above; this one is everything around it and the grade.
**The build host cannot be Linux, and the Mac needs a shadow SDK.** Xcode
26.4 and later ship a `libSystem.B.tbd` whose `targets:` line lists
`arm64e-macos` and no `arm64-macos`. zig 0.15.2's Mach-O linker matches the
bare `arm64` slice only, so against that SDK every libSystem symbol is
undefined — for the build runner too, which is a native link, so `zig build`
cannot even start (ziglang/zig#31658; fixed in 0.16, which ghostty's pin
cannot use). `deps/mac-sdk.sh` builds a shadow SDK that is the real one by
symlink in every path except `usr/lib/libSystem{,.B}.tbd`, where zig's own
bundled stub stands in, plus an `xcrun` shim on PATH that answers
`--show-sdk-path` with it — zig finds the SDK by running `xcrun`
(`std/zig/system/darwin.zig`) and so does ghostty's `apple_sdk` helper. It
is self-retiring: when the real stub lists `arm64-macos` again the script
builds nothing and a stale shim passes straight through. With it the whole
`zig build` compiles ghostty and its three C++ dependencies natively, and
the QUIC stack builds in about a minute once the dep script's `native` word
stopped assuming an x86 host. Cross-compiling the Mac binary from Linux is
still not possible: the QUIC prefix cross-builds, but the link needs the
Mac SDK, and there is no shadow of it on a Linux box.
**The pty master answers the Linux shape.** `tcgetattr`, `TIOCGPGRP`,
`TIOCSWINSZ` and `TIOCGWINSZ` on the master all returned 0 against a forked
child holding the slave as its controlling tty, so the Linux arm's shape is
reused unchanged: nothing reopens the slave by name and `forkPty` records
only the master and the pid.
**The default socket directory is `/tmp/mux-<uid>`, and the measurement is
why.** The longest path mux creates is
`<dir>/mux-agent-<pid>-<12 hex>/agent-<session>.sock`, which is the
directory plus 68 bytes plus the pid's digits at `session_name_max` 32,
against 103 usable bytes of `sun_path`. On the test Mac `$TMPDIR` is 48
bytes (121 needed), `~/Library/Caches/mux` 34 (107) and
`~/.local/state/mux/run` 36 (109): none of the three spec candidates fits.
`/tmp/mux-501` is 12 (85). So `sockpath.runtimeDir` takes
`$XDG_RUNTIME_DIR` when it is set — every isolated rig sets it, on both
OSes — and otherwise `/tmp/mux-<uid>`, created 0700 and re-checked on every
ask for owner, exact mode and no symlink in the last step. `/tmp` is sticky
and world-writable, so the per-uid directory is what carries the privacy,
as tmux's `/tmp/tmux-UID` does; the check here is the stricter of the two,
because tmux asks only that no other-user bit is set and this asks for
exactly 0700. Linux keeps NO fallback: a guess cannot make two binaries
agree on one daemon, so the caller names it with `--sock`.
**Four operations are a different mechanism, not a different spelling**, and
the `forkDetached` pin is a fifth difference of the same kind in the test
rather than in the arm. The four are `closeFrom`, `anonFd`, `sendNoSig` and
`peerCred`, which is the count and the list the step-3 entry above gives.
`closeFrom` walks the fd table one close at a time from the floor to
`getdtablesize()`, because Darwin has no `close_range`; that is a few
hundred cheap EBADFs once per session start, between fork and exec, so
nothing is opening fds underneath the walk. `anonFd` is an unlinked
`mkstemp` file in `/tmp` at mode 0600, because there is no `memfd_create`:
private by mode where memfd is private by having no name, for the width of
two syscalls on an empty file. That is a real change of medium — the
upgrade manifest's key bytes touch a disk that memfd never did — and it is
accepted because the file is 0600, unlinked before anyone could open it by
name, empty until written, and on a FileVault volume by default; `shm_open`
was the alternative and Darwin's does not support read/write, so the
manifest writer would have had to mmap. `sendNoSig` sets `SO_NOSIGPIPE` per
call because Darwin has no `MSG_NOSIGNAL` and "do not raise SIGPIPE" is a
property of the socket rather than the send, and a REFUSED set is the
answer: Darwin's `sosetopt` rejects every option with EINVAL once a socket
is shut down both ways, which is exactly the hung-up state, so the arm
reads that EINVAL as the peer being gone and returns `error.BrokenPipe`.
`peerCred` needs two calls: Darwin's `LOCAL_PEERCRED` answers a `struct
xucred` with no pid in it, so the uid comes from `getpeereid` and the pid
from `LOCAL_PEERPID`. And the fifth, in the test: the `forkDetached` pin
asks `getsid(2)` rather than `ps`, because macOS's `ps` has no `sid` column
and its `sess` column is a kernel address that reads 0 for anyone but root,
so the natural port of the Linux pin compared 0 against 0 and passed
whatever the child did.
**A Darwin panic the gate found, and the shape of its fix.** Attaching a
Mac client to a Linux daemon crashed with `attempt to unwrap error:
SocketNotConnected` in `client_os_macos.sendNoSig`. Darwin answers ENOTCONN
on a send to a peer that is mid-close, and `std.posix.send` lists
`SocketNotConnected` among the errors it maps to `unreachable`. Both Darwin
arms call `sendto` instead — which returns that error rather than panicking
on it — and map it to `BrokenPipe`, which is what both callers already
handle and what `send` would have answered had it not panicked first. There
is no test. The window is a race, and a deterministic pin would need the
Linux arms to map ENOTCONN too, which this branch does not change: "the
Linux outcome does not change" was a constraint on the whole port. A
root-owned error map, written once above both arms, is the shape if that is
ever done; today the switch is duplicated verbatim in the two arms, because
a shared file would need a third module row and would break the leaf
property the platform layer's rows have.
**Two fixes in the daemon's own suite, made when it first ran on a Mac.**
The hangup-upgrade refusal test raced the stubborn shell's `trap`: on macOS
`/bin/sh` execs bash 3.2, which takes 12 to 21 ms to arm a trap where dash
takes 2 or less, so a fixed sleep that held on Linux did not hold there. The
FIXTURE owns readiness now — `TestDaemon.startStubborn` and
`attachStubborn` block on a per-pid marker the shell writes once its trap is
armed — and five call sites each became one line. Separately the
repeated-end test grew a duration pin: it times the accepted end to the
shell's death on `std.time.Timer` and requires at least `Pty.term_grace_ms`
and less than three times it, measured at 661 ms on Linux and 668 to 682 ms
on macOS against a 500 ms grace. Both were graded by mutation on Linux
before the Mac run.
**The shell harness asks a bare Mac, so its Darwin arm is perl.**
`test/os_oracle.sh` gained a `Darwin` case beside the existing one: one
spelling per question a pin asks, so a group file names the question and
never the OS. Three of those spellings are perl rather than a Homebrew
binary, and the reason is what the `make mac` gate runs on — a PRISTINE
macOS guest with no developer tooling, where nothing `g`-prefixed exists.
`real_path` is `Cwd::realpath` with a hand-written fallback, because
`Cwd::realpath` is not the same function on every perl: on 5.42 with Cwd
3.94 it hands back the spelling of a directory that is not there, where
`readlink -f` answers nothing with rc 1, so the arm resolves the parent
itself and `oracle_selftest` pins both branches. `now_ms` is
`Time::HiRes`, not `/bin/date`: macOS 26's date answers `%N` and older ones
do not, so spelling the system date would make the suite's clock depend on
how new the OS is, and the failure on an older one is a literal `N` inside
an arithmetic expansion partway through a run. It costs 4.5 ms a call
against gdate's 2.1 and python3's 21.6 (measured 2026-09-03), which is
inside the noise of every bracket in the e2e, whose tightest budget is
1500 ms, and NOT inside the throughput gate's 10 ms ceiling around a 6 ms
leg — a `make throughput` on a Mac will need `SOLO_MAX_MS` raised.
`timeout` is the third. GNU's binary is used when it is on PATH,
Homebrew's `gtimeout` next, and a perl `alarm` wrapper last, so a group
file keeps spelling `timeout` through all three. The perl arm answers
GNU's exit contract for the five cases this harness asks — 0, 124 for the
budget, the child's own status, 127 for a missing command, 128+N for a
signal — and `oracle_selftest` pins all five. It differs from GNU in three
measured ways, none of which a caller here can reach: a command that
exists but is not executable answers 127 where GNU answers 126; a signal
sent to the WRAPPER is not relayed to the child; and the child is not put
in a process group, so GNU's kill-the-group on expiry becomes
kill-the-child and a grandchild outlives the budget.
**The two gates.** `make mac` (`test/mac.sh`) drives eight legs on a tart
clone of the pristine `mux-mac-base` guest, boots it in under twenty
seconds (17.6 s on the run the ledger records) and
has run green twice unaided from a Linux box. `make xos` (`test/xos.sh`) is
the cross-OS gate: ten legs proving a Mac client against a Linux daemon and
a Linux client against a Mac daemon, over both the ssh handoff and QUIC,
plus the cross-arch upgrade refusal in both directions. It has run green
three times from Linux — 21.6 s, 20.8 s, and 1:12 with a Mac recompile in
it. The Linux side is the VM `mux-lan`, an x86_64 Ubuntu box; macOS
delivered inbound UDP on 4433 to a daemon started from a non-GUI ssh
session, so QUIC INTO a Mac works and needed no firewall exception.
**Two bugs `make xos` surfaced that predate this branch.** First,
`mux d endpoint` announces the key it resolved from the DEFAULT path and
never the listener's `--key`, so a daemon started as
`mux d start -d --quic ADDR --key ELSEWHERE` loses every ssh handoff. It is
not Darwin's and it is not fixed here; the gate uses the default key on the
Linux box and the bug is filed. Second, `parsePreflight` compared `uname -m`
verbatim against the zig target tag, so a Mac reporting `arm64` refused to
upgrade itself against an `aarch64` image. That one IS fixed on this branch,
because a port whose Mac cannot upgrade itself is not a port; the map is
exact and a unit test is the only pin, since one Mac cannot host a
Mac-to-Mac run.
**A lesson about the fixture itself.** Two agents sharing the Mac's single
checkout raced each other's `git checkout` in the middle of a gate run, and
the run graded a tree that was not the one it was asked about. The gate
re-reads the tip after every run now. One box with one working copy is a
shared mutable resource and has to be treated as one.
**`make e2e` on the Mac: the verdict.** Sixteen groups, run one at a time
with `E2E_ONLY` and once as the whole suite, on macOS 26.6.2 (Apple M1) with
Homebrew's coreutils present.
| group | alone | seconds | in the whole suite |
|---|---|---|---|
| 01_boot | rc 0 | 34 | pass |
| 02_predict | rc 0 | 53 | pass |
| 03_side | cannot run alone | — | pass |
| 04_handoff | rc 0 | 17 | pass |
| 05_session | rc 0 | 8 | pass |
| 06_web | rc 0 | 21 | pass |
| 07_wallcli | rc 0 | 30 | pass |
| 08_mouse | rc 0 | 22 | pass |
| 09_hosts | rc 0 | 87 | pass |
| 10_agent | rc 0 | 15 | pass |
| 11_select | cannot run alone | — | RED, by design |
| 12_panes | rc 0 | 59 | not reached |
| 13_birth | rc 0 | 37 | not reached |
| 14_upgrade | rc 0 | 10 | not reached |
| 15_askpass | rc 2 | 31 | RED, by design |
| 16_push | rc 0 | 4 | not reached |
The whole-suite column stops at `11_select` because the suite is linear
and `set -e`: the groups after it are graded by their standalone runs
only. The suite reached 78 of its 113 scenarios before that leg.
**Not one of the failures was a defect in mux.** Ten Linux-only
assumptions in the HARNESS accounted for all of them, and every fix was
made and re-run green on both OSes:
- `proxy_pid` matched a `comm` column. BSD ps prints comm as the
executable's full path where Linux prints the basename, and truncates it
to the column width in a multi-column format, so no proxy was ever found
and every scenario that tears a transport failed. It reads `args` now.
- The handoff shim compared a shebang script's process name to `ssh`.
Linux names such a process after the SCRIPT and Darwin after the
INTERPRETER, so `ssh` and `sh` are both correct and the word says nothing
about the product. Only the parent word, `mux`, is pinned now.
- Three counter checks compared `wc -l` to a string. BSD wc pads its count
to a column width, so " 1" was not "1".
- The scenario log stamped `date +%s.%N`, and BSD date has no `%N`. It
takes the oracle's `now_ms`. This one had no witness: the log is opt-in.
- Every temporary path came out with `//` in it, because macOS sets
`$TMPDIR` to a per-user directory ending in a slash, and two groups
compare such a path as a string. The slash is stripped once, before the
first of the 191 call sites that spell it.
- `pid_args` answered a different SHAPE on each arm: Linux reads a
NUL-terminated `/proc` entry and left a trailing space that
`ps -o args=` does not. The one caller comparing a whole argv with `=`
had to spell a space it could only have learned from Linux.
`oracle_selftest` pins the shape now, which is the right place for it: a
helper the two OSes answer differently makes every pin that reads it
agree with anything.
- Seven assertions across two groups grep a socket path out of a tile's
LABEL BAR. A bar is as wide as its tile and `labelText` keeps the state
word and cuts the label's TAIL, which is the half carrying the session
name — and the macOS temporary directory's name alone was 48 characters
on the test Mac, so an 80-column bar had none of it left. Those daemons'
sockets are spelled under `/tmp` now, which is where a Linux run with no
`$TMPDIR` has always put them, and the leak sweep reads both directories
when they differ so the paths that moved are still swept.
- The pager leg spelled the row numbers `less` leaves on screen. Attaching
makes less repaint, and where the repaint puts the top of an at-the-end
view depends on how many of the 24 rows the build spends on its status
line: less 704 leaves it at 178 and less 668 at 179, three runs each. So
the leg was off by one row on the Mac while the wheel worked perfectly.
The leg READS the anchor now. A short attach waits for the paint and
detaches, which leaves the daemon's grid holding the repainted view; the
dump after it is the top row the wheel is about to move, taken while
nothing is attached and nothing can change it, and the assertion is row
anchor-24 present and row anchor gone. That is exactly 24 rows on both
builds and it can still fail on 23, which matters: a leg that accepted
154 OR 155 — which is what was delivered first — could no longer fail on
a wheel that moved 23 rows, and 23 is what one lost notch looks like.
- One symptom on this leg looked like a lost input byte and is not mux's.
An earlier attempt anchored the view by TYPING `178g` into the pager
after the attach; on macOS the leading `1` never arrived and less went to
line 78, three runs out of three, while Linux went to 178 three out of
three. Two probes settled where it goes. A session running `cat`, which
echoes exactly what reaches its pty, received a first send of `abcdef`
intact on both OSes, three runs each — so mux delivers the byte. And a
`\x0c` sent first, with `178g` as the SECOND send, reached less whole on
the Mac, three out of three. So whatever discards it is on the far side
of the pty and not an input byte mux drops — by elimination, since less's
own source was not read; a `TCSAFLUSH` on re-entering raw mode is what
behaves this way and is the guess. It stays OPEN, which is the word the
README and the ledger use for it: elimination says where the byte is not
lost, and nobody has yet said where it is. Nothing in the product changes
and the leg no longer types into the pager at all.
Recorded because the shape — one byte, first write, one OS — is exactly
what a real input bug would look like, and the next person to see it
should know these two probes exist.
- The agent-directory reap was read straight off a bound socket. The reap
is not a boot step — it happens inside the successor daemon's first
`makeDir`, when its first session is born — so the check raced the thing
it asserted, and lost on a Mac where Linux had always won.
- A backtick in an UNQUOTED heredoc is a command substitution wherever it
appears, comments included, and three ptyclient scripts quoted words
that way in their prose. On Linux the words are "command not found" and
the run carries on none the wiser. On macOS `/usr/bin/expect` exists, so
the shell started it, it read the rest of the heredoc as its own script,
and the group hung for its entire 600 s budget. This one cost four gate
runs, not one. The first sweep found two sites and MISSED the third,
because its detector required the heredoc's delimiter to end the line and
that opener has a trailing space after it; the replacement comment
written for the third site had a backtick in it too. The rule is written
above each of the three heredocs. A lint is better and it is small —
match the opener anywhere on the line, skip a quoted delimiter, and refuse
a backtick in the body — so `test/bans.sh` has one, wired into `make
check`, with a planted positive in both opener spellings and a quoted
delimiter as the negative. The comments stay; the lint is what gates it.
**Two groups stay RED, and both are red BY DESIGN — the same design.**
`sockpath.runtimeDir` has no fallback on Linux and falls back to
`/tmp/mux-<uid>` on Darwin, so every leg that pins what mux does with
`$XDG_RUNTIME_DIR` unset is pinning a state macOS never reaches.
`11_select` has a leg that unsets it and requires `mux d stats` and bare
`mux` to REFUSE, naming the variable. On Darwin they resolve the fallback
and report what is or is not listening on `/tmp/mux-501/muxd.sock`, which
is the arm working. `15_askpass`'s last leg unsets it and requires a wall
to start no prompt listener, so ssh keeps its own prompts. On Darwin the
wall arms the socket — the dial log shows it being handed over — the popup
opens, it eats the detach key and the leg times out.
Neither was patched, because a leg that asserts the Linux rule is not
wrong; it needs a Darwin half that asserts the fallback, and writing those
is not this branch's work. One consequence is worth stating plainly:
`11_select` stops at that leg, so the scenarios AFTER it in that group have
never run on macOS at all. Everything in every other group has, both alone
and in sequence.
**Not done, and not claimed.** There is no x86_64-macos build: the arm is
written for `aarch64-macos` and nothing has compiled or run on an Intel Mac.
There is no universal binary. The Mac binary cannot be cross-compiled from
Linux, for the SDK reason above, so `make install` on a Mac names the native
target — the only one that builds there — and a Mac release is cut on a Mac.
And only `make test` and `make e2e` have run there. `make agent`, `make
soak`, `make throughput` and `make xversion` have never been run on a Mac,
so nothing is known about them beyond that they pass on Linux.
`test/agent.sh` in particular still spells `${TMPDIR:-/tmp}` the way the
e2e suite did before this branch and defines its own `now_ms` through
`python3` instead of taking the oracle's, so the first Mac run of it should
be expected to find the same class of thing the e2e suite just did.
Spec: `docs/superpowers/specs/2026-09-03-macos-port-design.md`.
## 2026-09-04 — the box gates take their boxes by name
`make mac` and `make xos` were written against one developer's machines and
said so in their own source: a Mac called `squirtle`, a Linux VM at
`ubuntu@192.168.0.37`, and a `MUX_MAC_IP` literal for the QUIC legs. Nobody
else could run either gate. Worse, the two disagreed about what a gate may
do to a machine it finds: `xos` scrubbed the Linux box to bare metal and
tiptoed around the Mac — refusing to run at all if a mux was already going
there, and carrying a `MAC_CFG_MADE` flag so it could put back a
`~/.config/mux` the product had created — while `mac` cloned a fresh tart
guest per run and reached it only through the host's NAT, over an
ssh-through-ssh string in every command.
Both gates now take three ssh targets, and neither has a default for one:
`MAC_BOX` (macOS, no toolchain, the box under test), `LINUX_BOX` (x86_64
Linux, `xos` only) and `MAC_BUILDER` (a Mac with zig and the shadow SDK,
which BUILDS and is neither installed onto nor scrubbed; it defaults to
`MAC_BOX`). A missing name is a preflight refusal with rc 2 that names the
variable and the line to run, before anything is written anywhere. There is
no default because a gate that guesses a box eventually guesses somebody's
laptop — which is the same reason the "it is a person's real machine"
branches are gone rather than kept: MAC_BOX and LINUX_BOX are VMs the gate
MAY scrub, stated once, and `box_scrub` is the whole of what that means.
The shared pieces live in `test/box_lib.sh`, sourced by both gates after
`test/os_oracle.sh`: `box_preflight` (one `ssh -n` per box, `arm64`
normalised to `aarch64` exactly as `main.archMatches` does for `mux d
upgrade`), `box_pair` (idempotent key install, then a real dial from A to B
so the pairing is proved and B's host key recorded where the plain ssh mux
spawns will read it), `box_scrub`, and `box_ssh`/`box_scp`/`box_stream`,
which replaced the three copies of the script-on-stdin runner the two files
had between them.
The scrub runs at the START of a run and again at the END, so a failed run
leaves nothing behind and the next one starts from the same place whatever
happened. It writes NO shell rc file, which reverses what the old reset did.
The old one inserted a `~/.local/bin` PATH line into the Linux box's
`.bashrc`, and the first draft of this work added the Darwin mirror of it, a
`~/.zshenv` line, on the reasoning that the entry dial's `~/.local/bin`
lookup is part of what these gates test. That reasoning was wrong. Every
remote spelling the product sends already carries the prefix itself —
`handoff.local_bin_append`, `PATH="$PATH:$HOME/.local/bin"`, on the entry
dial's word, on the upgrade preflight and on the upgrade's push — so a line
in a box's shell rc tests nothing mux needs, and it is the fixture
configuring the machine. A bare box that finds mux only because the gate
edited its shell rc is exactly the configured developer machine the pristine
guest exists to rule out. The gate's own remote scripts spell
`$HOME/.local/bin/mux` in full instead, checked leg by leg, and `xos`'s
install leg asserts the product's actual question: a plain
`ssh BOX 'PATH="$PATH:$HOME/.local/bin"; command -v mux'` — that prefix
character for character — answers `~/.local/bin/mux` on both boxes.
The one arrangement the scrub still makes is `loginctl enable-linger` on the
Linux arm, because without it logind takes `/run/user/<uid>` down with the
ssh session that started the daemon and reaps the detached daemon with it.
It exits with its own code so a box without passwordless sudo is told that,
rather than being handed "the reset was refused".
The Mac guest's lifecycle moved out of `test/mac.sh` and into
`test/provision-mac.sh`, run from the Linux box: it clones the hand-made
`mux-mac-base` on a tart host, boots the clone with `--net-bridged`, polls
`tart ip --resolver=arp` (6 s when measured), installs this box's public key
through the host's own, verifies a DIRECT ssh from here, and prints one line
— `export MAC_BOX=admin@IP` — with everything else on stderr, so
`eval "$(test/provision-mac.sh)"` works. Bridged and not NAT is what removes
the host hop: the guest took `192.168.0.170` on the LAN, this box reaches it
directly, and the Linux VM can see its port 22, which is what lets the two
boxes dial each other for the cross-OS legs.
One consequence has to be handled rather than configured away. The guest's
host key is new on every clone, so the harness's own ssh to `MAC_BOX` keeps
none (`StrictHostKeyChecking=no`, `UserKnownHostsFile=/dev/null`) — but
mux's OWN entry dial spawns a plain `ssh HOST` that reads the real
`~/.ssh/known_hosts` and cannot be handed those options. `xos`'s preflight
therefore does `ssh-keygen -R IP` and then one accept-new dial, so leg 7
finds the key where the product looks for it; `box_pair` does the same
forget-and-relearn on the far box, because a re-provisioned guest at the
same address would otherwise be refused by a stale entry for the rest of
that VM's life.
The binary reaches `MAC_BOX` as a stream — `ssh BUILDER cat` into
`ssh MAC_BOX cat` — because neither Mac needs a key of the other's for that
and only the orchestrating box can reach both. `cat` carries no mode, so
`box_stream` chmods 755 and both gates compare the builder's sha256 against
the far side's: the assertion that the bytes arrived is the hash, not the
copy tool.
Leg counts: `xos` is still ten legs. `mac` is seven, not eight — the
clone-and-boot leg left with the VM lifecycle, and what replaced it is the
preflight line, which is not an `ok`. The user's real Mac is no longer a
gate target at all; it can still be `MAC_BUILDER`, which is the one role
that touches nothing.
## 2026-09-04 — a release is one tarball per OS, and a Mac cuts the Mac one
`RELEASE_TARGET` now follows the host, the way `MUX_TARGET` already did.
That reverses the rule the Makefile carried until today, which was that what
a release IS must not follow the host that cuts it, and the reversal is
worth stating because the old rule was right when it was written.
It was written when there was one release, a Linux one, and it guarded
against one accident: `MUX_TARGET` had just been hoisted to the top of the
Makefile behind a `uname -s` switch, which put `make release` on a Mac one
`?=` away from building a Mach-O binary and shipping it as the Linux
tarball. What removes that danger is not the rule but the NAME. The target
is in the filename, so a Mac cuts `mux-vN-aarch64-macos.tar.gz` and a Linux
box cuts `mux-vN-x86_64-linux-musl.tar.gz`, and no step downstream — the
publish, the release listing, the README's install line, `mux d upgrade`'s
stream — can take one for the other. The old rule's second argument, that
macOS ships no `sha256sum`, was never an argument about what a release is;
it is a spelling, and `SHA256` is `shasum -a 256` on Darwin now.
The reason to reverse it is that mux runs on two operating systems, and
there is no cross-compiling the Mac binary: zig 0.15.2 asks `xcrun` for the
SDK only for a NATIVE target, so `-Dtarget=aarch64-macos` gets no system
include path and `@cImport` of `<util.h>` fails. So the Mac release exists
only if a Mac cuts it. `RELEASE_BUILD_TARGET` is the second variable that
falls out of this: `native` is the only `-Dtarget=` word that builds on a
Mac, and `aarch64-macos` is the only name the artifact may carry, so the
name and the build word are separate variables that happen to be the same
string on Linux. `make release` on Linux is byte-for-byte the recipe it was.
`make release-mac` (tools/release-mac.sh) is how the Mac tarball gets cut
from a Linux box. It refuses unless the tag exists locally AND points at
HEAD — a release is cut from a tag, never a branch tip — pushes exactly that
one tag (`--no-follow-tags` and an explicit refspec: a developer with
`push.followTags = true`, which is a common global setting, otherwise sends
every reachable annotated tag, measured as five extra tags in the hand
check), and then hands ONE remote shell script to `MAC_BUILDER`. That
script fetches the tag as a tag, detaches onto it, runs `make release` and
publishes with the builder's own `git-collab`. The publish is the builder's
because the builder is the box holding the bytes; nothing here runs
git-collab locally, and nothing here touches main.
`MAC_BUILDER` has no default, and the refusal says so. A release published
to whatever machine happened to be in someone's ssh config is worse than a
refusal, and the variable is the same one the macOS gates take, so the
builder is a box that is already known to build this repo.
The builder needs three one-time arrangements beyond what the gates need,
all of them on the builder itself: its checkout's `origin` must be the real
remote (a gate-only checkout can have been cloned from anywhere), and it
needs `git-collab init` and `git-collab init-key` so it has an identity to
publish under. That key is then trusted from the developer's own box with
`git-collab key add --global --label BUILDER`; without the trust step the
builder's publish is refused for the identity, which reads nothing like a
build failure.
One outcome to expect rather than debug: a publish of a version that
already exists is REFUSED, and that is correct. The answer is to download
the published tarball from `https://HOST/mux/releases/vN/FILE`, extract it
and `cmp` the BINARY against the local one — the gzip wrapper never matches,
because it carries an mtime — and never `--force` on the strength of a
matching sha alone. That rule was established at v0.0.1-16 (2026-08-29),
where a publish was refused against an identical binary built 76 seconds
earlier and the download-and-cmp is what established it was identical.
## 2026-09-04 — cells on the wire: the size measurement
The design's go/no-go gate is what a cell row costs against the VT row it
replaces. `engine.zig`'s test "cells: wire size vs VT rows" measures it on
five 80x24 screens fed real output shapes, comparing `dumpVtRow` against
`encodeViewportRow` over the whole viewport:
```
cells-measure prose: vt=988 cells=1009 ratio=1.02
cells-measure vim: vt=3132 cells=2164 ratio=0.69
cells-measure htop: vt=4167 cells=3130 ratio=0.75
cells-measure shell: vt=128 cells=88 ratio=0.69
cells-measure curses: vt=1920 cells=1752 ratio=0.91
```
**Gate MET, with room.** The gate asked for prose <= 1.5 and the styled
screens <= 1.2. Prose is 1.02 and every styled screen is under 1.0 — cells
are SMALLER than the VT they replace on all four of them. Cells go on the
wire for terminal clients too; the spec's alternative (VT frames for terminal
clients, cells for native and browser) is not needed.
`curses` is the fifth screen and was added late, because the other four all
paint with `\r\n` and so hold one dimension constant: they write every column
up to the last glyph. Real ncurses programs — htop and vim among them — erase
the screen and then jump field to field with CUP, leaving cells between the
fields that nothing ever wrote. Those hold codepoint 0, not a space. Sending
one as empty text drops its whole run out of the ascii form and charges a head
byte to every cell of that run, and a CUP-painted row is mostly such cells:
`curses` measured 1.25 that way and 0.91 once an unwritten narrow cell became
a space on the wire, the same as a typed one. The screen paints identically
either way and the plain dump already renders an interior blank as a space,
so this is a wire encoding, not a rendering change. A spacer cell keeps its
empty text — the wide cell beside it carries the glyph.
### First cut, 16-byte absolute header (refused)
```
cells-measure prose: vt=988 cells=1308 ratio=1.32
cells-measure vim: vt=3132 cells=4533 ratio=1.45
cells-measure htop: vt=4167 cells=6488 ratio=1.56
cells-measure shell: vt=128 cells=127 ratio=0.99
```
The first run header was absolute: 16 bytes of `flags, fg, bg, ul` on every
run whether or not the run changed them. That failed the gate on both
style-dense screens. A syntax-highlighted source line or an htop row changes
style eight to ten times, so such a row paid 130 to 160 bytes of header
against roughly 70 bytes of text, where VT spends four to seven bytes on the
same change — because SGR is a DELTA against the previous style and that
header was not.
So the header became a delta too: `u16 count` ++ `u8 mask` ++ only the fields
the mask names. A run that turns bold on and keeps its colour spends three
bytes on style rather than twelve, and a colour spends a tag byte plus only
the bytes that tag needs — none, a palette index, or three. The row is still
the unit: every row starts again from the default style, so a row remains
self-contained and no reader needs the row before it. That one change is the
whole difference between the two tables above.
### The other measured line: a space is a space
An earlier cut of the encoder sent a space cell as the EMPTY text a blank
cell carries, on the reasoning that both paint the same and the empty form is
shorter. It is shorter for one cell and much longer for the row. Only a
one-byte narrow cell qualifies for the run's ascii form, in which a cell is
its single byte and nothing else, so one empty cell drops the whole run out
of that form and every remaining cell then pays a head byte as well as its
text. Prose is a third spaces, so blanking them cost 36% of the screen
(2.08 rather than 1.32 under the absolute header). Sending the space as a
space is also the better answer for copy: an interior space is a space.
Trailing blanks were never the question — the row already ends at the last
cell that is not one.
## 2026-09-04 — cells on the wire
The client stopped parsing VT. The daemon's ghostty-vt is now the only
terminal emulator in the system: the wire carries the cells its grid holds,
the client copies them into a grid of its own (`src/engine/grid.zig`), and
every painter reads that grid. The design is
`docs/superpowers/specs/2026-09-04-cells-on-the-wire-design.md`.
### The two findings the change rests on
A byte of session output used to be parsed three times before anyone saw it.
The daemon's ghostty-vt parsed it into the authoritative grid; `composeDelta`
rendered a changed row back into VT — CUP, EL2 and SGR around
`dumpVtRow`'s bytes — and the client fed that to a SECOND ghostty-vt to keep
its replica; the host terminal then parsed the client's paint. The middle
parse was the only reason a client linked an emulator at all, and the
rule-4 exemption on `protocol.zig` said so in its own words.
The second finding is why this is a trade rather than a cost. On the common
path the terminal client never used its replica: `paintDeltaClipped` wrote
the daemon's row bytes to the tty verbatim and re-dumped from the replica
only for a row under a selection or a pane narrower than the grid. So for the
terminal client the emulator was carried to serve two special cases, and one
of those cases — the over-wide pane, where a grid-wide row's surplus landed
on the neighbour — disappears entirely, because every row now paints from the
grid clipped to the pane.
### The wire
A row is a `CellRow`: `u16 LE ncells`, then runs until `ncells` cells have
been read. Cells past `ncells` are default-style blanks, so the encoder stops
at the last cell that is not one; a cell holding a bare background colour is
NOT blank, or a coloured `EL` would vanish. A run is `u16 LE count` ++ `u8
mask` ++ only the fields the mask names — bit 0 flags, bit 1 fg, bit 2 bg,
bit 3 ul, bit 7 ascii — and a present colour is a tag byte plus the bytes
that tag needs: none, a palette index, or three. Absent fields are unchanged
from the previous run of the SAME row, and every row starts again from the
default style, so a row is self-contained and no reader needs the row before
it. A non-ascii cell is `u8 head` = `wide << 6 | text_len` then that many
bytes of UTF-8, the whole grapheme cluster, capped at 63 bytes.
The header is a DELTA because SGR is one. The first cut was absolute — 16
bytes of flags, fg, bg and ul on every run — and the measurement gate refused
it: a syntax-highlighted line or an htop row changes style eight to ten times
and paid 130 to 160 bytes of header against roughly 70 bytes of text, where
VT spends four to seven bytes on the same change. The two measurement tables
are in the entry above; the delta header is the whole difference between
them.
### The frames took new numbers
`snapshot` is `0x95`, `delta` `0x96`, `scrollback_chunk` `0x97`. The old
`0x81`, `0x85` and `0x87` are retired with a comment and never reused. The
reason is what a mixed pair does: an unknown frame type is DROPPED by both
readers, so a client of this build meeting a daemon of `v0.0.1-16` shows a
blank tile and nothing else. Keeping the numbers would have fed VT bytes to a
cell reader and cell bytes to a VT reader, and both paint noise on a screen a
person is reading. A blank tile is a bug report; a screen of garbage is a
support call.
So this is a clean break, stated as policy: a client of this build and a
daemon of `v0.0.1-16` exchange no replay frames at all. The migration is
`mux d upgrade` on every box, daemon-first — a wall of upgraded clients
against an old daemon shows blank tiles until the daemon follows, and the
release note has to say that in those words. `make xversion` has no old side
to grade until the next release
is cut, so it is out of this branch's gate; the gate re-enters the moment a
release exists that speaks these frames, and `test/xversion.sh` carries a
comment saying `XVER_OLD_WORKTREE` must be this branch or newer.
### The real bytes
`make bench` types 120 characters into a session with small gaps and compares
the delta bytes actually sent against the measured full-snapshot equivalent.
Both runs are the same workload on the same machine, in an isolated
`XDG_STATE_HOME`, each tree building in its own cache:
```
main b2cc0a99 snapshot_bytes=5644 deltas=120 delta_bytes=7751 snapshot_equiv_bytes=690505 ratio 1%
cells 4b471f1 snapshot_bytes=86 deltas=120 delta_bytes=7860 snapshot_equiv_bytes=17930 ratio 43%
```
The number that matters is the one that barely moved. Steady-state traffic is
7751 bytes against 7860, one and a half percent more for 120 single-character
deltas, which is what the synthetic screens predicted: a cell row costs about
what the VT row it replaces cost.
Everything else in the table moved because the SNAPSHOT collapsed. An attach
to a fresh 80x24 session was 5644 bytes of VT and is 86 bytes of cells: a
blank row is `ncells = 0`, two bytes, where the VT snapshot spent a CUP, an
EL and a style reset on each of the 24 rows whether anything was on them or
not. `snapshot_equiv_bytes` accrues that same figure once per delta, so it
fell by the same factor, and the ratio the bench prints is a fraction whose
DENOMINATOR shrank 38-fold. Reading the 1% and the 43% as a regression gets
it exactly backwards; both runs sent the same bytes, and the second one made
a cold attach to that session sixty-five times cheaper.
That left the bench's own kill criterion sitting seven points away from
tripping — it failed at 50% — on a change that only improved things. The
criterion was written when a snapshot was expensive and a delta had to prove
it was worth having. It is restated the same day (user decision) as absolute
bytes per keystroke: `delta_bytes / 120`, failing at 160. A keystroke changes
one row and a delta re-sends that row whole — an 18-byte delta header, a
6-byte row header, and the cells — measured at 64 per keystroke on the VT
wire and 65 on cells. A differ that re-sent the whole viewport per keystroke
would cost about 340 on a blank 80x24 screen (23 empty rows at 8 bytes each
plus the typed row), so 160 catches that and leaves the honest figure room.
The bench still prints the attach snapshot's bytes beside it, for reading.
`mux_core.wasm` is 19075 bytes, from 356344. That is the whole terminal
emulator leaving the browser client: the wasm build has no ghostty dependency
at all now, and what remains is the wire contract, the grid and the replay
core. The 356344 figure is main's artefact as built on 2026-09-03 from a tree
close to this branch's base, read off disk rather than rebuilt.
### Trailing spaces are the one difference two correct grids may have
`Grid.dumpPlain` must equal `Engine.dumpPlain` for the same screen byte for
byte, MODULO trailing spaces per row. That qualifier is not a concession, it
is the old behaviour written down. Ghostty's VT formatter trimmed trailing
whitespace, so the VT rows the old wire carried never held a typed trailing
space and no replica ever had one either; the e2e convergence check has
always stripped them as a formatting difference between two correct grids.
The cell encoder's default-space cutoff — the row ends at the last cell that
is not a default blank — reproduces exactly that, and it is also what keeps a
space-padded curses row off the wire.
The rule is now stated in three places and all three must keep saying the
same thing: the header on `grid.dumpRowsPlain`, which `Grid.dumpPlain` is one
caller of; `server_test_harness.trimRowTails`, which the attach tests compare
through; and `converged_quiet` in `test/e2e_lib.sh`. The cost, unchanged from
the VT path, is that selecting a line that ends in a typed space loses that
space.
### What the break exposed and what it left behind
Selection had no pin. `paint.rowToVt` paints a span inverted and PLAIN,
snapped outward to whole glyphs, the way `Engine.dumpVtRowSpan` did — and
nothing anywhere asserted that a selected cell drops the style it carries.
The test now exists (`test: pin that a selection paints its cells plain,
whatever style they carry`), written because the flip needed it, not because
anything failed.
`mux a` was a reader of the wire that nobody had counted as one. It attaches
at 0x0, holds no grid and draws nothing, so it looked like a pure frame
client — but `mux a run` reports what a command printed, and it got that by
fetching the rows between the OSC 133 marks and stripping the SGR out of
them. Cells went through that stripper untouched, so `output` came back with
a NUL and a 0x80 ahead of every line. `make ci` caught it on the one agent
leg that compares a transcript exactly rather than with `contains`. The whole
e2e suite ran green past it, because no e2e leg reads `mux a`'s output field
at all. The fix decodes the chunk through `grid.decodeRows`, the same call the
terminal client's scrollback page makes, and renders it with
`grid.dumpRowsPlain`, which is `Grid.dumpPlain`'s body lifted out for a caller
that owns rows without a grid. The lesson for the next wire change is the
module table: every row that imports `term.protocol` is a reader of the wire,
whether or not it paints.
`onFetchScrollback` now echoes the CLAMPED start and count rather than the
request's own. `web/mux.js` drops a reply whose echo differs from the
outstanding request and rolls the scroll position back after its timeout, so
in the one window where the daemon clamps — the client's history count stale
AHEAD of the daemon's, after a `\e[3J` or an alt-screen flip, with a wheel
request already in flight — the browser now waits out that timeout where it
used to paint a short page labelled with the request's start. The old page
was mislabelled, which is the thing the JS check exists to prevent, so this
is the better of the two. If the wait is ever unwanted the fix is JS-side:
accept a reply whose start matches the request and whose count is no larger.
The CLI ignores the echoed start entirely.
A scrollback chunk in flight across a NARROWING resize is skipped rather than
painted. `interact.scrollbackPage` decodes at the replica's current width, so
a chunk encoded at the old wider grid fails to decode and the scroll view
stays unpainted until the next key. The old path blitted whatever VT arrived.
The refusal is correct — those rows would not fit the pane — and the race is
rare; it is written down here so nobody later reads the `.skip` as a bug.
`term` links libc for two of its own TESTS. Two `protocol.zig` tests need a
socket whose peer refuses to read, and the pinned Zig 0.15.2 has no
`socketpair` in `std.posix`, so they call `std.c`'s. The flag reaches only
the native module graph: `mux_core.wasm` and the wasm check modules are built
by a separate path that passes no `link_libc`, and no client binary gains a
link it did not already declare. The row used to get libc for free through
ghostty-vt; dropping the emulator made the dependency visible rather than
new. If "term is platform-free" should be stated by the table rather than by
a comment, the fix is a `test_link_libc` column applied to the test module
only, or moving those two tests into `link`.
### Why not ghostty's own protocol
Mitchell Hashimoto has said ghostty's binary protocol between its client and
its server is "mostly just libghostty's" — a page memcpy. That is the right
shape for one thing mux already does and the wrong shape for the wire. It is
right for `mux d upgrade`, where the same binary hands its own state across
an exec and the struct layout is identical on both sides by construction;
`dumpState` and the manifest memfd stay exactly that. It is wrong for the
wire, where the two ends are separately built, separately versioned binaries
and a client must not depend on ghostty's page layout to paint a row. So mux
keeps `dumpState` for the exec and cells for the wire, and the cell format is
mux's own: a client can be written against it without ghostty in the picture,
which is the point of the change.
## 2026-09-04 — cell text is filtered at the decoder, not at the painter
On main the client fed the daemon's bytes into its own ghostty engine and
painted from that engine's cell dump. That engine was a sanitiser as a side
effect: an escape sequence in the stream became at most a cell ghostty chose
to keep, and nothing the daemon sent could reach the host terminal AS an
escape.
The cells wire removed that engine. `paint.rowToVtFrom` writes a decoded
cell's text between its own SGRs and `renderClipped` writes the result to the
client's real terminal fd, so a cell's bytes now reach the user's emulator
verbatim. A daemon could spell `\x1b]52;c;<base64>\x07` across the cells of a
row — one byte per cell in an ascii run, or up to 63 in a head-form cluster —
and the emulator would honour it: a clipboard write, an alt-screen flip, a
title change. `CellRowWriter.cell` asserts only that a cluster fits in 63
bytes, and `asciiCell` gated the WRITER alone; the reader returned whatever
byte was there.
The check lives in `CellRowReader.next`, so every consumer of the wire — the
tui painter, the wasm core, muxa, any future one — is covered by construction
rather than by each painter remembering. An ascii-run cell must satisfy
`asciiByte` (0x20..0x7E), which is now one function the writer's run decision
and the reader's refusal share; a head-form cluster must carry no byte below
0x20 and no 0x7F. Either is `error.BadPayload`, which becomes `.resync` on a
delta and `error.SnapshotAborted` on a snapshot.
An honest ghostty page cell never holds a C0 or C1 byte, so this can only fire
on a malformed or hostile daemon — which is exactly the case validating a
payload is for. It is refused in the reader rather than stripped in the
painter because a stripped row is a row the client and the daemon disagree
about silently, and the dump-parity rule would go with it.
## 2026-09-04 — binary output can leave controls in Ghostty cells
The preceding entry's assumption that Ghostty never retains control
codepoints was incorrect: feeding a single DEL byte creates a cell holding
U+007F, and binary output can also leave C0 codepoints. The writer emitted
those cells, but the reader rejected them. A native client then exited with
`SnapshotAborted`; reconnecting failed again while the same cells remained.
Both the cell writer and the shared reader now replace an entire head-form
cluster containing C0 or DEL with U+FFFD. The reader consumes the original
encoded length, preserving subsequent cells, rows, styles and widths. This
also lets a new client attach to snapshots retained by an existing daemon.
Terminal consumers still never receive raw control bytes from those cells.
ASCII runs containing controls and structurally malformed rows remain
errors; the snapshot corruption checks are unchanged.
## 2026-09-05 — a deleted socket path: a trail in the log, a re-bind, and a QUIC door
Issues 04b3019d and 145807a2, from one incident on 2026-09-04: a daemon's
`muxd.sock` vanished from `/run/user/1000` between 13:21 and 13:45. The
daemon (three days up, upgraded in place that morning) kept its three
sessions and kept listening on the unlinked inode; nothing could reach it
by name; the next `mux` auto-started a second daemon on the same path; and
the day's log held ghostty stream warnings and nothing else. Who deleted
the file was never established.
### What was decided
**The daemon logs its socket.** One `mux d: socket PATH: ...` line per
event on stderr, which `forkDetached` points at the xdg log: the claim's
branch (`nothing there` / `cleared a dead daemon's leftover`), the bind
with dev+ino, an adopt across an upgrade, the loss of the path (`now
missing` or `now dev=D ino=I` for a replacement), a re-bind or its refusal
by error name, and the unlink guard's verdict at exit (`unlinked` / `left
in place: it names another daemon's socket now` / `left in place: already
gone`). The shared modules stay silent — `serve` is also askpass's binder
inside a wall, where a print lands on a pane — so `sockpath.claim` returns
a `Claimed`, `serve.Bound.close` and `unlinkIfOurs` return an `Unlink`,
and the daemon prints. The wall's `startLocalDaemon` appends ONE line to
the same log before it forks, naming what its dial answered
(`FileNotFound` for an absent path, `ConnectionRefused` for a dead socket
file): that line is what would have dated the second daemon against the
first one's loss. `sockpath.probe` is `answers` with the error kept.
**The daemon takes the path back.** `Server.watchSockPath`, once a second
from `pumpOnce`: one `fstatat` against `bound.path_id`. A missing path is
re-bound through `serve.bind(.refuse_live)` — the same claim a start makes,
so "no socket stealing" holds by construction — and the old listener fd is
closed (accepted connections are their own fds; the old backlog held
nothing reachable by name). A successor holding the path is
`DaemonAlreadyRunning` from claim, logged once per reason, and the watch
keeps looking: the successor's own `mux d stop` unlinks its file and the
next tick reclaims. The window for a wall to auto-start a second daemon
shrinks from forever to one tick. Rejected: a stat on every accept (a
path-less daemon receives no accepts by path, so it would never fire — the
timer is the only place the check can live) and re-binding without the
claim (that IS socket stealing).
**The admin verbs get a second door.** `mux d stop|dump|stats --quic
HOST[:PORT] [--key FILE]`, for the daemon a successor keeps path-less. The
daemon has served those three verbs on a QUIC client slot since `mux a
--quic` existed (`handleDaemonVerb` runs before the client-slot switch), so
this is client-side only: `AdminTarget` in main.zig, the key by the same
three-way rule as `mux a`. A QUIC ask is bounded (5 s) where the socket's
is not, because UDP has no "nothing listening" errno. Stop's verdict over
QUIC is the daemon's CONNECTION_CLOSE (`Server.deinit` tears client slots
down before anything else), and a port that never answered is rc 1 — the
socket's idempotent 0 is honest only where the errno says "absent", and
silence over UDP may be a firewall or the wrong key. `--sock` with `--quic`
on those verbs is a parse refusal; `start` still takes both. `upgrade` stays
`--sock`: `upgrade_req` is served to observers only, and its manifest is
local to the box.
### Measured
- Unit: the server test deletes the path, sees the re-bind within one tick,
dials it and gets a `sessions_reply`; binds a successor, sees
`DaemonAlreadyRunning` recorded and the successor's inode untouched;
closes the successor without unlinking and sees the leftover cleared and
the path reclaimed.
- e2e, boot group: a real detached daemon, `rm` of its socket, `mux d stats
--sock` answering again within the budget, the loss and re-bind lines in
the real log with the inode read off the bind line, the marker still on
the grid and the pid unchanged.
- e2e, session group: the `--quic` daemon with its socket deleted; stats,
dump and stop over UDP; the stop graded by the pid going and the file
gone; a second stop on the dead port rc 1.
- `make check` flaked once on `Pty.adopt`'s 5 s exit-code wait while a
sibling worktree's `make ci` was loading the box; green on the re-run,
and unrelated to any file this branch touched.
## 2026-09-05 — a burst of dials past the observer table waits in the backlog
Found rebasing the socket-lifecycle branch onto main: `make ci` failed the
refused-tile leg of `test/e2e_13_birth.sh` at one run in four, on a tree
whose every other run was green, and on main's own build. The leg seeds an
eight-tile wall on one daemon, fills the table, and births a ninth to be
refused; the ninth was SEATED instead, on a slot one of the eight had let
go of at birth. strace on the wall showed the eight pumps dialling within
150 µs of each other and one of them taking SIGPIPE on its first write 34 µs
after its connect returned: the daemon's accept loop had run ahead of the
attach frames, filled the four observer slots with peers whose frame was
still in flight, and closed the fifth at accept — `conn.close(); // out of
slots`. The pump ended that tile quietly, nothing painted, and the wall
stood at seven tiles with a slot to spare.
**Decided:** `Server.freeObserverSlot` is asked BEFORE the accept, and
`pumpOnce` polls the listener only while it answers. A fifth simultaneous
dial waits in the kernel's listen backlog (128 deep) until a promotion or
an idle drop frees a slot — the backlog is the queue and the observer table
is the set of connections the daemon is reading, and accepting a connection
it cannot read was never a service. The idle rule stands: four silent peers
now delay the fifth by up to `observer_idle_ms` rather than close it with
no diagnostic. The harness's "three at a time" fill rule in `test/e2e_lib.sh`
was the workaround for this, measured 2026-08; it stays as written, since
the arithmetic beside it is built for the batch, but it is no longer what
keeps a fill alive.
**Measured** (`burst8.sh`: eight piped attaches to one fresh daemon at
once, thirty trials, count of "connection to the daemon lost"):
| daemon | lost / 240 |
|---|---|
| this branch before the fix, Debug | 118 |
| main `ad746c0f`, Debug | 118 |
| main's parent's blocking dial swapped back in, Debug | 118 |
| installed v0.0.1-17, ReleaseSafe | 18 |
| this branch after the fix, Debug | 0 |
The blocking-dial row is the A/B that cleared main's new
`open_wait.connectUnix`: the loss is the daemon's, and a release build only
narrows the window. Pinned by the observer-burst test in
`server_test_attach.zig`: eight connects with no frame behind them, exactly
four held after twenty pumps, and all eight seated and served once the
frames arrive.
## 2026-09-05 — the terminal wall retires; `mux` becomes one session per invocation
Decided with the native client (`muxg`) in hand, recorded as issue
`94ed7dfd`.
**The TUI keeps one of its two jobs.** It is the only client that runs
where there is no window — over ssh, inside mosh, on a headless box, as
the piped `mux` a script reads an exit code from — and it is the pty the
e2e harness grades the daemon through, so it stays a product. What it
loses is the wall: tiles, rails, the split and resize chords, the layout
tree and the layout file, the once-a-second pane grade, the two-level
picker, focus, and relocating mouse reports into wall coordinates. The
native tiling design (2026-09-05) chose its own workspace and persistence
and refused to migrate the wall, so there were two walls and the browser
hub's grade made a third copy; the 2026-09-04 layout loss and the top two
backlog items were all wall-file problems.
**The shape after.** `mux [TARGET]` is strictly one session per
invocation: no picker, no tiles. It keeps the pump and redial, prediction,
the askpass popup, `-A`, mouse passthrough, drag select, scroll mode, the
end two-step on its own connection, the ssh handoff to QUIC, and the exit
code. `mux web` stays — it is the only phone and tablet surface — and
inherits ownership of `client.layout`, the layout file and the host grade,
since it is their only remaining consumer; `wall_host.planHostDiff` and
`webhub.applyList` collapse to the hub's copy. Dying with the wall:
`wallview.zig` (most), `wall_layout.zig`, `wall_picker.zig`, the wall parts
of `interact.zig` and `wall_host.zig`, the six `wall_test_*.zig`, e2e groups
07_wallcli, 09_hosts, 12_panes, 13_birth and the tile legs of 06_web, and
the wall invariants in CLAUDE.md. Backlog that closes with it: 32d168d1,
the seed half of 55ca6e6d, 2fc534d6, 05fc3090, 7b04a2ac, 68d4700e, most of
e3548ab1.
**Until then the wall is FROZEN: bug fixes only, no new wall features.**
Nothing is deleted before the GUI covers what the wall covers today, each
gap an issue: `d61fdc4c` muxg on macOS (a Mac has no client but the TUI —
nothing builds or runs the GUI there, though the Mac release recipe names
it), `8b16e26b` mouse selection and scroll wheel, `50ca9ba5` agent
forwarding per pane, `f3cf5785` ssh prompts through an askpass popup,
`fb4a0ee4` prediction overlay, `0b725c82` ending a session (the GUI can
detach and sends no `end_req`), `08ff372f` release bundling (the tarball
ships a dynamic `muxg` with none of its libraries and no message naming
the missing one).