aa247b30

docs: M14 closed — ssh→QUIC handoff

a73x 2026-08-12 06:34

Commit message

docs: M14 closed — ssh→QUIC handoff

Roadmap: M14 into "Now" (M1–M14) with its own verdict section; the
handoff bullet leaves the candidate list and the section becomes M15,
gaining the two items M14 created (the drain()/ECONNREFUSED fix and the
transport-recipe union).

decisions.md: the M14 section — verdict, the regrade table (3/3, each
catch legible at the check that predicted it), the LAN-box kill
criterion with its numbers, what shipped, the measured deadline and why
2000 stands, six findings, the bde1af5 errata, and what M14 banks.

README: the HOST line described `--via "ssh HOST muxd proxy"`, which has
not been what `mux HOST` does since Task 6. The shell-interpolation
warning stays — it is still true.

e2e.sh: (d)'s timing comment claimed the elapsed time IS the deadline.
It is the deadline plus launch overhead, measured at 26-129ms across 13
runs; the comment now says so, and says why the floor is still safe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

README.md

Old		New
@@ -53,12 +53,18 @@ mux HOST # attach; Ctrl-\ detaches, running it again reattaches
53	`muxd start` spawns the daemon detached and waits until it answers, so it	53	`muxd start` spawns the daemon detached and waits until it answers, so it
54	is safe to run every time — if one is already up it says so and exits 0.	54	is safe to run every time — if one is already up it says so and exits 0.
55		55
56	`mux HOST` is sugar for `mux --via "ssh HOST muxd proxy"` — ssh config	56	`mux HOST` runs `ssh HOST muxd endpoint` — ssh config (aliases, ports,
57	(aliases, ports, ProxyJump) all keeps working, since `mux` never parses	57	ProxyJump) all keeps working, since `mux` never parses HOST. That ssh
58	HOST. It is interpolated into a shell command, so HOST must be trusted:	58	fetches the daemon's QUIC port and key once and the session moves onto
59	feeding it unreviewed from an inventory file or a cloud API is handing that	59	QUIC, leaving no ssh process behind; the coordinates are cached under
60	source a shell. `--via` accepts any command that exposes the daemon's	60	`~/.cache/mux/hosts/`, so later attaches dial QUIC straight away and skip
61	socket on stdio, for custom socket paths or transports.	61	ssh entirely. Where inbound UDP is blocked, the attach lands over ssh one
		62	deadline (~2s) later, saying so in a single line, and the session is the
		63	same session either way. HOST is interpolated into a shell command, so
		64	it must be trusted: feeding it unreviewed from an inventory file or a
		65	cloud API is handing that source a shell. `--via` accepts any command that
		66	exposes the daemon's socket on stdio, for custom socket paths or
		67	transports.
62		68
63	If the link drops, the client reconnects and resumes by itself; `Ctrl-\`	69	If the link drops, the client reconnects and resumes by itself; `Ctrl-\`
64	gives up waiting.	70	gives up waiting.

docs/decisions.md

@@ @@ -2203,6 +2203,287 @@ before and after; no product code was left mutated. @@
+  mutation sweep, ASAN/valgrind over the QUIC tests, a `make gate` for
+  loaded runs, and the M12 pty-scenario timeout notes.
+## 2026-08-11 (M14 — the ssh→QUIC handoff)
+**Verdict: cleared, both legs.** `mux HOST` stopped being sugar for an
+ssh pipe and became a transport negotiation: one ssh to `muxd endpoint`,
+whose first stdout line is a mandatory announce of the daemon's QUIC port
+and key, then QUIC carries the session and the coordination ssh is
+reaped. The coordinates are cached, so the second attach spawns no ssh at
+all. Leg 1: the suite grew to **20 scenario checkpoints over 33
+convergence points**, both pinned as literals, with `SOAK_N=10 make soak`
+at **10/10 on the code that ships** (d9b17d5). Leg 2: three
+resurrections, three catches, each legible at the check that predicted
+it. The kill criterion held on the real LAN box (`192.168.0.109`) with
+`muxd` on the remote PATH as the one manual step. Spec:
+superpowers/specs/2026-08-11-m14-ssh-quic-handoff-design.md.
+### The regrade table
+One sitting, one tree, one set of binaries; `make build && make test &&
+make e2e` green before, green after, `git status` clean between every
+row. Each mutant was required to **compile** — a build failure
+masquerading as a kill grades nothing.
+| Resurrection | Predicted catch | Verdict |
+| --- | --- | --- |
+| `serviceObserver`'s `endpoint_req` arm deleted | unit: `Server: endpoint_req binds a listener lazily…` fails first and cheapest; e2e (a) fails too | **CAUGHT** (20s unit, 128s e2e) |
+| `endpointCmd`'s announce write deleted | e2e (a) fails at its bounded wait with the scenario's own message, the announce absent from the dump | **CAUGHT** (149s) |
+| `openHandoff`'s fallback stderr line deleted | e2e (d) fails its grep while (c)'s absent-grep control stays green in the same run | **CAUGHT** (156s) |
+**The observer arm — the unit layer answered first, as predicted, and
+named the missing thing itself.** `make test` failed at
+`src/server.zig:4634` with `return error.NoEndpointReply`, which is the
+whole diagnosis in one identifier, 20 seconds after the mutation. The
+e2e corroboration landed too, but **not at the assertion the prediction
+named**: (a) died at `e2e FAIL: cold handoff: QUIC took over, so ssh
+must be gone: pid 2931354 is still running 2s later`, not at the later
+cache/QUIC assertion, because that is the earliest check in (a) the
+mutation reaches. The prediction was right about the scenario and wrong
+about the line — which is finding 3 below, at assertion granularity
+rather than scenario granularity.
+**The announce write — the failure says why, not just that.** Both lines
+of the scenario's own message printed:
+```
+e2e FAIL: cold handoff: no marker in 25s — the attach never converged.
+          A client that blocks here read no announce line off the ssh pipe.
+muxd endpoint: starting…
+up (0.1s) pid=2940304
+mux: aborted before attaching
+```
+The diagnostic dump is what separates this kill from Task 7's
+stderr-routing kill, which produces the same first line: here `muxd
+endpoint`'s own preamble is present and the `endpoint …` announce is
+simply absent from it. The mutation as the plan first wrote it — moving
+the write below `return proxy.run(…)` — does not compile: Zig rejects
+unreachable code outright, so the sanctioned alternative was taken and
+the write deleted. Recorded because "each mutant must compile" and "make
+the announce unreachable" were, in this language, incompatible
+instructions.
+**The fallback line — the control proved it was a control.** (d) failed
+with `e2e FAIL: key mismatch printed no fallback line in 15s; its
+stderr was:` and an empty capture, in a run whose immediately preceding
+line was `e2e OK: a stale cache self-heals: one refetch (2231ms), no
+fallback line, real port cached`. (c) asserts that line is *absent* and
+(d) asserts it is *present*; deleting the line flipped exactly one of
+them. A pair like that is the only way an absent-grep assertion can be
+shown to be load-bearing rather than vacuous.
+### The kill criterion, on the LAN box (2026-08-11)
+`ubuntu@192.168.0.109`, a static musl `muxd` scp'd to the box and put on
+PATH — the one allowed manual step. Both ends zeroed first: remote
+daemon stopped, `~/.config/mux/key` removed, local cache entry removed,
+every local `mux` run under hermetic `XDG_*` so the developer's own key
+and cache were never touched. All three legs then ran with **zero
+further manual steps**.
+| Leg | Wall | What was observed |
+| --- | --- | --- |
+| Cold | **244ms** attach → first marker | ssh child (`ssh ubuntu@192.168.0.109 muxd endpoint`, pid 2959493) gone by `kill -0` at the moment the marker painted, while `mux` stayed alive and a second marker sent 8s later still painted |
+| Warm | **13ms** attach → marker | **no ssh child at any point** — polled across the whole session, not just the attach |
+| Blocked, warm cache | **4264ms** | fallback line printed, session landed over ssh, exit 0 |
+| Blocked, cold cache | **2177ms** | same line, one deadline + 177ms |
+The cold leg's announce was `endpoint 56407 8560…a307`, written to the
+cache verbatim; on the box `ss -uapn` showed `UNCONN 0.0.0.0:56407
+users:(("muxd",pid=17958,fd=6))` and `muxd stats` reported `clients=1
+deltas=3` — a session being served by deltas over a listener that did
+not exist before the attach asked for it. The blocked legs dropped
+inbound UDP with `iptables -A INPUT -p udp --dport 56407 -j DROP` and
+both printed the exact coordinates they had tried: `mux:
+quic://192.168.0.109:56407 unreachable, attaching over ssh`. The rule
+was removed on the way out and the removal proved by listing the chain
+(`-P INPUT ACCEPT`, nothing matching `--dport 56407`), not by trusting
+`iptables -D`'s exit status. Teardown was `muxd stop` over ssh: `muxd:
+stopped`, exit 0, then `pgrep -a muxd` empty, no UDP listener, socket
+file gone.
+**The two blocked numbers are the interesting result, and they are not
+the same case.** From a **cold** cache the blackhole costs **one**
+deadline, because the ssh that carried the announce is still open and
+*is* the fallback transport. From a **warm** cache it costs **two**: the
+cached dial spends a deadline, then the refetch over ssh returns the same
+(still-blocked) port and spends another before falling back. The spec's
+criterion says "within one deadline" and the cold leg meets it as
+written; the warm leg is recorded at 4264ms rather than reconciled,
+because it is what a user on a UDP-blocked network actually pays on
+every attach after their first. It is the direct cost of refusing
+negative caching, which remains the right call — network state changes,
+and a cached "UDP is blocked" would strand a user on ssh after their
+firewall was fixed.
+**A stale daemon from M10 had to be killed by pid, not stopped.** The
+box was still running the Aug-9 binary, which predates `stop_req`
+entirely, so `muxd stop` printed usage and exited 2. It was killed by
+its tracked pid and the absence confirmed with `pgrep`. Not part of the
+criterion — state zeroing, not attaching — but the shape is worth
+knowing: the stop verb is only as available as the binary on the far
+side, and a box that has been up across a milestone boundary may not
+have it.
+### What shipped
+`endpoint_req = 0x08` / `endpoint_reply = 0x89` (`src/protocol.zig`),
+with an `encode`/`decode` pair and golden wire bytes pinned as literals,
+following the rule M10 extracted — assert the literal, never the
+constant the code under test reads.
+The daemon's **lazy QUIC bind** (`src/server.zig`). `endpoint_req` is
+answered by binding a listener on demand if one is not already up, and
+the load-bearing arm is `serviceObserver` for M13's reason: `muxd
+endpoint` never attaches, so it is an observer for its whole life. A
+`quic_owned` flag records whether the listener was created by this path,
+so `deinit` hands back exactly what it took and a daemon started with an
+explicit `--quic` keeps ownership of its own.
+`src/handoff.zig` — one grammar, one place. Cache read/write, announce
+`format`/`parse`, the dial-host strip and the deadline constant all live
+in a pure module with no sockets and no processes, so the entire surface
+tests without a daemon. **The announce is mandatory in both
+directions**: `endpoint none` is an explicit negative rather than
+silence. The spec was amended (dd6c511) to say so after the alternative
+was found to be unresolvable — at the reading end, "the remote has no
+QUIC to offer" and "ssh is being slow" are the same zero bytes, and the
+client blocks on one line either way.
+`muxd endpoint` (`src/main.zig`) — `ensureForAttach`, then key
+resolution matching the daemon's exactly (`MUX_KEY_FILE`, then the
+default path, and only the default is ever created), then the announce
+as the **first** stdout bytes, then the byte-for-byte `muxd proxy` pump.
+Every unhappy path announces `none` and stays on ssh rather than
+panicking: a bug in a preamble must not take down a session ssh was
+about to carry perfectly well.
+Client-side, the handoff sits at the **Transport** layer
+(`src/client.zig`), not above it, so a reconnect re-runs it rather than
+resurrecting a dead recipe. Cache first, ssh second. The fallback line
+is attach-only (`report_fallback`) so a flapping link cannot turn it
+into a log. `readAnnounceAbortable` has **no timeout by design** — it is
+bounded by ssh itself, the same trust `--via` already extends to the
+command it spawns — and it honors the carry contract, holding
+non-abort bytes typed during the wait rather than dropping them.
+### The deadline, pinned from measurement (Task 4, 2026-08-11)
+**There is no fast-failure case.** A wrong PSK against a live listener
+ran 2037ms of a 2000ms budget, 3049 of 3000, 8031 of 8000: mutual auth
+means the listener does not answer a peer it cannot authenticate, so
+silence is all the client ever gets. A blackholed UDP port did the same
+(4001 of 4000, 8001 of 8000). So did an unbound port **on loopback**,
+ms of 2000 — the ICMP refusal is real, but it lands on the `sendto`
+inside `Client.drain`, whose `catch return` discards it, so the
+`ConnectionRefused` branch in `readable` never sees it. Every failure
+runs the budget out, and the deadline is the only thing that ends any of
+them.
+That fixes the shape of the trade, and 2000 stands. Below it: a real
+handshake is **4ms** local, **6.9ms** cold attach on the M8 LAN, and
+**234ms** at 75ms RTT — 3.1× RTT, because every fresh Initial costs a
+Retry round trip — so 2000 covers RTT to ~645ms, past any terrestrial
+link. Above it: with negative caching refused, every attach on a
+UDP-blocked network pays this in full, so it wants to be as low as the
+floor allows. 1000 would halve the tax and also halve the RTT ceiling to
+~320ms — buying latency on a degraded network at the price of abandoning
+QUIC paths that would have worked. Wrong way round.
+### Findings
+- **A wedged `zig test` step prints nothing at all.** Not a truncated
+  report, not a partial list — nothing, for the whole timeout. So the
+  ordering rule from M13 ("a regrade catch must be legible") needs a
+  second clause: a legible catch must also be ordered **before** any
+  test the same regression can hang. The ownership pin is the case that
+  taught it. Left where it was, the mutation produced a 600-second empty
+  wedge with no output to read; moved ahead of the test that hangs, the
+  same mutation produces 8 named failures in 14 seconds.
+- **The suite was running the developer's login shell.** Hermetic
+  `XDG_*` homes cannot make a suite hermetic while it still executes
+  arbitrary user rc files: a zsh plugin manager cloning a repository at
+  shell startup ate a convergence window and failed a scenario that had
+  nothing to do with it. `SHELL` is now pinned suite-wide. The general
+  form is that "hermetic" has to include the *code* the environment
+  causes to run, not only the paths it points at.
+- **A mutation only proves the earliest scenario that catches it.**
+  First-failure exit means every later scenario is untested against that
+  break, and a campaign that reports "five kills" is silently reporting
+  five *first* kills. Task 7's five left scenario (e) and (c)'s
+  self-heal arm unproven until the accounting was audited scenario by
+  scenario; re-running against each in turn brought the real total to
+  eight. Kill counts are claims about ordering as much as about
+  coverage.
+- **Review prescriptions remain fallible in both directions, and the
+  pipeline caught both.** A spec reviewer prescribed a "complementary"
+  post-hoc liveness check on the reaped ssh; it was **unfalsifiable** —
+  dead processes do not resurrect, and a reused pid presents as alive —
+  so it could only ever pass or lie, and an independent review proved
+  that and deleted it. The same review corrected the lead's own `set
+  +e` alignment. M13's rule holds: an implementer rejecting a bad
+  prescription with evidence is the process working in the direction it
+  is easier to forget it runs.
+- **`waitReady` answers the detach chord *during* a dial.** That is
+  correct behavior — a user who presses Ctrl-\ while waiting asked to
+  stop — but it means any scenario built to spend a deadline must hold
+  its detach bytes out past the budget, or the client aborts early and
+  the scenario passes having tested nothing. A timing assertion whose
+  subject can be cancelled by its own fixture is not an assertion.
+- **A fixed port must dodge the kernel's ephemeral range.** Read
+  `/proc/sys/net/ipv4/ip_local_port_range` and stay out of it: a port
+  chosen for being dead can come alive mid-suite when something
+  unrelated is handed it, and the scenario that depended on nothing
+  listening there then fails for a reason no one will find by reading
+  it.
+### Errata
+The commit message of bde1af5 states `kill -0`'s polarity backwards. The
+code and its comments are correct; only the prose in that message is
+wrong. Recorded here rather than fixed by rewriting history, which is
+the standing preference — a wrong sentence with a correction beside it
+is more honest than a history that never contained it.
+### Process note
+Subagent pipeline as M11–M13: one implementer and two reviewers per
+task, quality and spec, every new assertion written mutation-first. One
+new shape is worth recording. A spec reviewer was blocked by the
+permission classifier on `make e2e`, three times, situationally rather
+than by rule. It did not work around the block, invent an equivalent
+command, or report a verdict it had not earned: it stopped, reverted its
+own edit, and handed the exact change to the implementer to run. That is
+the correct response to a tool refusal, and it is recorded because the
+tempting alternatives all produce a green report that means nothing.
+### Banked by M14
+- **`drain()` swallows `ECONNREFUSED`** — the Task 4 incidental, and an
+  M15 candidate. Fixing it would make unreachable ports fail in ~1 RTT
+  instead of a full deadline. **Re-pin, do not delete:** e2e (d)'s
+  `elapsed >= floor` bound and `client.zig`'s dead-coordinates test
+  bound both currently lean on the quirk, and both would need re-timing
+  in the same change.
+- **The transport recipe as a `union(enum)`** — from Task 6's quality
+  review. `Transport.open` now takes four mutually-exclusive nullables,
+  two of them sharing a type; `union(enum) { sock, via, quic, hand }`
+  makes the invariant unstatable-wrong rather than documented. M15.
+- **Per-client keys, certificates and TOFU** — still parked, unchanged
+  since M10. The handoff hands out the *same* shared key over ssh, which
+  is exactly the trust model M10 chose and no worse; per-client
+  credentials remain the thing that would change it.
+- **Negative caching of "UDP blocked"** — still refused, by design, and
+  the LAN box's 4264ms warm-blocked attach is now the measured price of
+  that refusal rather than an estimate of it.
+- **The WAN-box RTT on record disagrees with itself** — ~290ms in one
+  place against the 15.7–16.5ms baseline at decisions.md:314. Verify
+  against the live box before the next measurement leans on either.
+- **Scenario (e) has two unexercised assertions** — its absent-fallback
+  grep and its no-budget bound were never driven red. Cheap candidates
+  for the next regrade, and cheaper than the audit that found them.
+## Open (owed by later milestones)
+- Scrollback retention *tuning*. The policy itself was decided in M1 and

docs/roadmap.md

Old		New
@@ -5,7 +5,7 @@ The forward view, one item per line, ranked. History and evidence live in
5	this file at each milestone close and whenever the queue reorders; the	5	this file at each milestone close and whenever the queue reorders; the
6	queue's order is set by the user, not by this file.	6	queue's order is set by the user, not by this file.
7		7
8	Now: M1–M13 complete; `v0.0.1-2` published as a Linux tarball; in	8	Now: M1–M14 complete; `v0.0.1-2` published as a Linux tarball; in
9	field trial on real VMs, and the trial is now producing the queue.	9	field trial on real VMs, and the trial is now producing the queue.
10	Trial feedback outranks everything below — what actually hurts in use is	10	Trial feedback outranks everything below — what actually hurts in use is
11	better data than any of this ranking, and the proof is that the three	11	better data than any of this ranking, and the proof is that the three
@@ -117,18 +117,45 @@ truncation is reserved for `muxd start`; and three review prescriptions
117	were rejected with evidence, which is the adversarial flow running in	117	were rejected with evidence, which is the adversarial flow running in
118	the direction it is easier to forget.	118	the direction it is easier to forget.
119		119
120	## M14 candidates — and still outranked by trial feedback	120	## M14 — ssh→QUIC handoff — complete
121		121
122	The trial-friction bundle that used to head this section shipped as M13.	122	Verdict: cleared, both legs. `mux HOST` no longer chooses a
123	What remains is what was already ranked behind it, plus two items M13	123	transport; it earns one. An ssh to `muxd endpoint` fetches the daemon's
124	created. Nothing here is a field finding; the next one that arrives	124	QUIC port and key as a mandatory one-line announce, the session moves
125	outranks all of it.	125	onto QUIC and the coordination ssh is reaped, and the coordinates are
126		126	cached so every later attach dials direct with no ssh process at all.
127	- ssh→QUIC handoff: `muxd endpoint` prints port+key over ssh;	127	Leg 1 held at 20 scenario checkpoints over 33 convergence points,
128	client caches both, tries QUIC first under a ~2s deadline, and the	128	both pinned as literals, with `SOAK_N=10 make soak` at **10/10 on the
129	coordination ssh stays alive as the fallback proxy. Full sketch —	129	code that ships** (d9b17d5). Leg 2 is three for three, each catch
130	including the key-outlives-the-daemon model and the stale-cache	130	legible at the check that predicted it. The kill criterion held on the
131	invariant — in the M10 spec's context section.	131	real LAN box with the one allowed manual step: cold attach 244ms
		132	with the ssh child observed dead while the session went on answering,
		133	warm attach 13ms with zero ssh spawned, and a blackholed UDP port
		134	still landing the session over ssh, one deadline and one line later
		135	(decisions.md, M14, which carries the numbers and the regrade table).
		136	En route: the announce had to become mandatory in both directions
		137	because silence and a slow ssh are indistinguishable at the reading end
		138	— `endpoint none` is now an explicit negative — and the deadline was
		139	pinned from measurement showing there is no fast-failure case at
		140	all, not even a refused loopback port.
		141
		142	## M15 candidates — and still outranked by trial feedback
		143
		144	What remains is what was already ranked behind M13 and M14, plus two
		145	items M14 created. Nothing here is a field finding; the next one that
		146	arrives outranks all of it.
		147
		148	- `drain()` swallows `ECONNREFUSED` — created by M14 and the
		149	cheapest win on the list. On a connected UDP socket the queued ICMP
		150	refusal is consumed by the `sendto` in `Client.drain`, whose `catch
		151	return` discards it, so the `ConnectionRefused` branch never fires and
		152	an unreachable port costs the full 2s deadline instead of ~1 RTT.
		153	Fixing it re-times two existing bounds rather than deleting them
		154	(decisions.md, M14).
		155	- The transport recipe as a `union(enum)` — also from M14.
		156	`Transport.open` takes four mutually-exclusive nullables, two of them
		157	sharing a type; `union(enum) { sock, via, quic, hand }` would make the
		158	invariant unstatable-wrong instead of merely documented.
132	- First-backoff tuning: a reconnect after a tear can pay a 200ms–2s	159	- First-backoff tuning: a reconnect after a tear can pay a 200ms–2s
133	backoff step; measured ~4x win available on fast links, ~2x on slow	160	backoff step; measured ~4x win available on fast links, ~2x on slow
134	(decisions.md, M7).	161	(decisions.md, M7).

test/e2e.sh

Old		New
@@ -2581,9 +2581,9 @@ HT0=$(date +%s%N)
2581	SHELL=/bin/sh XDG_RUNTIME_DIR="$HRUN2" PATH="$HPATH" timeout 40 \	2581	SHELL=/bin/sh XDG_RUNTIME_DIR="$HRUN2" PATH="$HPATH" timeout 40 \
2582	"$MUX" "127.0.0.1" > "$OUT.h4" 2> "$OUT.h4.err" &	2582	"$MUX" "127.0.0.1" > "$OUT.h4" 2> "$OUT.h4.err" &
2583	H4PID=$!	2583	H4PID=$!
2584	# Timed to the LINE rather than to the client's exit: the line is printed	2584	# Timed to the LINE rather than to the client's exit, so no session sleeps
2585	# the moment the budget runs out, so the elapsed time here IS the deadline	2585	# are folded in. The elapsed time is the dial's budget plus launch overhead
2586	# the dial spent, with no session sleeps folded into it.	2586	# (measured 26-129ms), which only inflates it — the floor is safe.
2587	# Only the stderr is printed on failure, deliberately: the complaint is	2587	# Only the stderr is printed on failure, deliberately: the complaint is
2588	# about a line that is missing from it, and the capture beside it is a	2588	# about a line that is missing from it, and the capture beside it is a
2589	# screenful of escape sequences that would bury the answer.	2589	# screenful of escape sequences that would bury the answer.