024048ae
docs: M6 transport verdict — the swap held, measured; milestone complete
a73x 2026-08-08 14:08
Commit message
README.md
| Old | New | ||
|---|---|---|---|
| @@ -5,13 +5,17 @@ authoritatively in a daemon and replicated in the client — state sync | |||
| 5 | instead of escape-sequence replay. See `docs/handoff.md` for the design | 5 | instead of escape-sequence replay. See `docs/handoff.md` for the design |
| 6 | and `docs/decisions.md` for decisions made. | 6 | and `docs/decisions.md` for decisions made. |
| 7 | 7 | ||
| 8 | Status: **M5 — two clients.** All handoff milestones complete; all three | 8 | Status: **M6 — transport.** All handoff milestones complete; all kill |
| 9 | kill criteria cleared. The prototype's two founding questions are both | 9 | criteria cleared. The prototype's two founding questions are both |
| 10 | answered yes: ghostty-vt serves as an authoritative headless grid in a | 10 | answered yes: ghostty-vt serves as an authoritative headless grid in a |
| 11 | daemon without forking it, and detach/reattach as state sync is correct | 11 | daemon without forking it, and detach/reattach as state sync is correct |
| 12 | and fast — a killed client reattaches into a live full-screen `nvim` | 12 | and fast — a killed client reattaches into a live full-screen `nvim` |
| 13 | session in ~5 ms from one snapshot, and steady-state delta traffic is 1% | 13 | session in ~5 ms from one snapshot, and steady-state delta traffic is 1% |
| 14 | of the snapshot-equivalent cost. | 14 | of the snapshot-equivalent cost. M6 answered the third: transport is a |
| 15 | swap, not a redesign. Measured over a real WAN link, the protocol adds | ||
| 16 | ~4 ms to a raw ssh byte round-trip (constant, not a multiple of it), and | ||
| 17 | reattach-after-kill costs ~1.3× the round trip in protocol time on top | ||
| 18 | of ssh's own channel setup. | ||
| 15 | 19 | ||
| 16 | Requires Zig 0.15.x (ghostty pin); the Makefile points at the pinned | 20 | Requires Zig 0.15.x (ghostty pin); the Makefile points at the pinned |
| 17 | toolchain, override with `make ZIG=...`. | 21 | toolchain, override with `make ZIG=...`. |
| @@ -24,6 +28,16 @@ toolchain, override with `make ZIG=...`. | |||
| 24 | ./zig-out/bin/muxd stats # wire stats: deltas vs snapshot bytes | 28 | ./zig-out/bin/muxd stats # wire stats: deltas vs snapshot bytes |
| 25 | make bench # typing-workload byte-ratio measurement | 29 | make bench # typing-workload byte-ratio measurement |
| 26 | 30 | ||
| 31 | Attach over a network: | ||
| 32 | |||
| 33 | ./zig-out/bin/mux --via "ssh host /path/muxd proxy --sock /path/muxd.sock" | ||
| 34 | |||
| 35 | `--via` runs the protocol over any command that exposes the daemon's | ||
| 36 | socket on its stdio — `muxd proxy` is one such command, and it is a | ||
| 37 | frame-agnostic byte pump that contains no protocol knowledge at all. | ||
| 38 | `test/wan.sh` is the env-driven harness that measured this over a real | ||
| 39 | WAN link (see the script header for the variables it needs). | ||
| 40 | |||
| 27 | Multiple mux clients may attach to one session; the grid follows the most | 41 | Multiple mux clients may attach to one session; the grid follows the most |
| 28 | recent attacher/resize (latest wins). | 42 | recent attacher/resize (latest wins). |
| 29 | 43 | ||
docs/decisions.md
| Old | New | ||
|---|---|---|---|
| @@ -63,6 +63,9 @@ | |||
| 63 | confusing connection error. | 63 | confusing connection error. |
| 64 | - **Blocking frame I/O.** One local client on a Unix socket; a stuck client | 64 | - **Blocking frame I/O.** One local client on a Unix socket; a stuck client |
| 65 | can stall the daemon. Buffered nonblocking I/O is owed by M4 (network). | 65 | can stall the daemon. Buffered nonblocking I/O is owed by M4 (network). |
| 66 | **Superseded in M6** for the write half (queued per-client sends with a | ||
| 67 | cap); reads stay blocking and POLLIN-gated, deliberately — see the M6 | ||
| 68 | section. | ||
| 66 | - **Detach chord: Ctrl-\ (0x1c).** No keybinding layer in the prototype. | 69 | - **Detach chord: Ctrl-\ (0x1c).** No keybinding layer in the prototype. |
| 67 | - **Render: home + ED(2) full repaint under CSI ?2026 sync.** Wasteful by | 70 | - **Render: home + ED(2) full repaint under CSI ?2026 sync.** Wasteful by |
| 68 | design (deltas are M4); sync-output makes it artifact-free on modern | 71 | design (deltas are M4); sync-output makes it artifact-free on modern |
| @@ -194,9 +197,12 @@ | |||
| 194 | - **Known costs and edges, accepted for the prototype:** blocking | 197 | - **Known costs and edges, accepted for the prototype:** blocking |
| 195 | per-client writes mean one stalled client can head-of-line-block all | 198 | per-client writes mean one stalled client can head-of-line-block all |
| 196 | others (fix = buffered/nonblocking writes, owed before any network | 199 | others (fix = buffered/nonblocking writes, owed before any network |
| 197 | transport). Simultaneous connects funnel through 4 observer slots | 200 | transport). **Superseded in M6:** client sends are queued per client |
| 198 | before promotion, so >4 clients connecting in the same instant see | 201 | and flushed non-blocking, with a cap that drops a stalled peer instead |
| 199 | unexplained EOFs (connect-then-confirm serializes fine in practice). A | 202 | of waiting on it; the M6 section records the design. Simultaneous |
| 203 | connects funnel through 4 observer slots before promotion, so >4 | ||
| 204 | clients connecting in the same instant see unexplained EOFs | ||
| 205 | (connect-then-confirm serializes fine in practice). A | ||
| 200 | client's corrupt-delta resync re-attaches at its own size, which — by | 206 | client's corrupt-delta resync re-attaches at its own size, which — by |
| 201 | latest-wins — resizes the session for everyone. Scroll paging divides | 207 | latest-wins — resizes the session for everyone. Scroll paging divides |
| 202 | daemon history rows by the local tty height, so page counts are | 208 | daemon history rows by the local tty height, so page counts are |
| @@ -216,6 +222,208 @@ | |||
| 216 | checkpoint/restore, which handoff §6 explicitly defers as a different | 222 | checkpoint/restore, which handoff §6 explicitly defers as a different |
| 217 | problem (process location, not terminal rendering). | 223 | problem (process location, not terminal rendering). |
| 218 | 224 | ||
| 225 | ## 2026-08-07 (M6) | ||
| 226 | |||
| 227 | - **Client writes are queued and never stall the daemon.** Each client | ||
| 228 | slot carries a pending byte buffer; sends `appendFrame` into it and | ||
| 229 | flush opportunistically with `MSG_DONTWAIT | MSG_NOSIGNAL`, with | ||
| 230 | POLLOUT requested only while the queue is non-empty. A queue past | ||
| 231 | `pending_cap` (8 MiB, a `Server` field so tests can shrink it) drops | ||
| 232 | the client rather than waiting on it: one stalled WAN peer must not | ||
| 233 | head-of-line-block the session, and unbounded buffering would trade a | ||
| 234 | stall for an OOM. Stats count bytes when *queued*, so "sent" now means | ||
| 235 | "accepted into the queue" — the cap is what bounds the lie (no client | ||
| 236 | can be more than one cap ahead of the truth). Reads stay blocking and | ||
| 237 | POLLIN-gated, deliberately: input frames are tiny and the observed | ||
| 238 | hazard is entirely write-side (snapshots). Observer replies stay on | ||
| 239 | blocking `writeFrame` — they are local one-shot tools. | ||
| 240 | - **exit_status is queued like everything else, then drained under a | ||
| 241 | 250ms budget** at shutdown. It is the one frame with nothing behind it | ||
| 242 | to carry it out, so the queue alone would lose it; the bounded drain | ||
| 243 | delivers it unless the peer is fully stalled, and refuses to hang the | ||
| 244 | daemon's exit on a peer that is. A client that misses it still learns | ||
| 245 | the session ended, from EOF. | ||
| 246 | - **Session epoch: a random nonzero u64 per daemon instance**, 0 | ||
| 247 | reserved for "none". It rides in attach (now 20 bytes: cols, rows, | ||
| 248 | have_seq, have_epoch) and in the snapshot prefix (now 24: seq, | ||
| 249 | history, cols, rows, epoch). The delta fast path requires | ||
| 250 | `have_epoch == self.epoch` as well as a serviceable seq, so a client | ||
| 251 | can never be served a delta belonging to a different daemon instance. | ||
| 252 | **Deltas deliberately carry no epoch**: a delta stream cannot outlive | ||
| 253 | the instance that opened it — the connection dies with the daemon, and | ||
| 254 | the proxy exits rather than reconnecting for exactly this reason — so | ||
| 255 | the only place an epoch can be checked is where a client asserts what | ||
| 256 | it already has. Distinctness is pinned by a daemon-restart test, not | ||
| 257 | by trusting the RNG in prose. Both payload widenings are breaking wire | ||
| 258 | changes; binaries upgrade in lockstep, consistent with the standing | ||
| 259 | no-negotiation policy. | ||
| 260 | - **`muxd proxy` + `mux --via CMD`: the transport is an opaque byte | ||
| 261 | pipe.** `muxd proxy` is a bidirectional pump between its stdio and the | ||
| 262 | local `muxd.sock` that imports only `std` — no protocol import, no | ||
| 263 | frame parsing, and the build graph enforces it. `mux --via CMD` spawns | ||
| 264 | `/bin/sh -c CMD` and speaks the existing framed protocol over the | ||
| 265 | child's stdin/stdout (stderr inherited, so ssh's errors reach the | ||
| 266 | user). Any command that exposes the daemon socket over stdio works; | ||
| 267 | ssh is just the first one. | ||
| 268 | |||
| 269 | ### The transport verdict | ||
| 270 | |||
| 271 | **"Transport is a swap, not a redesign" — HELD, structurally and by | ||
| 272 | measurement.** | ||
| 273 | |||
| 274 | Structurally: the protocol crossed to a remote link with *zero* framing | ||
| 275 | changes. `readExact` already looped, so arbitrary chunk boundaries — | ||
| 276 | which an SSH channel produces constantly and a Unix socket almost never | ||
| 277 | did — needed nothing. Payloads stayed unstructured, so the M4 msgpack | ||
| 278 | tripwire did not trip. The only *protocol* work in M6 was the two debts | ||
| 279 | the log had already recorded as owed before any network (queued writes, | ||
| 280 | epoch). | ||
| 281 | |||
| 282 | The network did force two client-side changes, named here first because | ||
| 283 | they are the counterexamples the claim has to survive. Neither touches | ||
| 284 | the wire. (1) **Torn and EOF'd frames now report "connection to muxd | ||
| 285 | lost"** instead of surfacing an error trace: over a Unix socket a | ||
| 286 | mid-frame tear was near-unreachable, and over ssh it is the ordinary | ||
| 287 | failure — the transport dying mid-snapshot is a message, not a crash. | ||
| 288 | (2) **Alt-screen entry is deferred until the first frame arrives.** | ||
| 289 | Entering at setup would erase whatever the `--via` command wrote to its | ||
| 290 | inherited stderr — ssh reports auth and connection failures hundreds of | ||
| 291 | milliseconds after spawn — and would blank the screen for the whole of a | ||
| 292 | hang like `mux --via "sleep 30"`. Both are the client learning that its | ||
| 293 | transport can now fail slowly, visibly, and in someone else's words. | ||
| 294 | That is a UX consequence of distance, not a redesign of the protocol, | ||
| 295 | which is exactly the distinction the verdict claims. | ||
| 296 | |||
| 297 | Empirically, measured by `test/wan.sh` against a real WAN box through an | ||
| 298 | ssh jump host (figures in milliseconds, medians): | ||
| 299 | |||
| 300 | - **Link baseline** (raw byte round-trip through `ssh cat`, warm | ||
| 301 | channel): **15.7–16.5ms**. | ||
| 302 | - **Keystroke echo through the whole stack** (ssh → proxy → daemon → | ||
| 303 | pty → bash echo → engine → delta → proxy → ssh → client paint): | ||
| 304 | **baseline + ~4.2ms**. That figure is *constant and | ||
| 305 | RTT-independent* — under `netem delay 75ms loss 1%` the baseline rose | ||
| 306 | to 91.3ms and echo to 95.4ms, the same ~4ms of protocol cost on a link | ||
| 307 | nearly six times slower. The protocol adds a fixed pump tick, not a | ||
| 308 | multiple of the round trip. | ||
| 309 | - **Reattach after `kill -9`**: wall clock **56.7–59.5ms** on the fast | ||
| 310 | link, decomposing into a **35.3–37.5ms** ssh channel-open/exec floor | ||
| 311 | (measured, not assumed — `viafloor` in the harness) and a | ||
| 312 | **19.8–22.0ms protocol share** — **1.2–1.4× round-trip** on the clean | ||
| 313 | link. Under netem the same share works out to **~1.05×**, because the | ||
| 314 | protocol's cost is roughly fixed while the round trip it is measured | ||
| 315 | against grew; the clean-link ratio is the one to quote, being the | ||
| 316 | larger and therefore the harder test. | ||
| 317 | - **State correctness over the WAN**: the pre-kill marker was present in | ||
| 318 | the *first paint* of every rep, clean link and netem alike. | ||
| 319 | |||
| 320 | **Kill criterion (a), structural — PASS.** The remote path works with | ||
| 321 | the proxy containing zero protocol knowledge. | ||
| 322 | |||
| 323 | **Kill criterion (b), measured — PASS on both halves.** Echo: 4.2ms | ||
| 324 | against a budget of baseline + 120ms, margin **+116ms**. Reattach: the | ||
| 325 | protocol share, 1.2–1.4× round-trip on the clean link (~1.05× under | ||
| 326 | netem), against a 2× budget. | ||
| 327 | |||
| 328 | **The reattach ruling, recorded so the gate is never mistaken for a | ||
| 329 | softened threshold.** The plan's wording is "within ~2×RTT *of the | ||
| 330 | attach request*", so the criterion governs the cost the protocol adds. | ||
| 331 | The wall-clock reading fails on any fast link *regardless of protocol | ||
| 332 | design*: ssh's channel-open-plus-exec floor alone measured 2.2–2.4× the | ||
| 333 | round-trip on this link, past the entire budget before one protocol byte | ||
| 334 | moves. A gate that fails no matter what the code does cannot falsify | ||
| 335 | anything, so it is not the gate. | ||
| 336 | |||
| 337 | **The order this happened in, disclosed because it is the kind of thing | ||
| 338 | a reader should not have to reconstruct:** the harness gated the strict | ||
| 339 | wall-clock reading first, and those runs printed FAIL. The gate was | ||
| 340 | changed to the protocol share *after* and *because of* the `viafloor` | ||
| 341 | decomposition, which is what showed the failure to be ssh's channel | ||
| 342 | setup rather than anything the protocol did. A threshold moved after | ||
| 343 | seeing a red result deserves the suspicion it attracts; what defends | ||
| 344 | this one is that the decomposition is itself measured, the wall clock is | ||
| 345 | still printed and still reads FAIL on a fast link, and the reasoning | ||
| 346 | above holds independently of which number anyone hoped for. | ||
| 347 | |||
| 348 | Both readings stay on the record — `wan.sh` prints the wall clock and | ||
| 349 | its decomposition and gates on the protocol share — because the wall | ||
| 350 | clock is what a user actually waits through. Closing that gap means | ||
| 351 | killing the per-attach channel setup, | ||
| 352 | not the protocol: QUIC 0-RTT or a reused connection is the recorded | ||
| 353 | path, and it is transport work, which is the verdict restated. | ||
| 354 | |||
| 355 | - **Proxy write-side head-of-line blocking — known, measured, | ||
| 356 | accepted.** The proxy is a byte pump with no notion of priority and no | ||
| 357 | write buffering of its own, so a flooding session shares the pipe with | ||
| 358 | a keystroke's echo. Measured under a ~100 deltas/s flood (one | ||
| 359 | full-screen-scrolling line every 10ms): echo degraded by +0.5–5ms on | ||
| 360 | the clean link and +3–15ms under netem. It is bounded rather than | ||
| 361 | unbounded because the daemon's `pending_cap` terminates a hopeless | ||
| 362 | client honestly instead of queueing forever. The fix, if it ever | ||
| 363 | matters, is buffered non-blocking writes in the proxy — the same | ||
| 364 | change the daemon just made. | ||
| 365 | - **Ctrl-C had been dead in every script-started session since M1 — | ||
| 366 | found by the WAN run, not by any test.** A non-interactive shell sets | ||
| 367 | SIGINT/SIGQUIT to `SIG_IGN` for anything it backgrounds with `&`, | ||
| 368 | which is how every script starts the daemon (`e2e.sh`, `wan.sh`, any | ||
| 369 | deploy). `SIG_IGN` is the one disposition that survives `exec`, so it | ||
| 370 | rode through forkpty into the session shell, and a shell keeps | ||
| 371 | signals ignored-on-entry ignored for the jobs it spawns. Symptoms | ||
| 372 | looked healthy the whole way down: `isig` on, `^C` echoed, foreground | ||
| 373 | process group correct — and `sleep 300` immune. Fixed by resetting | ||
| 374 | both to `SIG_DFL` in the pty child before `exec`. The test aims the | ||
| 375 | signal at a *job of* the session shell, not at the shell itself: an | ||
| 376 | interactive shell catches SIGINT to abandon the current line, so it | ||
| 377 | abandons the marker either way and cannot discriminate — that form was | ||
| 378 | verified to pass against the unfixed code. **SIGPIPE is reset in the | ||
| 379 | same place for the same reason**, as hardening rather than a fix: the | ||
| 380 | daemon ignores SIGPIPE for its own sockets, and today only | ||
| 381 | initialization order keeps that ignore from riding into session shells | ||
| 382 | by the identical survives-exec route. Resetting it beside INT and QUIT | ||
| 383 | makes the property hold regardless of order, which is what the SIGINT | ||
| 384 | bug taught — a disposition inherited across exec is a latent defect | ||
| 385 | whether or not the current call order happens to hide it. | ||
| 386 | - **Two test limits, recorded rather than papered over.** (1) The | ||
| 387 | proxy's 300KB backpressure test pins byte-exact transfer in both | ||
| 388 | directions but *cannot* kill a partial-write mutation: blocking | ||
| 389 | stream-socket writes never return short, so the `writeAll` loop is | ||
| 390 | guarding signal interruption, not partial writes, and the mutation has | ||
| 391 | no observable behaviour to catch. (2) Zig's runtime installs a noop | ||
| 392 | SIGPIPE handler by default, so the explicit `SIG_IGN` installs are | ||
| 393 | defence in depth — the EPIPE error paths are reachable without them. | ||
| 394 | What the explicit ignore really buys is that `SIG_IGN` survives exec | ||
| 395 | and a handler does not (which is why `client.zig` installs its ignore | ||
| 396 | only *after* spawning the transport child). `std.options.keep_sigpipe` | ||
| 397 | staying at its default is a dependency nothing pins. | ||
| 398 | - **The WAN box is ephemeral and lives entirely in the environment** | ||
| 399 | (`MUX_WAN_SSH`, `MUX_WAN_SCP`, `MUX_WAN_HOST`, `MUX_WAN_NETEM`), never | ||
| 400 | hardcoded in a committed file. `wan.sh` refuses with a usage message | ||
| 401 | when they are unset. | ||
| 402 | - **Banked cleanup (post-M6):** proxy write-side head-of-line blocking | ||
| 403 | (buffered proxy writes); no test covers the `--via` child's lifecycle | ||
| 404 | (kill/wait on exit, ssh dying mid-session); client reconnect-with- | ||
| 405 | epoch logic — `session_epoch` is parsed from the snapshot prefix and | ||
| 406 | then unused, so the epoch machinery is only half-exercised by the | ||
| 407 | shipped client; `std.net.Address.initUnix` is bounded by `sun_path` | ||
| 408 | (108 bytes) regardless of the 280-byte buffers around it, so the test | ||
| 409 | suite is unrunnable from a checkout nested deeper than that allows — | ||
| 410 | this bit two independent reviewers, which makes it a real defect and | ||
| 411 | not a footnote; `wan.sh`'s netem stanza adds delay on egress only, so | ||
| 412 | `delay 75ms` is ~75ms of added round trip and not ~150ms (the script | ||
| 413 | says so; the plan's parenthetical said otherwise). | ||
| 414 | - **Owed before this is more than a prototype, in order:** TLS or QUIC | ||
| 415 | for any deployment that is not tunnelled through SSH — `--via` borrows | ||
| 416 | ssh's authentication and encryption entirely, and has none of its own; | ||
| 417 | **reconnect-with-epoch** in the client, which is what turns the epoch | ||
| 418 | from a correctness fence into a feature (resume the stream instead of | ||
| 419 | re-attaching from zero) and is also what would let a transport survive | ||
| 420 | a blip; and **prediction / local echo**, deferred since the handoff and | ||
| 421 | now holding a measured baseline to be judged against — handoff §5 lists | ||
| 422 | it as post-prototype and conservative by default (predict in line mode, | ||
| 423 | fall back to server-confirmed inside TUIs), and the ~4ms of protocol | ||
| 424 | cost measured here is what a predictor must beat to be worth its | ||
| 425 | complexity on a link this good. | ||
| 426 | |||
| 219 | ## Open (owed by later milestones) | 427 | ## Open (owed by later milestones) |
| 220 | 428 | ||
| 221 | - Scrollback retention *tuning*. The policy itself was decided in M1 and | 429 | - Scrollback retention *tuning*. The policy itself was decided in M1 and |
| @@ -231,6 +439,11 @@ All five milestones are complete and all three kill criteria are | |||
| 231 | cleared. The two questions handoff §0 says this prototype exists to | 439 | cleared. The two questions handoff §0 says this prototype exists to |
| 232 | answer are both answered yes. | 440 | answer are both answered yes. |
| 233 | 441 | ||
| 442 | *(Written at the close of M5. M6 added a sixth milestone and a fourth | ||
| 443 | criterion — the transport kill criterion, both halves cleared and | ||
| 444 | measured over a real WAN link — without disturbing anything below. See | ||
| 445 | the M6 section and its transport verdict.)* | ||
| 446 | |||
| 234 | - **M1 kill criterion — cleared.** The grid is extracted from upstream | 447 | - **M1 kill criterion — cleared.** The grid is extracted from upstream |
| 235 | ghostty-vt with no fork: an unmodified pinned dependency, driven | 448 | ghostty-vt with no fork: an unmodified pinned dependency, driven |
| 236 | headlessly, serialized through its own formatters. | 449 | headlessly, serialized through its own formatters. |
| @@ -263,3 +476,7 @@ Everything downstream of these answers — network transport, mesh, | |||
| 263 | multiplayer, panes, checkpoint/restore — remains deliberately out of | 476 | multiplayer, panes, checkpoint/restore — remains deliberately out of |
| 264 | scope, and the known gaps owed before any of it are recorded above (no | 477 | scope, and the known gaps owed before any of it are recorded above (no |
| 265 | session epoch, blocking per-client writes, hand-rolled wire format). | 478 | session epoch, blocking per-client writes, hand-rolled wire format). |
| 479 | |||
| 480 | *(M6 closed the first two of those three and put the protocol on a real | ||
| 481 | WAN link; the hand-rolled wire format stands, and its tripwire has still | ||
| 482 | not tripped. See the M6 section and its transport verdict.)* | ||
docs/superpowers/plans/2026-08-07-m6-transport.md
| Old | New | ||
|---|---|---|---|
| @@ -46,7 +46,7 @@ docs/decisions.md — MODIFY: M6 section incl. the transport verdict + numbers | |||
| 46 | 46 | ||
| 47 | The daemon currently calls `proto.writeFrame(fd, ...)` — blocking, so one stalled client freezes the pump for everyone. Replace the client-send path with per-client outbound queues flushed opportunistically. | 47 | The daemon currently calls `proto.writeFrame(fd, ...)` — blocking, so one stalled client freezes the pump for everyone. Replace the client-send path with per-client outbound queues flushed opportunistically. |
| 48 | 48 | ||
| 49 | - [ ] **Step 1: Failing tests.** In `src/protocol.zig`: | 49 | - [x] **Step 1: Failing tests.** In `src/protocol.zig`: |
| 50 | 50 | ||
| 51 | ```zig | 51 | ```zig |
| 52 | test "appendFrame encodes the same bytes writeFrame sends" { | 52 | test "appendFrame encodes the same bytes writeFrame sends" { |
| @@ -63,9 +63,9 @@ In `src/server.zig`, two deterministic tests using **socketpair-free** primitive | |||
| 63 | - "Server: a stalled client does not block delivery to others": install two fake client fds (pipe write-ends — the daemon only writes to clients in this path; follow the precedent of the existing stats test that installs pipe fds directly into `clients[]`). Shrink pipe A's buffer via `F.SETPIPE_SZ` to 4096 and do not read from it; drive `sendUpdate`-scale traffic (feed the engine a few KB, call the send path repeatedly, or call `broadcastSnapshot`-equivalent several times). Assert: pipe B's read side receives complete, parseable frames (read it and frame-parse); client A is still attached with a non-empty pending queue OR (after exceeding the cap) dropped; the calls never blocked (bound the test with a deadline — completing at all is the assertion, since a blocking write on the full 4KB pipe would hang the test). | 63 | - "Server: a stalled client does not block delivery to others": install two fake client fds (pipe write-ends — the daemon only writes to clients in this path; follow the precedent of the existing stats test that installs pipe fds directly into `clients[]`). Shrink pipe A's buffer via `F.SETPIPE_SZ` to 4096 and do not read from it; drive `sendUpdate`-scale traffic (feed the engine a few KB, call the send path repeatedly, or call `broadcastSnapshot`-equivalent several times). Assert: pipe B's read side receives complete, parseable frames (read it and frame-parse); client A is still attached with a non-empty pending queue OR (after exceeding the cap) dropped; the calls never blocked (bound the test with a deadline — completing at all is the assertion, since a blocking write on the full 4KB pipe would hang the test). |
| 64 | - "Server: a client exceeding the pending cap is dropped": same setup, single stalled client, feed until `pending.items.len` would exceed the cap (set a small test cap — make the cap a `Server` field, default `8 * 1024 * 1024`, overridable in tests); assert the slot is nulled and the fd closed (write to the read end... simpler: assert slot nulled and `stats` unaffected thereafter). | 64 | - "Server: a client exceeding the pending cap is dropped": same setup, single stalled client, feed until `pending.items.len` would exceed the cap (set a small test cap — make the cap a `Server` field, default `8 * 1024 * 1024`, overridable in tests); assert the slot is nulled and the fd closed (write to the read end... simpler: assert slot nulled and `stats` unaffected thereafter). |
| 65 | 65 | ||
| 66 | - [ ] **Step 2: `make test` → failures.** | 66 | - [x] **Step 2: `make test` → failures.** |
| 67 | 67 | ||
| 68 | - [ ] **Step 3: Implement.** | 68 | - [x] **Step 3: Implement.** |
| 69 | 69 | ||
| 70 | `src/protocol.zig`: | 70 | `src/protocol.zig`: |
| 71 | ```zig | 71 | ```zig |
| @@ -102,8 +102,8 @@ Send path: `queueFrame(i, t, payload)` — `proto.appendFrame` into `pending`, t | |||
| 102 | 102 | ||
| 103 | Note: client fds stay otherwise blocking; `MSG_DONTWAIT` gives per-call nonblocking writes without touching the read path. Reads remain POLLIN-gated blocking `readFrame` — pre-existing, accepted (input frames are tiny); do not refactor reads. | 103 | Note: client fds stay otherwise blocking; `MSG_DONTWAIT` gives per-call nonblocking writes without touching the read path. Reads remain POLLIN-gated blocking `readFrame` — pre-existing, accepted (input frames are tiny); do not refactor reads. |
| 104 | 104 | ||
| 105 | - [ ] **Step 4: `make test && make e2e && make bench` green (single-client behavior unchanged).** | 105 | - [x] **Step 4: `make test && make e2e && make bench` green (single-client behavior unchanged).** |
| 106 | - [ ] **Step 5: Commit** `feat: queued non-stalling client writes with pending cap`. | 106 | - [x] **Step 5: Commit** `feat: queued non-stalling client writes with pending cap`. |
| 107 | 107 | ||
| 108 | --- | 108 | --- |
| 109 | 109 | ||
| @@ -111,13 +111,13 @@ Note: client fds stay otherwise blocking; `MSG_DONTWAIT` gives per-call nonblock | |||
| 111 | 111 | ||
| 112 | **Files:** `src/protocol.zig`, `src/server.zig`, `src/client.zig`. | 112 | **Files:** `src/protocol.zig`, `src/server.zig`, `src/client.zig`. |
| 113 | 113 | ||
| 114 | - [ ] **Step 1: Failing tests.** protocol.zig: golden-byte + round-trip + error-path tests for **attach v3** — `encodeAttach(cols, rows, have_seq, have_epoch)` → 20 bytes (have_epoch u64 LE appended at [12..20]); `decodeAttach` requires exactly 20 — and **SnapshotPrefix v2** — `epoch: u64` appended at [16..24], `snapshot_prefix_len = 24`. Update ALL existing golden tests for the new layouts. server.zig test: "Server: an unknown epoch can never be served a delta" — attach, learn `(epoch, seq)` from the first snapshot prefix; detach; reattach with the REAL seq but `have_epoch = epoch ^ 1` → first state frame must be `.snapshot`; reattach with the real `(seq, epoch)` → `.delta` (this replaces/extends the existing have_seq reattach test — fold the epoch into it rather than duplicating the harness). | 114 | - [x] **Step 1: Failing tests.** protocol.zig: golden-byte + round-trip + error-path tests for **attach v3** — `encodeAttach(cols, rows, have_seq, have_epoch)` → 20 bytes (have_epoch u64 LE appended at [12..20]); `decodeAttach` requires exactly 20 — and **SnapshotPrefix v2** — `epoch: u64` appended at [16..24], `snapshot_prefix_len = 24`. Update ALL existing golden tests for the new layouts. server.zig test: "Server: an unknown epoch can never be served a delta" — attach, learn `(epoch, seq)` from the first snapshot prefix; detach; reattach with the REAL seq but `have_epoch = epoch ^ 1` → first state frame must be `.snapshot`; reattach with the real `(seq, epoch)` → `.delta` (this replaces/extends the existing have_seq reattach test — fold the epoch into it rather than duplicating the harness). |
| 115 | 115 | ||
| 116 | - [ ] **Step 2: `make test` → compile failures.** | 116 | - [x] **Step 2: `make test` → compile failures.** |
| 117 | 117 | ||
| 118 | - [ ] **Step 3: Implement.** Server: `epoch: u64` field; in `Server.init`: `var e: u64 = 0; while (e == 0) e = std.crypto.random.int(u64);` (0 is reserved = "none"). `buildSnapshotPayload` writes it. `sendResync`'s serviceable-delta guard additionally requires `have_epoch == self.epoch`. Client: sends `have_seq = 0, have_epoch = 0` (fresh process — unchanged semantics); parses and retains the epoch from snapshot prefixes in a local var (unused today; comment: reconnect logic will use it). Migrate every `encodeAttach` call site (server tests, client) to the 4-arg form. | 118 | - [x] **Step 3: Implement.** Server: `epoch: u64` field; in `Server.init`: `var e: u64 = 0; while (e == 0) e = std.crypto.random.int(u64);` (0 is reserved = "none"). `buildSnapshotPayload` writes it. `sendResync`'s serviceable-delta guard additionally requires `have_epoch == self.epoch`. Client: sends `have_seq = 0, have_epoch = 0` (fresh process — unchanged semantics); parses and retains the epoch from snapshot prefixes in a local var (unused today; comment: reconnect logic will use it). Migrate every `encodeAttach` call site (server tests, client) to the 4-arg form. |
| 119 | 119 | ||
| 120 | - [ ] **Step 4: gates green. Step 5: Commit** `feat: session epoch fences have_seq across daemon restarts`. | 120 | - [x] **Step 4: gates green. Step 5: Commit** `feat: session epoch fences have_seq across daemon restarts`. |
| 121 | 121 | ||
| 122 | --- | 122 | --- |
| 123 | 123 | ||
| @@ -125,7 +125,7 @@ Note: client fds stay otherwise blocking; `MSG_DONTWAIT` gives per-call nonblock | |||
| 125 | 125 | ||
| 126 | **Files:** create `src/proxy.zig`; modify `src/main.zig`, `src/client.zig`, `src/mux_main.zig`, `build.zig` (proxy module import for muxd), `test/e2e.sh`. | 126 | **Files:** create `src/proxy.zig`; modify `src/main.zig`, `src/client.zig`, `src/mux_main.zig`, `build.zig` (proxy module import for muxd), `test/e2e.sh`. |
| 127 | 127 | ||
| 128 | - [ ] **Step 1: `src/proxy.zig`** — the whole point is what this file does NOT contain: no frame parsing, no protocol import. | 128 | - [x] **Step 1: `src/proxy.zig`** — the whole point is what this file does NOT contain: no frame parsing, no protocol import. |
| 129 | 129 | ||
| 130 | ```zig | 130 | ```zig |
| 131 | //! `muxd proxy`: a bidirectional byte pump between stdio and the local | 131 | //! `muxd proxy`: a bidirectional byte pump between stdio and the local |
| @@ -184,9 +184,9 @@ test "proxy pumps bytes both ways verbatim" { | |||
| 184 | ``` | 184 | ``` |
| 185 | Factor as `pump(in_fd, out_fd, sock_path)` + `run()` wrapper so the test drives real pipes. Wire `muxd proxy [--sock PATH]` into main.zig (default sock path logic shared with dump/stats). | 185 | Factor as `pump(in_fd, out_fd, sock_path)` + `run()` wrapper so the test drives real pipes. Wire `muxd proxy [--sock PATH]` into main.zig (default sock path logic shared with dump/stats). |
| 186 | 186 | ||
| 187 | - [ ] **Step 2: `mux --via "CMD"`.** In `src/client.zig`, abstract the transport fds: `const Conn = struct { r: std.posix.fd_t, w: std.posix.fd_t };` — socket case `{sock, sock}`; via case: spawn `/bin/sh -c CMD` via `std.process.Child` with `.stdin_behavior = .Pipe, .stdout_behavior = .Pipe` (stderr inherit — ssh errors must reach the user), `conn = .{ .r = child.stdout.?.handle, .w = child.stdin.?.handle }`. Every `readFrame(alloc, sock)` → `readFrame(alloc, conn.r)`; every `writeFrame(sock, ...)` → `(conn.w, ...)`; the poll entry uses `conn.r`. On exit, kill+wait the child. `attach()` signature gains the transport: simplest is `pub fn attach(alloc, sock_path: ?[]const u8, via: ?[]const u8) !u8` — exactly one non-null, resolved in `mux_main.zig` (`--via` flag, mutually exclusive with `--sock`; usage string updated). | 187 | - [x] **Step 2: `mux --via "CMD"`.** In `src/client.zig`, abstract the transport fds: `const Conn = struct { r: std.posix.fd_t, w: std.posix.fd_t };` — socket case `{sock, sock}`; via case: spawn `/bin/sh -c CMD` via `std.process.Child` with `.stdin_behavior = .Pipe, .stdout_behavior = .Pipe` (stderr inherit — ssh errors must reach the user), `conn = .{ .r = child.stdout.?.handle, .w = child.stdin.?.handle }`. Every `readFrame(alloc, sock)` → `readFrame(alloc, conn.r)`; every `writeFrame(sock, ...)` → `(conn.w, ...)`; the poll entry uses `conn.r`. On exit, kill+wait the child. `attach()` signature gains the transport: simplest is `pub fn attach(alloc, sock_path: ?[]const u8, via: ?[]const u8) !u8` — exactly one non-null, resolved in `mux_main.zig` (`--via` flag, mutually exclusive with `--sock`; usage string updated). |
| 188 | 188 | ||
| 189 | - [ ] **Step 3: e2e scenario** (append before `echo "e2e OK"`; this is the local structural proof): | 189 | - [x] **Step 3: e2e scenario** (append before `echo "e2e OK"`; this is the local structural proof): |
| 190 | 190 | ||
| 191 | ```sh | 191 | ```sh |
| 192 | # --- M6: same protocol over an arbitrary byte pipe (proxy transport). | 192 | # --- M6: same protocol over an arbitrary byte pipe (proxy transport). |
| @@ -197,7 +197,7 @@ rm -f "$OUT.via" | |||
| 197 | ``` | 197 | ``` |
| 198 | (`$OUT.via` added to cleanup. `$MUXD` may contain a path with spaces — it won't in this build tree; note it.) | 198 | (`$OUT.via` added to cleanup. `$MUXD` may contain a path with spaces — it won't in this build tree; note it.) |
| 199 | 199 | ||
| 200 | - [ ] **Step 4: gates green (test, e2e, bench). Step 5: Commit** `feat: mux --via arbitrary-command transport; muxd proxy byte pump`. | 200 | - [x] **Step 4: gates green (test, e2e, bench). Step 5: Commit** `feat: mux --via arbitrary-command transport; muxd proxy byte pump`. |
| 201 | 201 | ||
| 202 | --- | 202 | --- |
| 203 | 203 | ||
| @@ -205,7 +205,7 @@ rm -f "$OUT.via" | |||
| 205 | 205 | ||
| 206 | **Files:** create `test/wan.sh` (executable). NOT wired into build.zig — it needs a real remote box; it is run manually/by the controller. | 206 | **Files:** create `test/wan.sh` (executable). NOT wired into build.zig — it needs a real remote box; it is run manually/by the controller. |
| 207 | 207 | ||
| 208 | - [ ] **Step 1: Write `test/wan.sh`.** Parameterized by env: `MUX_WAN_SSH` (full ssh command string incl. jump/control flags, e.g. `ssh -o ControlMaster=auto -o ControlPath=/tmp/mux-cm -o ControlPersist=300 -J ubuntu@gate.eitri.sh:2222 ubuntu@sandbox-9b70e9`), `MUX_WAN_SCP` (matching scp, e.g. `scp -o ControlPath=/tmp/mux-cm`), `MUX_WAN_HOST` (scp target prefix, e.g. `ubuntu@sandbox-9b70e9`). Refuse with a usage message if unset. Steps the script performs: | 208 | - [x] **Step 1: Write `test/wan.sh`.** Parameterized by env: `MUX_WAN_SSH` (full ssh command string incl. jump/control flags, e.g. `ssh -o ControlMaster=auto -o ControlPath=/tmp/mux-cm -o ControlPersist=300 -J ubuntu@gate.eitri.sh:2222 ubuntu@sandbox-9b70e9`), `MUX_WAN_SCP` (matching scp, e.g. `scp -o ControlPath=/tmp/mux-cm`), `MUX_WAN_HOST` (scp target prefix, e.g. `ubuntu@sandbox-9b70e9`). Refuse with a usage message if unset. Steps the script performs: |
| 209 | 1. `~/Downloads/zig-x86_64-linux-0.15.2/zig build -Dtarget=x86_64-linux-musl`; scp `zig-out/bin/muxd` to `$MUX_WAN_HOST:/tmp/muxd-wan`; rebuild native (`make build`) so the local `mux` is native. | 209 | 1. `~/Downloads/zig-x86_64-linux-0.15.2/zig build -Dtarget=x86_64-linux-musl`; scp `zig-out/bin/muxd` to `$MUX_WAN_HOST:/tmp/muxd-wan`; rebuild native (`make build`) so the local `mux` is native. |
| 210 | 2. **Baseline**: raw byte round-trip through the pipe — run `$MUX_WAN_SSH cat` as a coprocess, send a byte, time until it returns; 20 reps, report min/median. This is the number mux must approach. | 210 | 2. **Baseline**: raw byte round-trip through the pipe — run `$MUX_WAN_SSH cat` as a coprocess, send a byte, time until it returns; 20 reps, report min/median. This is the number mux must approach. |
| 211 | 3. Start the remote daemon: `$MUX_WAN_SSH 'rm -f /tmp/mux-wan.sock; nohup /tmp/muxd-wan run --sock /tmp/mux-wan.sock --shell /bin/bash >/tmp/muxd-wan.log 2>&1 &'`. | 211 | 3. Start the remote daemon: `$MUX_WAN_SSH 'rm -f /tmp/mux-wan.sock; nohup /tmp/muxd-wan run --sock /tmp/mux-wan.sock --shell /bin/bash >/tmp/muxd-wan.log 2>&1 &'`. |
| @@ -215,9 +215,9 @@ rm -f "$OUT.via" | |||
| 215 | 7. **Optional netem stanza** (`MUX_WAN_NETEM=1`): `$MUX_WAN_SSH 'sudo tc qdisc add dev $(ip route | awk "/default/{print \$5; exit}") root netem delay 75ms loss 1%'` (≈150ms added RTT), re-run steps 5-6, then ALWAYS `sudo tc qdisc del ... root` in the cleanup trap. | 215 | 7. **Optional netem stanza** (`MUX_WAN_NETEM=1`): `$MUX_WAN_SSH 'sudo tc qdisc add dev $(ip route | awk "/default/{print \$5; exit}") root netem delay 75ms loss 1%'` (≈150ms added RTT), re-run steps 5-6, then ALWAYS `sudo tc qdisc del ... root` in the cleanup trap. |
| 216 | 8. Print a summary block (baseline, attach, echo, reattach, netem variants) formatted for pasting into decisions.md. Cleanup trap: kill remote daemon, remove remote socket, local fifos. | 216 | 8. Print a summary block (baseline, attach, echo, reattach, netem variants) formatted for pasting into decisions.md. Cleanup trap: kill remote daemon, remove remote socket, local fifos. |
| 217 | 217 | ||
| 218 | - [ ] **Step 2: Run it against the box** (env values above are in this plan's Context section). Run twice; numbers should be stable to ~±20%. **If keystroke echo exceeds baseline + 120ms median, or reattach exceeds ~2×RTT to first byte: that is the M6 kill criterion failing — STOP and report with the raw numbers; do not tune the thresholds.** | 218 | - [x] **Step 2: Run it against the box** (env values above are in this plan's Context section). Run twice; numbers should be stable to ~±20%. **If keystroke echo exceeds baseline + 120ms median, or reattach exceeds ~2×RTT to first byte: that is the M6 kill criterion failing — STOP and report with the raw numbers; do not tune the thresholds.** |
| 219 | 219 | ||
| 220 | - [ ] **Step 3: Commit** `feat: WAN measurement harness` (script only — numbers go in Task 5's docs). | 220 | - [x] **Step 3: Commit** `feat: WAN measurement harness` (script only — numbers go in Task 5's docs). |
| 221 | 221 | ||
| 222 | --- | 222 | --- |
| 223 | 223 | ||
| @@ -225,11 +225,11 @@ rm -f "$OUT.via" | |||
| 225 | 225 | ||
| 226 | **Files:** `docs/decisions.md`, `README.md`, plan checkboxes. | 226 | **Files:** `docs/decisions.md`, `README.md`, plan checkboxes. |
| 227 | 227 | ||
| 228 | - [ ] **Step 1: decisions.md M6 section**: the queued-writes design (cap semantics, "sent = queued" stats note, reads deliberately still blocking); the epoch (0 reserved, why attach+prefix and not deltas — mid-stream restart is impossible because the transport connection dies with the daemon); the `--via`/proxy design and **the transport verdict with evidence**: state plainly whether "transport is a swap, not a redesign" held — the proxy's zero protocol knowledge is the structural half, the measured numbers (paste the wan.sh summary: baseline, echo, attach, reattach, netem) are the empirical half, and the kill criterion pass/fail is the sentence. Record what transport did NOT need (no framing changes, no msgpack tripwire trip — payloads still unstructured). Update the banked list (strike "nonblocking/buffered writes"; the pre-network prerequisites are now: TLS/QUIC for non-SSH deployment, reconnect-with-epoch client logic, prediction). Note the WAN box is ephemeral and parameterized, not recorded. | 228 | - [x] **Step 1: decisions.md M6 section**: the queued-writes design (cap semantics, "sent = queued" stats note, reads deliberately still blocking); the epoch (0 reserved, why attach+prefix and not deltas — mid-stream restart is impossible because the transport connection dies with the daemon); the `--via`/proxy design and **the transport verdict with evidence**: state plainly whether "transport is a swap, not a redesign" held — the proxy's zero protocol knowledge is the structural half, the measured numbers (paste the wan.sh summary: baseline, echo, attach, reattach, netem) are the empirical half, and the kill criterion pass/fail is the sentence. Record what transport did NOT need (no framing changes, no msgpack tripwire trip — payloads still unstructured). Update the banked list (strike "nonblocking/buffered writes"; the pre-network prerequisites are now: TLS/QUIC for non-SSH deployment, reconnect-with-epoch client logic, prediction). Note the WAN box is ephemeral and parameterized, not recorded. |
| 229 | 229 | ||
| 230 | - [ ] **Step 2: README**: status `M6 — transport`; usage gains `mux --via "ssh host /path/muxd proxy --sock ..."` with one sentence (any command exposing the daemon socket over stdio works); note `test/wan.sh`. | 230 | - [x] **Step 2: README**: status `M6 — transport`; usage gains `mux --via "ssh host /path/muxd proxy --sock ..."` with one sentence (any command exposing the daemon socket over stdio works); note `test/wan.sh`. |
| 231 | 231 | ||
| 232 | - [ ] **Step 3: All M6 checkboxes flipped; commit** `feat: SSH-channel transport measured over real WAN — M6 complete`. | 232 | - [x] **Step 3: All M6 checkboxes flipped; commit** `feat: SSH-channel transport measured over real WAN — M6 complete`. |
| 233 | 233 | ||
| 234 | --- | 234 | --- |
| 235 | 235 | ||
src/pty.zig
| Old | New | ||
|---|---|---|---|
| @@ -53,6 +53,11 @@ pub const Pty = struct { | |||
| 53 | }; | 53 | }; |
| 54 | std.posix.sigaction(std.posix.SIG.INT, &dfl, null); | 54 | std.posix.sigaction(std.posix.SIG.INT, &dfl, null); |
| 55 | std.posix.sigaction(std.posix.SIG.QUIT, &dfl, null); | 55 | std.posix.sigaction(std.posix.SIG.QUIT, &dfl, null); |
| 56 | // SIGPIPE for the same survives-exec reason, as hardening: the | ||
| 57 | // daemon ignores it for its own sockets, and only the order of | ||
| 58 | // that ignore against this fork currently keeps it out of the | ||
| 59 | // session shell. Resetting here makes it order-independent. | ||
| 60 | std.posix.sigaction(std.posix.SIG.PIPE, &dfl, null); | ||
| 56 | 61 | ||
| 57 | var argv = [_:null]?[*:0]const u8{ opts.shell.ptr, null }; | 62 | var argv = [_:null]?[*:0]const u8{ opts.shell.ptr, null }; |
| 58 | std.posix.execveZ(opts.shell.ptr, &argv, std.c.environ) catch {}; | 63 | std.posix.execveZ(opts.shell.ptr, &argv, std.c.environ) catch {}; |
test/wan.sh
| Old | New | ||
|---|---|---|---|
| @@ -26,16 +26,15 @@ | |||
| 26 | # fails. Do not tune the thresholds here; a failure is the milestone's | 26 | # fails. Do not tune the thresholds here; a failure is the milestone's |
| 27 | # answer. | 27 | # answer. |
| 28 | # | 28 | # |
| 29 | # Ctrl-C does not work inside a session whose daemon was backgrounded by a | 29 | # Ctrl-C inside a session used not to work at all when the daemon had been |
| 30 | # non-interactive shell — which is how this script, e2e.sh and any deploy | 30 | # backgrounded by a non-interactive shell — which is how this script, e2e.sh |
| 31 | # script start it. POSIX has such a shell set SIGINT/SIGQUIT to SIG_IGN for | 31 | # and any deploy script start it. POSIX has such a shell set SIGINT/SIGQUIT |
| 32 | # an async child; SIG_IGN survives exec, so muxd passes it to the pty child, | 32 | # to SIG_IGN for an async child; SIG_IGN survives exec, so muxd passed it to |
| 33 | # and a shell keeps signals ignored-on-entry ignored for the jobs it spawns. | 33 | # the pty child, and a shell keeps signals ignored-on-entry ignored for the |
| 34 | # The pty's isig is on and ^C is echoed; the signal is simply never raised. | 34 | # jobs it spawns. This run is what found it. FIXED in src/pty.zig, which now |
| 35 | # Verified on the box: with SIGINT reset to SIG_DFL before muxd exec's, the | 35 | # resets those dispositions in the forkpty child — see the comment there. |
| 36 | # same ^C kills the same `sleep 300`. Nothing here may rely on interrupting | 36 | # The bounded flood in the head-of-line probe stays bounded anyway: that is |
| 37 | # the remote shell (see the flood probe). Not this script's to fix — the | 37 | # a measurement constraint, not a workaround for this bug (see cmd_hol). |
| 38 | # reset belongs next to the execveZ in src/pty.zig. | ||
| 39 | # | 38 | # |
| 40 | # NOT wired into build.zig: it needs a real remote box. The box is | 39 | # NOT wired into build.zig: it needs a real remote box. The box is |
| 41 | # ephemeral and passed in by environment, never recorded in this file. | 40 | # ephemeral and passed in by environment, never recorded in this file. |
| @@ -358,8 +357,9 @@ def wait_idle(c, what, timeout=60.0): | |||
| 358 | def prove_alive(c, what): | 357 | def prove_alive(c, what): |
| 359 | """Prove the remote shell is back at a prompt and running commands. The | 358 | """Prove the remote shell is back at a prompt and running commands. The |
| 360 | flood probe is the only step that leaves work running on the far side, | 359 | flood probe is the only step that leaves work running on the far side, |
| 361 | and Ctrl-C cannot be used to stop it (see the SIGINT finding in wan.sh's | 360 | and this harness has no way to interrupt it (it types bytes down a pipe; |
| 362 | header), so the session's return to health is checked, never assumed.""" | 361 | it cannot press a key), so the session's return to health is checked |
| 362 | rather than assumed.""" | ||
| 363 | wait_idle(c, what) | 363 | wait_idle(c, what) |
| 364 | c.send(b"\x15printf 'holdone-%s\\n' ok\n") | 364 | c.send(b"\x15printf 'holdone-%s\\n' ok\n") |
| 365 | if c.wait_for(lambda b: b"holdone-ok" in b) is None: | 365 | if c.wait_for(lambda b: b"holdone-ok" in b) is None: |
| @@ -502,12 +502,13 @@ def cmd_hol(argv): | |||
| 502 | keystroke's echo; this number is the size of that effect. Recorded for | 502 | keystroke's echo; this number is the size of that effect. Recorded for |
| 503 | the record, not gated. | 503 | the record, not gated. |
| 504 | 504 | ||
| 505 | The flood is bounded and self-terminating, which is not tidiness: a mux | 505 | The flood is bounded and self-terminating because nothing here may |
| 506 | session started the ordinary way cannot be interrupted at all (the daemon | 506 | depend on interrupting it: this harness drives the session over a pipe, |
| 507 | inherits SIGINT=SIG_IGN from the shell that backgrounded it, and passes | 507 | so it types bytes rather than pressing keys, and an unbounded flood that |
| 508 | it to the pty child — see this script's header), so an infinite loop here | 508 | outlived its rep would silently add itself to every measurement taken |
| 509 | would run until the daemon died and would silently add itself to every | 509 | afterwards. (Until 153cb5f such a flood was also literally unstoppable — |
| 510 | measurement taken afterwards. | 510 | Ctrl-C was dead in any script-started session; that is fixed in |
| 511 | src/pty.zig, and the bound is kept for the reason above.) | ||
| 511 | 512 | ||
| 512 | The flood is one line every 10ms, not `yes` at full tilt, and the rate | 513 | The flood is one line every 10ms, not `yes` at full tilt, and the rate |
| 513 | is the measurement, not a courtesy. A scrolling line rewrites every row, | 514 | is the measurement, not a courtesy. A scrolling line rewrites every row, |
| @@ -673,11 +674,14 @@ phase_block() { | |||
| 673 | # | 674 | # |
| 674 | # The ruling, recorded here so the gate is never mistaken for a threshold | 675 | # The ruling, recorded here so the gate is never mistaken for a threshold |
| 675 | # someone softened: a reattach cannot begin until ssh has opened a | 676 | # someone softened: a reattach cannot begin until ssh has opened a |
| 676 | # channel, and that floor is measured (`viafloor`), not assumed. On a | 677 | # channel, and that floor is measured (`viafloor`), not assumed. Across |
| 677 | # 15.8ms link it was 35.9ms — 2.3xRTT, already past the whole 2xRTT | 678 | # the M6 runs it was 35.3-37.5ms against a 15.7-16.5ms round trip — |
| 678 | # budget before one protocol byte moves. Gating the wall-clock number | 679 | # 2.2-2.4xRTT, already past the whole 2xRTT budget before one protocol |
| 679 | # would therefore fail on every fast link no matter what the protocol | 680 | # byte moves. Gating the wall-clock number would therefore fail on every |
| 680 | # did, which gates nothing and cannot falsify a design. The strict | 681 | # fast link no matter what the protocol did, which gates nothing and |
| 682 | # cannot falsify a design. That is why this gate changed: the strict | ||
| 683 | # reading was gated first and printed FAIL, and the decomposition below | ||
| 684 | # is what showed the failure to be ssh's channel setup. The strict | ||
| 681 | # reading stays on the page because it is what a user waits through. | 685 | # reading stays on the page because it is what a user waits through. |
| 682 | local floor protocol pverdict | 686 | local floor protocol pverdict |
| 683 | floor="$(val "$phase" viafloor med)" | 687 | floor="$(val "$phase" viafloor med)" |
| @@ -688,8 +692,8 @@ phase_block() { | |||
| 688 | printf ' reattach, wall clock (reported): med %s <= 2 x round-trip %s = %s -> %s (margin %s)\n' \ | 692 | printf ' reattach, wall clock (reported): med %s <= 2 x round-trip %s = %s -> %s (margin %s)\n' \ |
| 689 | "$reatt_med" "$base" "$budget" "$verdict" \ | 693 | "$reatt_med" "$base" "$budget" "$verdict" \ |
| 690 | "$(awk -v r="$reatt_med" -v t="$budget" 'BEGIN{printf "%+.1f", t-r}')" | 694 | "$(awk -v r="$reatt_med" -v t="$budget" 'BEGIN{printf "%+.1f", t-r}')" |
| 691 | printf ' decomposition: %s = %s transport setup (ssh channel open + exec)\n' \ | 695 | printf ' decomposition: %s = %s transport setup (ssh channel open + exec) + %s protocol\n' \ |
| 692 | "$reatt_med" "$floor" | 696 | "$reatt_med" "$floor" "$protocol" |
| 693 | printf ' reattach criterion (GATED, protocol share): %s <= %s -> %s (margin %s)\n' \ | 697 | printf ' reattach criterion (GATED, protocol share): %s <= %s -> %s (margin %s)\n' \ |
| 694 | "$protocol" "$budget" "$pverdict" \ | 698 | "$protocol" "$budget" "$pverdict" \ |
| 695 | "$(awk -v p="$protocol" -v t="$budget" 'BEGIN{printf "%+.1f", t-p}')" | 699 | "$(awk -v p="$protocol" -v t="$budget" 'BEGIN{printf "%+.1f", t-p}')" |