a73x

bc625c43

refactor: handoff and hub keep the measurement, drop the diary

a73x   2026-08-30 19:45

Commit message
refactor: handoff and hub keep the measurement, drop the diary

55 essays to 10. The QUIC deadline still says what it bounds and why 2000
beats 1000; it no longer recites four measurement sessions to get there.
The hub's frame guards still name the panic they prevent, without the
commit that introduced it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XSUFuYHqU9wr4J5NC8EkWV
docscheck.blocks
Old New
@@ -7,7 +7,7 @@ delta.zig 8
7 docscheck.zig 4 7 docscheck.zig 4
8 engine.zig 17 8 engine.zig 17
9 flags.zig 2 9 flags.zig 2
10 handoff.zig 26 10 handoff.zig 6
11 hosts.zig 5 11 hosts.zig 5
12 interact.zig 24 12 interact.zig 24
13 keymap.zig 1 13 keymap.zig 1
@@ -57,5 +57,5 @@ wall_test_wall.zig 7
57 wallview.zig 0 57 wallview.zig 0
58 wasm_core.zig 5 58 wasm_core.zig 5
59 webhub_main.zig 4 59 webhub_main.zig 4
60 webhub.zig 29 60 webhub.zig 4
61 xdg.zig 4 61 xdg.zig 4
src/client/handoff.zig
Old New
@@ -1,51 +1,29 @@
1 //! The ssh→QUIC handoff's shared vocabulary: the announce line 1 //! The ssh→QUIC handoff's shared vocabulary: the announce line
2 //! `mux d endpoint` prints and `mux` parses, the per-host cache file that 2 //! `mux d endpoint` prints and `mux` parses, the per-host cache that
3 //! remembers it, the strip that turns an ssh destination into a dialable 3 //! remembers it, and the step table `next` that orders a handoff out of them.
4 //! host, and the step table `next` that orders a handoff out of them. 4 //! Pure by design — no sockets, no processes — so it tests without a daemon.
5 //! Pure by design — no sockets, no processes — so the whole surface, the
6 //! policy included, tests without a daemon.
7 const std = @import("std"); 5 const std = @import("std");
8 // Only for the private-parent discipline the cache file shares with the 6 // Only for the private-parent discipline the cache file shares with the
9 // key file. No XDG resolution happens here: writeCache is handed a path. 7 // key file. No XDG resolution happens here: writeCache is handed a path.
10 const xdg = @import("xdg"); 8 const xdg = @import("xdg");
11 9
12 /// The QUIC attach budget for one attempt, warm path and cold path 10 /// The QUIC attach budget for one attempt, warm and cold alike.
13 /// alike — and, since the module-seam refactor, for a direct `quic://` dial
14 /// too. Pinned from measurement on 2026-08-11 when the ssh→QUIC handoff
15 /// landed (decisions.md), re-measured 2026-08-12 after the one fast-failure
16 /// case was taught to work.
17 /// 11 ///
18 /// Measured: silence is what this bound is for, and silence is the 12 /// SILENCE is what it bounds, and silence is the common case: a wrong PSK
19 /// common case. A wrong PSK against a live listener ran 2037ms of a 13 /// against a live listener never answers, because mutual auth means the
20 /// 2000ms budget, 3049 of 3000, 8031 of 8000 — mutual auth means the 14 /// listener does not reply to a peer it cannot authenticate, and a blackholed
21 /// listener does not answer a peer it cannot authenticate — and the 15 /// UDP port is the same. Nothing else ends those dials.
22 /// re-measurement agrees at 2012ms. A blackholed UDP port does the same
23 /// (4001 of 4000, 8001 of 8000; 2002ms on re-measurement). Neither ends
24 /// any other way, so this bound is the only thing that ends them.
25 /// 16 ///
26 /// The exception is a REFUSED port, and it used to be silent only by 17 /// 2000 fixes the trade. Below it: a handshake is 3.1x RTT — every fresh
27 /// accident: the ICMP unreachable arrived, but it was delivered to the 18 /// Initial costs a Retry round trip — so this covers RTT to ~645ms, past any
28 /// send in Client.drain, whose `catch return` dropped it. Both socket 19 /// terrestrial link. Above it: there is no negative caching of "UDP blocked",
29 /// paths act on it now, so an unbound loopback port fails in 1ms where 20 /// so every fallback attach on such a network pays this in full. 1000 would
30 /// it once spent 2051 of 2000. 21 /// halve the tax and halve the RTT ceiling, abandoning paths that work.
31 ///
32 /// Silence spending the budget fixes the shape of the trade, and 2000
33 /// stands. Below: a real handshake is 4ms local, 6.9ms cold attach on
34 /// the LAN box, and 234ms at 75ms RTT — 3.1x RTT, since every fresh
35 /// Initial costs a Retry round trip — so 2000 covers RTT to ~645ms,
36 /// past any terrestrial link. Above: with no negative caching of "UDP
37 /// blocked" by design, every fallback attach on such a network pays this
38 /// in full, so it wants to be as low as the floor allows. 1000 would
39 /// halve the tax and also halve the RTT ceiling to ~320ms, buying
40 /// latency on a degraded network at the price of abandoning QUIC paths
41 /// that would have worked. Wrong way round.
42 pub const deadline_ms: u32 = 2000; 22 pub const deadline_ms: u32 = 2000;
43 23
44 /// The PSK's length in bytes. The same 32 as `quic.Key`, spelled again 24 /// The PSK's length in bytes. The same 32 as `quic.Key`, spelled again rather
45 /// rather than imported: this module stays free of the C stack, which is 25 /// than imported so this module stays free of the C stack. A drift between the
46 /// what lets it test without one. The conversion at the dial site is by 26 /// two does not compile: the dial site converts by value into a fixed array.
47 /// value into `quic.Key`'s fixed-size array, so a drift between the two
48 /// spellings does not compile.
49 pub const key_len = 32; 27 pub const key_len = 32;
50 28
51 /// What a `mux d endpoint` announce carries. 29 /// What a `mux d endpoint` announce carries.
@@ -87,13 +65,9 @@ pub const ReadLineError = error{
87 /// host`) and narrow enough to sit on a picker row beside a spelling. 65 /// host`) and narrow enough to sit on a picker row beside a spelling.
88 pub const reason_max = 120; 66 pub const reason_max = 120;
89 67
90 /// The last COMPLETE line a stderr pipe carried, kept across any 68 /// The last COMPLETE line a stderr pipe carried, kept across the pipe's
91 /// fragmentation the pipe imposes. 69 /// fragmentation. A struct rather than a buffer the reader appends to: ssh says
92 /// 70 /// several things and dies on the last, and a caller wants one sentence.
93 /// Why a struct and not a buffer the reader appends to: ssh says several
94 /// things and dies on the last one, the reader sees arbitrary chunks, and
95 /// what a caller wants is one sentence. Pure — no fd, no allocator — so
96 /// the whole policy is testable without a process.
97 pub const Reason = struct { 71 pub const Reason = struct {
98 buf: [reason_max]u8 = undefined, 72 buf: [reason_max]u8 = undefined,
99 len: usize = 0, 73 len: usize = 0,
@@ -117,21 +91,11 @@ pub const Reason = struct {
117 } 91 }
118 continue; 92 continue;
119 } 93 }
120 // PRINTABLE ASCII, and nothing else. This string is painted 94 // PRINTABLE ASCII only: this string is painted into a picker row
121 // into a picker row inside a wall's alternate screen, RAW — 95 // RAW, never through the VT engine, so any other byte could move a
122 // it never goes through the VT engine that makes a session's 96 // cursor in somebody else's tile. The high half goes too — xterm
123 // remote bytes safe — so a byte that reached the terminal 97 // honours UTF-8-encoded C1 (`\xc2\x9b` is CSI), so stopping at 0x7f
124 // could move a cursor or open a sequence in somebody else's 98 // lets an escape through in a second spelling.
125 // tile, which is the bug this whole change closes.
126 //
127 // The high half goes too, not just C0: UTF-8-encoded C1 is
128 // two bytes (`\xc2\x9b` is CSI) and xterm honours it in UTF-8
129 // mode, so a rule that stopped at 0x7f would let an escape
130 // through in a second spelling. Dropping the whole half also
131 // ends the split-codepoint question at `reason_max`. The
132 // picker's own spelling editor already takes printable ASCII
133 // only; ssh's diagnostics are ASCII, and a hostname with
134 // other bytes in it loses them from this row alone.
135 if (c < 0x20 or c > 0x7e) continue; 99 if (c < 0x20 or c > 0x7e) continue;
136 // Cut, not wrapped: the head of an ssh diagnostic is the part 100 // Cut, not wrapped: the head of an ssh diagnostic is the part
137 // that names the cause. 101 // that names the cause.
@@ -152,11 +116,9 @@ pub const Reason = struct {
152 } 116 }
153 }; 117 };
154 118
155 /// What one dial left behind for whoever has somewhere to show it. 119 /// What one dial left behind for whoever has somewhere to show it: two facts
156 /// 120 /// with one lifetime, since a failed dial is reported in ssh's own words AND
157 /// Two facts with one lifetime: a failed dial is REPORTED in ssh's own 121 /// attributed to the ssh that said them.
158 /// words and ATTRIBUTED to the ssh that said them, and a caller that had
159 /// to thread two out-params would be a caller free to thread one.
160 pub const Dial = struct { 122 pub const Dial = struct {
161 reason: Reason = .{}, 123 reason: Reason = .{},
162 /// The coordination ssh this dial spawned, 0 when it spawned none. 124 /// The coordination ssh this dial spawned, 0 when it spawned none.
@@ -171,20 +133,13 @@ pub const CacheError = error{
171 CacheMalformed, 133 CacheMalformed,
172 }; 134 };
173 135
174 /// `endpoint <port> <64 lowercase hex chars>\n` into `buf`. 136 /// `endpoint <port> <64 lowercase hex chars>\n` into `buf`. One writer and two
175 /// 137 /// readers — the ssh pipe and the cache file, which stores this exact line — so
176 /// One writer — `mux d endpoint` — and two readers: the client reading the 138 /// a cache written by one version and read by another agrees or fails loudly.
177 /// ssh pipe, and the client reading its own cache file, which stores this
178 /// exact line. One grammar, not two, so a cache written by one version and
179 /// read by another can only agree or fail loudly.
180 pub fn formatAnnounce(buf: []u8, ep: Endpoint) ![]const u8 { 139 pub fn formatAnnounce(buf: []u8, ep: Endpoint) ![]const u8 {
181 // The writer refuses what the reader refuses. `endpoint_reply` carries 140 // The writer refuses what the reader refuses: `endpoint_reply` carries 0 as
182 // 0 as a legal wire value meaning "could not", so the daemon side is 141 // "could not", and the caller owes it a translation into `announce_none`.
183 // holding a u16 that may be 0 and owes it a translation into 142 // Refusing here puts the failure on the line that forgot, not on another box.
184 // `announce_none`. Refusing here puts the failure on the line that
185 // forgot, rather than emitting a syntactically fine announce whose
186 // only complaint arrives on another machine, from the parser, about a
187 // message this process wrote.
188 if (ep.port == 0) return error.AnnouncePortZero; 143 if (ep.port == 0) return error.AnnouncePortZero;
189 // `{x}` on a byte slice is per-byte lowercase hex — 64 characters for 144 // `{x}` on a byte slice is per-byte lowercase hex — 64 characters for
190 // 32 bytes, leading zeros and all. Verified against 0.15.2 rather than 145 // 32 bytes, leading zeros and all. Verified against 0.15.2 rather than
@@ -193,19 +148,11 @@ pub fn formatAnnounce(buf: []u8, ep: Endpoint) ![]const u8 {
193 return std.fmt.bufPrint(buf, "endpoint {d} {x}\n", .{ ep.port, &ep.key }); 148 return std.fmt.bufPrint(buf, "endpoint {d} {x}\n", .{ ep.port, &ep.key });
194 } 149 }
195 150
196 /// The announce line back into an `Endpoint`, or null for `endpoint none`. 151 /// The announce line back into an `Endpoint`, or null for `endpoint none`, with
197 /// 152 /// or without its trailing newline. Deliberately no stricter than its parts:
198 /// Accepts the line with or without its trailing newline: the client's 153 /// nothing but this module's writer produces these lines, a looser reader
199 /// announce reader hands back the line stripped, a cache file still has it on. 154 /// cannot admit anything a dial would not reject, and every call site bounds
200 /// 155 /// the input by `announce_max_len`.
201 /// Deliberately no stricter than its parts. The port token is whatever
202 /// `std.fmt.parseInt` accepts, so `+443`, `00443` and `4_433` all parse;
203 /// the key is hex in either case, as `quic.Key.load` also accepts. Nothing
204 /// but this module's own writer produces these lines, a looser reader
205 /// cannot admit anything a dial would not immediately reject, and the
206 /// input is length-bounded by `announce_max_len` at every call site:
207 /// each reader sizes its buffer — or, for `readCache`, its size check —
208 /// from that constant. Tightening it would only add rules to get wrong.
209 pub fn parseAnnounce(line: []const u8) ParseError!?Endpoint { 156 pub fn parseAnnounce(line: []const u8) ParseError!?Endpoint {
210 const prefix = "endpoint "; 157 const prefix = "endpoint ";
211 158
@@ -265,11 +212,9 @@ pub const Recipe = struct {
265 cache_path: ?[]const u8, 212 cache_path: ?[]const u8,
266 213
267 pub fn deinit(self: Recipe, alloc: std.mem.Allocator) void { 214 pub fn deinit(self: Recipe, alloc: std.mem.Allocator) void {
268 // Exactly once, by the arena that built it: the wall hands one 215 // Exactly once, by the arena that built it: the wall hands one recipe to
269 // recipe to every Tile of a host, and both the CLI tile and the 216 // every Tile of a host and both fronts ALIAS its argvs, so a per-Tile
270 // hub's checkout ALIAS its argvs rather than copying them. Two 217 // free would be a double one.
271 // levels deep, so a future non-arena owner has two levels to free
272 // and a per-Tile free would be a double one.
273 freeArgv(alloc, self.ssh_argv); 218 freeArgv(alloc, self.ssh_argv);
274 freeArgv(alloc, self.asked_argv); 219 freeArgv(alloc, self.asked_argv);
275 if (self.cache_path) |c| alloc.free(c); 220 if (self.cache_path) |c| alloc.free(c);
@@ -302,39 +247,23 @@ fn sshArgv(
302 batch: bool, 247 batch: bool,
303 remote: []const u8, 248 remote: []const u8,
304 ) ![]const []const u8 { 249 ) ![]const []const u8 {
305 // Only the remote word differs between the reading command and the 250 // Only the remote WORD differs between the reading command and the asking
306 // asking one, and a drift between the two spellings would start a 251 // one; a drift would start a daemon somewhere the attach does not look.
307 // daemon somewhere other than where the attach then looks for it.
308 // 252 //
309 // BatchMode is for the recipes NOBODY is sitting in front of — the 253 // BatchMode is for the recipes NOBODY is sitting in front of: ssh prompts on
310 // wall's per-host poll, and `mux hosts`. ssh asks for a password or a 254 // /dev/tty, and a poll running every second under a full-screen wall would
311 // host-key confirmation on /dev/tty, and a poll that runs every second 255 // ask forever, over the panes. ConnectTimeout rides with it — a blackholed
312 // under a full-screen wall would ask forever, over the panes. An 256 // host sits in the kernel's TCP retry schedule for two minutes, which
313 // interactive attach is the opposite case and must keep prompting. 257 // `client.listSessions`' own budget starts too late to bound.
314 //
315 // ConnectTimeout rides with it for the same reason. A blackholed host
316 // — dropped SYNs, no RST — sits in the kernel's TCP retry schedule for
317 // about two minutes, and `client.listSessions` starts its own budget
318 // only after the open returns: without a bound here `mux hosts` hangs
319 // per dead host, serially, and a poll thread holds a stale tile for the
320 // whole stall with no abort fd that could interrupt it. The interactive
321 // attach keeps ssh's own patience — that wait is the user's to abandon.
322 const batch_opt: []const []const u8 = if (batch) 258 const batch_opt: []const []const u8 = if (batch)
323 &.{ "-o", "BatchMode=yes", "-o", "ConnectTimeout=5" } 259 &.{ "-o", "BatchMode=yes", "-o", "ConnectTimeout=5" }
324 else 260 else
325 &.{}; 261 &.{};
326 // ONE argv word, and it is the only one any shell ever reads: ssh joins 262 // ONE argv word, read by the REMOTE user's shell — which is what expands
327 // everything past the host and hands it to the REMOTE user's shell, 263 // `$PATH` here. Ours does not, so nothing in it survives a local quote
328 // which is what expands `$PATH` here. Ours does not — the client execs 264 // round. sshd runs that shell non-login, so it never sources the profile
329 // this argv — so nothing in the word has to survive a local round of 265 // putting ~/.local/bin on PATH. APPENDED: a fallback place to look, never a
330 // quote removal. 266 // shadow over whatever `mux` the remote PATH already resolves.
331 //
332 // sshd runs that remote shell non-login and non-interactive, so it
333 // never sources the profile putting ~/.local/bin (make install's
334 // target) on PATH, and without the suffix a `mux` the user can run by
335 // hand is invisible here. APPENDED, deliberately: a fallback place to
336 // look, never a shadow over whatever `mux` the remote PATH already
337 // resolves (or, under the e2e ssh shim, over the binary under test).
338 const word = try std.fmt.allocPrint(alloc, "PATH=\"$PATH:$HOME/.local/bin\" {s}", .{remote}); 267 const word = try std.fmt.allocPrint(alloc, "PATH=\"$PATH:$HOME/.local/bin\" {s}", .{remote});
339 defer alloc.free(word); 268 defer alloc.free(word);
340 var argv: [7][]const u8 = undefined; 269 var argv: [7][]const u8 = undefined;
@@ -345,11 +274,9 @@ fn sshArgv(
345 return dupeArgv(alloc, argv[0 .. 3 + batch_opt.len]); 274 return dupeArgv(alloc, argv[0 .. 3 + batch_opt.len]);
346 } 275 }
347 276
348 /// ONE owner for the handoff recipe: `mux HOST` and a `mux web` HOST tile 277 /// ONE owner for the handoff recipe: `mux HOST` and a `mux web` tile build the
349 /// build the identical thing, and a drift between two spellings of the 278 /// identical thing, and two spellings would point them at different remote
350 /// ssh line would quietly point the two binaries at different remote 279 /// commands. Here rather than client.zig, which stays free of XDG.
351 /// commands. Building it here (rather than in client.zig) is what keeps
352 /// the client free of XDG and of allocating a command line.
353 pub fn recipeFor(alloc: std.mem.Allocator, host: []const u8, batch: bool) !Recipe { 280 pub fn recipeFor(alloc: std.mem.Allocator, host: []const u8, batch: bool) !Recipe {
354 const cmd = try sshArgv(alloc, host, batch, "mux d endpoint"); 281 const cmd = try sshArgv(alloc, host, batch, "mux d endpoint");
355 errdefer freeArgv(alloc, cmd); 282 errdefer freeArgv(alloc, cmd);
@@ -362,19 +289,12 @@ pub fn recipeFor(alloc: std.mem.Allocator, host: []const u8, batch: bool) !Recip
362 }; 289 };
363 } 290 }
364 291
365 /// The announce line for `ep` at `path`: mode 0600, parent directories 292 /// The announce line for `ep` at `path`: mode 0600, parents created, immediate
366 /// created, immediate parent tightened to 0700 — the key travels in this 293 /// parent tightened to 0700 — the key travels in this file. OVERWRITES, unlike
367 /// file. 294 /// `xdg.writeNewKey`: everything here is re-derivable from one ssh.
368 ///
369 /// OVERWRITES. A cache is the latest truth, not a credential: unlike
370 /// `xdg.writeNewKey`, which refuses because overwriting would destroy the
371 /// only copy of something, everything here is re-derivable from one ssh.
372 pub fn writeCache(path: []const u8, ep: Endpoint) !void { 295 pub fn writeCache(path: []const u8, ep: Endpoint) !void {
373 // The same discipline the key file gets, from the same place: the 296 // The same discipline the key file gets: the file's own 0600 hides the key,
374 // file's own 0600 hides the key, but a 0755 directory still publishes 297 // but a 0755 directory still publishes which hosts this user attaches to.
375 // which hosts this user attaches to, by name. Shared rather than
376 // copied because the reason it is subtle — Dir.chmod needs `.iterate`
377 // — is worth having written down once.
378 try xdg.makePrivateParent(path); 298 try xdg.makePrivateParent(path);
379 var buf: [announce_max_len]u8 = undefined; 299 var buf: [announce_max_len]u8 = undefined;
380 const line = try formatAnnounce(&buf, ep); 300 const line = try formatAnnounce(&buf, ep);
@@ -389,19 +309,11 @@ pub fn writeCache(path: []const u8, ep: Endpoint) !void {
389 try f.writeAll(line); 309 try f.writeAll(line);
390 } 310 }
391 311
392 /// The endpoint remembered at `path`. 312 /// The endpoint remembered at `path`. A group- or other-readable file is
393 /// 313 /// refused before its contents are read, as `quic.Key.load` refuses a
394 /// A group- or other-readable file is refused before its contents are 314 /// permissive key: the key is in here. Three errors, because every caller can
395 /// read, exactly as `quic.Key.load` refuses a permissive key file: the 315 /// only attach cold — `CacheMissing` stays separate from `CachePermissive` and
396 /// key is in here, and a cache that anyone can read has cached a 316 /// `CacheMalformed` because it is the ordinary first run.
397 /// credential in public. Everything a caller can do about this file is
398 /// the same — attach cold — so this function names three errors:
399 /// `CacheMissing` for "no cache yet", `CachePermissive` for one held
400 /// wrong, `CacheMalformed` for every way the contents can be unusable.
401 /// Missing stays separate from the other two because it is the ordinary
402 /// first run rather than something to look into. Ordinary filesystem
403 /// failures — an unreadable file, a bad path — pass through
404 /// untranslated, as they do from `quic.Key.load`.
405 pub fn readCache(path: []const u8) !Endpoint { 317 pub fn readCache(path: []const u8) !Endpoint {
406 const f = std.fs.cwd().openFile(path, .{}) catch |err| switch (err) { 318 const f = std.fs.cwd().openFile(path, .{}) catch |err| switch (err) {
407 error.FileNotFound => return error.CacheMissing, 319 error.FileNotFound => return error.CacheMissing,
@@ -416,11 +328,9 @@ pub fn readCache(path: []const u8) !Endpoint {
416 const n = try f.readAll(&buf); 328 const n = try f.readAll(&buf);
417 if (n > announce_max_len) return error.CacheMalformed; 329 if (n > announce_max_len) return error.CacheMalformed;
418 330
419 // Every way the contents can be wrong arrives as one error. The 331 // Every way the contents can be wrong arrives as ONE error: the granular
420 // granular `Announce*` names are for the ssh-pipe path, where a caller 332 // `Announce*` names are for the ssh-pipe path, where a caller can report
421 // can report what a daemon actually said; a cache has one decision to 333 // what a daemon said. A cache has one decision — use it or refetch.
422 // make — use it or refetch — and `quic.Key.load` already set this
423 // taste by folding hexToBytes failures into KeyFileMalformed.
424 const ep = parseAnnounce(buf[0..n]) catch return error.CacheMalformed; 334 const ep = parseAnnounce(buf[0..n]) catch return error.CacheMalformed;
425 // Nobody writes `endpoint none` here: there are no coordinates to 335 // Nobody writes `endpoint none` here: there are no coordinates to
426 // remember, so the cold path simply leaves the cache alone. A file 336 // remember, so the cold path simply leaves the cache alone. A file
@@ -448,14 +358,9 @@ pub const Step = union(enum) {
448 /// Terminal: the transport the last dial returned. The driver kills 358 /// Terminal: the transport the last dial returned. The driver kills
449 /// the coordination ssh, if it started one. 359 /// the coordination ssh, if it started one.
450 use_quic, 360 use_quic,
451 /// Terminal: the live ssh child IS the session. True asks for the 361 /// Terminal: the live ssh child IS the session. True asks for the fallback
452 /// fallback line. 362 /// line. Not a leftover: through `ssh -J gate box` the announced UDP port is
453 /// 363 /// unreachable BY CONSTRUCTION, so the pipe is that host's only session.
454 /// These arms are not a leftover of a bygone network. Through
455 /// `ssh -J gate box` the announced UDP port is unreachable from here
456 /// BY CONSTRUCTION — the jump host is the only route and it carries
457 /// TCP — so the pipe is the only session such a host can ever have.
458 /// Every one of them takes this step.
459 use_pipe: bool, 364 use_pipe: bool,
460 /// Terminal: the error the driver recorded for the step that failed. 365 /// Terminal: the error the driver recorded for the step that failed.
461 fail, 366 fail,
@@ -503,12 +408,10 @@ pub const State = struct {
503 phase: enum { init, warm_dial, ssh, announce, cache, cold_dial } = .init, 408 phase: enum { init, warm_dial, ssh, announce, cache, cold_dial } = .init,
504 }; 409 };
505 410
506 /// The next step, given what the last one produced. `null` is the first 411 /// The next step, given what the last one produced; `null` is the first call.
507 /// call, which has produced nothing yet. 412 /// An (outcome, phase) pair no row covers is `unreachable`: the driver is the
508 /// 413 /// only caller and answers each step from a fixed set, so a pair outside the
509 /// An (outcome, phase) pair no row covers is `unreachable`: the driver is 414 /// table is a bug in the loop rather than input to judge.
510 /// the only caller, it answers each step from a fixed set, and a pair
511 /// outside the table is a bug in the loop rather than input to judge.
512 pub fn next(s: *State, o: ?Outcome) Step { 415 pub fn next(s: *State, o: ?Outcome) Step {
513 const outcome = o orelse { 416 const outcome = o orelse {
514 std.debug.assert(s.phase == .init); 417 std.debug.assert(s.phase == .init);
@@ -705,12 +608,9 @@ test "announce: format → parse round-trip, with and without the newline" {
705 const crlf = (try parseAnnounce(crlf_buf[0 .. line.len + 1])).?; 608 const crlf = (try parseAnnounce(crlf_buf[0 .. line.len + 1])).?;
706 try std.testing.expectEqual(ep.port, crlf.port); 609 try std.testing.expectEqual(ep.port, crlf.port);
707 610
708 // The longest line the grammar can produce, formatted into a buffer 611 // The longest line the grammar can produce, into a buffer sized by the
709 // sized by the constant that claims to bound it. Every caller sizes 612 // constant that bounds it. A drift there fails on whichever port happens to
710 // its buffer from announce_max_len, so if that number ever drifts 613 // be five digits, on someone else's box; here it is one assertion.
711 // from the grammar the failure lands on whichever port happens to be
712 // five digits — in production, on someone else's box. Here it is one
713 // assertion.
714 var max_buf: [announce_max_len]u8 = undefined; 614 var max_buf: [announce_max_len]u8 = undefined;
715 const max_line = try formatAnnounce(&max_buf, .{ .port = 65535, .key = [_]u8{0xAB} ** 32 }); 615 const max_line = try formatAnnounce(&max_buf, .{ .port = 65535, .key = [_]u8{0xAB} ** 32 });
716 try std.testing.expectEqual(@as(usize, announce_max_len), max_line.len); 616 try std.testing.expectEqual(@as(usize, announce_max_len), max_line.len);
@@ -733,12 +633,9 @@ test "announce: a key with leading zero bytes still hexes to 64 chars" {
733 } 633 }
734 634
735 test "announce: the WRITER refuses port 0, where the mistake is still local" { 635 test "announce: the WRITER refuses port 0, where the mistake is still local" {
736 // `endpoint_reply` carries 0 as a legal wire value meaning "could 636 // `endpoint_reply` carries 0 as "could not", so the reader must turn it
737 // not", so the reader holds a u16 that may be 0 and must turn it into 637 // into `endpoint none` rather than a line. Refusing at the WRITER means a
738 // `endpoint none` rather than a line. Refusing at the writer means a 638 // caller that forgets fails on its own line, not on the client.
739 // caller that forgets fails on its own line, instead of shipping a
740 // line that only fails much later, on the client, as a parse error
741 // about a message this side produced.
742 var buf: [announce_max_len]u8 = undefined; 639 var buf: [announce_max_len]u8 = undefined;
743 try std.testing.expectError( 640 try std.testing.expectError(
744 ParseError.AnnouncePortZero, 641 ParseError.AnnouncePortZero,
@@ -784,13 +681,9 @@ fn expectArgv(want: []const []const u8, got: []const []const u8) !void {
784 test "recipeFor: the remote command carries ~/.local/bin itself — sshd's non-login shell never sources the profile that would" { 681 test "recipeFor: the remote command carries ~/.local/bin itself — sshd's non-login shell never sources the profile that would" {
785 const r = try recipeFor(std.testing.allocator, "user@box", false); 682 const r = try recipeFor(std.testing.allocator, "user@box", false);
786 defer r.deinit(std.testing.allocator); 683 defer r.deinit(std.testing.allocator);
787 // APPENDED, not prepended: this adds a place to look when `mux` is 684 // APPENDED, not prepended: a place to look when `mux` is nowhere on the
788 // nowhere on the remote PATH; it must never let a stale ~/.local/bin 685 // remote PATH, never a shadow over one it already resolves. The remote
789 // shadow a `mux` the PATH already resolves. 686 // command is ONE word — no local shell strips anything off it.
790 //
791 // The remote command is ONE word, unquoted: no local shell strips
792 // anything off it, so the quotes a shell line needed would arrive at
793 // sshd as literal bytes.
794 try expectArgv(&.{ 687 try expectArgv(&.{
795 "ssh", 688 "ssh",
796 "user@box", 689 "user@box",
@@ -841,13 +734,9 @@ test "recipeFor: a batch recipe cannot prompt, an interactive one still can" {
841 argvWord(quiet.ssh_argv, "BatchMode=yes").? < 734 argvWord(quiet.ssh_argv, "BatchMode=yes").? <
842 argvWord(quiet.ssh_argv, h).?, 735 argvWord(quiet.ssh_argv, h).?,
843 ); 736 );
844 // The other half of "nobody is sitting in front of this": a 737 // The other half of "nobody is sitting in front of this": a blackholed
845 // blackholed host — dropped SYNs, no RST — leaves ssh in the 738 // host leaves ssh in the kernel's TCP retry schedule for two minutes, and
846 // kernel's TCP retry schedule for about two minutes, and 739 // `client.listSessions` starts its budget only AFTER the open returns.
847 // `client.listSessions` starts its own budget only AFTER the open
848 // returns. So `mux hosts` appears to hang per dead host, serially,
849 // and a poll thread carries a stale tile for the whole stall with
850 // no abort fd to interrupt it.
851 try std.testing.expect(argvWord(quiet.ssh_argv, "ConnectTimeout=5") != null); 740 try std.testing.expect(argvWord(quiet.ssh_argv, "ConnectTimeout=5") != null);
852 try std.testing.expect( 741 try std.testing.expect(
853 argvWord(quiet.ssh_argv, "ConnectTimeout=5").? < 742 argvWord(quiet.ssh_argv, "ConnectTimeout=5").? <
@@ -933,11 +822,9 @@ test "cache: a looser file standing in the cache's place is re-tightened" {
933 const ep: Endpoint = .{ .port = 4433, .key = [_]u8{0xCD} ** 32 }; 822 const ep: Endpoint = .{ .port = 4433, .key = [_]u8{0xCD} ** 32 };
934 try writeCache(path, ep); 823 try writeCache(path, ep);
935 824
936 // createFile's `.mode` applies at creation only, so writing over the 825 // `createFile`'s `.mode` applies at creation only, so writing over the file
937 // file does NOT re-tighten it. Without an explicit chmod the 0644 826 // does NOT re-tighten it: without the chmod a 0644 survives, `readCache`
938 // survives, readCache refuses the cache it just wrote, and the host is 827 // refuses the cache it just wrote, and the host is permanently cold.
939 // permanently cold with nothing in any log to point at. That chmod is
940 // one deletable line, which is exactly why it needs an assertion.
941 const f = try std.fs.cwd().openFile(path, .{}); 828 const f = try std.fs.cwd().openFile(path, .{});
942 defer f.close(); 829 defer f.close();
943 const st = try f.stat(); 830 const st = try f.stat();
@@ -988,12 +875,9 @@ test "cache: refuses a permissive file, a missing one, and `endpoint none`" {
988 } 875 }
989 try std.testing.expectError(CacheError.CacheMalformed, readCache(path)); 876 try std.testing.expectError(CacheError.CacheMalformed, readCache(path));
990 877
991 // Content that is not the grammar at all reports the SAME error. The 878 // Content that is not the grammar at all reports the SAME error: the
992 // granular parse names exist for the ssh-pipe path, where the caller 879 // granular names exist for the ssh-pipe path, and a cache has one decision.
993 // can say something useful about a daemon that answered oddly; for a 880 // Two vocabularies for one condition means every caller learns both.
994 // cache there is one decision — use it or refetch — so there is one
995 // error. Two vocabularies for one condition would just mean every
996 // caller had to know both.
997 try std.fs.cwd().writeFile(.{ .sub_path = path, .data = "garbage\n" }); 881 try std.fs.cwd().writeFile(.{ .sub_path = path, .data = "garbage\n" });
998 { 882 {
999 const f = try std.fs.cwd().openFile(path, .{}); 883 const f = try std.fs.cwd().openFile(path, .{});
@@ -1056,11 +940,9 @@ test "handoff step: the abort key inside the warm dial asked to stop, not to try
1056 test "handoff step: a warm dial that failed hands the question to ssh, and remembers what went silent" { 940 test "handoff step: a warm dial that failed hands the question to ssh, and remembers what went silent" {
1057 var s: State = .{ .cached = epA(), .asked = false, .has_cache = true, .phase = .warm_dial }; 941 var s: State = .{ .cached = epA(), .asked = false, .has_cache = true, .phase = .warm_dial };
1058 try expectStep("spawn_ssh", next(&s, .failed)); 942 try expectStep("spawn_ssh", next(&s, .failed));
1059 // Remembered here or nowhere: by the time the announce comes back, the 943 // Remembered here or nowhere: by the time the announce comes back the cache
1060 // cache holds whatever ssh just said and cannot say what was tried. 944 // holds what ssh just said. `!= null` before the compare, not a `.?`: an
1061 // `!= null` before the compare, and not a `.?`: an unwrap of null 945 // unwrap panics, taking the binary and every later test down with it.
1062 // PANICS, which takes the whole test binary down and every later test
1063 // with it — including the walk that would have named the same defect.
1064 try std.testing.expect(s.failed_on != null and std.meta.eql(s.failed_on.?, epA())); 946 try std.testing.expect(s.failed_on != null and std.meta.eql(s.failed_on.?, epA()));
1065 } 947 }
1066 948
@@ -1200,12 +1082,10 @@ test "handoff walk: a warm cache that answers is one dial and nothing else" {
1200 } 1082 }
1201 1083
1202 test "handoff walk: a warm miss whose refetch names the SAME endpoint pays one deadline, not two" { 1084 test "handoff walk: a warm miss whose refetch names the SAME endpoint pays one deadline, not two" {
1203 // The case this table was built for. On a network where the announced 1085 // The case this table was built for: where the announced UDP port cannot be
1204 // UDP port cannot be reached — blocked, jump-hosted, a listener holding 1086 // reached, the warm dial spends the budget and ssh answers with the very
1205 // another key — the warm dial spends the budget, ssh answers with the 1087 // coordinates that went silent — dialling them again spends it twice.
1206 // very coordinates that just went silent, and a client that dialled 1088 // EXACTLY ONE `dial_quic` in this trace.
1207 // them again spent it twice (measured 4264ms against a spec of "within
1208 // one deadline"). EXACTLY ONE `dial_quic` in this trace.
1209 var s: State = .{ .cached = epA(), .asked = true, .has_cache = true }; 1089 var s: State = .{ .cached = epA(), .asked = true, .has_cache = true };
1210 try expectWalk( 1090 try expectWalk(
1211 "dial_quic 4433/ab, spawn_ssh, read_announce, write_cache 4433/ab, use_pipe(line)", 1091 "dial_quic 4433/ab, spawn_ssh, read_announce, write_cache 4433/ab, use_pipe(line)",
src/client/webhub.zig
Old New
@@ -1,17 +1,10 @@
1 //! The `mux web` hub's HTTP/WebSocket layer: route table, 1 //! The `mux web` hub's HTTP/WebSocket layer: route table, Origin gate and WS
2 //! Origin gate, and the WS endpoint naming — the decisions std.http does 2 //! endpoint naming — the decisions std.http does NOT make for us. Assets are
3 //! NOT make for us. The connection loop and per-tile pump build on this; 3 //! `@embedFile`'d by webhub_main.zig and injected, so this tests without wasm.
4 //! the assets are @embedFile'd by webhub_main.zig (the exe
5 //! root) and injected here, which keeps this module testable without
6 //! building the wasm artifact.
7 //! 4 //!
8 //! Localhost only, by construction: the hub binds 127.0.0.1 and there is 5 //! Localhost only, by construction: remote viewing is `ssh -L`, authenticated
9 //! no flag to change that in v1 — remote viewing is `ssh -L`, 6 //! by ssh like everything else here. No popup, so no prompts — a birth on a
10 //! authenticated by ssh like everything else in this project. 7 //! host that wants a password fails with ssh's own reason on `/tiles`.
11 //!
12 //! No popup, so no prompts: every target here keeps `ask_sock` null and a
13 //! birth on a host that wants a password fails with ssh's own reason on
14 //! `/tiles`. A browser is not somewhere to type one.
15 8
16 const std = @import("std"); 9 const std = @import("std");
17 const proto = @import("term").protocol; 10 const proto = @import("term").protocol;
@@ -19,13 +12,10 @@ const client = @import("client");
19 12
20 pub const default_port: u16 = 7681; 13 pub const default_port: u16 = 7681;
21 14
22 /// ONE number bounds two things, a property of std.http.Server: the 15 /// ONE number bounds two things, a property of std.http.Server: the connection
23 /// buffer handed to the connection's Reader is both the max HTTP header 16 /// Reader's buffer is both the max HTTP header size and the max inbound
24 /// size AND the max inbound WebSocket message (readSmallMessage rejects 17 /// WebSocket message. 64 KiB — the browser chunks pastes at 32 KiB, so nothing
25 /// fragmented messages outright and caps at the buffer length). 64 KiB: 18 /// approaches it. Hub→browser has no such bound, so snapshots are safe.
26 /// headers never approach it, keystrokes are bytes, and the browser side
27 /// chunks pastes at 32 KiB so no message approaches it either. Hub→
28 /// browser has no such bound (u64 lengths) — snapshots are safe.
29 pub const ws_buffer_len = 64 * 1024; 19 pub const ws_buffer_len = 64 * 1024;
30 20
31 /// Any webpage may dial ws://127.0.0.1:PORT — localhost binding does not 21 /// Any webpage may dial ws://127.0.0.1:PORT — localhost binding does not
@@ -105,13 +95,10 @@ const HubTile = struct {
105 missed_once: bool = false, 95 missed_once: bool = false,
106 }; 96 };
107 97
108 /// The wall at runtime: the hosts file's daemons, their live sessions as 98 /// The wall at runtime: the hosts file's daemons, their live sessions as tiles,
109 /// tiles, and the ids the browser names them by. 99 /// and the ids the browser names them by. Ids are handed out once and never
110 /// 100 /// reused — the browser holds them across changes it did not make, and a
111 /// Ids are handed out once and never reused, because the browser holds 101 /// recycled one would re-point a `/ws/<n>` at a different shell.
112 /// them across a change it did not make: an index — or a recycled id —
113 /// would silently re-point a `/ws/<n>` at a different shell the moment a
114 /// session ended somewhere else.
115 pub const Hub = struct { 102 pub const Hub = struct {
116 alloc: std.mem.Allocator, 103 alloc: std.mem.Allocator,
117 /// One lock for the tile list. Every writer is a poller's wake or a 104 /// One lock for the tile list. Every writer is a poller's wake or a
@@ -134,15 +121,10 @@ pub const Hub = struct {
134 } 121 }
135 122
136 pub fn deinit(self: *Hub) void { 123 pub fn deinit(self: *Hub) void {
137 // A started hub cannot be torn down. Its pollers are detached, hold 124 // A started hub cannot be torn down: its pollers are detached, hold
138 // `*HubHost` into the array freed below, and have no stop to be 125 // `*HubHost` into the array freed below, and have no stop to ask for.
139 // asked for — a hub polls until the process leaves the accept loop, 126 // An assert rather than a comment, because the day someone adds a
140 // which is why `HubHost.keep` is a constant where the CLI wall's 127 // shutdown path this free is a use-after-free per host.
141 // answers `shared.running` and the picker's `forgotten`. `mux web`
142 // never reaches here; the tests that do never `start`. An assert
143 // rather than a comment because the day someone adds a shutdown
144 // path, the free below is a use-after-free in as many threads as
145 // there are hosts, and nothing else would say so.
146 std.debug.assert(!self.polling); 128 std.debug.assert(!self.polling);
147 for (self.tiles.items) |t| self.alloc.free(t.session); 129 for (self.tiles.items) |t| self.alloc.free(t.session);
148 self.tiles.deinit(self.alloc); 130 self.tiles.deinit(self.alloc);
@@ -249,12 +231,9 @@ pub const Hub = struct {
249 /// Caller holds the mutex. 231 /// Caller holds the mutex.
250 fn vanish(self: *Hub, i: usize) void { 232 fn vanish(self: *Hub, i: usize) void {
251 const t = self.tiles.orderedRemove(i); 233 const t = self.tiles.orderedRemove(i);
252 // Wake the pump before freeing what it might still be reading for: 234 // Wake the pump before freeing what it might still be reading for.
253 // shutdown unblocks its recv, and it unregisters the fd itself. 235 // The raw syscall, ignoring errno: `std.posix.shutdown` calls BADF and
254 // The raw syscall, ignoring errno: std.posix.shutdown calls BADF 236 // NOTSOCK unreachable, and neither is a reason to abort a removal.
255 // and NOTSOCK unreachable, and neither is a reason to abort a
256 // removal — a registered fd whose pump is already unwinding is
257 // exactly the race this wakes up.
258 if (t.ws_fd) |fd| _ = std.os.linux.shutdown(fd, std.os.linux.SHUT.RDWR); 237 if (t.ws_fd) |fd| _ = std.os.linux.shutdown(fd, std.os.linux.SHUT.RDWR);
259 std.debug.print("mux web: tile {d}: gone\n", .{t.id}); 238 std.debug.print("mux web: tile {d}: gone\n", .{t.id});
260 self.alloc.free(t.session); 239 self.alloc.free(t.session);
@@ -274,15 +253,10 @@ pub const Hub = struct {
274 // so there is nothing here to copy field by field — the three 253 // so there is nothing here to copy field by field — the three
275 // comptime field-count asserts a copy needed are gone with it. 254 // comptime field-count asserts a copy needed are gone with it.
276 const copy = Checkout{ .target = self.hosts[self.tiles.items[idx].host].spec.target }; 255 const copy = Checkout{ .target = self.hosts[self.tiles.items[idx].host].spec.target };
277 // FIRST registration wins: two browsers on one wall run two pumps 256 // FIRST registration wins: two browsers run two pumps per tile, and
278 // per tile, and overwriting would leave a vanishing able to wake 257 // overwriting would let the untracked pump's release clear the tracked
279 // only the last one — worse, the untracked pump's release would 258 // one's fd, so a vanishing would wake nobody. The second pump still
280 // then clear the tracked one's fd and the vanishing would wake 259 // serves its browser and its own WS read ends it.
281 // nobody. The second pump still runs and still serves its browser;
282 // it is simply not the shutdown-tracked one, and its own WS read
283 // ends it when the browser goes away. Best-effort single-fd
284 // tracking — a full fd list per tile is deliberately deferred
285 // until two-browser removal latency is shown to matter.
286 if (self.tiles.items[idx].ws_fd == null) self.tiles.items[idx].ws_fd = ws_fd; 260 if (self.tiles.items[idx].ws_fd == null) self.tiles.items[idx].ws_fd = ws_fd;
287 return copy; 261 return copy;
288 } 262 }
@@ -385,11 +359,9 @@ pub fn route(assets: Assets, path: []const u8) ?Asset {
385 } 359 }
386 360
387 // --------------------------------------------------------------------------- 361 // ---------------------------------------------------------------------------
388 // The wire between hub and browser: WebSocket binary 362 // The wire between hub and browser: WebSocket binary messages, one envelope
389 // messages, one envelope byte. 0x00 + mux protocol frame verbatim, both 363 // byte. 0x00 + a mux frame verbatim both ways; 0x01 + JSON control, hub→browser
390 // directions; 0x01 + UTF-8 JSON control message, hub→browser only. That 364 // only. The whole vocabulary — anything the protocol learns later transits.
391 // is the whole vocabulary — anything the mux protocol learns to say
392 // later transits untouched.
393 365
394 pub const env_frame: u8 = 0x00; 366 pub const env_frame: u8 = 0x00;
395 pub const env_control: u8 = 0x01; 367 pub const env_control: u8 = 0x01;
@@ -416,12 +388,10 @@ pub const FrameMsgError = error{
416 HubOwned, 388 HubOwned,
417 }; 389 };
418 390
419 /// One browser→hub WebSocket message → one mux frame. The hub parses the 391 /// One browser→hub WebSocket message → one mux frame. The hub parses the 5-byte
420 /// 5-byte header (it must — the WS leg is message-delimited while the 392 /// header — it must, since the WS leg is message-delimited and the daemon leg is
421 /// daemon leg is a byte stream) and NEVER the payload; the spec's 393 /// a byte stream — and NEVER the payload. The length must match exactly:
422 /// amendment 2 is this function's contract. The length must match the 394 /// WebSocket already delimits, so a trailing byte is a bug upstream.
423 /// message exactly: WebSocket already delimits, so a trailing byte is a
424 /// bug upstream, not framing to resynchronize.
425 pub fn parseFrameMessage(msg: []const u8) FrameMsgError!ParsedFrame { 395 pub fn parseFrameMessage(msg: []const u8) FrameMsgError!ParsedFrame {
426 if (msg.len < 1 or msg[0] != env_frame) return error.BadEnvelope; 396 if (msg.len < 1 or msg[0] != env_frame) return error.BadEnvelope;
427 const f = msg[1..]; 397 const f = msg[1..];
@@ -442,21 +412,14 @@ pub fn parseFrameMessage(msg: []const u8) FrameMsgError!ParsedFrame {
442 } 412 }
443 413
444 /// What the WS reader's ALREADY-BUFFERED bytes will do to the next 414 /// What the WS reader's ALREADY-BUFFERED bytes will do to the next
445 /// `readSmallMessage`, decided without touching the socket. 415 /// `readSmallMessage`, decided without touching the socket. It exists because
446 /// 416 /// that call BLOCKS on a partial frame, parking the tile thread with the daemon
447 /// This exists because readSmallMessage blocks: it fills from the socket 417 /// leg unattended — and poll cannot rescue it, since the bytes already in
448 /// whenever the frame it is decoding runs past what the reader holds. A 418 /// userspace are what made poll fire in the first place.
449 /// pump that calls it on a partial frame parks the tile thread with the
450 /// daemon leg unattended — and poll will NOT rescue it, because the
451 /// missing bytes are not in the kernel; the bytes that are already in
452 /// userspace were what made poll fire in the first place.
453 pub const HeadFrame = union(enum) { 419 pub const HeadFrame = union(enum) {
454 /// No complete frame at the head. Leave POLLIN armed and re-check 420 /// No complete frame at the head: leave POLLIN armed and re-check after the
455 /// after the next readable event: a frame that is short is a frame 421 /// next readable event, since a short frame's rest is still in flight. A
456 /// whose rest is still in flight, so more bytes ARE coming and poll 422 /// peer that then goes quiet is what the dead-leg ping below ends.
457 /// will fire again. (A peer that sends half a frame and then goes
458 /// quiet parks this tile in poll — which is exactly where the
459 /// dead-leg ping below finds it and ends the tile.)
460 incomplete, 423 incomplete,
461 /// A complete pong at the head, `bytes` long. readSmallMessage 424 /// A complete pong at the head, `bytes` long. readSmallMessage
462 /// SWALLOWS pongs and loops to the frame behind them, so handing it 425 /// SWALLOWS pongs and loops to the frame behind them, so handing it
@@ -466,14 +429,10 @@ pub const HeadFrame = union(enum) {
466 /// A whole frame is buffered: readSmallMessage returns it (or names 429 /// A whole frame is buffered: readSmallMessage returns it (or names
467 /// it a close) without touching the socket. 430 /// it a close) without touching the socket.
468 ready, 431 ready,
469 /// The frame does not FIT the reader's buffer, so no amount of 432 /// The frame does not FIT the reader's buffer, so no waiting makes it
470 /// waiting makes it readable without blocking — and a pump that kept 433 /// readable and a pump that waited would fill the buffer with a frame it can
471 /// waiting would eventually fill the buffer with a frame it can never 434 /// never finish. Said here so `readSmallMessage` is never handed one it
472 /// finish. readSmallMessage answers most of these with MessageTooBig 435 /// would block on. Our page's largest message is half the buffer.
473 /// and the pump ends the tile either way, so it is said here instead:
474 /// never hand that function a frame it would have to block on. Our
475 /// own page's largest message is a 32 KiB paste chunk against a
476 /// 64 KiB buffer, so reaching this means a peer that is not our page.
477 too_big, 436 too_big,
478 }; 437 };
479 438
@@ -509,51 +468,32 @@ pub fn headFrame(buffered: []const u8, capacity: usize) HeadFrame {
509 if (buffered.len < off + 4) return .incomplete; 468 if (buffered.len < off + 4) return .incomplete;
510 off += 4; 469 off += 4;
511 } 470 }
512 // HEADER AND PAYLOAD against the capacity, not the payload alone: the 471 // HEADER AND PAYLOAD against the capacity: the reader holds both, so a
513 // reader holds both, so a payload that only just fits still leaves a 472 // payload that only just fits still leaves a frame that never completes —
514 // frame that never completes. 473 // and `fillMore`'s rebase asserts on a FULL buffer, so saying `.too_big`
515 // 474 // first is what keeps the pump off that panic.
516 // Load-bearing for fillMore, not just tidiness. fillMore calls
517 // rebase(bufferedLen + 1), and std's defaultRebase asserts on exit
518 // that `buffer.len - seek >= capacity` — which cannot hold once the
519 // buffer is FULL. Saying .too_big before a frame can fill the buffer
520 // is what keeps the pump off that assert; without it, a peer that
521 // declares a frame just too large to fit sends the hub into an
522 // unreclaimable buffer and a panic.
523 // 475 //
524 // Written as a subtraction on the RIGHT because payload_len is a wire 476 // A subtraction on the RIGHT, because `payload_len` is a wire number:
525 // number: `off + payload_len` overflows on a declared length near 477 // `off + payload_len` overflows near u64 max, which is a Debug panic and a
526 // u64 max (a 14-byte message does it), which is a Debug panic — and 478 // wrap to a permanent `.incomplete` in ReleaseFast.
527 // the default build IS Debug — and a wrap to a permanent .incomplete
528 // hang in ReleaseFast. `capacity < off` is checked too: this is a pub
529 // function and nothing guarantees a caller's buffer outgrows a
530 // 14-byte header.
531 if (capacity < off or payload_len > capacity - off) return .too_big; 479 if (capacity < off or payload_len > capacity - off) return .too_big;
532 if (buffered.len - off < payload_len) return .incomplete; 480 if (buffered.len - off < payload_len) return .incomplete;
533 if (opcode == ws_opcode_pong) return .{ .pong = off + @as(usize, @intCast(payload_len)) }; 481 if (opcode == ws_opcode_pong) return .{ .pong = off + @as(usize, @intCast(payload_len)) };
534 return .ready; 482 return .ready;
535 } 483 }
536 484
537 /// Dead browser leg. The pump's poll is capped at 100 ms because the QUIC 485 /// Dead browser leg. The pump's poll is capped at 100 ms for the QUIC timer, so
538 /// timer needs the tick, so silence is measured on the wall clock rather 486 /// silence is measured on the wall clock: the hub pings after `ping_idle_ms` and
539 /// than by a poll timeout: after `ping_idle_ms` with nothing inbound the 487 /// gives up after `dead_intervals`. What this releases is the daemon CLIENT SLOT
540 /// hub pings, and after `dead_intervals` of them it gives up. What this 488 /// a half-open socket would otherwise hold until the daemon exits.
541 /// releases is not just a thread — it is the daemon client slot behind
542 /// it, which a browser that vanished with a half-open socket would
543 /// otherwise hold until the daemon exits.
544 pub const ping_idle_ms: i64 = 30_000; 489 pub const ping_idle_ms: i64 = 30_000;
545 pub const dead_intervals: i64 = 3; 490 pub const dead_intervals: i64 = 3;
546 491
547 /// The browser leg's liveness, carried BY POINTER across dialLoop — 492 /// The browser leg's liveness, carried BY POINTER across `dialLoop` — which is
548 /// which is the whole point of it being a struct. 493 /// the whole point of the struct. A reconnect is when a dead browser is most
549 /// 494 /// likely and nothing else is watching: leaving the timer to the pump blinds the
550 /// A reconnect is when a dead browser is most likely and when nothing 495 /// check for the outage, and resetting it on the way out hides a browser that
551 /// else is watching. Two shortcuts are both wrong: leaving the timer to 496 /// died mid-way. One ping in both loops against one clock.
552 /// the pump blinds the check for the length of the outage; resetting the
553 /// clock on the way out of dialLoop blinds it AND lets a long outage hide
554 /// a browser that died mid-way. So one ping runs in both loops against
555 /// one clock. A browser merely waiting answers the ping in its WebSocket
556 /// stack without waking the page, so silence here really is silence.
557 const Liveness = struct { 497 const Liveness = struct {
558 last_inbound_ms: i64, 498 last_inbound_ms: i64,
559 pings_sent: i64 = 0, 499 pings_sent: i64 = 0,
@@ -592,14 +532,10 @@ fn drainBrowser(
592 fill: bool, 532 fill: bool,
593 transport: ?*client.Transport, 533 transport: ?*client.Transport,
594 ) Drained { 534 ) Drained {
595 // poll reports what the KERNEL holds; one fill turns that readable 535 // poll reports what the KERNEL holds, and one fill turns that into buffered
596 // event into buffered bytes, and it cannot block — poll just said 536 // bytes without blocking. Readiness gates only this step: bytes already in
597 // there are bytes (or an EOF, which ends the tile right here). 537 // userspace are invisible to poll, so a drain that ran only on readiness
598 // Readiness gates only this step: bytes already in userspace are 538 // would strand what a mid-drain reconnect left — a close frame among them.
599 // invisible to poll (they are what made it fire last time), so a
600 // drain that ran only on readiness would strand whatever a mid-drain
601 // reconnect left behind — a close frame stranded that way holds this
602 // tile's daemon client slot until the reaper takes it.
603 if (fill) ws.input.fillMore() catch return .browser_dead; 539 if (fill) ws.input.fillMore() catch return .browser_dead;
604 while (true) { 540 while (true) {
605 // Every read is gated on headFrame: readSmallMessage may not be 541 // Every read is gated on headFrame: readSmallMessage may not be
@@ -651,12 +587,9 @@ fn redial(
651 live: *Liveness, 587 live: *Liveness,
652 dial: *Dial, 588 dial: *Dial,
653 ) bool { 589 ) bool {
654 // A dial that never saw a grid was REFUSED, and a refusal is a state, 590 // A dial that never saw a grid was REFUSED, and a refusal is a state: the
655 // not an event: the daemon closes on it, so the next dial opens on its 591 // next dial opens on its first try and the page walks back into the same no.
656 // first try and the page — which re-attaches on `up` — walks back into 592 // A dial that DID see a grid was torn, and a tear still heals at once.
657 // the same no. Charging the backoff here is what turns that loop into
658 // a poll. A dial that DID see a grid was torn, and a tear still heals
659 // at once.
660 if (!dial.saw_grid) dial.spin_ms = client.nextBackoffMs(dial.spin_ms); 593 if (!dial.saw_grid) dial.spin_ms = client.nextBackoffMs(dial.spin_ms);
661 // The per-dial reset belongs here, not at the three call sites: one of 594 // The per-dial reset belongs here, not at the three call sites: one of
662 // them forgot, and a forgotten reset stays silent until a torn 595 // them forgot, and a forgotten reset stays silent until a torn
@@ -706,11 +639,9 @@ pub fn pumpTile(
706 target_in: client.Target, 639 target_in: client.Target,
707 ) void { 640 ) void {
708 var target = target_in; 641 var target = target_in;
709 // No terminal to spam and a control channel that already narrates: 642 // No terminal to spam and a control channel that already narrates, so the
710 // the one fallback line the CLI allows itself is quieted here, the 643 // CLI's one fallback line is quieted. It also keeps a browser from starting
711 // same way reconnect() quiets retries. It is also what keeps a browser 644 // daemons: a hub tile redials for as long as the page is open.
712 // from starting daemons: a hub tile redials for as long as the page is
713 // open, and nobody is watching it do so.
714 if (target == .hand) target.hand.asked = false; 645 if (target == .hand) target.hand.asked = false;
715 646
716 var dial: Dial = .{}; 647 var dial: Dial = .{};
@@ -719,11 +650,9 @@ pub fn pumpTile(
719 ws.writeMessage(controlMessage(.connecting), .binary) catch return; 650 ws.writeMessage(controlMessage(.connecting), .binary) catch return;
720 var transport = dialLoop(alloc, target, ws, ws_fd, &live, 0) orelse return; 651 var transport = dialLoop(alloc, target, ws, ws_fd, &live, 0) orelse return;
721 defer transport.close(); 652 defer transport.close();
722 // `up` is not decoration: the browser re-attaches when it reads this, 653 // `up` is not decoration: the browser re-attaches when it reads this, and
723 // and the hub never re-attaches on its behalf. mux.js's ENV_CONTROL 654 // the hub never re-attaches on its behalf. mux.js's ENV_CONTROL handler is
724 // handler is the other half of that contract — the residual coupling 655 // the other half of that contract.
725 // this module has to the page it serves, named here where the message
726 // is written rather than left to be discovered.
727 ws.writeMessage(controlMessage(.up), .binary) catch return; 656 ws.writeMessage(controlMessage(.up), .binary) catch return;
728 657
729 outer: while (true) { 658 outer: while (true) {
@@ -754,13 +683,10 @@ pub fn pumpTile(
754 .incomplete => break :frames, 683 .incomplete => break :frames,
755 .closed => { 684 .closed => {
756 if (!redial(alloc, &transport, target, ws, ws_fd, &live, &dial)) return; 685 if (!redial(alloc, &transport, target, ws, ws_fd, &live, &dial)) return;
757 // fds[1].revents describes a socket state from 686 // `fds[1].revents` describes a socket state from BEFORE
758 // BEFORE the re-dial, and dialLoop may have eaten 687 // the re-dial, and `dialLoop` may have eaten the message
759 // the very message it described. Re-poll instead 688 // it described. Nothing is stranded by re-polling: the
760 // of reading on a readiness that is now a rumour; 689 // drain below runs on buffered bytes regardless.
761 // nothing is stranded by the skip, because the
762 // drain below runs on buffered bytes regardless of
763 // what the next poll says.
764 continue :outer; 690 continue :outer;
765 }, 691 },
766 }; 692 };
@@ -789,12 +715,9 @@ pub fn pumpTile(
789 }, 715 },
790 } 716 }
791 717
792 // Is anyone still there? A closed tab usually arrives as a close 718 // Is anyone still there? A closed tab arrives as a close frame or EOF,
793 // frame or EOF, but a laptop that slept, a killed browser, or a 719 // but a slept laptop or a dropped `ssh -L` leaves the socket half-open
794 // dropped ssh -L leaves the socket half-open and silent forever. 720 // and silent forever. Best-effort: it is measured between passes.
795 // Best-effort, not a bound: it is measured between passes, and a
796 // blocking upstream readFrame parks the pass it is in for as long
797 // as the transport takes.
798 if (!live.tick(ws)) return; 721 if (!live.tick(ws)) return;
799 } 722 }
800 } 723 }
@@ -807,11 +730,9 @@ fn dialLoop(
807 ws: *std.http.Server.WebSocket, 730 ws: *std.http.Server.WebSocket,
808 ws_fd: std.posix.fd_t, 731 ws_fd: std.posix.fd_t,
809 live: *Liveness, 732 live: *Liveness,
810 /// Where the wait resumes. Zero is the ordinary dial — a link that 733 /// Where the wait resumes. Zero is the ordinary dial, so the wait below is
811 /// just died usually reconnects now — and the wait below is skipped 734 /// skipped on the first pass. Nonzero is `redial` saying the last dial was
812 /// on the first pass exactly as it always was. Nonzero is `redial` 735 /// REFUSED: the target is reachable, so retrying at once buys the same no.
813 /// saying the last dial ended in a REFUSAL: the target is reachable,
814 /// so opening it again at once buys nothing but the same no.
815 backoff_start_ms: u64, 736 backoff_start_ms: u64,
816 ) ?client.Transport { 737 ) ?client.Transport {
817 var backoff_ms: u64 = backoff_start_ms; 738 var backoff_ms: u64 = backoff_start_ms;
@@ -849,11 +770,9 @@ fn dialLoop(
849 } 770 }
850 } 771 }
851 772
852 /// Labels are argv, not hostile input, but a path with a quote in it must 773 /// Labels are argv, not hostile input, but a path with a quote in it must not
853 /// not break the page. Deliberately NOT `mux a`'s jsonEscape: this one sends 774 /// break the page. Deliberately NOT `mux a`'s `jsonEscape`: this sends every
854 /// every control byte to `\u00XX` (one rule, no table to get wrong) while 775 /// control byte to `\u00XX`, one rule with no table to get wrong.
855 /// `mux a` spells the short forms. Both parse identically, the bytes differ,
856 /// and each is pinned by its own test.
857 fn appendJsonString(alloc: std.mem.Allocator, out: *std.ArrayList(u8), s: []const u8) !void { 776 fn appendJsonString(alloc: std.mem.Allocator, out: *std.ArrayList(u8), s: []const u8) !void {
858 try out.append(alloc, '"'); 777 try out.append(alloc, '"');
859 for (s) |c| switch (c) { 778 for (s) |c| switch (c) {
@@ -897,13 +816,10 @@ pub fn serveConn(
897 if (std.ascii.eqlIgnoreCase(h.name, "origin")) origin = h.value; 816 if (std.ascii.eqlIgnoreCase(h.name, "origin")) origin = h.value;
898 } 817 }
899 818
900 // std's `discardBody` ASSERTS that a kept-alive request whose method 819 // std's `discardBody` ASSERTS that a kept-alive request whose method may
901 // may carry a body declared a length — and `POST /anything` with 820 // carry a body declared a length, and `curl -X POST` sends neither —
902 // neither, which is exactly what `curl -X POST` sends, reached that 821 // which aborted the whole hub. Answering and closing covers every route
903 // assert and aborted the whole hub, every tile with it. Answering 822 // below; the page's own POST carries `content-length: 0` and keeps alive.
904 // such a request and closing is the one fix that covers every route
905 // below, including the asset router's 404. The page's own POST
906 // carries `content-length: 0` and keeps its connection.
907 const keep = !(method.requestHasBody() and 823 const keep = !(method.requestHasBody() and
908 req.head.content_length == null and 824 req.head.content_length == null and
909 req.head.transfer_encoding == .none); 825 req.head.transfer_encoding == .none);
@@ -931,12 +847,9 @@ pub fn serveConn(
931 // must never race a free of our own strings. 847 // must never race a free of our own strings.
932 var pump_arena = std.heap.ArenaAllocator.init(alloc); 848 var pump_arena = std.heap.ArenaAllocator.init(alloc);
933 defer pump_arena.deinit(); 849 defer pump_arena.deinit();
934 // Checkout BEFORE the upgrade, so an id that is not there is 850 // Checkout BEFORE the upgrade, so a missing id is answered in HTTP
935 // answered in HTTP — a 404 the page can read, rather than a 101 851 // — a 404 the page can read, not a 101 and a silent close. One name
936 // followed by a silent close it can only guess at. The fd is 852 // for the fd, so the release below names what checkout got.
937 // the same number either way: the upgrade rides this socket.
938 // One name for the fd we register with, so the release below
939 // provably names the same number it checked out under.
940 const ws_fd = stream.handle; 853 const ws_fd = stream.handle;
941 const checked = hub.checkoutTile(id, ws_fd) catch |err| switch (err) { 854 const checked = hub.checkoutTile(id, ws_fd) catch |err| switch (err) {
942 // Removed between the page's GET and this dial: the browser 855 // Removed between the page's GET and this dial: the browser
@@ -946,11 +859,9 @@ pub fn serveConn(
946 return; 859 return;
947 }, 860 },
948 }; 861 };
949 // Deferred, not called after pumpTile: an upgrade that fails 862 // Deferred, not called after `pumpTile`: a failed upgrade must
950 // must unregister too. Registered AFTER `defer stream.close()`, 863 // unregister too. Declared AFTER `defer stream.close()` so it runs
951 // so it runs BEFORE it — the ordering that keeps a concurrent 864 // BEFORE it, or a vanishing shuts down a recycled fd.
952 // a vanishing from shutting down an fd the kernel has already
953 // handed to somebody else.
954 defer hub.releaseTile(id, ws_fd); 865 defer hub.releaseTile(id, ws_fd);
955 var ws = req.respondWebSocket(.{ .key = key }) catch return; 866 var ws = req.respondWebSocket(.{ .key = key }) catch return;
956 ws.flush() catch return; 867 ws.flush() catch return;
@@ -972,27 +883,19 @@ pub fn serveConn(
972 } 883 }
973 884
974 // The one mutation left, Origin-gated: a text/plain POST is a CSRF 885 // The one mutation left, Origin-gated: a text/plain POST is a CSRF
975 // "simple request" any web page can fire at localhost without a 886 // "simple request" any page can fire at localhost with no preflight.
976 // preflight; the gate is what keeps this page's power this page's. 887 // GET /tiles stays ungated — no CORS headers, so a hostile page can make
977 // GET /tiles above stays ungated: we send no CORS headers, so a 888 // the request and never read the answer. EXACT match only, or
978 // hostile page can make the request but never read the answer. 889 // `/tilesgarbage` routes into a mutation.
979 //
980 // Exact match only: a prefix match would route `/tilesgarbage`
981 // into a mutation and `/tiles?x=1` into a confusing 403 here
982 // instead of the asset router's plain 404. The API defines exactly
983 // "/tiles" (GET) and "/tiles/<id>" (POST); reject everything else
984 // by not matching it at all.
985 const tiles_root = std.mem.eql(u8, path, "/tiles"); 890 const tiles_root = std.mem.eql(u8, path, "/tiles");
986 const tile_id_suffix = if (std.mem.startsWith(u8, path, "/tiles/") and path.len > "/tiles/".len) 891 const tile_id_suffix = if (std.mem.startsWith(u8, path, "/tiles/") and path.len > "/tiles/".len)
987 path["/tiles/".len..] 892 path["/tiles/".len..]
988 else 893 else
989 null; 894 null;
990 if (tiles_root or tile_id_suffix != null) { 895 if (tiles_root or tile_id_suffix != null) {
991 // Every answer here ends the connection, and SAYS so: a 896 // Every answer here ends the connection and SAYS so: `keep` would be
992 // mutation is one request, the page fires one and reads one. 897 // true for the page's own `content-length: 0` POST, promising a
993 // `keep` would be true for the page's own `content-length: 0` 898 // connection this route then closes anyway.
994 // POST, so the header would promise a connection this route
995 // then closes anyway.
996 if (!originAllowed(origin, port)) { 899 if (!originAllowed(origin, port)) {
997 req.respond("forbidden\n", .{ .status = .forbidden, .keep_alive = false }) catch {}; 900 req.respond("forbidden\n", .{ .status = .forbidden, .keep_alive = false }) catch {};
998 return; 901 return;
@@ -1090,10 +993,8 @@ test "origin: exactly our two spellings pass, everything else refuses" {
1090 .{ .origin = "http://evil.example", .port = 7681, .want = false }, 993 .{ .origin = "http://evil.example", .port = 7681, .want = false },
1091 .{ .origin = "http://127.0.0.1:7681.evil.example", .port = 7681, .want = false }, 994 .{ .origin = "http://127.0.0.1:7681.evil.example", .port = 7681, .want = false },
1092 .{ .origin = "http://[::1]:7681", .port = 7681, .want = false }, 995 .{ .origin = "http://[::1]:7681", .port = 7681, .want = false },
1093 // A missing Origin header is a refusal, not a shrug: browsers 996 // A missing Origin header is a refusal, not a shrug: browsers always
1094 // always send it on cross-origin WebSocket dials, so its absence 997 // send it on cross-origin dials, and "absent means yes" is how localhost
1095 // means a non-browser client that can speak to the daemon
1096 // directly anyway — and "absent means yes" is how localhost
1097 // servers get owned. 998 // servers get owned.
1098 .{ .origin = null, .port = 7681, .want = false }, 999 .{ .origin = null, .port = 7681, .want = false },
1099 .{ .origin = "", .port = 7681, .want = false }, 1000 .{ .origin = "", .port = 7681, .want = false },
@@ -1370,11 +1271,9 @@ test "head frame: every split boundary is INCOMPLETE, the whole frame is READY"
1370 std.mem.writeInt(u64, edge[2..10], cap - 14 + 1, .big); 1271 std.mem.writeInt(u64, edge[2..10], cap - 14 + 1, .big);
1371 try std.testing.expectEqual(HeadFrame.too_big, headFrame(&edge, cap)); 1272 try std.testing.expectEqual(HeadFrame.too_big, headFrame(&edge, cap));
1372 1273
1373 // The declared length is a WIRE number and gets no benefit of the 1274 // The declared length is a WIRE number and gets no benefit of the doubt:
1374 // doubt: u64 max in 14 bytes overflowed `off + payload_len`, which is 1275 // u64 max in 14 bytes overflows `off + payload_len` — a Debug panic, or a
1375 // a panic in Debug — the default build — and a wrap to a permanent 1276 // wrap to a permanent `.incomplete`. One hostile message per tile.
1376 // .incomplete in ReleaseFast. Either way one hostile message per tile
1377 // was enough.
1378 std.mem.writeInt(u64, edge[2..10], std.math.maxInt(u64), .big); 1277 std.mem.writeInt(u64, edge[2..10], std.math.maxInt(u64), .big);
1379 try std.testing.expectEqual(HeadFrame.too_big, headFrame(&edge, cap)); 1278 try std.testing.expectEqual(HeadFrame.too_big, headFrame(&edge, cap));
1380 std.mem.writeInt(u64, edge[2..10], std.math.maxInt(u64) - 13, .big); 1279 std.mem.writeInt(u64, edge[2..10], std.math.maxInt(u64) - 13, .big);
@@ -1410,11 +1309,9 @@ test "drain browser: the reader's OWN bytes decide, with or without a readable e
1410 // empty, poll silent forever. 1309 // empty, poll silent forever.
1411 var out_buf: [256]u8 = undefined; 1310 var out_buf: [256]u8 = undefined;
1412 1311
1413 // The regression this exists for. A close frame that is ALREADY 1312 // A close frame that is ALREADY buffered must end the tile on a pass where
1414 // buffered must end the tile on a pass where poll said nothing: it 1313 // poll said nothing — it gets there via a drain a mid-drain reconnect cut
1415 // gets there by arriving during a drain that a mid-drain reconnect 1314 // short, and holds this tile's daemon client slot until the reaper.
1416 // cut short, and while it sits there this tile's daemon client slot
1417 // stays open until the 90s reaper takes it.
1418 { 1315 {
1419 var bytes = [_]u8{ 0x88, 0x80, 0, 0, 0, 0 }; 1316 var bytes = [_]u8{ 0x88, 0x80, 0, 0, 0, 0 };
1420 var r: std.Io.Reader = .fixed(&bytes); 1317 var r: std.Io.Reader = .fixed(&bytes);