a73x

ede80939

feat: a deleted socket path is logged and taken back within a second

a73x   2026-09-05 19:04

Commit message
feat: a deleted socket path is logged and taken back within a second

2026-09-04, on a live box: a daemon's socket file vanished from the
runtime dir. The daemon kept its sessions on the unlinked inode, nothing
could reach it by name, the next `mux` auto-started a second daemon on
the path, and the day's log said nothing about any of it.

Two fixes. The daemon now writes one `mux d: socket PATH: ...` line per
socket event — the claim's branch (nothing there / cleared a dead
leftover), the bind with dev+ino, an adopt across an upgrade, the loss
of the path (missing, or replaced by another inode), each re-bind or
its refusal by name, and which way the unlink guard went at exit. The
wall appends one line of its own before it auto-starts a daemon, naming
what its dial answered (FileNotFound vs ConnectionRefused). For that,
sockpath.claim returns the branch it took, serve.Bound.close and
unlinkIfOurs return an Unlink verdict, and sockpath.probe is `answers`
with the error kept.

And Server.pumpOnce stats the path once a second against the bound
inode. A missing path is re-bound through the same serve.bind
(.refuse_live) a start uses, so "no socket stealing" holds: a successor
already holding the path is refused by claim, logged once per reason,
and the watch keeps looking — the successor's own stop unlinks its
file, and the next tick takes the path back.

Pinned by a server test (loss, re-bind, successor refused, successor's
leftover reclaimed) and an e2e leg in the boot group that deletes a
real detached daemon's socket and reads the trail off the real log.

Issues 04b3019d and 145807a2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
docs/superpowers/specs/2026-09-05-daemon-socket-lifecycle-design.md
Old New
@@ -0,0 +1,114 @@
1 # The daemon's socket path: a trail in the log, and a way back
2
3 Issues `04b3019d` (the daemon logs nothing about its socket) and `145807a2`
4 (a daemon whose socket file is deleted can never be reached by path again).
5 Both from the 2026-09-04 incident: a live daemon's `muxd.sock` vanished from
6 `/run/user/1000`, the daemon kept its three sessions on an unlinked inode, a
7 second daemon auto-started onto the same path, and nothing in the log said
8 any of it happened.
9
10 ## What the code does today
11
12 - `sockpath.claim` decides whether a path is ours to bind (nothing there /
13 dead leftover cleared / live daemon refused / not a socket) and returns
14 only `void` or an error — the branch it took is not observable.
15 - `serve.bind` binds and stamps `PathId` (dev+ino of the PATH). `Bound.close`
16 unlinks only if `stillAt` — and says nothing either way.
17 - `Server.acceptConn` accepts off the listener fd and never looks at the path.
18 - The daemon's log IS its stderr (`forkDetached` hands it the xdg log,
19 O_APPEND); every line is a `std.debug.print("mux d: ...")`. There is no
20 `std.log`; this design adds none.
21 - `handleDaemonVerb` serves `stats_req`, `sessions_req`, `endpoint_req`,
22 `debug_dump`, `end_req` and `stop_req` for CLIENT slots as well as
23 observers, so a QUIC peer can already ask them. `upgrade_req` is
24 observer-only (unix socket). `mux d stop/dump/stats` take `--sock` only
25 (`main.oneShotQuery`, `stopCmd`, over `dial.ask`).
26 - The wall's local entry (`mux_main.startLocalDaemon`) runs
27 `mux d start -d --sock PATH` when `sockpath.answers` says no, and records
28 nothing about why the dial failed.
29
30 ## Design
31
32 ### 1. The log trail (04b3019d)
33
34 One line each, `mux d: socket ...`, on the daemon's stderr:
35
36 | Event | Line |
37 |---|---|
38 | claim, at `Server.init` | `socket PATH: claimed (free)` / `claimed (cleared a dead leftover)`; refusals already propagate by error name and `main.run` prints them — add the path and the branch to that print so the log says WHICH errno refused |
39 | bind | `socket PATH: bound dev=D ino=I` (from `Bound.path_id`); an adopted listener logs `adopted dev=D ino=I` from `initFromManifest` |
40 | loss | `socket PATH: no longer names our listener (was dev=D ino=I, now missing)` or `(now dev=D2 ino=I2 — a successor)` — once per transition, from the watch in §2 |
41 | recovery | `socket PATH: re-bound dev=D ino=I` or `socket PATH: cannot re-bind: <errname>` |
42 | unlink at exit | `socket PATH: unlinked` / `left in place: not ours (dev=D2 ino=I2)` / `left in place: already gone` |
43
44 `sockpath.claim` returns a `Claimed` enum (`.free`, `.cleared_leftover`)
45 instead of `void`; `serve.Bound.close` and `unlinkIfOurs` return an
46 `Unlink` enum (`.unlinked`, `.spared_successor`, `.already_gone`). The
47 shared modules print nothing — `serve` is also askpass's binder inside a
48 wall, where a print lands on somebody's pane — the daemon does.
49
50 The wall's auto-start: `mux_main.startLocalDaemon` appends ONE line to the
51 xdg log before it forks — `mux: auto-starting a daemon on PATH: dial said
52 <errname>` — where the errname is what `sockpath.connectSocket` answered
53 (`FileNotFound` for a missing path, `ConnectionRefused` for a dead socket
54 file). `sockpath.answers` stays a bool; a sibling `sockpath.probe` returns
55 the error name, and `answers` is `probe == null`. This is the line that
56 would have dated the second daemon's birth against the first one's loss.
57
58 ### 2. Watching the path and taking it back (145807a2, option a)
59
60 `Server.pumpOnce` gains a once-a-second stat of `sock_path` against
61 `bound.path_id` (`sock_watch_ms`, same shape as the agent relay's timers).
62 Cost: one `fstatat` per second per daemon.
63
64 On loss:
65 - path missing → `serve.bind(path, .refuse_live)` a fresh listener, swap
66 it into `self.bound` (the old fd is closed; accepted connections are
67 their own fds and are untouched; anything in the old backlog was never
68 reachable by name anyway), log `re-bound`.
69 - something else at the path → `claim` refuses it (live successor:
70 `DaemonAlreadyRunning`; a non-socket: `SockPathNotASocket`; a dead
71 leftover it CLEARS and binds, which is the one case where the deleted
72 path came back as junk). Log the refusal by name, keep the watch running
73 — a successor that later stops unlinks its file and the next tick takes
74 the path back.
75
76 The rule "no socket stealing" holds unchanged: the re-bind goes through
77 the same `claim` a start does. The window in which a wall can auto-start a
78 second daemon shrinks from forever to one tick.
79
80 `mux d upgrade` carries the listener fd across the exec via the manifest;
81 a re-bound fd is just the current `bound.fd`, so nothing there changes.
82 `upgrade_req` is refused while a re-bind is pending? No pending state
83 exists: the re-bind is synchronous inside one tick.
84
85 ### 3. Admin verbs by QUIC (145807a2, option b) — pending the user's call
86
87 `mux d stop|dump|stats --quic HOST:PORT [--key PATH]` for a daemon whose
88 path is gone AND whose slot the successor now holds. Client-side only:
89 `oneShotQuery` and `stopCmd` take a `Target` (`sock | quic`) and open a
90 `link.Link` of the matching arm; the daemon already answers those verbs on
91 a client slot. `stopCmd`'s "did it die" wait polls the QUIC endpoint
92 instead of `sockpath.answers`. `upgrade` stays `--sock` (observer-only
93 verb; the manifest is local to the box either way).
94
95 Not in scope: an admin socket of its own, a signal handler.
96
97 ## Tests
98
99 - `sockpath`: `claim` returns `.free` on nothing, `.cleared_leftover` on a
100 dead socket file; `probe` names `FileNotFound` / `ConnectionRefused`.
101 - `serve`: `close` returns `.unlinked` / `.spared_successor` /
102 `.already_gone` (extends the existing three tests).
103 - `server_test_session` (harness `TestDaemon`): delete the socket path
104 under a running daemon, pump past one tick, dial the path again and get
105 a `sessions_reply`; then the successor case — bind a second listener at
106 the path, pump, assert the first did NOT unlink or replace it, unbind
107 the second, pump, assert the first has the path back.
108 - e2e (`e2e_03_side` or a new `e2e_NN_socket`): `rm` the socket of a live
109 daemon, `sleep 2`, `mux d stats --sock PATH` answers, and the log carries
110 the loss line and the re-bound line in that order; `mux d stop` then
111 unlinks and logs `unlinked`.
112 - Log-line unit test: the trail is asserted through `std.testing` capture
113 of a writer, so the daemon's `socketLog` takes a writer like every other
114 stdout-adjacent path in this repo (the fd-1 wedge rule).
src/cli/mux_main.zig
Old New
@@ -336,12 +336,14 @@ pub fn main(args: []const [:0]const u8) !u8 {
336 } 336 }
337 } 337 }
338 338
339 /// Return true when the hosts file includes the local socket but no process is 339 /// The reason the local socket needs a start — the dial's own error — when
340 /// currently answering on it. 340 /// the hosts file lists it and nothing answers on it; null otherwise. The
341 fn localNeedsStart(h: *const hosts.Hosts, sock: []const u8) bool { 341 /// reason rather than a bool because `startLocalDaemon` writes it down.
342 fn localNeedsStart(h: *const hosts.Hosts, sock: []const u8) ?anyerror {
342 var buf: [std.fs.max_path_bytes + hosts.sock_prefix.len]u8 = undefined; 343 var buf: [std.fs.max_path_bytes + hosts.sock_prefix.len]u8 = undefined;
343 const line = std.fmt.bufPrint(&buf, hosts.sock_prefix ++ "{s}", .{sock}) catch return false; 344 const line = std.fmt.bufPrint(&buf, hosts.sock_prefix ++ "{s}", .{sock}) catch return null;
344 return h.has(line) and !sockpath.answers(sock); 345 if (!h.has(line)) return null;
346 return sockpath.probe(sock);
345 } 347 }
346 348
347 /// Attach through a local Unix socket, used by both `mux --sock PATH` and the 349 /// Attach through a local Unix socket, used by both `mux --sock PATH` and the
@@ -365,7 +367,9 @@ fn attachLocal(
365 return 2; 367 return 2;
366 } 368 }
367 369
368 if (!sockpath.answers(sock_path) and !try startLocalDaemon(alloc, sock_path)) return 1; 370 if (sockpath.probe(sock_path)) |why| {
371 if (!try startLocalDaemon(alloc, sock_path, why)) return 1;
372 }
369 return wall.runAttach( 373 return wall.runAttach(
370 alloc, 374 alloc,
371 .{ .sock = sock_path }, 375 .{ .sock = sock_path },
@@ -379,7 +383,21 @@ fn attachLocal(
379 383
380 /// Start a detached local daemon by executing this binary as 384 /// Start a detached local daemon by executing this binary as
381 /// `mux d start -d --sock PATH`. Inherited stdio preserves daemon diagnostics. 385 /// `mux d start -d --sock PATH`. Inherited stdio preserves daemon diagnostics.
382 fn startLocalDaemon(alloc: std.mem.Allocator, sock_path: []const u8) !bool { 386 ///
387 /// `why` is what the dial answered, and it goes into the daemon log FIRST:
388 /// `FileNotFound` is a path with nothing at it, `ConnectionRefused` a
389 /// socket file nobody is listening on. When a running daemon's socket file
390 /// is deleted, this is the line that says a second daemon was started on
391 /// the path, when, and that the path was empty rather than dead at the
392 /// time — none of which the 2026-09-04 log could say (issue 04b3019d).
393 fn startLocalDaemon(alloc: std.mem.Allocator, sock_path: []const u8, why: anyerror) !bool {
394 const note = try std.fmt.allocPrint(
395 alloc,
396 "mux: auto-starting a daemon on {s}: the dial said {s}\n",
397 .{ sock_path, @errorName(why) },
398 );
399 defer alloc.free(note);
400 xdg.appendLogLine(alloc, note) catch {};
383 var exe_buf: [std.fs.max_path_bytes]u8 = undefined; 401 var exe_buf: [std.fs.max_path_bytes]u8 = undefined;
384 const argv = [_][]const u8{ spawn.selfExe(&exe_buf), "d", "start", "-d", "--sock", sock_path }; 402 const argv = [_][]const u8{ spawn.selfExe(&exe_buf), "d", "start", "-d", "--sock", sock_path };
385 var child = std.process.Child.init(&argv, alloc); 403 var child = std.process.Child.init(&argv, alloc);
@@ -428,7 +446,7 @@ fn wallOfHosts(alloc: std.mem.Allocator) !u8 {
428 // entry remains but its socket is inactive. If startup fails, wall polling 446 // entry remains but its socket is inactive. If startup fails, wall polling
429 // continues and can discover a daemon started by another process. 447 // continues and can discover a daemon started by another process.
430 if (sockpath.defaultSockPath(arena) catch null) |sock| { 448 if (sockpath.defaultSockPath(arena) catch null) |sock| {
431 if (localNeedsStart(&h, sock)) _ = try startLocalDaemon(alloc, sock); 449 if (localNeedsStart(&h, sock)) |why| _ = try startLocalDaemon(alloc, sock, why);
432 } 450 }
433 451
434 const key = std.posix.getenv(xdg.key_env); 452 const key = std.posix.getenv(xdg.key_env);
@@ -939,19 +957,22 @@ test "wall: a LISTED local daemon that nothing answers on is one this wall start
939 try std.testing.expect(try h.add(alloc, "box")); 957 try std.testing.expect(try h.add(alloc, "box"));
940 try std.testing.expect(try h.add(alloc, line)); 958 try std.testing.expect(try h.add(alloc, line));
941 959
942 // The state file contains the socket, but no process is listening yet. 960 // The state file contains the socket, but no process is listening yet —
943 try std.testing.expect(localNeedsStart(&h, sock)); 961 // and the answer names WHY, since that is what the log line says.
962 try std.testing.expectEqual(@as(?anyerror, error.FileNotFound), localNeedsStart(&h, sock));
944 963
945 const addr = try std.net.Address.initUnix(sock); 964 const addr = try std.net.Address.initUnix(sock);
946 var listener = try addr.listen(.{}); 965 var listener = try addr.listen(.{});
947 try std.testing.expect(!localNeedsStart(&h, sock)); 966 try std.testing.expectEqual(@as(?anyerror, null), localNeedsStart(&h, sock));
948 listener.deinit(); 967 listener.deinit();
968 // The socket file the dead listener left is a different reason.
969 try std.testing.expectEqual(@as(?anyerror, error.ConnectionRefused), localNeedsStart(&h, sock));
949 970
950 // A wall of only remote hosts starts nothing, however dead they are. 971 // A wall of only remote hosts starts nothing, however dead they are.
951 var remote: hosts.Hosts = .{}; 972 var remote: hosts.Hosts = .{};
952 defer remote.deinit(alloc); 973 defer remote.deinit(alloc);
953 try std.testing.expect(try remote.add(alloc, "box")); 974 try std.testing.expect(try remote.add(alloc, "box"));
954 try std.testing.expect(!localNeedsStart(&remote, sock)); 975 try std.testing.expectEqual(@as(?anyerror, null), localNeedsStart(&remote, sock));
955 } 976 }
956 977
957 test "hosts: a session count is the daemon's lines, not its bytes" { 978 test "hosts: a session count is the daemon's lines, not its bytes" {
src/client/askpass.zig
Old New
@@ -171,7 +171,7 @@ pub const Listener = struct {
171 .backlog = backlog, 171 .backlog = backlog,
172 .cloexec = true, 172 .cloexec = true,
173 }); 173 });
174 errdefer bound.close(path); 174 errdefer _ = bound.close(path);
175 const bell = try std.posix.pipe2(.{ .CLOEXEC = true }); 175 const bell = try std.posix.pipe2(.{ .CLOEXEC = true });
176 errdefer { 176 errdefer {
177 std.posix.close(bell[0]); 177 std.posix.close(bell[0]);
@@ -219,7 +219,7 @@ pub const Listener = struct {
219 // that got our pid back binds this same name, and unlinking by name 219 // that got our pid back binds this same name, and unlinking by name
220 // here would take ITS socket and leave that client's ssh prompting 220 // here would take ITS socket and leave that client's ssh prompting
221 // at nothing. 221 // at nothing.
222 self.bound.unlinkIfOurs(self.path); 222 _ = self.bound.unlinkIfOurs(self.path);
223 } 223 }
224 224
225 /// The keyboard's side. Copies the pending prompt out and marks it 225 /// The keyboard's side. Copies the pending prompt out and marks it
src/serve.zig
Old New
@@ -36,6 +36,27 @@ pub const BindOpts = struct {
36 cloexec: bool = true, 36 cloexec: bool = true,
37 }; 37 };
38 38
39 /// Which way the unlink guard went. The daemon logs it, because "the
40 /// socket was already gone when the daemon stopped" is the whole trail a
41 /// deleted path leaves, and the guard used to decline in silence
42 /// (issue 04b3019d, 2026-09-04).
43 pub const Unlink = enum {
44 /// The path named our socket and it is gone now.
45 unlinked,
46 /// The path names somebody else's socket — a successor's — and stays.
47 spared_successor,
48 /// Nothing at the path to unlink, or nothing this process may stat.
49 already_gone,
50 /// The path names our socket and the unlink was refused (a directory
51 /// we may no longer write, a read-only mount): the file is still
52 /// there, and the next start's claim will read it as a dead leftover.
53 /// It used to be reported as `already_gone`, which is the opposite of
54 /// what an operator needs to know.
55 unlink_failed,
56 /// `close` on a Bound whose descriptor was already closed: nothing done.
57 closed_before,
58 };
59
39 pub const Bound = struct { 60 pub const Bound = struct {
40 fd: std.posix.fd_t, 61 fd: std.posix.fd_t,
41 path_id: sockpath.PathId, 62 path_id: sockpath.PathId,
@@ -45,11 +66,11 @@ pub const Bound = struct {
45 /// its clients. The stat comes after the close because a successor only 66 /// its clients. The stat comes after the close because a successor only
46 /// claims once nothing is listening — that narrows the race to the 67 /// claims once nothing is listening — that narrows the race to the
47 /// stat→unlink gap, the floor Linux gives for deleting by name. 68 /// stat→unlink gap, the floor Linux gives for deleting by name.
48 pub fn close(self: *Bound, path: []const u8) void { 69 pub fn close(self: *Bound, path: []const u8) Unlink {
49 if (self.fd == -1) return; 70 if (self.fd == -1) return .closed_before;
50 std.posix.close(self.fd); 71 std.posix.close(self.fd);
51 self.fd = -1; 72 self.fd = -1;
52 self.unlinkIfOurs(path); 73 return self.unlinkIfOurs(path);
53 } 74 }
54 75
55 /// The guard without the close, for the one caller that must not close: 76 /// The guard without the close, for the one caller that must not close:
@@ -57,10 +78,14 @@ pub const Bound = struct {
57 /// detached pumps still live, one of which may be inside `declined` on 78 /// detached pumps still live, one of which may be inside `declined` on
58 /// that Listener. The name has to leave the filesystem there; the 79 /// that Listener. The name has to leave the filesystem there; the
59 /// descriptor may not. 80 /// descriptor may not.
60 pub fn unlinkIfOurs(self: *const Bound, path: []const u8) void { 81 pub fn unlinkIfOurs(self: *const Bound, path: []const u8) Unlink {
61 if (self.path_id.stillAt(path)) { 82 const now = sockpath.PathId.of(path) catch return .already_gone;
62 std.fs.cwd().deleteFile(path) catch {}; 83 if (!std.meta.eql(now, self.path_id)) return .spared_successor;
63 } 84 std.fs.cwd().deleteFile(path) catch |e| return switch (e) {
85 error.FileNotFound => .already_gone,
86 else => .unlink_failed,
87 };
88 return .unlinked;
64 } 89 }
65 }; 90 };
66 91
@@ -70,7 +95,10 @@ pub const Bound = struct {
70 /// one binder-side pre-check, and it lives with `mux d`. 95 /// one binder-side pre-check, and it lives with `mux d`.
71 pub fn bind(path: []const u8, opts: BindOpts) !Bound { 96 pub fn bind(path: []const u8, opts: BindOpts) !Bound {
72 switch (opts.policy) { 97 switch (opts.policy) {
73 .refuse_live => try sockpath.claim(path), 98 // The branch claim took is the DAEMON's to log, at its own claim
99 // one call earlier; this one is the same question asked again
100 // after the fork, and its answer is not news.
101 .refuse_live => _ = try sockpath.claim(path),
74 .clobber_own => std.fs.cwd().deleteFile(path) catch {}, 102 .clobber_own => std.fs.cwd().deleteFile(path) catch {},
75 } 103 }
76 const addr = try std.net.Address.initUnix(path); 104 const addr = try std.net.Address.initUnix(path);
@@ -109,7 +137,10 @@ test "refuse_live refuses a path a live listener owns; clobber_own takes its own
109 std.posix.close(first.fd); 137 std.posix.close(first.fd);
110 first.fd = -1; 138 first.fd = -1;
111 var second = try bind(path, .{ .policy = .clobber_own }); 139 var second = try bind(path, .{ .policy = .clobber_own });
112 second.close(path); 140 try testing.expectEqual(Unlink.unlinked, second.close(path));
141 // A second close is a no-op that says so, not a second unlink of
142 // whatever is at the path by then.
143 try testing.expectEqual(Unlink.closed_before, second.close(path));
113 } 144 }
114 145
115 test "close unlinks our socket but never a successor's" { 146 test "close unlinks our socket but never a successor's" {
@@ -125,10 +156,30 @@ test "close unlinks our socket but never a successor's" {
125 // the displaced one's teardown must not delete by that name. 156 // the displaced one's teardown must not delete by that name.
126 var succ = try bind(path, .{ .policy = .clobber_own }); 157 var succ = try bind(path, .{ .policy = .clobber_own });
127 158
128 old.close(path); // guard fires: the inode at path is succ's — no unlink 159 // guard fires: the inode at path is succ's — no unlink
160 try testing.expectEqual(Unlink.spared_successor, old.close(path));
129 _ = try std.fs.cwd().statFile(path); // successor's file survived 161 _ = try std.fs.cwd().statFile(path); // successor's file survived
130 succ.close(path); // ours: unlinked 162 try testing.expectEqual(Unlink.unlinked, succ.close(path)); // ours
131 try testing.expectError(error.FileNotFound, std.fs.cwd().statFile(path)); 163 try testing.expectError(error.FileNotFound, std.fs.cwd().statFile(path));
164
165 // And a socket whose path somebody deleted under it — the 2026-09-04
166 // incident shape — reports that the name was already gone, which is
167 // the one word the daemon's log needed and did not have.
168 var lost = try bind(path, .{ .policy = .clobber_own });
169 try std.fs.cwd().deleteFile(path);
170 try testing.expectEqual(Unlink.already_gone, lost.close(path));
171
172 // And an unlink the filesystem refuses is reported as that, never as
173 // "already gone": the file is still there for the next start to find.
174 // A directory without write permission is the cheapest refusal; root
175 // is exempt from that bit, so the case is skipped for uid 0.
176 if (std.posix.getuid() != 0) {
177 var held = try bind(path, .{ .policy = .clobber_own });
178 try std.posix.fchmodat(std.fs.cwd().fd, tmp.path(), 0o500, 0);
179 defer std.posix.fchmodat(std.fs.cwd().fd, tmp.path(), 0o700, 0) catch {};
180 try testing.expectEqual(Unlink.unlink_failed, held.close(path));
181 _ = try std.fs.cwd().statFile(path);
182 }
132 } 183 }
133 184
134 test "unlinkIfOurs takes the name without the descriptor, and spares a successor's" { 185 test "unlinkIfOurs takes the name without the descriptor, and spares a successor's" {
@@ -140,17 +191,17 @@ test "unlinkIfOurs takes the name without the descriptor, and spares a successor
140 // askpass's retire shape: the name goes, the fd stays open for the 191 // askpass's retire shape: the name goes, the fd stays open for the
141 // detached pumps that may still be reading this object. 192 // detached pumps that may still be reading this object.
142 var one = try bind(path, .{ .policy = .clobber_own }); 193 var one = try bind(path, .{ .policy = .clobber_own });
143 one.unlinkIfOurs(path); 194 try testing.expectEqual(Unlink.unlinked, one.unlinkIfOurs(path));
144 try testing.expectError(error.FileNotFound, std.fs.cwd().statFile(path)); 195 try testing.expectError(error.FileNotFound, std.fs.cwd().statFile(path));
145 try testing.expect(one.fd != -1); 196 try testing.expect(one.fd != -1);
146 std.posix.close(one.fd); 197 std.posix.close(one.fd);
147 198
148 var old = try bind(path, .{ .policy = .clobber_own }); 199 var old = try bind(path, .{ .policy = .clobber_own });
149 var succ = try bind(path, .{ .policy = .clobber_own }); 200 var succ = try bind(path, .{ .policy = .clobber_own });
150 old.unlinkIfOurs(path); 201 try testing.expectEqual(Unlink.spared_successor, old.unlinkIfOurs(path));
151 _ = try std.fs.cwd().statFile(path); 202 _ = try std.fs.cwd().statFile(path);
152 std.posix.close(old.fd); 203 std.posix.close(old.fd);
153 succ.close(path); 204 try testing.expectEqual(Unlink.unlinked, succ.close(path));
154 } 205 }
155 206
156 test "adopt re-stamps the id of the file as found, and close then unlinks it" { 207 test "adopt re-stamps the id of the file as found, and close then unlinks it" {
@@ -165,7 +216,7 @@ test "adopt re-stamps the id of the file as found, and close then unlinks it" {
165 const first = try bind(path, .{ .policy = .refuse_live }); 216 const first = try bind(path, .{ .policy = .refuse_live });
166 var after = try adopt(first.fd, path); 217 var after = try adopt(first.fd, path);
167 try testing.expectEqual(first.path_id, after.path_id); 218 try testing.expectEqual(first.path_id, after.path_id);
168 after.close(path); 219 try testing.expectEqual(Unlink.unlinked, after.close(path));
169 try testing.expectError(error.FileNotFound, std.fs.cwd().statFile(path)); 220 try testing.expectError(error.FileNotFound, std.fs.cwd().statFile(path));
170 } 221 }
171 222
@@ -182,11 +233,11 @@ test "cloexec is on by default and off only when asked" {
182 // unconditionally, so a bind() written without it was a silent leak. 233 // unconditionally, so a bind() written without it was a silent leak.
183 var on = try bind(path, .{ .policy = .clobber_own }); 234 var on = try bind(path, .{ .policy = .clobber_own });
184 try testing.expect(try std.posix.fcntl(on.fd, std.posix.F.GETFD, 0) & std.posix.FD_CLOEXEC != 0); 235 try testing.expect(try std.posix.fcntl(on.fd, std.posix.F.GETFD, 0) & std.posix.FD_CLOEXEC != 0);
185 on.close(path); 236 _ = on.close(path);
186 237
187 var off = try bind(path, .{ .policy = .clobber_own, .cloexec = false }); 238 var off = try bind(path, .{ .policy = .clobber_own, .cloexec = false });
188 try testing.expect(try std.posix.fcntl(off.fd, std.posix.F.GETFD, 0) & std.posix.FD_CLOEXEC == 0); 239 try testing.expect(try std.posix.fcntl(off.fd, std.posix.F.GETFD, 0) & std.posix.FD_CLOEXEC == 0);
189 off.close(path); 240 _ = off.close(path);
190 } 241 }
191 242
192 // Forces semantic analysis of every pub decl under `zig build test`, so an 243 // Forces semantic analysis of every pub decl under `zig build test`, so an
src/server/server.zig
Old New
@@ -58,6 +58,32 @@ pub const Observer = struct {
58 /// diagnostic. 10 s is for a `--via` relay's first frame over a slow link. 58 /// diagnostic. 10 s is for a `--via` relay's first frame over a slow link.
59 pub const observer_idle_ms_default: i64 = 10_000; 59 pub const observer_idle_ms_default: i64 = 10_000;
60 60
61 /// How often `Server.watchSockPath` stats the socket path. A second is
62 /// well under the wall's own dial-then-auto-start, which is what a lost
63 /// path has to beat. The Server field of the same name is what the watch
64 /// reads, so a test can tick it in milliseconds instead of waiting out
65 /// three real seconds.
66 pub const sock_watch_interval_ms_default: i64 = 1000;
67
68 /// `watchSockPath`'s state: when it last looked, whether the path is
69 /// currently lost (so the loss logs once, not once a second), and the last
70 /// re-bind refusal (so that logs once per reason).
71 pub const SockWatch = struct {
72 checked_ms: i64 = 0,
73 lost: bool = false,
74 refused: ?anyerror = null,
75 };
76
77 /// Every line about the daemon's own socket, in one shape, on the
78 /// daemon's stderr — which `forkDetached` pointed at the xdg log. The
79 /// 2026-09-04 incident (a live daemon's socket file deleted, a second
80 /// daemon started on the path) left a log with nothing in it about
81 /// either; these lines are the trail that would have dated it
82 /// (issue 04b3019d).
83 fn logSocket(path: []const u8, comptime fmt: []const u8, args: anytype) void {
84 std.debug.print("mux d: socket {s}: " ++ fmt ++ "\n", .{path} ++ args);
85 }
86
61 /// The clock the daemon's bounded deadlines measure against: a calendar 87 /// The clock the daemon's bounded deadlines measure against: a calendar
62 /// step must not move one, and a boot's clock survives an upgrade's exec. 88 /// step must not move one, and a boot's clock survives an upgrade's exec.
63 pub fn monoMs() i64 { 89 pub fn monoMs() i64 {
@@ -465,6 +491,9 @@ pub const Server = struct {
465 /// and an absent-means-false arm is the silent no-unlink 6090604 fixed. 491 /// and an absent-means-false arm is the silent no-unlink 6090604 fixed.
466 bound: serve.Bound, 492 bound: serve.Bound,
467 sock_path: []const u8, 493 sock_path: []const u8,
494 /// The once-a-second check that `sock_path` still names `bound`, and
495 /// the re-bind when it does not. See `watchSockPath`.
496 sock_watch: SockWatch = .{},
468 /// The attached interactive clients, across every session; each sees 497 /// The attached interactive clients, across every session; each sees
469 /// every update of the one session it is attached to. 498 /// every update of the one session it is attached to.
470 clients: [max_clients]?ClientSlot = @splat(null), 499 clients: [max_clients]?ClientSlot = @splat(null),
@@ -482,6 +511,7 @@ pub const Server = struct {
482 /// one-shot tool that reads one answer over the unix socket and exits. 511 /// one-shot tool that reads one answer over the unix socket and exits.
483 observers: [max_observers]?Observer = @splat(null), 512 observers: [max_observers]?Observer = @splat(null),
484 observer_idle_ms: i64 = observer_idle_ms_default, 513 observer_idle_ms: i64 = observer_idle_ms_default,
514 sock_watch_interval_ms: i64 = sock_watch_interval_ms_default,
485 /// `.none` is a first-class answer, not a failure: QUIC is opt-in per 515 /// `.none` is a first-class answer, not a failure: QUIC is opt-in per
486 /// invocation. 516 /// invocation.
487 quic: union(enum) { 517 quic: union(enum) {
@@ -558,7 +588,11 @@ pub const Server = struct {
558 // again, and the repeat is not redundant: it is the one that decides, 588 // again, and the repeat is not redundant: it is the one that decides,
559 // covering the window this early check opens by refusing before the 589 // covering the window this early check opens by refusing before the
560 // fork rather than at the bind. 590 // fork rather than at the bind.
561 try sockpath.claim(opts.sock_path); 591 const claimed = try sockpath.claim(opts.sock_path);
592 logSocket(opts.sock_path, "claimed ({s})", .{switch (claimed) {
593 .free => "nothing there",
594 .cleared_leftover => "cleared a dead daemon's leftover",
595 }});
562 596
563 // Shell integration, decided and written before the fork: whatever 597 // Shell integration, decided and written before the fork: whatever
564 // the child is going to be told has to exist on disk by the time it 598 // the child is going to be told has to exist on disk by the time it
@@ -591,6 +625,7 @@ pub const Server = struct {
591 } 625 }
592 626
593 const bound = try serve.bind(opts.sock_path, .{ .policy = .refuse_live }); 627 const bound = try serve.bind(opts.sock_path, .{ .policy = .refuse_live });
628 logSocket(opts.sock_path, "bound dev={d} ino={d}", .{ bound.path_id.dev, bound.path_id.ino });
594 var srv: Server = .{ 629 var srv: Server = .{
595 .alloc = alloc, 630 .alloc = alloc,
596 .spawn_plan = plan, 631 .spawn_plan = plan,
@@ -687,6 +722,7 @@ pub const Server = struct {
687 .shellint_dir = injection.dir, 722 .shellint_dir = injection.dir,
688 .agents = .{ .dir = agent_dir }, 723 .agents = .{ .dir = agent_dir },
689 }; 724 };
725 logSocket(sock_path, "adopted across upgrade dev={d} ino={d}", .{ srv.bound.path_id.dev, srv.bound.path_id.ino });
690 726
691 // Cumulative, so an upgrade is not mistaken for a restart by 727 // Cumulative, so an upgrade is not mistaken for a restart by
692 // anything sampling `mux d stats`. Saturating rather than @intCast 728 // anything sampling `mux d stats`. Saturating rather than @intCast
@@ -895,8 +931,16 @@ pub const Server = struct {
895 .none => {}, 931 .none => {},
896 } 932 }
897 // Close, then unlink only if the path still names *our* socket — the 933 // Close, then unlink only if the path still names *our* socket — the
898 // guard and its rationale live in serve.Bound.close now. 934 // guard and its rationale live in serve.Bound.close now. Which way
899 self.bound.close(self.sock_path); 935 // it went is logged: "already gone" at a stop is the only trace a
936 // path deleted under a running daemon leaves once it exits.
937 logSocket(self.sock_path, "{s}", .{switch (self.bound.close(self.sock_path)) {
938 .unlinked => "unlinked",
939 .spared_successor => "left in place: it names another daemon's socket now",
940 .already_gone => "left in place: already gone",
941 .unlink_failed => "left in place: the unlink was refused; the next start will clear it as a dead leftover",
942 .closed_before => "already closed",
943 }});
900 // Two passes over one deadline: every child is asked to go before any 944 // Two passes over one deadline: every child is asked to go before any
901 // is waited on, so a table of shells that ignore TERM costs one 945 // is waited on, so a table of shells that ignore TERM costs one
902 // `term_grace_ms` and not one each. 946 // `term_grace_ms` and not one each.
@@ -1031,6 +1075,7 @@ pub const Server = struct {
1031 } 1075 }
1032 1076
1033 if (fds[listener_idx].revents & std.posix.POLL.IN != 0) self.acceptConn(); 1077 if (fds[listener_idx].revents & std.posix.POLL.IN != 0) self.acceptConn();
1078 self.watchSockPath();
1034 1079
1035 // Re-check each slot: an earlier arm may have dropped a client whose 1080 // Re-check each slot: an earlier arm may have dropped a client whose
1036 // fd is still in `fds`. The null check suffices only because no slot 1081 // fd is still in `fds`. The null check suffices only because no slot
@@ -1136,6 +1181,60 @@ pub const Server = struct {
1136 conn.close(); // out of slots 1181 conn.close(); // out of slots
1137 } 1182 }
1138 1183
1184 /// Once a second: does `sock_path` still name the socket we bound? A
1185 /// unix listener survives the deletion of its path — the daemon keeps
1186 /// every session and keeps listening on an inode nothing can reach by
1187 /// name — and until this check existed that was permanent: nothing
1188 /// re-bound, no admin verb takes anything but `--sock`, and the next
1189 /// `mux` to dial the path found nothing and auto-started a second daemon
1190 /// on it, stranding the first (issue 145807a2, 2026-09-04).
1191 ///
1192 /// The re-bind goes through the same `serve.bind(.refuse_live)` a start
1193 /// does, so "no socket stealing" holds by construction: a successor that
1194 /// already holds the path is refused by `sockpath.claim` and this daemon
1195 /// stays path-less, logs it once, and keeps looking — a successor's own
1196 /// `mux d stop` unlinks its file, and the next tick takes the path back.
1197 /// Only a path with NOTHING at it, or a dead socket file, is re-bound.
1198 /// One `fstatat` per second is the whole cost.
1199 fn watchSockPath(self: *Server) void {
1200 const now = monoMs();
1201 if (now - self.sock_watch.checked_ms < self.sock_watch_interval_ms) return;
1202 self.sock_watch.checked_ms = now;
1203
1204 if (self.bound.path_id.stillAt(self.sock_path)) return;
1205
1206 if (!self.sock_watch.lost) {
1207 self.sock_watch.lost = true;
1208 if (sockpath.PathId.of(self.sock_path)) |other| {
1209 logSocket(self.sock_path, "no longer names our listener (was dev={d} ino={d}, now dev={d} ino={d})", .{
1210 self.bound.path_id.dev, self.bound.path_id.ino, other.dev, other.ino,
1211 });
1212 } else |_| {
1213 logSocket(self.sock_path, "no longer names our listener (was dev={d} ino={d}, now missing)", .{
1214 self.bound.path_id.dev, self.bound.path_id.ino,
1215 });
1216 }
1217 }
1218
1219 const fresh = serve.bind(self.sock_path, .{ .policy = .refuse_live }) catch |err| {
1220 // Once per reason, not once per second: a successor holding the
1221 // path for a week would otherwise write 600k identical lines.
1222 const e: anyerror = err;
1223 if (self.sock_watch.refused == null or self.sock_watch.refused.? != e) {
1224 self.sock_watch.refused = e;
1225 logSocket(self.sock_path, "cannot re-bind: {s}", .{@errorName(e)});
1226 }
1227 return;
1228 };
1229 // Accepted connections are descriptors of their own and are not
1230 // touched; the old listener's backlog held nothing reachable by
1231 // name, which is the whole reason we are here.
1232 std.posix.close(self.bound.fd);
1233 self.bound = fresh;
1234 self.sock_watch = .{ .checked_ms = now };
1235 logSocket(self.sock_path, "re-bound dev={d} ino={d}", .{ fresh.path_id.dev, fresh.path_id.ino });
1236 }
1237
1139 fn setNonblocking(fd: std.posix.fd_t) !void { 1238 fn setNonblocking(fd: std.posix.fd_t) !void {
1140 const fl = try std.posix.fcntl(fd, std.posix.F.GETFL, 0); 1239 const fl = try std.posix.fcntl(fd, std.posix.F.GETFL, 0);
1141 const nb: u32 = @bitCast(std.posix.O{ .NONBLOCK = true }); 1240 const nb: u32 = @bitCast(std.posix.O{ .NONBLOCK = true });
src/server/server_agent.zig
Old New
@@ -32,7 +32,7 @@ pub const AgentSock = struct {
32 /// name puts two owners on one path, and only the newest may be deleted 32 /// name puts two owners on one path, and only the newest may be deleted
33 /// by it. 33 /// by it.
34 pub fn release(self: *AgentSock, alloc: std.mem.Allocator) void { 34 pub fn release(self: *AgentSock, alloc: std.mem.Allocator) void {
35 self.bound.close(self.path); 35 _ = self.bound.close(self.path);
36 alloc.free(self.path); 36 alloc.free(self.path);
37 } 37 }
38 }; 38 };
src/server/server_test_session.zig
Old New
@@ -1765,3 +1765,72 @@ test "Server: a repeated end_req does not push the SIGKILL deadline out" {
1765 return error.GraceDeadlineMoved; 1765 return error.GraceDeadlineMoved;
1766 } 1766 }
1767 } 1767 }
1768
1769 fn sockLost(s: *Server) bool {
1770 return s.sock_watch.lost;
1771 }
1772
1773 fn sockRefused(s: *Server) bool {
1774 return s.sock_watch.refused != null;
1775 }
1776
1777 test "Server: a socket path deleted under a running daemon is taken back within a tick, and never from a successor" {
1778 const alloc = std.testing.allocator;
1779 const sockpath = @import("sockpath");
1780
1781 var td = try h.TestDaemon.init(alloc, "lost", .{ .shell = "/bin/sh" });
1782 defer td.deinit();
1783 // Tick the watch on every pump: the second it waits in production is
1784 // the wall's auto-start budget, not anything this test is grading.
1785 td.srv.sock_watch_interval_ms = 0;
1786 const path: []const u8 = td.sock_path;
1787 const first_id = td.srv.bound.path_id;
1788
1789 // The 2026-09-04 shape: the FILE goes and the listener stays. Nothing
1790 // can reach the daemon by name, and until the watch existed nothing
1791 // ever would again.
1792 try std.fs.cwd().deleteFile(path);
1793 try std.testing.expect(!sockpath.answers(path));
1794
1795 // Within a tick the daemon has re-bound the path, and the file there is
1796 // the one it now holds — a new inode, ours.
1797 try std.testing.expect(try h.pumpUntil(&td.srv, 3000, path, sockpath.answers));
1798 try std.testing.expect(!std.meta.eql(first_id, td.srv.bound.path_id));
1799 try std.testing.expect(td.srv.bound.path_id.stillAt(path));
1800 try std.testing.expect(!td.srv.sock_watch.lost);
1801
1802 // And it SERVES there: this is not a file that reappeared.
1803 {
1804 const c = try dial.dial(path);
1805 defer c.close();
1806 try proto.writeFrame(c.handle, .sessions_req, "");
1807 const reply = (try awaitFrame(alloc, &td.srv, c.handle, .sessions_reply, 400)) orelse
1808 return error.NoReplyOnReboundPath;
1809 reply.deinit(alloc);
1810 }
1811
1812 // The successor case: the path goes again, and before the next tick a
1813 // second listener has bound it — the second daemon a wall's auto-start
1814 // made in the incident. "No socket stealing" holds for the re-bind as
1815 // it does for a start: the successor's file is refused, not replaced.
1816 const ours_before = td.srv.bound.path_id;
1817 try std.fs.cwd().deleteFile(path);
1818 const addr = try std.net.Address.initUnix(path);
1819 var succ = try addr.listen(.{});
1820 const succ_id = try sockpath.PathId.of(path);
1821 try std.testing.expect(try h.pumpUntil(&td.srv, 3000, &td.srv, sockLost));
1822 try std.testing.expect(try h.pumpUntil(&td.srv, 3000, &td.srv, sockRefused));
1823 try std.testing.expectEqual(@as(?anyerror, error.DaemonAlreadyRunning), td.srv.sock_watch.refused);
1824 try std.testing.expect(std.meta.eql(succ_id, try sockpath.PathId.of(path)));
1825 try std.testing.expect(std.meta.eql(ours_before, td.srv.bound.path_id));
1826
1827 // The successor stops without unlinking (a SIGKILL's leftover). The
1828 // next tick finds a dead socket file, which a claim may clear, and the
1829 // path is ours again.
1830 succ.deinit();
1831 try std.testing.expect(try h.pumpUntil(&td.srv, 3000, path, sockpath.answers));
1832 try std.testing.expect(td.srv.bound.path_id.stillAt(path));
1833 try std.testing.expect(!std.meta.eql(ours_before, td.srv.bound.path_id));
1834 try std.testing.expect(!td.srv.sock_watch.lost);
1835 try std.testing.expectEqual(@as(?anyerror, null), td.srv.sock_watch.refused);
1836 }
src/sockpath.zig
Old New
@@ -200,11 +200,27 @@ pub fn connectSocket(path: []const u8) !std.net.Stream {
200 /// Whether anything LISTENS at `path` now: a read, so every connect 200 /// Whether anything LISTENS at `path` now: a read, so every connect
201 /// error is a no. The decision needing the errno is `claim`. 201 /// error is a no. The decision needing the errno is `claim`.
202 pub fn answers(path: []const u8) bool { 202 pub fn answers(path: []const u8) bool {
203 const s = connectSocket(path) catch return false; 203 return probe(path) == null;
204 }
205
206 /// `answers` with the reason kept: null when something listens, otherwise
207 /// the connect's own error. `FileNotFound` is a path with nothing at it and
208 /// `ConnectionRefused` a socket file nobody is listening on — and which of
209 /// the two a wall saw the moment it auto-started a daemon is the one fact
210 /// that dates a deleted socket against the second daemon that took its
211 /// path (issue 145807a2, 2026-09-04). `answers` stays the bool every
212 /// caller reads; this is for the one that writes the reason down.
213 pub fn probe(path: []const u8) ?anyerror {
214 const s = connectSocket(path) catch |err| return err;
204 s.close(); 215 s.close();
205 return true; 216 return null;
206 } 217 }
207 218
219 /// What `claim` found, for the daemon's log: a start that cleared a dead
220 /// daemon's leftover is a different event from one that found nothing, and
221 /// the log used to say neither.
222 pub const Claimed = enum { free, cleared_leftover };
223
208 /// Make the socket path ours to bind, or refuse it. Without this, daemons 224 /// Make the socket path ours to bind, or refuse it. Without this, daemons
209 /// started against one path each unlink and bind fresh: every one keeps running 225 /// started against one path each unlink and bind fresh: every one keeps running
210 /// with its sessions intact, but only the newest is reachable and the rest are 226 /// with its sessions intact, but only the newest is reachable and the rest are
@@ -226,20 +242,20 @@ pub fn answers(path: []const u8) bool {
226 /// and hits `unreachable` on — a panic, in a daemon, over a file the user 242 /// and hits `unreachable` on — a panic, in a daemon, over a file the user
227 /// named. Asking the stat first means the connect is only ever made to 243 /// named. Asking the stat first means the connect is only ever made to
228 /// something that IS a socket, and neither kernel has a surprise there. 244 /// something that IS a socket, and neither kernel has a surprise there.
229 pub fn claim(path: []const u8) !void { 245 pub fn claim(path: []const u8) !Claimed {
230 const st = std.posix.fstatat(std.posix.AT.FDCWD, path, 0) catch |err| switch (err) { 246 const st = std.posix.fstatat(std.posix.AT.FDCWD, path, 0) catch |err| switch (err) {
231 // Nothing at the path, or a symlink to nothing: free as far as this 247 // Nothing at the path, or a symlink to nothing: free as far as this
232 // function can tell, and `bind` gets the last word on the entry. 248 // function can tell, and `bind` gets the last word on the entry.
233 error.FileNotFound => return, 249 error.FileNotFound => return .free,
234 else => |e| return e, 250 else => |e| return e,
235 }; 251 };
236 if (!std.posix.S.ISSOCK(st.mode)) return error.SockPathNotASocket; 252 if (!std.posix.S.ISSOCK(st.mode)) return error.SockPathNotASocket;
237 253
238 if (std.net.connectUnixSocket(path)) |probe| { 254 if (std.net.connectUnixSocket(path)) |live| {
239 probe.close(); 255 live.close();
240 return error.DaemonAlreadyRunning; 256 return error.DaemonAlreadyRunning;
241 } else |err| switch (err) { 257 } else |err| switch (err) {
242 error.FileNotFound => return, // vanished under us; path is free 258 error.FileNotFound => return .free, // vanished under us; path is free
243 // A socket file nobody is listening on: a dead daemon's leftover, 259 // A socket file nobody is listening on: a dead daemon's leftover,
244 // which the stat above has already confirmed is a socket. 260 // which the stat above has already confirmed is a socket.
245 error.ConnectionRefused => {}, 261 error.ConnectionRefused => {},
@@ -255,6 +271,7 @@ pub fn claim(path: []const u8) !void {
255 error.FileNotFound => {}, 271 error.FileNotFound => {},
256 else => |e| return e, 272 else => |e| return e,
257 }; 273 };
274 return .cleared_leftover;
258 } 275 }
259 276
260 test "sockpath.max_sun_path is the kernel's field less its NUL, not a number of ours" { 277 test "sockpath.max_sun_path is the kernel's field less its NUL, not a number of ours" {
@@ -300,6 +317,13 @@ test "answers: a live listener, a stale socket file, and a path with nothing on
300 listener.deinit(); 317 listener.deinit();
301 try std.testing.expect(!answers(path)); 318 try std.testing.expect(!answers(path));
302 try std.fs.cwd().access(path, .{}); 319 try std.fs.cwd().access(path, .{});
320 // The reason `answers` swallowed, for the auto-start's log line: a dead
321 // socket file is ECONNREFUSED, and only after the claim clears it is
322 // the path simply absent.
323 try std.testing.expectEqual(@as(?anyerror, error.ConnectionRefused), probe(path));
324 try std.testing.expectEqual(Claimed.cleared_leftover, try claim(path));
325 try std.testing.expectEqual(@as(?anyerror, error.FileNotFound), probe(path));
326 try std.testing.expectEqual(Claimed.free, try claim(path));
303 327
304 // A path longer than `sun_path` is a no as flatly as a missing one: 328 // A path longer than `sun_path` is a no as flatly as a missing one:
305 // nothing is listening there and nothing could be. A yes here would 329 // nothing is listening there and nothing could be. A yes here would
src/xdg.zig
Old New
@@ -90,7 +90,10 @@ pub fn keyPathFrom(alloc: std.mem.Allocator, xdg_config_home: ?[]const u8, home:
90 return pathFrom(alloc, xdg_config_home, home, ".config", "key"); 90 return pathFrom(alloc, xdg_config_home, home, ".config", "key");
91 } 91 }
92 92
93 /// Truncated by a verb that asks for a daemon; spawn.zig owns which and why. 93 /// The one daemon log on the box, `$XDG_STATE_HOME/mux/muxd.log`. Every
94 /// writer opens it O_APPEND and none truncates it: a detached daemon's
95 /// stderr (main.zig's fork) and the wall's auto-start note below share it,
96 /// and one daemon per socket path means several may be writing at once.
94 pub fn logPath(alloc: std.mem.Allocator) ![]const u8 { 97 pub fn logPath(alloc: std.mem.Allocator) ![]const u8 {
95 return statePath(alloc, "muxd.log"); 98 return statePath(alloc, "muxd.log");
96 } 99 }
@@ -99,6 +102,37 @@ pub fn logPathFrom(alloc: std.mem.Allocator, xdg_state_home: ?[]const u8, home:
99 return pathFrom(alloc, xdg_state_home, home, ".local/state", "muxd.log"); 102 return pathFrom(alloc, xdg_state_home, home, ".local/state", "muxd.log");
100 } 103 }
101 104
105 /// Append one line to the daemon log from a process that is NOT a daemon.
106 /// The one caller is the wall about to auto-start a daemon: the reason its
107 /// dial failed belongs beside the lines that daemon is about to write,
108 /// because a deleted socket and the second daemon that took its path are
109 /// one story and were in no file at all (issue 04b3019d). O_APPEND, like
110 /// the fork's own open of this file: one log serves every daemon on the
111 /// box. Best effort — a log that cannot be opened is not a reason to
112 /// refuse the user a shell, so the caller ignores the error.
113 pub fn appendLogLine(alloc: std.mem.Allocator, line: []const u8) !void {
114 const path = try logPath(alloc);
115 defer alloc.free(path);
116 try appendLogLineTo(path, line);
117 }
118
119 /// The append without the environment, so a test can aim it at a
120 /// directory of its own (tests cannot setenv).
121 pub fn appendLogLineTo(path: []const u8, line: []const u8) !void {
122 if (std.fs.path.dirname(path)) |dir| try std.fs.cwd().makePath(dir);
123 // O_APPEND rather than a seek-then-write: a daemon is writing this
124 // file at the same time, and only the append flag makes the two
125 // interleave by line rather than overwrite each other.
126 const f: std.fs.File = .{ .handle = try std.posix.open(path, .{
127 .ACCMODE = .WRONLY,
128 .CREAT = true,
129 .APPEND = true,
130 .CLOEXEC = true,
131 }, 0o600) };
132 defer f.close();
133 try f.writeAll(line);
134 }
135
102 /// Where `mux HOST` remembers the last announce. A host containing a path 136 /// Where `mux HOST` remembers the last announce. A host containing a path
103 /// separator is refused; the caller attaches uncached rather than failing. 137 /// separator is refused; the caller attaches uncached rather than failing.
104 pub fn hostCachePath(alloc: std.mem.Allocator, host: []const u8) ![]const u8 { 138 pub fn hostCachePath(alloc: std.mem.Allocator, host: []const u8) ![]const u8 {
@@ -315,6 +349,33 @@ test "hostCachePathFrom: same shape against XDG_CACHE_HOME, and refuses a host w
315 try std.testing.expectError(error.UncacheableHost, hostCachePathFrom(a, "../k", null, "/home/u")); 349 try std.testing.expectError(error.UncacheableHost, hostCachePathFrom(a, "../k", null, "/home/u"));
316 } 350 }
317 351
352 test "appendLogLineTo: creates the log's directory, appends behind what is there, and never truncates" {
353 const testtmp = @import("testtmp");
354 var tmp = try testtmp.TmpDir.make();
355 defer tmp.cleanup();
356 var buf: [280]u8 = undefined;
357 const path = try std.fmt.bufPrint(&buf, "{s}/state/mux/muxd.log", .{tmp.path()});
358
359 // The wall's note is the FIRST line in a fresh state dir: the daemon it
360 // is about to start has not created the directory yet.
361 try appendLogLineTo(path, "mux: auto-starting a daemon on /r/mux.sock: the dial said FileNotFound\n");
362 // A daemon's own line lands between two of ours; the second note must
363 // follow it, not overwrite from offset zero.
364 try appendLogLineTo(path, "mux d: socket /r/mux.sock: bound\n");
365 try appendLogLineTo(path, "mux: auto-starting a daemon on /r/mux.sock: the dial said ConnectionRefused\n");
366
367 const got = try std.fs.cwd().readFileAlloc(std.testing.allocator, path, 4096);
368 defer std.testing.allocator.free(got);
369 try std.testing.expectEqualStrings(
370 "mux: auto-starting a daemon on /r/mux.sock: the dial said FileNotFound\n" ++
371 "mux d: socket /r/mux.sock: bound\n" ++
372 "mux: auto-starting a daemon on /r/mux.sock: the dial said ConnectionRefused\n",
373 got,
374 );
375 const st = try std.fs.cwd().statFile(path);
376 try std.testing.expectEqual(@as(u32, 0o600), @as(u32, @intCast(st.mode & 0o777)));
377 }
378
318 test "makePrivateParent: a path with no directory part is a no-op" { 379 test "makePrivateParent: a path with no directory part is a no-op" {
319 // The branch neither caller's tests reach: nothing to create, nothing 380 // The branch neither caller's tests reach: nothing to create, nothing
320 // to tighten, and crucially no error — a bare filename must not make 381 // to tighten, and crucially no error — a bare filename must not make
test/e2e_01_boot.sh
Old New
@@ -1009,6 +1009,41 @@ grep -qxF "$SEED" "$MUXLOG" || {
1009 softkill "$TPID" || true 1009 softkill "$TPID" || true
1010 TPID="" 1010 TPID=""
1011 1011
1012 # --- the socket path deleted under a live daemon: a trail, and a way back --
1013 #
1014 # 2026-09-04, on a live box: a daemon's socket file vanished from the
1015 # runtime dir. The daemon kept its sessions and kept listening on the
1016 # unlinked inode, nothing could reach it by name, and the next `mux` to
1017 # dial the path started a SECOND daemon on it. The day's log said nothing
1018 # about any of it. Two fixes, both graded here against a real detached
1019 # daemon and the real log file: the daemon writes one line per socket
1020 # event, and it takes a deleted path back within a second — so the second
1021 # daemon never gets its chance.
1022 grep -q "mux d: socket $SOCK8: claimed (nothing there)" "$MUXLOG" || {
1023 echo "e2e FAIL: the log has no claim line for $SOCK8"; tail -20 "$MUXLOG"; exit 1; }
1024 grep -q "mux d: socket $SOCK8: bound dev=[0-9]* ino=[0-9]*" "$MUXLOG" || {
1025 echo "e2e FAIL: the log has no bind line for $SOCK8"; tail -20 "$MUXLOG"; exit 1; }
1026 # `|` as the delimiter: the socket path is full of `/`.
1027 INO_BEFORE=$(sed -n "s|.*socket $SOCK8: bound dev=[0-9]* ino=\([0-9]*\).*|\1|p" "$MUXLOG" | tail -1)
1028 [ -n "$INO_BEFORE" ] || { echo "e2e FAIL: could not read the bound inode off the log"; exit 1; }
1029 rm "$SOCK8"
1030 # The path is dead to dials for now — this is the state the box was in for
1031 # hours — and the daemon is still there: the OS, not the daemon, says so.
1032 kill -0 "$SPID" || { echo "e2e FAIL: the daemon died with its socket file"; exit 1; }
1033 wait_until 30 "the daemon never took $SOCK8 back after its file was deleted" \
1034 '"$MUX" d stats --sock "$SOCK8" > /dev/null 2>&1' \
1035 'tail -20 "$MUXLOG"'
1036 grep -q "mux d: socket $SOCK8: no longer names our listener (was dev=[0-9]* ino=$INO_BEFORE, now missing)" "$MUXLOG" || {
1037 echo "e2e FAIL: the log does not record the loss of $SOCK8"; tail -20 "$MUXLOG"; exit 1; }
1038 grep -q "mux d: socket $SOCK8: re-bound dev=[0-9]* ino=[0-9]*" "$MUXLOG" || {
1039 echo "e2e FAIL: the log does not record the re-bind of $SOCK8"; tail -20 "$MUXLOG"; exit 1; }
1040 # The same daemon, not a replacement: the marker typed into its shell
1041 # before the deletion is still on its grid, and the pid never changed.
1042 "$MUX" d dump --sock "$SOCK8" | grep -q "start-works" || {
1043 echo "e2e FAIL: the re-bound path reaches a daemon without the marker"; exit 1; }
1044 kill -0 "$SPID" || { echo "e2e FAIL: the daemon that re-bound is not pid $SPID"; exit 1; }
1045 ok "a deleted socket path is logged and taken back within a tick (04b3019d, 145807a2)"
1046
1012 # Race: two concurrent starts, both exit 0, still one session (the marker 1047 # Race: two concurrent starts, both exit 0, still one session (the marker
1013 # survives — a second daemon on the path would have started a fresh shell). 1048 # survives — a second daemon on the path would have started a fresh shell).
1014 softkill "$SPID" && wait_gone "$SOCK8" 1049 softkill "$SPID" && wait_gone "$SOCK8"
test/e2e_03_side.sh
Old New
@@ -183,6 +183,14 @@ set -e
183 grep -q '^mux d: starting' "$OUT.pa.err" || { 183 grep -q '^mux d: starting' "$OUT.pa.err" || {
184 echo "e2e FAIL: local auto-start printed no 'mux d'-prefixed starting line" 184 echo "e2e FAIL: local auto-start printed no 'mux d'-prefixed starting line"
185 cat "$OUT.pa.err"; exit 1; } 185 cat "$OUT.pa.err"; exit 1; }
186 # And the wall wrote WHY into the daemon log before it started one, naming
187 # what the dial answered: a path with nothing at it is FileNotFound, and
188 # that word beside the daemon's own bind line is the whole difference
189 # between "a fresh box" and "a live daemon's socket file was deleted"
190 # (issue 04b3019d). The log lives under the leg's own XDG_STATE_HOME.
191 grep -qF "mux: auto-starting a daemon on $SOCK15: the dial said FileNotFound" "$HOSTROOM/mux/muxd.log" || {
192 echo "e2e FAIL: the auto-start left no 'the dial said' line in $HOSTROOM/mux/muxd.log:"
193 cat "$HOSTROOM/mux/muxd.log" 2>/dev/null; exit 1; }
186 PAPID=$(sed -n 's/.* pid=\([0-9]*\).*/\1/p' "$OUT.pa.err" | head -1) 194 PAPID=$(sed -n 's/.* pid=\([0-9]*\).*/\1/p' "$OUT.pa.err" | head -1)
187 defer_kill "$PAPID" 195 defer_kill "$PAPID"
188 [ -n "$PAPID" ] || { echo "e2e FAIL: mux up-line carries no pid"; cat "$OUT.pa.err"; exit 1; } 196 [ -n "$PAPID" ] || { echo "e2e FAIL: mux up-line carries no pid"; cat "$OUT.pa.err"; exit 1; }