7b207001

docs: macOS port design — a platform layer, then a second arm

a73x 2026-09-03 07:03

Commit message

docs: macOS port design — a platform layer, then a second arm

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SakwJEwD9dXBoRP5kWbemW

docs/superpowers/specs/2026-09-03-macos-port-design.md

@@ @@ -0,0 +1,291 @@ @@
+# macOS port: a platform layer, then a second arm
+Date 2026-09-03. HEAD at survey time `e6fff02f`.
+## Goal
+`mux` builds and runs on macOS (Apple Silicon, `aarch64-macos`) with full
+parity: daemon, pty, client, wall, agent surface, web hub, QUIC, and
+`mux d upgrade`. The work is three steps in order, and each step is a
+deliverable on its own:
+. **Identify the seams.** Done; the inventory is below.
+. **Push them behind an interface on Linux.** A platform layer, `src/os/`,
+   with one backend per side. Linux behaviour does not change. `make ci`
+   stays green throughout, and a source ban keeps the seams from growing
+   back.
+. **Write the macOS arm.** Two files and a build branch, plus the harness
+   and dep-script arms. The build host for that step is deliberately
+   undecided (see "Deferred").
+## Findings the design rests on
+Measured on this box, 2026-09-03, with the pinned zig 0.15.2:
+- `zig cc -target aarch64-macos` compiles and links a libc program using
+  `posix_openpt`, `kqueue`, `libproc` and `dyld` without an SDK. Zig ships
+  the Darwin libc headers; `<util.h>` (BSD `openpty`/`forkpty`) is NOT
+  among them, so the pty opens through `posix_openpt` + `grantpt` +
+  `unlockpt` + `ptsname`, which glibc has too.
+- Zig's self-hosted Mach-O linker works; build.zig's `use_lld = true` on
+  every executable is the one linker blocker ("using LLD to link macho
+  files is unsupported").
+- The QUIC stack (ngtcp2 1.25.0 + wolfSSL 5.9.2) cross-builds for
+  `aarch64-macos` with `zig cc` as the C compiler, given two additions to
+  `build-deps.sh`: `-DWOLFSSL_SYS_CA_CERTS=no` (mux is PSK-only, and the
+  system-CA path wants Security.framework) and a
+  `CMAKE_FIND_ROOT_PATH` fence so ngtcp2 stops finding the host's
+  `libwolfssl.so`. All three archives came out.
+- `std.posix.socket` and `std.posix.accept` already emulate `SOCK_CLOEXEC`,
+  `SOCK_NONBLOCK` and `accept4` on Darwin with a trailing `fcntl`, so the
+  socket sites compile and behave unchanged.
+- The event loop is `poll(2)` everywhere, by two recorded decisions; threads
+  are `std.Thread` only. Nothing to port there.
+- ghostty-vt links three C++ libraries (simdutf, highway, utfcpp) whose
+  build.zig files each call ghostty's `apple_sdk.addPaths` on a Darwin
+  target. That helper resolves the HOST libc, so from a Linux host it
+  injects `-isystem /usr/include` and the C++ compile fails on glibc
+  headers. It is upstream code; build.zig cannot reach it.
+## Seam inventory
+Ninety-odd sites, grouped by the side that owns them. The full report with
+every `file:line` is the survey in the session scratchpad; the spec keeps
+the shape, and the plan will re-grep every line before touching it.
+**Server side** (the daemon and everything it execs):
+| Seam | Today | Portable form |
+|---|---|---|
+| pty open | `forkpty` via `<pty.h>` | `posix_openpt`+`grantpt`+`unlockpt`+`ptsname`, then `fork`, `setsid`, `TIOCSCTTY`; same on both OSes |
+| child bail-out | `std.os.linux.exit_group` (5 sites, `pty.zig` and `main.zig`) | `std.c._exit` |
+| fd barrier in the child | `close_range(3, max)` syscall | Linux keeps `close_range`; Darwin loops `close(3..getdtablesize())` |
+| line discipline, foreground pgid | `tcgetattr`/`TIOCGPGRP` on the MASTER | Linux unchanged; Darwin is a hardware probe (below) |
+| upgrade manifest carrier | `memfd_create` (2 sites + 8 tests) | Linux keeps memfd; Darwin is an unlinked file in the 0700 runtime dir |
+| stale-image verdict | `readlink /proc/self/exe` ends ` (deleted)` | dev+inode of the image opened at boot vs the path now; one rule on both OSes |
+| non-blocking send | `MSG_NOSIGNAL` | Linux unchanged; Darwin sets `SO_NOSIGPIPE` once at accept |
+| adopted QUIC fd check | `std.os.linux.getsockopt SO_TYPE` | `std.c.getsockopt` |
+| peer credentials on accept | `SO_PEERCRED` | Linux unchanged; Darwin `getpeereid` + `LOCAL_PEERPID` |
+| daemon fork | `std.os.linux.setsid` in `forkDaemon` | `std.c.setsid`; the fork moves with it |
+**Client side** (the wall, askpass, the hub):
+| Seam | Today | Portable form |
+|---|---|---|
+| askpass caller check | `SO_PEERCRED` (uid and pid) | as above |
+| parent of a pid | parse `/proc/PID/stat` | Linux unchanged; Darwin `proc_pidinfo(PROC_PIDTBSDINFO)` |
+| terminal size | `std.os.linux.ioctl TIOCGWINSZ` | `std.c.ioctl` with the platform's constant |
+| test pty pair | `/dev/ptmx` + `TIOCSPTLCK` + `TIOCGPTN` | the same `posix_openpt` recipe the server uses |
+**Shared root** (`xdg`, `sockpath`, `spawn`): every site here is a Linux
+spelling of a portable call and needs no backend once respelled.
+`std.os.linux.{getpid,geteuid,socketpair}` become `std.c.*`;
+`access("/proc/PID")` becomes `kill(pid, 0)`; `max_sun_path` is derived at
+comptime from `std.c.sockaddr.un`. Two policy items remain and each is one
+`switch (builtin.os.tag)` in its owning file: the default runtime
+directory (`sockpath`) and `selfExe`'s deleted-inode fallback (`spawn`,
+which moves into the platform folder).
+**Harness** (`test/*.sh`, 13 files): `/proc` reads for a pid's exe, comm,
+cmdline, fd table, RSS and the UDP port table; GNU `timeout`, `stat -c`,
+`ps --ppid`, `sed -i`, `sha256sum`, `nproc`. One site has no macOS answer:
+`e2e_09_hosts.sh` walks other processes' `environ`, which SIP forbids.
+**Build**: `quicDeps` names two prefixes (`native`, `musl`);
+`build-deps.sh` has a two-way `case`, `-march=x86_64_v3`, `sha256sum`,
+`nproc`; `use_lld = true` on every exe; `make install` and `release`
+hard-wire `x86_64-linux-musl`.
+## Design
+### The folder table gains a platform layer
+A new folder, `src/os/`, is the bottom of the import graph: it imports
+nothing of ours. It holds two rows, one per side, so neither side can
+reach the other's syscalls and an app that links the engine and a client
+never links `fork` or a pty:
+| Row | Root | Children | Imported by |
+|---|---|---|---|
+| `server_os` | `src/os/server_os.zig` | `server_os_linux.zig`, later `server_os_macos.zig` | `pty`, `daemon`, `mux` |
+| `client_os` | `src/os/client_os.zig` | `client_os_linux.zig`, later `client_os_macos.zig` | `client`, `wall` |
+Each root is the interface: `pub const impl = switch (builtin.os.tag) {
+.linux => @import("server_os_linux.zig"), .macos => ..., else =>
+@compileError("mux has no platform arm for this OS") }` and one `pub`
+declaration per operation that forwards to it. The root file is where the
+contract is written, one doc comment per operation naming the failure the
+operation prevents (the fd barrier's comment moves here, for instance).
+The children spell syscalls and nothing else. A macOS build with a
+missing operation is a compile error naming the operation, never a
+runtime surprise.
+`spawn` moves from `src/cli/` into `src/os/` as `spawn.zig`, keeping its
+row name. It was placed under `cli` to dodge rule 4; asking the OS for a
+terminal is platform code, and the folder now exists.
+`forkDaemon` and the daemon's peer-credential read move out of
+`src/cli/main.zig` into `server_os` (`forkDetached`, `peerCred`), and
+folder rule 6's `except` follows the fork to
+`src/os/server_os_linux.zig`. `main.zig` keeps the argv it spells and the
+stderr it lends. This is the one change to the CLI folder, and it moves
+the entry back toward parse-and-dispatch.
+### The interface, by operation
+`server_os`:
+- `exitNow(code) noreturn` — a child's bail-out with no atexit and no
+  stdio flush, so a forked daemon or session child never flushes the
+  parent's buffers twice.
+- `closeFrom(first_fd)` — the fd barrier. Everything at or above `first_fd`
+  is closed in the child before exec, so the upgrade manifest's key bytes
+  and an adopted listener cannot ride into a shell.
+- `openPty(winsize) !struct{master, slave_path}` and `becomeSession(slave)`
+  — the two halves of what `forkpty` did, so the fork itself stays in the
+  child code that resets signals and injects env.
+- `ptyMode(master) !Mode` and `ptyFgPgid(master) !pid` — the line
+  discipline and foreground group. On Linux they read the master. The
+  Darwin arm is written after the probe below decides whether the master
+  answers or the slave must be reopened.
+- `anonFd(name) !fd` — the upgrade manifest carrier: an fd no path names
+  after the call returns, private to this uid, that survives an exec of
+  the same process. Linux is `memfd_create`; Darwin is `mkstemp` under the
+  runtime dir followed by `unlink`.
+- `selfImageStale() bool` — has the file at the running image's path been
+  replaced since boot. Implemented ONCE, in the root, as dev+inode of an
+  fd opened at boot against `stat` of `std.fs.selfExePath` now; the
+  `(deleted)` string goes away and the wire word stays.
+- `sendNoSig(fd, bytes, flags)` — a non-blocking send that cannot raise
+  SIGPIPE. Linux passes `MSG_NOSIGNAL`; Darwin's `acceptClient` sets
+  `SO_NOSIGPIPE` and `sendNoSig` is a plain send.
+- `peerCred(fd) !struct{uid, pid}` — who is on the other end of a unix
+  socket.
+- `forkDetached(argv, stderr_fd) !pid` — the one `posix.fork` under `src/`.
+- `sockType(fd) !u32` — for the adopted-QUIC-fd check.
+`client_os`:
+- `peerCred(fd)` — same shape; the askpass uid-and-pid check.
+- `parentOf(pid) !pid` — the askpass process-tree walk.
+- `winSize(fd) !Winsize` — the wall's terminal size.
+- `openPtyPair(winsize)` — test-only; the same recipe as `server_os.openPty`
+  written twice on purpose, because the sides do not import each other.
+Shared-root items that stay in their owners as a `switch (builtin.os.tag)`:
+- `sockpath.runtimeDir()` — Linux is `XDG_RUNTIME_DIR` with no fallback,
+  exactly as today. The Darwin arm is a single deliberate spelling chosen
+  in step 3 under the 104-byte `sun_path` budget (see the probe). The
+  askpass listener gate in `wallview` reads this function instead of the
+  env var, so the two agree by construction.
+- `spawn.selfExe()` — `std.fs.selfExePath` on both; the deleted-inode
+  fallback arm is Linux-only and says so.
+### Folder rule 7
+A new `SourceBan`: the needles `std.os.linux`, `/proc`, `memfd`,
+`close_range`, `exit_group`, `PEERCRED`, `TIOCSPTLCK`, `TIOCGPTN` may not
+appear in a production line under `src/`, `src/engine/`, `src/client/`,
+`src/tui/`, `src/server/` or `src/cli/`. `src/os/` is not in the list, so
+the children may spell anything; the roots have no reason to. The one
+foreseeable exemption is `spawn.zig`'s Linux fallback arm, which is in
+`src/os/` and needs none. The rule is what turns "we pushed the seams
+behind an interface" from a claim into a gate: a new Linux-ism outside
+`src/os/` fails `make check`. The scan reads comment lines as well as
+code, as rules 4 to 6 do, so step 2 also rewords the comments that cite
+`/proc` or `memfd` outside `src/os/` to name the operation instead
+(`server_os.anonFd`, `pid_alive`); a comment that names the mechanism is
+a comment that goes stale when the mechanism differs per OS.
+### The harness oracle
+`test/e2e_lib.sh` gains helpers that answer the "ask the OS about the OS"
+questions by name, so the pins stop spelling `/proc`:
+`pid_exe PID`, `pid_comm PID`, `pid_args PID`, `pid_alive PID`,
+`pid_fds PID` (one line per open fd, with the target as the kernel names
+it), `pid_rss_kb PID`, `udp_ports` (bound local UDP ports), `file_mode
+PATH`, `file_size PATH`, `with_timeout SECS CMD...`.
+Step 2 implements the Linux arm from `/proc` and `stat -c` and rewrites
+every pin to call the helper; step 3 adds `case "$(uname)"` arms from
+`lsof`, `ps -o`, `stat -f` and `gtimeout`. The `environ` walk in
+`e2e_09_hosts.sh` is rewritten in step 2 against `pid_args`, since a stray
+`mux d start` is visible in its argv and `environ` has no portable
+reading. The fixture rule stays: a helper that holds a dimension constant
+is blind to it, so each helper's own test asserts on a process with an
+off-origin pid, an fd table larger than three, and a non-empty UDP table.
+### Build and deps
+- `quicDeps` and `build-deps.sh` take a target word from the resolved
+  target: `native`, `musl`, `aarch64-macos`. The prefix is
+  `deps/quic/out/<word>`, so `make clean-deps` and wan.sh keep their
+  contract. The macOS arm of the script adds the two wolfSSL flags the
+  probe found and drops `-march=x86_64_v3` for a non-x86 target.
+  `sha256sum` and `nproc` become `shasum -a 256` and `sysctl -n hw.ncpu`
+  behind a `case "$(uname)"`.
+- `use_lld` is set only when the target is not Darwin.
+- `make install` and `make release` gain a `MUX_TARGET` variable defaulting
+  to `x86_64-linux-musl`, so a macOS build is `make install
+  MUX_TARGET=aarch64-macos` and the release tarball carries the target
+  triple as it does today.
+### Step 3: the macOS arm
+Two children, `server_os_macos.zig` and `client_os_macos.zig`, and the
+harness and script arms above. Two questions are answered on hardware
+before either file is written, each as a throwaway probe whose output is
+recorded in `docs/decisions.md`:
+. **Does the pty master answer `tcgetattr` and `TIOCGPGRP` on Darwin?**
+   If yes the Linux arm's shape is reused. If not, `ptyMode` and
+   `ptyFgPgid` reopen the slave path by name, which `openPty` records for
+   that purpose.
+. **Which default socket directory fits.** `$TMPDIR` is `/var/folders/…`,
+   about fifty bytes, against a 103-byte usable `sun_path` that the
+   per-session agent socket and `mux-ask-PID.sock` are dialled through
+   from inside a session. The probe measures the longest path mux creates
+   under each candidate (`$TMPDIR`, `~/Library/Caches/mux`, `~/.local/state
+   /mux/run`) and picks the one with room; the spelling then lands in
+   `sockpath.runtimeDir` and README.
+Behaviours that differ on purpose on Darwin, each named in
+`docs/decisions.md` when written: `closeFrom` is O(fd table) rather than
+one syscall; `anonFd` is an unlinked file, private to the uid by the
+directory mode rather than by anonymity; the harness cannot read another
+process's environment.
+### Testing
+- Step 2 is a refactor: `make ci` green after every task, and the
+  cross-version gate (`make xversion`) against the current tip proves the
+  wire and the upgrade manifest did not change shape.
+- Each interface operation gets a unit test in its root file that runs on
+  whatever OS built it, against a real fd, a real child, a real pty: the
+  fd barrier is asserted by a child shell testing `/dev/fd/N`, which both
+  OSes have; `anonFd` by `fstat` reporting zero links; `selfImageStale` by
+  renaming a copy of the test binary over its own path in a temp dir.
+- Rule 7 is asserted the way rules 4 to 6 are: a doc-gate test that plants
+  each needle in a scratch file under each banned folder and reads the
+  fatal.
+- Step 3's gate is `make ci` run on the Mac plus the harness helpers'
+  own tests there, and a `make vm`-shaped journey script driven from
+  whichever host builds.
+## Deferred
+**Where the macOS binary is built.** Cross-compiling from Linux is blocked
+by ghostty's `apple_sdk` lookup (finding above). The options are a native
+build on the Mac driven over ssh the way `make vm` drives the VM,
+prebuilt macOS archives for the three C++ deps passed through ghostty's
+`-fsys=` flags, or a patched ghostty pin. The decision is taken after step
+lands, when the Linux side no longer moves under it, and it is not
+needed to write either `_macos.zig` file.
+**Splitting the shared root** into a paths layer and a connection layer is
+worth doing and is not this work.
+**x86_64-macos and a universal binary** are one more target word each and
+are out of scope until someone has the hardware.