docs/superpowers/specs/2026-09-03-macos-port-design.md

Ref: main worktree-web-client-spec   Size: 18.0 KiB   History

# macOS port: a platform layer, then a second arm

Date 2026-09-03. HEAD at survey time `e6fff02f`.

## Goal

`mux` builds and runs on macOS (Apple Silicon, `aarch64-macos`) with full
parity: daemon, pty, client, wall, agent surface, web hub, QUIC, and
`mux d upgrade`. The work is three steps in order, and each step is a
deliverable on its own:

1. **Identify the seams.** Done; the inventory is below.
2. **Push them behind an interface on Linux.** A platform layer, `src/os/`,
   with one backend per side. Linux behaviour does not change. `make ci`
   stays green throughout, and a source ban keeps the seams from growing
   back.
3. **Write the macOS arm.** Two files and a build branch, plus the harness
   and dep-script arms. The build host for that step is deliberately
   undecided (see "Deferred").

## Findings the design rests on

Measured on this box, 2026-09-03, with the pinned zig 0.15.2:

- `zig cc -target aarch64-macos` compiles and links a libc program using
  `posix_openpt`, `kqueue`, `libproc` and `dyld` without an SDK. Zig ships
  the Darwin libc headers; `<util.h>` (BSD `openpty`/`forkpty`) is NOT
  among them, so the pty opens through `posix_openpt` + `grantpt` +
  `unlockpt` + `ptsname`, which glibc has too.
- Zig's self-hosted Mach-O linker works; build.zig's `use_lld = true` on
  every executable is the one linker blocker ("using LLD to link macho
  files is unsupported").
- The QUIC stack (ngtcp2 1.25.0 + wolfSSL 5.9.2) cross-builds for
  `aarch64-macos` with `zig cc` as the C compiler, given two additions to
  `build-deps.sh`: `-DWOLFSSL_SYS_CA_CERTS=no` (mux is PSK-only, and the
  system-CA path wants Security.framework) and a
  `CMAKE_FIND_ROOT_PATH` fence so ngtcp2 stops finding the host's
  `libwolfssl.so`. All three archives came out.
- `std.posix.socket` and `std.posix.accept` already emulate `SOCK_CLOEXEC`,
  `SOCK_NONBLOCK` and `accept4` on Darwin with a trailing `fcntl`, so the
  socket sites compile and behave unchanged.
- The event loop is `poll(2)` everywhere, by two recorded decisions; threads
  are `std.Thread` only. Nothing to port there.
- ghostty-vt links three C++ libraries (simdutf, highway, utfcpp) whose
  build.zig files each call ghostty's `apple_sdk.addPaths` on a Darwin
  target. That helper resolves the HOST libc, so from a Linux host it
  injects `-isystem /usr/include` and the C++ compile fails on glibc
  headers. It is upstream code; build.zig cannot reach it.

## Seam inventory

Ninety-odd sites, grouped by the side that owns them. The full report with
every `file:line` is the survey in the session scratchpad; the spec keeps
the shape, and the plan will re-grep every line before touching it.

**Server side** (the daemon and everything it execs):

| Seam | Today | Portable form |
|---|---|---|
| pty open | `forkpty` via `<pty.h>` | `posix_openpt`+`grantpt`+`unlockpt`+`ptsname`, then `fork`, `setsid`, `TIOCSCTTY`; same on both OSes |
| child bail-out | `std.os.linux.exit_group` (5 sites, `pty.zig` and `main.zig`) | `std.c._exit` |
| fd barrier in the child | `close_range(3, max)` syscall | Linux keeps `close_range`; Darwin loops `close(3..getdtablesize())` |
| line discipline, foreground pgid | `tcgetattr`/`TIOCGPGRP` on the MASTER | Linux unchanged; Darwin is a hardware probe (below) |
| upgrade manifest carrier | `memfd_create` (2 sites + 8 tests) | Linux keeps memfd; Darwin is an unlinked file in the 0700 runtime dir |
| stale-image verdict | `readlink /proc/self/exe` ends ` (deleted)` | dev+inode of the image opened at boot vs the path now; one rule on both OSes |
| non-blocking send | `MSG_NOSIGNAL` | Linux unchanged; Darwin sets `SO_NOSIGPIPE` once at accept |
| adopted QUIC fd check | `std.os.linux.getsockopt SO_TYPE` | `std.c.getsockopt` |
| peer credentials on accept | `SO_PEERCRED` | Linux unchanged; Darwin `getpeereid` + `LOCAL_PEERPID` |
| daemon fork | `std.os.linux.setsid` in `forkDaemon` | `std.c.setsid`; the fork moves with it |

**Client side** (the wall, askpass, the hub):

| Seam | Today | Portable form |
|---|---|---|
| askpass caller check | `SO_PEERCRED` (uid and pid) | as above |
| parent of a pid | parse `/proc/PID/stat` | Linux unchanged; Darwin `proc_pidinfo(PROC_PIDTBSDINFO)` |
| terminal size | `std.os.linux.ioctl TIOCGWINSZ` | `std.c.ioctl` with the platform's constant |
| test pty pair | `/dev/ptmx` + `TIOCSPTLCK` + `TIOCGPTN` | the same `posix_openpt` recipe the server uses |

**Shared root** (`xdg`, `sockpath`, `spawn`): every site here is a Linux
spelling of a portable call and needs no backend once respelled.
`std.os.linux.{getpid,geteuid,socketpair}` become `std.c.*`;
`access("/proc/PID")` becomes `kill(pid, 0)`; `max_sun_path` is derived at
comptime from `std.c.sockaddr.un`. Two policy items remain and each is one
`switch (builtin.os.tag)` in its owning file: the default runtime
directory (`sockpath`) and `selfExe`'s deleted-inode fallback (`spawn`,
which moves into the platform folder).

**Harness** (`test/*.sh`, 13 files): `/proc` reads for a pid's exe, comm,
cmdline, fd table, RSS and the UDP port table; GNU `timeout`, `stat -c`,
`ps --ppid`, `sed -i`, `sha256sum`, `nproc`. One site has no macOS answer:
`e2e_09_hosts.sh` walks other processes' `environ`, which SIP forbids.

**Build**: `quicDeps` names two prefixes (`native`, `musl`);
`build-deps.sh` has a two-way `case`, `-march=x86_64_v3`, `sha256sum`,
`nproc`; `use_lld = true` on every exe; `make install` and `release`
hard-wire `x86_64-linux-musl`.

## Design

### The folder table gains a platform layer

A new folder, `src/os/`, is the bottom of the import graph: it imports
nothing of ours. It holds two rows, one per side, so neither side can
reach the other's syscalls and an app that links the engine and a client
never links `fork` or a pty:

| Row | Root | Children | Imported by |
|---|---|---|---|
| `server_os` | `src/os/server_os.zig` | `server_os_linux.zig`, later `server_os_macos.zig` | `pty`, `daemon`, `mux` |
| `client_os` | `src/os/client_os.zig` | `client_os_linux.zig`, later `client_os_macos.zig` | `client`, `wall` |

Each root is the interface: `pub const impl = switch (builtin.os.tag) {
.linux => @import("server_os_linux.zig"), .macos => ..., else =>
@compileError("mux has no platform arm for this OS") }` and one `pub`
declaration per operation that forwards to it. The root file is where the
contract is written, one doc comment per operation naming the failure the
operation prevents (the fd barrier's comment moves here, for instance).
The children spell syscalls and nothing else. A macOS build with a
missing operation is a compile error naming the operation, never a
runtime surprise.

`spawn` moves from `src/cli/` into `src/os/` as `spawn.zig`, keeping its
row name. It was placed under `cli` to dodge rule 4; asking the OS for a
terminal is platform code, and the folder now exists.

`forkDaemon` and the daemon's peer-credential read move out of
`src/cli/main.zig` into `server_os` (`forkDetached`, `peerCred`), and
folder rule 6's `except` follows the fork to
`src/os/server_os_linux.zig`. `main.zig` keeps the argv it spells and the
stderr it lends. This is the one change to the CLI folder, and it moves
the entry back toward parse-and-dispatch.

### The interface, by operation

`server_os`:

- `exitNow(code) noreturn` — a child's bail-out with no atexit and no
  stdio flush, so a forked daemon or session child never flushes the
  parent's buffers twice.
- `closeFrom(first_fd)` — the fd barrier. Everything at or above `first_fd`
  is closed in the child before exec, so the upgrade manifest's key bytes
  and an adopted listener cannot ride into a shell.
- `forkPty(winsize) !struct{pid, master}` — forkpty(3) behind one name,
  returning pid 0 in the child exactly as forkpty does, so the child code
  that resets signals and injects env stays in `pty.zig` where the fork is
  visible. (Amended 2026-09-03 while planning: the first draft split this
  into `openPty` plus `becomeSession`; forkpty exists on both OSes and the
  Darwin header gap is one `extern "c" fn forkpty` line in the macOS arm,
  so hand-rolling the session-leader dance bought nothing.)
- `ptyMode(master) !Mode` and `ptyFgPgid(master) !pid` — the line
  discipline and foreground group. On Linux they read the master. The
  Darwin arm is written after the probe below decides whether the master
  answers or the slave must be reopened.
- `anonFd(name) !fd` — the upgrade manifest carrier: an fd no path names
  after the call returns, private to this uid, that survives an exec of
  the same process. Linux is `memfd_create`; Darwin is `mkstemp` under the
  runtime dir followed by `unlink`.
- `selfImageStale() bool` — has the file at the running image's path been
  replaced since boot. Implemented ONCE, in the root, as dev+inode of an
  fd opened at boot against `stat` of `std.fs.selfExePath` now; the
  `(deleted)` string goes away and the wire word stays.
- `sendNoSig(fd, bytes, flags)` — a non-blocking send that cannot raise
  SIGPIPE. Linux passes `MSG_NOSIGNAL`; Darwin's `acceptClient` sets
  `SO_NOSIGPIPE` and `sendNoSig` is a plain send.
- `peerCred(fd) !struct{uid, pid}` — who is on the other end of a unix
  socket.
- `forkDetached(argv, stderr_fd) !pid` — the one `posix.fork` under `src/`.
- `sockType(fd) !u32` — for the adopted-QUIC-fd check.

`client_os`:

- `peerCred(fd)` — same shape; the askpass uid-and-pid check.
- `parentOf(pid) !pid` — the askpass process-tree walk.
- `winSize(fd) !Winsize` — the wall's terminal size.
- `openPtyPair(winsize)` — test-only; the same recipe as `server_os.openPty`
  written twice on purpose, because the sides do not import each other.

Shared-root items that stay in their owners as a `switch (builtin.os.tag)`:

- `sockpath.runtimeDir()` — Linux is `XDG_RUNTIME_DIR` with no fallback,
  exactly as today. The Darwin arm is a single deliberate spelling chosen
  in step 3 under the 104-byte `sun_path` budget (see the probe). The
  askpass listener gate in `wallview` reads this function instead of the
  env var, so the two agree by construction.
- `spawn.selfExe()` — `std.fs.selfExePath` on both; the deleted-inode
  fallback arm is Linux-only and says so.

### Folder rule 7

A new `SourceBan`: the needles `std.os.linux`, `/proc`, `memfd`,
`close_range`, `exit_group`, `PEERCRED`, `TIOCSPTLCK`, `TIOCGPTN` may not
appear in a production line under `src/`, `src/engine/`, `src/client/`,
`src/tui/`, `src/server/` or `src/cli/`. `src/os/` is not in the list, so
the children may spell anything; the roots have no reason to. The one
foreseeable exemption is `spawn.zig`'s Linux fallback arm, which is in
`src/os/` and needs none. The rule is what turns "we pushed the seams
behind an interface" from a claim into a gate: a new Linux-ism outside
`src/os/` fails `make check`. The scan reads comment lines as well as
code, as rules 4 to 6 do, so step 2 also rewords the comments that cite
`/proc` or `memfd` outside `src/os/` to name the operation instead
(`server_os.anonFd`, `pid_alive`); a comment that names the mechanism is
a comment that goes stale when the mechanism differs per OS.

### The harness oracle

`test/e2e_lib.sh` gains helpers that answer the "ask the OS about the OS"
questions by name, so the pins stop spelling `/proc`:

`pid_exe PID`, `pid_comm PID`, `pid_args PID`, `pid_alive PID`,
`pid_fds PID` (one line per open fd, with the target as the kernel names
it), `pid_rss_kb PID`, `udp_ports` (bound local UDP ports), `file_mode
PATH`, `file_size PATH`, `with_timeout SECS CMD...`.

Step 2 implements the Linux arm from `/proc` and `stat -c` and rewrites
every pin to call the helper; step 3 adds `case "$(uname)"` arms from
`lsof`, `ps -o`, `stat -f` and `gtimeout`. The `environ` walk in
`e2e_09_hosts.sh` is rewritten in step 2 against `pid_args`, since a stray
`mux d start` is visible in its argv and `environ` has no portable
reading. The fixture rule stays: a helper that holds a dimension constant
is blind to it, so each helper's own test asserts on a process with an
off-origin pid, an fd table larger than three, and a non-empty UDP table.

### Build and deps

- `quicDeps` and `build-deps.sh` take a target word from the resolved
  target: `native`, `musl`, `aarch64-macos`. The prefix is
  `deps/quic/out/<word>`, so `make clean-deps` and wan.sh keep their
  contract. The macOS arm of the script adds the two wolfSSL flags the
  probe found and drops `-march=x86_64_v3` for a non-x86 target.
  `sha256sum` and `nproc` become `shasum -a 256` and `sysctl -n hw.ncpu`
  behind a `case "$(uname)"`.
- `use_lld` is set only when the target is not Darwin.
- `make install` and `make release` gain a `MUX_TARGET` variable defaulting
  to `x86_64-linux-musl`, so a macOS build is `make install
  MUX_TARGET=aarch64-macos` and the release tarball carries the target
  triple as it does today.

### Step 3: the macOS arm

Two children, `server_os_macos.zig` and `client_os_macos.zig`, and the
harness and script arms above. Two questions are answered on hardware
before either file is written, each as a throwaway probe whose output is
recorded in `docs/decisions.md`:

1. **Does the pty master answer `tcgetattr` and `TIOCGPGRP` on Darwin?**
   If yes the Linux arm's shape is reused. If not, `ptyMode` and
   `ptyFgPgid` reopen the slave path by name, which `openPty` records for
   that purpose.
2. **Which default socket directory fits.** `$TMPDIR` is `/var/folders/…`,
   about fifty bytes, against a 103-byte usable `sun_path` that the
   per-session agent socket and `mux-ask-PID.sock` are dialled through
   from inside a session. The probe measures the longest path mux creates
   under each candidate (`$TMPDIR`, `~/Library/Caches/mux`, `~/.local/state
   /mux/run`) and picks the one with room; the spelling then lands in
   `sockpath.runtimeDir` and README.

Behaviours that differ on purpose on Darwin, each named in
`docs/decisions.md` when written: `closeFrom` is O(fd table) rather than
one syscall; `anonFd` is an unlinked file, private to the uid by the
directory mode rather than by anonymity; the harness cannot read another
process's environment.

### Testing

- Step 2 is a refactor: `make ci` green after every task, and the
  cross-version gate (`make xversion`) against the current tip proves the
  wire and the upgrade manifest did not change shape.
- Each interface operation gets a unit test in its root file that runs on
  whatever OS built it, against a real fd, a real child, a real pty: the
  fd barrier is asserted by a child shell testing `/dev/fd/N`, which both
  OSes have; `anonFd` by `fstat` reporting zero links; `selfImageStale` by
  renaming a copy of the test binary over its own path in a temp dir.
- Rule 7 is asserted the way rules 4 to 6 are: a doc-gate test that plants
  each needle in a scratch file under each banned folder and reads the
  fatal.
- Step 3's gate is `make ci` run on the Mac plus the harness helpers'
  own tests there, and a `make vm`-shaped journey script driven from
  whichever host builds.

## Deferred

**Where the macOS binary is built.** Cross-compiling from Linux is blocked
by ghostty's `apple_sdk` lookup (finding above). The options are a native
build on the Mac driven over ssh the way `make vm` drives the VM,
prebuilt macOS archives for the three C++ deps passed through ghostty's
`-fsys=` flags, or a patched ghostty pin. The decision is taken after step
2 lands, when the Linux side no longer moves under it, and it is not
needed to write either `_macos.zig` file.

**Splitting the shared root** into a paths layer and a connection layer is
worth doing and is not this work.

**x86_64-macos and a universal binary** are one more target word each and
are out of scope until someone has the hardware.

## Hardware findings, 2026-09-03 (amendment after step 2)

Measured on `squirtle` (Apple M1, macOS 26.6.2, Xcode 26.x) the day step 2
merged. Each of these settles a question the sections above left open, and
the step-3 plan (`docs/superpowers/plans/2026-09-03-macos-port-step3-darwin-arm.md`)
carries them as rulings.

- **Build host: native on the Mac, over ssh.** Not by choice among the
  three options in "Deferred" but by finding: zig 0.15.2 cannot link a
  NATIVE target against the SDK Xcode 26.4+ ships, because that SDK's
  `libSystem.B.tbd` lists `arm64e-macos` and no `arm64-macos`, and the
  0.15 Mach-O linker matches the bare slice only (ziglang/zig#31658 on
  Codeberg; fixed upstream in 0.16, which ghostty's pin cannot take). The
  build runner is a native link, so `zig build` cannot start. An explicit
  `-target aarch64-macos` links against zig's bundled stub and runs, which
  is the whole fix: a shadow SDK — the real one by symlink except
  `usr/lib/libSystem{,.B}.tbd`, where zig's stub stands in — reached through
  an `xcrun` shim on PATH. With it the full `zig build` compiled ghostty and
  its three C++ dependencies natively and stopped at exactly the three
  step-3 `@compileError`s. The QUIC stack built natively in about a minute
  once the dep script's `native` word stopped assuming an x86 host.
- **Probe 1, the pty master:** `tcgetattr`, `TIOCGPGRP`, `TIOCSWINSZ` and
  `TIOCGWINSZ` on the master all answered 0 against a forked child holding
  the slave as its controlling tty. The Linux arm's shape is reused; nothing
  reopens the slave by name, and `openPty` need not record it.
- **Probe 2, the socket directory:** the longest path mux creates is
  `<dir>/mux-agent-<pid>-<12 hex>/agent-<session>.sock`, dir + 68 bytes +
  pid digits at `session_name_max` 32, against 103 usable. `$TMPDIR` is 48
  bytes on this box (121), `~/Library/Caches/mux` 34 (107),
  `~/.local/state/mux/run` 36 (109): none of the three fits. `/tmp/mux-501`
  is 12 (85). The Darwin spelling is therefore `$XDG_RUNTIME_DIR` when set
  (every isolated rig sets it, on both OSes) and otherwise `/tmp/mux-<uid>`,
  created `0700` and re-checked on every ask for owner, mode and no symlink
  — tmux's `/tmp/tmux-UID` rule, for tmux's reason.
- **The upgrade carrier** on Darwin is a `mkstemp` file in `/tmp`, mode
  `0600`, unlinked before `anonFd` returns; "private by mode" replaces
  "private by having no name" for the width of two syscalls on an empty file.
- **Darwin `sendNoSig` sets `SO_NOSIGPIPE` per call.** No adopt-socket hook
  is added to the roots: the option is idempotent and the accept sites stay
  outside the platform rows.