docs/superpowers/specs/2026-09-04-cells-on-the-wire-design.md

Ref: main worktree-web-client-spec   Size: 11.1 KiB   History

# Cells on the wire — design

2026-09-04. Clean break: the replay frames change shape and number, and
no daemon or client of an earlier version pairs with one of this. The
migration is `mux d upgrade` on every box, which the release note says.

## Goal

The client stops parsing VT. Today a byte of session output is parsed
three times before a person sees it: the daemon's ghostty-vt (the
authoritative grid), the client's ghostty-vt (the replica, fed the delta's
bytes), and the host terminal (fed the client's paint). After this change
the daemon's parse is the only one in the system: the wire carries the
CELLS the daemon's grid holds, the client copies them into a grid of its
own, and paints from that grid — to a terminal today, to a renderer when a
native or browser client wants one. The client no longer links ghostty-vt
at all; `mux_core.wasm` shrinks by the whole emulator.

## Findings the design rests on

- **The delta row is painted bytes by construction.** `protocol.composeDelta`
  stamps `CUP` + `EL2` around a row `Engine.dumpVtRow` rendered as SGR, and
  `replica.apply` FEEDS that to a second engine (`replica.zig:82-87`). The
  rule-4 exemption on `protocol.zig` says so in its own words. Nothing else
  makes the client an emulator.
- **The terminal client already passes rows through verbatim.** On the common
  path `paint.paintDeltaClipped` writes the daemon's row bytes to the tty
  unchanged (`paint.zig:194`) and re-dumps from the replica only for a row
  under the selection or an over-wide pane. So for the TERMINAL client this
  is a trade — a stateless serialize replaces a VT state machine — and the
  over-wide special case (a daemon row is grid-wide; a narrower pane's
  surplus lands on the neighbour) disappears, because every row now paints
  from the grid clipped to the pane.
- **The browser painter already consumes cells.** `wasm_core.paintRow` reads
  ghostty's page cells into a 4×u32 record (codepoint, fg, bg, flags|wide)
  that `web/mux.js` draws on a canvas. The cell the wire carries is that
  record plus underline colour and the grapheme cluster; the JS side does not
  change.
- **Modes and title are already a side channel.** `term_modes` (a u32 bitset:
  bracketed paste, the mouse modes) and `term_title` frames carry the state
  the client acts on; the replica engine is consulted for exactly two mode
  bits — `onAltScreen` and `cursorKeys` for the wheel-to-arrows rule
  (`interact.zig:1747`). Those two become bits of `term_modes`.
- **The wire size is the open question.** A run-length styled row is roughly
  SGR-shaped, but dense plain text costs a length byte per cell that SGR does
  not pay. An `ascii` run form (one byte per cell, no header) closes most of
  that. The number is measured in Task 2 before the flip is built, and the
  stats counters (`snapshot_bytes`, `delta_bytes`, `make bench`) grade the
  real workload after.

## Design

### The wire

Frame kinds keep their names and take NEW numbers, so a binary of either
side that meets the other shows a blank tile and never garbage: an unknown
type is dropped by both readers, while a VT reader fed cell bytes, or a cell
reader fed VT, would paint noise.

| kind | number | payload |
|---|---|---|
| `snapshot` | `0x95` | `SnapshotPrefix` (24 B, unchanged) ++ `u16 LE cursor_x` ++ `u16 LE cursor_y` ++ `rows` × `CellRow`, row 0 first |
| `delta` | `0x96` | `DeltaHeader` (18 B, unchanged) ++ `row_count` × (`u16 LE row` ++ `u32 LE len` ++ `CellRow`) — the delta row header is unchanged |
| `scrollback_chunk` | `0x97` | `u32 LE start` ++ `u16 LE count` ++ `count` × `CellRow` |

`0x81`, `0x85`, `0x87` are retired with a comment and never reused.

**`CellRow`** — `u16 LE ncells` then runs until `ncells` cells are read.
Cells past `ncells` up to the grid width are default-style blanks, so the
encoder stops at the last cell that is not one (a cell whose content is a
bare background colour is NOT blank).

A **run** is a header that carries only what CHANGED since the previous run
of the same row — the shape SGR has, which is why VT rows are small — then
`count` cells. The first run of a row starts from the default style; a row
is self-contained, so nothing carries between rows or frames.

- `u16 LE count` ++ `u8 mask` ++ the present fields in order. `mask` bit 0:
  `flags` present, `u16 LE` follows; bit 1: `fg` present; bit 2: `bg`
  present; bit 3: `ul` present; bit 7: `ascii`. A present colour is `u8
  code` — 0 none, 1 palette then one index byte, 2 RGB then three bytes.
  A default first run is 3 bytes; a palette foreground change is 5.
  (The first cut was a 16-byte absolute header, and the measurement gate
  refused it: vim 1.45×, htop 1.56× — see decisions.md, 2026-09-04.)
- `flags` bits 0–10 are ghostty's `Style.Flags` verbatim — bold 0, italic 1,
  faint 2, blink 3, inverse 4, invisible 5, strikethrough 6, overline 7,
  underline style 8–10 (none 0, single 1, double 2, curly 3, dotted 4,
  dashed 5). `ascii` means every cell of the run is ONE byte in
  `0x20..0x7E`, narrow, with no per-cell header.
- In memory (`CellStyle`, the grid, the wasm viewport) colours stay packed
  as `wasm_core.packColor` does today: `0` none, `1<<24 | index` palette,
  `2<<24 | r<<16 | g<<8 | b` RGB; the wire spells the same three cases as
  the code byte above.
- A non-`ascii` cell is `u8 head` = `wide << 6 | text_len` then `text_len`
  bytes of UTF-8 — the whole grapheme cluster, capped at 63 bytes (the
  encoder keeps the leading codepoints that fit). `wide` is 0 narrow, 1
  wide, 2 spacer_tail, 3 spacer_head; a spacer has `text_len` 0. An empty
  text is a blank glyph in that style.

The delta tracker hashes the encoded `CellRow`, so "this row changed" on
the daemon means "these bytes differ" on the wire.

### `term_modes` gains two bits

`alt_screen` (the alternate screen is active) and `cursor_keys` (DECCKM).
The daemon samples them where it samples bracketed paste
(`Server.sampleTermModes`); the client's wheel rule reads them from
`client_core`'s modes instead of from an engine.

### The client grid — `src/engine/grid.zig`, a child of `term`

Platform-free like `replica.zig`: no posix, no clocks, compiles for
wasm32-freestanding. `Grid` is `cols × rows` of `Cell` (`style`, `wide`,
and a slice into the row's own text buffer), a cursor, and:

- `applyRow(y, bytes)` — decode one `CellRow` into row `y`;
- `decodeRows(alloc, bytes, count, cols)` — a dense run of rows (snapshot
  body, scrollback chunk) as `[]Row`, for the caller to own;
- `dumpPlain(alloc)` — the text `Engine.dumpPlain` produces for the same
  screen, byte for byte, which is what every harness compares;
- `clipCol` and `snapWide` — the two wide-glyph rules `engine.zig` holds
  today, on the `wide` flag instead of a page pin.

`Replica` holds a `*Grid` instead of an `*Engine`. A snapshot resizes,
clears, sets the cursor and applies `rows` dense rows; a delta applies each
listed row and sets the cursor. A snapshot whose body will not decode is
`error.BadPayload` out of `apply` — the pump ends the tile with it — never a
`.resync`, because a resync re-attaches and reads the same bytes again.

### Painting from the grid — `src/tui/paint.zig`

`rowToVt(alloc, row, view, span)` is the one serializer: it walks the
row's cells to `clipCol`, emits an SGR when the style changes, writes a
wide glyph once and skips its spacer, writes a space for an empty text, and
stops at the last non-default cell (the caller's ECH has cleared the rest).
A `span` is painted inverted and plain, snapped outward to whole glyphs,
exactly as `Engine.dumpVtRowSpan` does today. `renderClipped`,
`paintDeltaClipped` and `renderScrollback` call it; the verbatim branch and
the over-wide branch of `paintDeltaClipped` go, because there is no
daemon-rendered row to pass through any more.

### The daemon — `src/engine/engine.zig`, `delta.zig`, `server.zig`

`Engine.encodeViewportRow(alloc, y)` and `Engine.encodeScrollback(alloc,
start, count)` read ghostty's page cells (content, `page.styles`,
`lookupGrapheme`, `wide`) into `CellRow` bytes. `delta.buildSnapshot`
composes prefix + cursor + rows; `DeltaTracker` hashes encoded rows;
`Server.buildSnapshotPayload`, `accrueSnapshotEquiv` and
`onFetchScrollback` call them. `dumpState` stays — it is how a daemon
carries its own state across `mux d upgrade`'s exec, and what `debug_dump`
prints — and so do `dumpVt`/`dumpPlain` (the e2e render oracle, `mux d
dump`). `dumpVtRow`, `dumpVtRowClipped`, `dumpVtRowSpan`, `dumpScrollback`
and `protocol.composeDelta` are deleted with the exemption line that
explained them.

### The module table

`term` (`term.zig` → `protocol`, `replica`, `grid`) no longer imports
ghostty-vt and stays the wasm root. A new row `engine` (`engine.zig` →
`delta`) imports `term` and links ghostty-vt. `daemon` and the e2e `render`
fixture import `engine`; `client`, `wall`, `webhub`, `wsclient` and the wasm
core import only `term`. Tests that author a screen still do it through an
`Engine` — `Engine.mirrorInto(grid)` encodes every row into a `Grid` — so
`wall`, `client` and `wsclient` carry `engine` as a TEST import and the
production client links no emulator. The ghostty wasm dependency is removed
from `build.zig`.

## What this removes

The client-side VT parse; the `protocol.zig` rule-4 exemption; the
verbatim-versus-redump branch in `paintDeltaClipped` and its "only safe
while the pane is as wide as the grid" rule; the scratch `Engine` the wasm
core keeps for scrollback; ghostty-vt from every client binary and from the
wasm build.

## The measurement gate

Task 2 encodes four synthetic screens both ways (dense prose, a vim-like
screen, an htop-like screen, a mostly-blank shell) and prints bytes and
ratio. The flip (Task 5) is built only if dense prose is ≤ 1.5× the VT rows
and the styled screens ≤ 1.2×; otherwise the branch stops there and reports,
because the alternative plan — VT frames for terminal clients, cells only
for native and browser ones — is a different and smaller project. After the
flip, `make bench` on `main` and on this branch records the real numbers in
`docs/decisions.md`.

## Testing

- `protocol.zig`: golden bytes for every cell form and every malformed
  shape.
- `engine.zig`: the encoder round-trips through the decoder for ASCII, wide,
  grapheme, bg-only, styled and trailing-blank rows; the ORACLE test feeds
  VT into an `Engine`, mirrors it into a `Grid`, and compares `dumpPlain` and
  the cursor — the engine is the oracle, the grid must agree.
- `server_test_*`: unchanged in intent; the harness's replica is a `Grid`.
- `paint.zig`, `interact.zig`, `wall_test_*`: the pins that exist, with
  screens authored through `mirrorInto`. The span-bounded clear and `col_off`
  pins must each fire once against the new serializer.
- `make ci` is the delivery gate. `make xversion` has no old side to grade
  until the next release and is out of this branch's gate, recorded in
  `docs/decisions.md`.

## Deferred

- OSC 8 hyperlinks: the wire has no field; today's formatter emits none on
  the delta path either, so parity holds.
- The JS painter draws a cell's first codepoint; the wire now carries the
  whole cluster, so the canvas can draw it when someone wants to.
- A native client. This change is what makes one cheap to write.