110e95aa
Add input-latency bench design spec
a73x 2026-04-18 06:38
Closed-loop input-latency benchmark design for waystty, complementing the existing output-only bench. Covers cold (idle) and hot (contended PTY) latency via in-process KeyEvent injection, PUA-codepoint sentinels, and wp_presentation_time feedback for the display endpoint. Keeps MAILBOX present mode throughout for fidelity to normal operation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
diff --git a/docs/superpowers/specs/2026-04-18-input-latency-bench-design.md b/docs/superpowers/specs/2026-04-18-input-latency-bench-design.md new file mode 100644 index 0000000..bf6c94f --- /dev/null +++ b/docs/superpowers/specs/2026-04-18-input-latency-bench-design.md @@ -0,0 +1,145 @@ # Input-Latency Bench Design ## Goal Measure waystty's keystroke-to-display latency in a reproducible, closed-loop, automated way. This complements the existing output-only bench (`docs/superpowers/specs/2026-04-10-performance-benchmarking-design.md`), which deliberately excluded input responsiveness. The output is two headline metrics plus a per-stage breakdown: - **Cold latency** — terminal is idle; press a key, measure until the echo appears on screen. Tests the best-case path. - **Hot latency** — the child process is producing continuous output; press a key while output floods the PTY. Measures contended-PTY responsiveness, which is where terminals diverge in feel. ## Non-goals - Real photon latency (requires external hardware; not automatable). - Virtual-keyboard-protocol injection through the compositor (adds compositor and xkb as variables; worth having as a separate integration check later). - Scriptable screen-state assertions beyond sentinel detection (YAGNI; the primitives this design uses are extensible to that later). - Changing waystty's normal present mode. MAILBOX stays the default. ## Architecture New env var `WAYSTTY_INPUT_BENCH={cold|hot|both}` activates input-bench mode. When active, waystty: 1. Spawns a minimal PTY child instead of `$SHELL`: - `cold` → `sleep infinity`. Kernel PTY line discipline echoes each byte (ECHO flag on the slave); no reader needed. This removes `cat`'s canonical-mode buffering as a confound. - `hot` → `sh -c 'yes "spam spam spam spam spam"'`. Continuous output contends with sentinel echo bytes on the PTY. - `both` → runs cold, dumps stats, restarts child, runs hot. 2. Disables real Wayland keyboard input (nothing is pushed to `keyboard.event_queue` from `src/wayland.zig`). Protects measurements from ambient user input or stuck modifier state. 3. Runs the `BenchDriver` (new, in `src/bench_input.zig`) alongside the normal main loop. Driver owns the sentinel allocator, the in-flight sample, the pending-feedback map, and the stats collector. 4. On exit (N samples reached or timeout), prints stats to stderr in the existing output-bench format and exits cleanly. MAILBOX present mode is preserved throughout. The driver handles `wp_presentation_feedback.discarded` by keeping feedback listeners live on subsequent frames, since a discarded frame's sentinel is still in the grid on the next frame. ## Module 1: Sentinel allocation Each sample uses a unique codepoint from the Unicode Private Use Area U+E000…U+EFFF (4096 distinct sentinels). PUA is chosen because it never appears in normal output from `sleep infinity` or `yes`, so a grid scan for a specific codepoint cannot collide with unrelated output. If a run exceeds 4096 samples, the allocator wraps (no collision concern because only one sample is in flight at a time). PUA codepoints render as `.notdef` (tofu) for most fonts; this is fine — the grid cell carries the codepoint regardless of glyph appearance. ## Module 2: Injection A fabricated `wayland_client.Keyboard.KeyEvent` is pushed onto `keyboard.event_queue`. The event has `utf8` filled with the sentinel's UTF-8 encoding and `utf8_len` set appropriately. No other fields (keysym, modifiers) are needed for the UTF-8 path at `src/main.zig:393`: ```zig if (ev.utf8_len > 0) { _ = try p.write(ev.utf8[0..ev.utf8_len]); } ``` `t_inject` is captured with `std.time.Instant.now()` immediately before the push. On Linux this uses `CLOCK_MONOTONIC`, matching `wp_presentation_feedback`'s clock. The real `src/wayland.zig` keyboard binding remains connected so the protocol stays valid, but its callback drops events when bench mode is active. ## Module 3: Presentation feedback Requires binding `wp_presentation_time` (global `wp_presentation`). The repo does not currently bind this protocol; this is real implementation work, not plumbing. Flow per frame: 1. Immediately before each `vkQueuePresentKHR` that follows any pending injection, call `wp_presentation.feedback(surface)` to get a new `wp_presentation_feedback` object. Record the association: `{frame_counter → feedback_object}`. 2. The feedback object fires one of two events eventually: - `presented(tv_sec_hi, tv_sec_lo, tv_nsec, refresh, seq_hi, seq_lo, flags)` — frame was latched by the compositor. Compute `t_present = tv_sec * 1e9 + tv_nsec` (CLOCK_MONOTONIC nanoseconds). - `discarded` — frame was superseded. Drop the feedback object but keep listeners on subsequent frames; the sentinel is still in the grid for the next frame. 3. The Vulkan WSI owns the Wayland `wl_surface.commit` call; `wp_presentation.feedback` attaches to the next commit on the surface. As long as the feedback request precedes `vkQueuePresentKHR`, it captures the right commit. ## Module 4: Pair-on-arrival matching An in-flight sample has two asynchronous completions: - **Grid-side:** scan each frame's `term.snapshot()` for the sentinel codepoint. Record the first frame K where the sentinel is present. - **Feedback-side:** when `presented` fires for frame K's feedback object, record `t_present_K`. The sample completes when *both* are known for the same frame K. Latency = `t_present_K - t_inject`. Store in a ring buffer (`u32` microseconds, capped at ~68 min range — more than enough). If frame K's feedback is `discarded`, move on to frame K+1's feedback (still looking for sentinel in K+1's grid — the sentinel cell will be there until scrolled off). A sample times out if neither grid nor feedback match within `max_frames_per_sample = 120` (~2s at 60Hz). Timeouts are counted as a separate stat, not folded into the latency distribution. Grid scan cost: scanning all cells (typical ~200×50 = 10k cells) per frame for a single codepoint is trivial (~10µs). No optimization needed. ## Module 5: Sample scheduling After a sample matches (both sides landed), schedule the next injection on the next main-loop iteration following the *next* frame. This is "one-frame floor" — prevents two injections inside the same frame — without inventing a magic millisecond number. Target sample count per scenario: **500**. At ~1 sample per refresh + echo round-trip, cold should complete in ~10-15s, hot can run longer due to contention. ## Module 6: Output On exit, prints to stderr, extending the existing format: ``` === waystty input latency (500 cold, 500 hot) === scenario min avg p50 p99 max (ms) timeouts cold 1.2 4.8 4.1 9.2 14.1 0 hot 3.4 12.7 11.5 28.4 42.0 3 ───────────────────────────────────────── ``` Per-stage breakdown (for p99 samples only) reuses the existing frame-timing ring buffer at the frame K of those samples — snapshots which stage of the output path dominated the p99 tail: ``` p99 cold sample breakdown (µs): inject → pty_write_seen: 820 pty_write → term.write: 140 term.write → frame K start: 620 frame K total render: 6800 frame K submit → present: 820 ``` (This is an additional module, not gating. Ship it after the headline metric works.) ## Module 7: Makefile target ```makefile bench-input: $(ZIG) build -Doptimize=$(OPT) WAYSTTY_INPUT_BENCH=both ./zig-out/bin/waystty 2>bench-input.log || true @echo "--- input latency ---" @grep -A 12 "waystty input latency" bench-input.log || echo "(no timing data found)" ``` Mirrors the existing `bench` / `profile` targets: `OPT` defaults to `ReleaseFast`; output in `bench-input.log`. ## Files changed - `src/bench_input.zig` — new. `BenchDriver` struct, sentinel allocator, in-flight sample state, pending-feedback map, stats printer. - `src/main.zig` — env parsing, driver init/tick hookup, PTY child switcheroo, scenario sequencer, sentinel scan in the post-frame hook, bench-keyboard-suppression gate. - `src/wayland.zig` — bind `wp_presentation` global, plumb `wp_presentation_feedback` creation and event callbacks. - `src/renderer.zig` — expose swapchain present call site so `wp_presentation.feedback(surface)` can be called immediately before `vkQueuePresentKHR`. - `Makefile` — `bench-input` target. ## Testing - **Unit:** `BenchDriver` grid scan finds the active sentinel at arbitrary row/col positions in a synthetic grid. - **Unit:** sentinel allocator produces 4096 distinct codepoints before wrapping. - **Unit:** pair-on-arrival state machine completes when events arrive in either order; advances past `discarded`. - **Manual:** `make bench-input` produces cold and hot numbers with cold < hot. Sanity: cold p50 on the order of one refresh interval (wait-to-vsync plus compositor latch) is expected; hot p99 noticeably larger than cold p99 confirms contention is being observed. - **Manual:** `WAYSTTY_INPUT_BENCH=cold` against a debug build; verify via logs that each injected sentinel round-trips and matches within a few frames. ## Open implementation risks - **`wp_presentation_time` binding is non-trivial.** Neither the global nor the feedback callbacks exist in the repo. Budget real time for this, including the WSI-commit coordination (feedback request must precede `vkQueuePresentKHR`). - **Mesa Wayland WSI internals.** Confirm empirically that `wp_presentation.feedback(surface)` called immediately before `vkQueuePresentKHR` attaches to the driver's commit. If not, we may need `VK_KHR_present_wait` or a different coordination strategy. - **`sleep infinity` tty state.** Confirm kernel echoes bytes on a PTY whose slave-side reader is `sleep` (which doesn't open the tty in any special mode). If echo is off, `stty echo` on the slave fd before exec resolves it.