a73x

110e95aa

Add input-latency bench design spec

a73x   2026-04-18 06:38

Commit message
Add input-latency bench design spec

Closed-loop input-latency benchmark design for waystty, complementing
the existing output-only bench. Covers cold (idle) and hot (contended
PTY) latency via in-process KeyEvent injection, PUA-codepoint sentinels,
and wp_presentation_time feedback for the display endpoint. Keeps
MAILBOX present mode throughout for fidelity to normal operation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
docs/superpowers/specs/2026-04-18-input-latency-bench-design.md
Old New
@@ -0,0 +1,145 @@
1 # Input-Latency Bench Design
2
3 ## Goal
4
5 Measure waystty's keystroke-to-display latency in a reproducible, closed-loop, automated way. This complements the existing output-only bench (`docs/superpowers/specs/2026-04-10-performance-benchmarking-design.md`), which deliberately excluded input responsiveness.
6
7 The output is two headline metrics plus a per-stage breakdown:
8
9 - **Cold latency** — terminal is idle; press a key, measure until the echo appears on screen. Tests the best-case path.
10 - **Hot latency** — the child process is producing continuous output; press a key while output floods the PTY. Measures contended-PTY responsiveness, which is where terminals diverge in feel.
11
12 ## Non-goals
13
14 - Real photon latency (requires external hardware; not automatable).
15 - Virtual-keyboard-protocol injection through the compositor (adds compositor and xkb as variables; worth having as a separate integration check later).
16 - Scriptable screen-state assertions beyond sentinel detection (YAGNI; the primitives this design uses are extensible to that later).
17 - Changing waystty's normal present mode. MAILBOX stays the default.
18
19 ## Architecture
20
21 New env var `WAYSTTY_INPUT_BENCH={cold|hot|both}` activates input-bench mode.
22
23 When active, waystty:
24
25 1. Spawns a minimal PTY child instead of `$SHELL`:
26 - `cold` → `sleep infinity`. Kernel PTY line discipline echoes each byte (ECHO flag on the slave); no reader needed. This removes `cat`'s canonical-mode buffering as a confound.
27 - `hot` → `sh -c 'yes "spam spam spam spam spam"'`. Continuous output contends with sentinel echo bytes on the PTY.
28 - `both` → runs cold, dumps stats, restarts child, runs hot.
29 2. Disables real Wayland keyboard input (nothing is pushed to `keyboard.event_queue` from `src/wayland.zig`). Protects measurements from ambient user input or stuck modifier state.
30 3. Runs the `BenchDriver` (new, in `src/bench_input.zig`) alongside the normal main loop. Driver owns the sentinel allocator, the in-flight sample, the pending-feedback map, and the stats collector.
31 4. On exit (N samples reached or timeout), prints stats to stderr in the existing output-bench format and exits cleanly.
32
33 MAILBOX present mode is preserved throughout. The driver handles `wp_presentation_feedback.discarded` by keeping feedback listeners live on subsequent frames, since a discarded frame's sentinel is still in the grid on the next frame.
34
35 ## Module 1: Sentinel allocation
36
37 Each sample uses a unique codepoint from the Unicode Private Use Area U+E000…U+EFFF (4096 distinct sentinels). PUA is chosen because it never appears in normal output from `sleep infinity` or `yes`, so a grid scan for a specific codepoint cannot collide with unrelated output. If a run exceeds 4096 samples, the allocator wraps (no collision concern because only one sample is in flight at a time).
38
39 PUA codepoints render as `.notdef` (tofu) for most fonts; this is fine — the grid cell carries the codepoint regardless of glyph appearance.
40
41 ## Module 2: Injection
42
43 A fabricated `wayland_client.Keyboard.KeyEvent` is pushed onto `keyboard.event_queue`. The event has `utf8` filled with the sentinel's UTF-8 encoding and `utf8_len` set appropriately. No other fields (keysym, modifiers) are needed for the UTF-8 path at `src/main.zig:393`:
44
45 ```zig
46 if (ev.utf8_len > 0) {
47 _ = try p.write(ev.utf8[0..ev.utf8_len]);
48 }
49 ```
50
51 `t_inject` is captured with `std.time.Instant.now()` immediately before the push. On Linux this uses `CLOCK_MONOTONIC`, matching `wp_presentation_feedback`'s clock.
52
53 The real `src/wayland.zig` keyboard binding remains connected so the protocol stays valid, but its callback drops events when bench mode is active.
54
55 ## Module 3: Presentation feedback
56
57 Requires binding `wp_presentation_time` (global `wp_presentation`). The repo does not currently bind this protocol; this is real implementation work, not plumbing.
58
59 Flow per frame:
60
61 1. Immediately before each `vkQueuePresentKHR` that follows any pending injection, call `wp_presentation.feedback(surface)` to get a new `wp_presentation_feedback` object. Record the association: `{frame_counter → feedback_object}`.
62 2. The feedback object fires one of two events eventually:
63 - `presented(tv_sec_hi, tv_sec_lo, tv_nsec, refresh, seq_hi, seq_lo, flags)` — frame was latched by the compositor. Compute `t_present = tv_sec * 1e9 + tv_nsec` (CLOCK_MONOTONIC nanoseconds).
64 - `discarded` — frame was superseded. Drop the feedback object but keep listeners on subsequent frames; the sentinel is still in the grid for the next frame.
65 3. The Vulkan WSI owns the Wayland `wl_surface.commit` call; `wp_presentation.feedback` attaches to the next commit on the surface. As long as the feedback request precedes `vkQueuePresentKHR`, it captures the right commit.
66
67 ## Module 4: Pair-on-arrival matching
68
69 An in-flight sample has two asynchronous completions:
70
71 - **Grid-side:** scan each frame's `term.snapshot()` for the sentinel codepoint. Record the first frame K where the sentinel is present.
72 - **Feedback-side:** when `presented` fires for frame K's feedback object, record `t_present_K`.
73
74 The sample completes when *both* are known for the same frame K. Latency = `t_present_K - t_inject`. Store in a ring buffer (`u32` microseconds, capped at ~68 min range — more than enough).
75
76 If frame K's feedback is `discarded`, move on to frame K+1's feedback (still looking for sentinel in K+1's grid — the sentinel cell will be there until scrolled off).
77
78 A sample times out if neither grid nor feedback match within `max_frames_per_sample = 120` (~2s at 60Hz). Timeouts are counted as a separate stat, not folded into the latency distribution.
79
80 Grid scan cost: scanning all cells (typical ~200×50 = 10k cells) per frame for a single codepoint is trivial (~10µs). No optimization needed.
81
82 ## Module 5: Sample scheduling
83
84 After a sample matches (both sides landed), schedule the next injection on the next main-loop iteration following the *next* frame. This is "one-frame floor" — prevents two injections inside the same frame — without inventing a magic millisecond number.
85
86 Target sample count per scenario: **500**. At ~1 sample per refresh + echo round-trip, cold should complete in ~10-15s, hot can run longer due to contention.
87
88 ## Module 6: Output
89
90 On exit, prints to stderr, extending the existing format:
91
92 ```
93 === waystty input latency (500 cold, 500 hot) ===
94 scenario min avg p50 p99 max (ms) timeouts
95 cold 1.2 4.8 4.1 9.2 14.1 0
96 hot 3.4 12.7 11.5 28.4 42.0 3
97 ─────────────────────────────────────────
98 ```
99
100 Per-stage breakdown (for p99 samples only) reuses the existing frame-timing ring buffer at the frame K of those samples — snapshots which stage of the output path dominated the p99 tail:
101
102 ```
103 p99 cold sample breakdown (µs):
104 inject → pty_write_seen: 820
105 pty_write → term.write: 140
106 term.write → frame K start: 620
107 frame K total render: 6800
108 frame K submit → present: 820
109 ```
110
111 (This is an additional module, not gating. Ship it after the headline metric works.)
112
113 ## Module 7: Makefile target
114
115 ```makefile
116 bench-input:
117 $(ZIG) build -Doptimize=$(OPT)
118 WAYSTTY_INPUT_BENCH=both ./zig-out/bin/waystty 2>bench-input.log || true
119 @echo "--- input latency ---"
120 @grep -A 12 "waystty input latency" bench-input.log || echo "(no timing data found)"
121 ```
122
123 Mirrors the existing `bench` / `profile` targets: `OPT` defaults to `ReleaseFast`; output in `bench-input.log`.
124
125 ## Files changed
126
127 - `src/bench_input.zig` — new. `BenchDriver` struct, sentinel allocator, in-flight sample state, pending-feedback map, stats printer.
128 - `src/main.zig` — env parsing, driver init/tick hookup, PTY child switcheroo, scenario sequencer, sentinel scan in the post-frame hook, bench-keyboard-suppression gate.
129 - `src/wayland.zig` — bind `wp_presentation` global, plumb `wp_presentation_feedback` creation and event callbacks.
130 - `src/renderer.zig` — expose swapchain present call site so `wp_presentation.feedback(surface)` can be called immediately before `vkQueuePresentKHR`.
131 - `Makefile` — `bench-input` target.
132
133 ## Testing
134
135 - **Unit:** `BenchDriver` grid scan finds the active sentinel at arbitrary row/col positions in a synthetic grid.
136 - **Unit:** sentinel allocator produces 4096 distinct codepoints before wrapping.
137 - **Unit:** pair-on-arrival state machine completes when events arrive in either order; advances past `discarded`.
138 - **Manual:** `make bench-input` produces cold and hot numbers with cold < hot. Sanity: cold p50 on the order of one refresh interval (wait-to-vsync plus compositor latch) is expected; hot p99 noticeably larger than cold p99 confirms contention is being observed.
139 - **Manual:** `WAYSTTY_INPUT_BENCH=cold` against a debug build; verify via logs that each injected sentinel round-trips and matches within a few frames.
140
141 ## Open implementation risks
142
143 - **`wp_presentation_time` binding is non-trivial.** Neither the global nor the feedback callbacks exist in the repo. Budget real time for this, including the WSI-commit coordination (feedback request must precede `vkQueuePresentKHR`).
144 - **Mesa Wayland WSI internals.** Confirm empirically that `wp_presentation.feedback(surface)` called immediately before `vkQueuePresentKHR` attaches to the driver's commit. If not, we may need `VK_KHR_present_wait` or a different coordination strategy.
145 - **`sleep infinity` tty state.** Confirm kernel echoes bytes on a PTY whose slave-side reader is `sleep` (which doesn't open the tty in any special mode). If echo is off, `stty echo` on the slave fd before exec resolves it.