a73x

106552cd

test: guard native raw-output responsiveness with release builds

a73x   2026-09-05 11:40

Commit message
test: guard native raw-output responsiveness with release builds
Makefile
Old New
@@ -31,7 +31,7 @@ SHA256 ?= shasum -a 256
31 endif 31 endif
32 MUX_TARGET ?= x86_64-linux-musl 32 MUX_TARGET ?= x86_64-linux-musl
33 33
34 .PHONY: build check ci test e2e soak bench agent native native-e2e throughput vm coverage deps clean clean-deps xversion xversion-build install release release-mac mac-sdk mac xos provision-mac 34 .PHONY: build check ci test e2e soak bench agent native native-e2e native-stress throughput vm coverage deps clean clean-deps xversion xversion-build install release release-mac mac-sdk mac xos provision-mac
35 35
36 # The QUIC stack (deps/quic) is built on demand by build.zig, so no target 36 # The QUIC stack (deps/quic) is built on demand by build.zig, so no target
37 # here needs to depend on this one. It exists to make the one-time cost 37 # here needs to depend on this one. It exists to make the one-time cost
@@ -277,6 +277,9 @@ native: mac-sdk
277 native-e2e: mac-sdk 277 native-e2e: mac-sdk
278 $(ZIG) build native-e2e -Doptimize=ReleaseSafe 278 $(ZIG) build native-e2e -Doptimize=ReleaseSafe
279 279
280 native-stress: mac-sdk
281 $(ZIG) build native-stress -Doptimize=ReleaseSafe
282
280 # Cross-version gate (test/xversion.sh): this tree's client against a 283 # Cross-version gate (test/xversion.sh): this tree's client against a
281 # previous version's daemon and back, each daemon in a container. 284 # previous version's daemon and back, each daemon in a container.
282 # 285 #
README.md
Old New
@@ -25,11 +25,26 @@ packages; OpenGL functions are loaded through SDL, with no direct libGL
25 link. It is outside the default build and CI gates: 25 link. It is outside the default build and CI gates:
26 26
27 ```sh 27 ```sh
28 make native # build muxg and run its no-window unit tests 28 make native # Debug development build and no-window unit tests
29 make native-e2e # ReleaseSafe pane, picker, resize and restore checks; needs python3 29 make native-e2e # ReleaseSafe pane, picker, resize and restore checks; needs python3
30 make native-stress # Linux: raw cat /dev/random, shared and separate daemon neighbours
30 ./zig-out/bin/muxg [TARGET] [--session NAME] [--sock PATH] [--via CMD] [--key PATH] [--font-px N] 31 ./zig-out/bin/muxg [TARGET] [--session NAME] [--sock PATH] [--via CMD] [--key PATH] [--font-px N]
31 ``` 32 ```
32 33
34 For interactive use and performance measurements, build **both** binaries in
35 ReleaseSafe. Debug enables expensive terminal integrity checks in the daemon;
36 a release GUI alone does not avoid that cost. A separate output directory keeps
37 subsequent development checks from replacing these binaries:
38
39 ```sh
40 deps/zig/zig build install native -Doptimize=ReleaseSafe -p dist/native-release
41 ./dist/native-release/bin/mux d start -d
42 ./dist/native-release/bin/muxg
43 ```
44
45 The daemon command starts a new daemon; an already running daemon retains its
46 existing build until it is upgraded or restarted.
47
33 `muxg` displays daemon sessions in native terminal panes. With no arguments it 48 `muxg` displays daemon sessions in native terminal panes. With no arguments it
34 restores the saved native workspace; first launch opens the host/session picker. 49 restores the saved native workspace; first launch opens the host/session picker.
35 For local sessions, start the daemon with `mux d start -d`. It never starts a local 50 For local sessions, start the daemon with `mux d start -d`. It never starts a local
RETRO.md
Old New
@@ -272,6 +272,9 @@ within the release review or should remain an explicit follow-up.
272 Owner: the sprint lead and implementation/review pair. Close each with evidence 272 Owner: the sprint lead and implementation/review pair. Close each with evidence
273 at the final milestone retrospective. 273 at the final milestone retrospective.
274 274
275 - [ ] Include `make native-stress` and a real Sway raw-output run in the final
276 release review; build both demo binaries together in an isolated ReleaseSafe
277 prefix and report sampled input latency separately from frame timing.
275 - [ ] Obtain explicit Sprint 4 demo acceptance before beginning the release review. 278 - [ ] Obtain explicit Sprint 4 demo acceptance before beginning the release review.
276 - [ ] Start with a bounded review/cleanup of the accumulated native workflow, then 279 - [ ] Start with a bounded review/cleanup of the accumulated native workflow, then
277 retain a closing cleanup pass; avoid introducing tabs UI or terminal tiling changes. 280 retain a closing cleanup pass; avoid introducing tabs UI or terminal tiling changes.
@@ -283,3 +286,37 @@ at the final milestone retrospective.
283 shared sessions, state ownership/failures, high DPI, and cleanup of owned fixtures. 286 shared sessions, state ownership/failures, high DPI, and cleanup of owned fixtures.
284 - [ ] Record final demo acceptance separately from automation and any hands-on 287 - [ ] Record final demo acceptance separately from automation and any hands-on
285 ergonomic feedback, update the spec's completion status, and commit the milestone. 288 ergonomic feedback, update the spec's completion status, and commit the milestone.
289
290 ### Sprint 4 follow-up: raw-output responsiveness
291
292 The user's `cat /dev/random` question exposed two validation gaps: printable
293 floods do not exercise arbitrary terminal controls, and checking the GUI's build
294 mode does not establish the daemon's mode. The first ad-hoc run paired a
295 ReleaseSafe GUI with a Debug daemon left in `zig-out` by the repository gates.
296 Its same-daemon neighbour took up to 843 ms to appear, while a separate-daemon
297 neighbour appeared within 121 ms. Binary inspection confirmed Debug; the
298 daemon's terminal engine enables expensive integrity checks in that mode.
299
300 Rebuilding only the daemon in ReleaseSafe resolved the observed difference.
301 The same real Sway scenario consumed about 56 MB over 30 seconds, versus 0.88 MB
302 in Debug, with both neighbours observed within 121 ms. Close/reopen, menus and
303 divider resizing passed. Those latency figures include polling delay and are
304 upper bounds, not precise intrinsic timings. Independent offscreen replication
305 agreed; there was no evidence warranting a production scheduler change.
306
307 The follow-up adds `make native-stress`, whose build graph supplies both release
308 binaries and refuses Debug. The Linux test uses raw output, independently checks
309 producer identity and sustained daemon reads, probes shared and separate daemon
310 neighbours, and repeats lifecycle/resize checks during the flood. A tighter
311 observation interval and a 250 ms input-to-painted limit guard the reported
312 failure. Demo instructions now use a separate release prefix. The original
313 measurement is retained here as a corrected finding, not a product defect.
314
315 The permanent test passed offscreen and on Sway at 200%: with 5 ms observation
316 polling, shared-daemon response was at most 65.2 ms and separate-daemon response
317 51.1 ms on Sway, while about 57 MB ran over 30 seconds. Frame p99 stayed below
318 17.3 ms, and the lifecycle/resize checks passed. The Debug build target failed
319 before launching fixtures as intended. Adversarial review accepted the test
320 after adding its Makefile target to `.PHONY`.
321 As a negative control, deliberately bypassing the build target and pairing the
322 release GUI with the Debug daemon failed the new latency assertion at 878 ms.
build.zig
Old New
@@ -1180,6 +1180,18 @@ pub fn build(b: *std.Build) void {
1180 const native_e2e_step = b.step("native-e2e", "Run the native client's end-to-end leg (opt-in)"); 1180 const native_e2e_step = b.step("native-e2e", "Run the native client's end-to-end leg (opt-in)");
1181 native_e2e_step.dependOn(&native_lifecycle.step); 1181 native_e2e_step.dependOn(&native_lifecycle.step);
1182 1182
1183 // Both paths come from this build graph: a ReleaseSafe GUI beside a stale
1184 // Debug daemon gives misleading latency numbers under raw terminal output.
1185 const native_stress_step = b.step("native-stress", "Check raw-output native pane responsiveness (Linux, release build)");
1186 if (optimize != .ReleaseSafe and optimize != .ReleaseFast) {
1187 native_stress_step.dependOn(&b.addFail("native-stress requires -Doptimize=ReleaseSafe or ReleaseFast for both binaries").step);
1188 } else {
1189 const native_stress = b.addSystemCommand(&.{ "python3", "-B", "test/native_stress.py" });
1190 native_stress.addArtifactArg(mux_exe);
1191 native_stress.addArtifactArg(muxg_exe);
1192 native_stress_step.dependOn(&native_stress.step);
1193 }
1194
1183 const soak = b.addSystemCommand(&.{"test/soak.sh"}); 1195 const soak = b.addSystemCommand(&.{"test/soak.sh"});
1184 // The same list the e2e step passes, in the same order: soak IS that 1196 // The same list the e2e step passes, in the same order: soak IS that
1185 // suite run N times, so an argument added to one and not the other 1197 // suite run N times, so an argument added to one and not the other
docs/native-sprint-workflow.md
Old New
@@ -19,6 +19,12 @@ packages. Keep one sprint active, ending in a working demo and explicit acceptan
19 acceptance criteria, and each other's names. Root owns integration, independent 19 acceptance criteria, and each other's names. Root owns integration, independent
20 acceptance, user communication, and commits. Delegate another task only when 20 acceptance, user communication, and commits. Delegate another task only when
21 it can progress independently. Load skills only where they help the work. 21 it can progress independently. Load skills only where they help the work.
22
23 Cost default: explicitly select GPT-5.6 Luna for bounded implementation and
24 GPT-5.6 Terra for adversarial review. Start with task-specific context instead
25 of inheriting the full conversation. Escalate a concrete unresolved problem
26 to a stronger model when needed. Root coordinates and integrates; assign each
27 investigation one owner so agents do not repeat it independently by accident.
22 4. **Build a functional slice.** Establish the smallest model/interface contract, 28 4. **Build a functional slice.** Establish the smallest model/interface contract,
23 then connect actual input, rendering, and daemon behavior. Keep changes small 29 then connect actual input, rendering, and daemon behavior. Keep changes small
24 enough to review. Avoid expanding into later sprint features. 30 enough to review. Avoid expanding into later sprint features.
@@ -35,6 +41,14 @@ packages. Keep one sprint active, ending in a working demo and explicit acceptan
35 exercise SSH/QUIC when changes affect their behavior. Create and clean up only 41 exercise SSH/QUIC when changes affect their behavior. Create and clean up only
36 owned fixture sessions and daemons. 42 owned fixture sessions and daemons.
37 43
44 For responsiveness, run `make native-stress` on Linux. It builds both binaries
45 in ReleaseSafe from the same build graph and drives raw `cat /dev/random` in
46 one pane while probing neighbours on the same daemon and a separate daemon.
47 It checks ongoing output, input through completed painted state, menus,
48 divider resizing, and close/reopen under load. Daemon inspection happens after
49 the timed paint observation. Report sampled latency as an upper bound; frame
50 timing alone does not establish input responsiveness.
51
38 For lifecycle work, independently observe the daemon after Detach and End, 52 For lifecycle work, independently observe the daemon after Detach and End,
39 including another attached client. Change focus and open another modal while 53 including another attached client. Change focus and open another modal while
40 a destructive request is pending; a later confirmation must still identify 54 a destructive request is pending; a later confirmation must still identify
@@ -55,6 +69,7 @@ packages. Keep one sprint active, ending in a working demo and explicit acceptan
55 make ci 69 make ci
56 deps/zig/zig build native native-test -Doptimize=ReleaseSafe 70 deps/zig/zig build native native-test -Doptimize=ReleaseSafe
57 make native-e2e 71 make native-e2e
72 make native-stress # Linux raw-output responsiveness
58 ``` 73 ```
59 74
60 `make ci` excludes native tests. Use the pinned compiler. A sandbox denial is 75 `make ci` excludes native tests. Use the pinned compiler. A sandbox denial is
@@ -70,6 +85,17 @@ packages. Keep one sprint active, ending in a working demo and explicit acceptan
70 before closing the window so another application cannot enter the recording. 85 before closing the window so another application cannot enter the recording.
71 A slideshow of screenshots is useful evidence but is not a screen recording. 86 A slideshow of screenshots is useful evidence but is not a screen recording.
72 87
88 Build both demo binaries together in an isolated release prefix:
89
90 ```sh
91 deps/zig/zig build install native -Doptimize=ReleaseSafe -p dist/native-release
92 ```
93
94 Pass `dist/native-release/bin/mux` and `dist/native-release/bin/muxg` to Rig.
95 `make ci` can leave a Debug daemon in `zig-out`; `make native` builds only the
96 GUI. Checking just `muxg --version` therefore cannot establish the daemon's
97 build mode. Use the build graph's artifact paths for automated stress tests.
98
73 For a close/reopen demo, stop the recorder before closing the first window and 99 For a close/reopen demo, stop the recorder before closing the first window and
74 resume on the new owned window after launch. Join those continuous clips and 100 resume on the new owned window after launch. Join those continuous clips and
75 disclose the restart cuts; do not capture the intervening desktop. 101 disclose the restart cuts; do not capture the intervening desktop.
test/native_stress.py
Old New
@@ -0,0 +1,156 @@
1 #!/usr/bin/env python3
2 """Raw cat /dev/random with same-daemon and separate-daemon GUI neighbours.
3
4 Run through `make native-stress`: its build graph supplies both release binaries.
5 Linux /proc provides independent producer identity and daemon read-progress
6 oracles. Producer write accounting misses GNU cat's splice path.
7 """
8 import hashlib
9 import json
10 import os
11 from pathlib import Path
12 import shlex
13 import signal
14 import subprocess
15 import sys
16 import time
17
18 from native_lifecycle import LifecycleRig, start_persistent
19 from native_resize import by_id
20 from native_tiling import eventually, require
21
22
23 class StressRig(LifecycleRig):
24 # Sample completed painted state more closely than the general integration
25 # harness's 40 ms polling. These remain observed upper bounds, not GPU timings.
26 def artifact(self, command, suffix):
27 self.serial += 1
28 path = self.root / (str(self.serial) + suffix)
29 self.send(command + ':' + str(path))
30 deadline = time.monotonic() + 5
31 while not path.exists() and self.gui.poll() is None and time.monotonic() < deadline:
32 time.sleep(.005)
33 require(path.exists(), 'GUI did not produce ' + command)
34 return path
35
36 def painted(self, pane, marker):
37 deadline = time.monotonic() + 2
38 while time.monotonic() < deadline:
39 state = self.state()
40 if marker in by_id(state)[pane]['painted_text']:
41 return state
42 time.sleep(.005)
43 raise AssertionError('neighbour did not paint ' + marker)
44
45
46 def main():
47 require(sys.platform == 'linux', 'native-stress currently requires Linux /proc')
48 r = StressRig(*sys.argv[1:3])
49 producer = None
50 report = {'artifacts': str(r.root), 'samples': []}
51 report_path = r.root / 'stress-result.json'
52
53 def cat_alive():
54 try:
55 return Path(f'/proc/{producer}/cmdline').read_bytes().startswith(b'cat\0/dev/random\0')
56 except FileNotFoundError:
57 return False
58
59 def read_count(pid):
60 fields = dict(line.split(': ', 1) for line in Path(f'/proc/{pid}/io').read_text().splitlines())
61 return int(fields['rchar'])
62
63 def probe(pane, sock, session, iteration):
64 r.focus(pane)
65 marker = f'RAW-NEIGHBOUR-{iteration}-{pane}'
66 before = time.monotonic()
67 # The contiguous marker is absent from shell input, so echo alone cannot
68 # satisfy the check before the shell executes our command.
69 r.shell("printf '\\033[32mRAW-NEIGHBOUR-%s\\033[0m\\n' " + shlex.quote(f'{iteration}-{pane}'))
70 state = r.painted(pane, marker)
71 delay_ms = (time.monotonic() - before) * 1000
72 report['samples'].append({'pane': pane, 'session': session,
73 'observed_input_to_painted_ms': round(delay_ms, 1)})
74 require(delay_ms < 250, f'neighbour exceeded 250 ms: {delay_ms:.1f} ms')
75 require(all(p['phase'] == 'attached' for p in state['panes']), 'pane lost attachment')
76 # Independently check the authoritative grid outside the latency interval.
77 r.wait_marker(sock, session, marker)
78
79 try:
80 version = subprocess.check_output([r.muxg, '--version'], text=True)
81 require('ReleaseSafe' in version or 'ReleaseFast' in version, 'stress requires release binaries')
82 report['binaries'] = {path: hashlib.sha256(Path(path).read_bytes()).hexdigest()
83 for path in (r.mux, r.muxg)}
84 refs = start_persistent(r)
85 ids = list(refs)
86 flooded, separate, shared = ids
87 report['panes'] = {'flooded': flooded, 'separate_daemon': separate, 'shared_daemon': shared}
88 daemon_pid = next(proc.pid for sock, proc in r.daemons if sock == refs[flooded][0])
89 r.focus(flooded)
90 pid_path = r.root / 'random.pid'
91 r.shell('sh -c ' + shlex.quote('printf "%s\\n" $$ > ' + shlex.quote(str(pid_path)) +
92 '; exec cat /dev/random'))
93 eventually(lambda: pid_path.exists() and pid_path.stat().st_size, 'cat did not start')
94 producer = int(pid_path.read_text())
95 eventually(cat_alive, 'owned producer did not exec cat /dev/random')
96 started = time.monotonic()
97 baseline = read_count(daemon_pid)
98 last_count = baseline
99 round_no = 0
100 restarted = False
101 while time.monotonic() - started < 30:
102 require(cat_alive(), 'raw producer exited early')
103 for pane in (separate, shared):
104 probe(pane, *refs[pane], round_no)
105 if not restarted and time.monotonic() - started >= 10:
106 r.chord('p')
107 r.wait_state(lambda s: s['recovery'] is not None)
108 r.key('escape')
109 r.wait_state(lambda s: s['recovery'] is None)
110 r.drag('beside', dx=-r.state()['cell_w'] * 3)
111 r.kernel_sizes()
112 report['before_reopen_frames'], report['before_reopen_p99_us'] = r.frames()
113 r.quit()
114 r.launch_gui([], 'gui-raw-restored')
115 r.kernel_sizes()
116 report['reopened_during_flood'] = True
117 restarted = True
118 time.sleep(.15)
119 count = read_count(daemon_pid)
120 require(count - last_count > 16 * 1024, 'raw flood made insufficient progress this round')
121 last_count = count
122 round_no += 1
123 report['active_seconds'] = round(time.monotonic() - started, 1)
124 report['daemon_bytes_read'] = last_count - baseline
125 report['final_frames'], report['final_p99_us'] = r.frames()
126 require(restarted and cat_alive(), 'reopen did not run during active raw output')
127 require(report['daemon_bytes_read'] > 256 * 1024, 'raw flood made insufficient total progress')
128 require(report['before_reopen_frames'] > 0 and report['final_frames'] > 0, 'no frames during flood')
129 require(max(report['before_reopen_p99_us'], report['final_p99_us']) < 20000,
130 'raw flood exceeded 20 ms frame p99 budget')
131 os.kill(producer, signal.SIGTERM)
132 producer = None
133 for pane in (separate, shared):
134 probe(pane, *refs[pane], 'after-stop')
135 r.quit()
136 r.assert_cli_untouched()
137 report['passed'] = True
138 print(json.dumps(report, indent=2), flush=True)
139 print('native stress OK; report:', report_path, flush=True)
140 except BaseException:
141 r.failure_artifacts()
142 raise
143 finally:
144 try:
145 if producer is not None and cat_alive():
146 os.kill(producer, signal.SIGTERM)
147 except ProcessLookupError:
148 pass
149 try:
150 r.close()
151 finally:
152 report_path.write_text(json.dumps(report, indent=2) + '\n')
153
154
155 if __name__ == '__main__':
156 main()