a73x

cbd7007f

fix(agent.sh): type-visible JSON asserts; setup defects fail loudly; composed-narrative pin

a73x   2026-08-13 19:55

Commit message
fix(agent.sh): type-visible JSON asserts; setup defects fail loudly; composed-narrative pin

Three doors this suite could have passed through while green, found in review.
Each is closed and each closure is measured against the version that had it.

`jget` printed `str(v)`, which spells the number 0, the string "0", the
boolean true and the string "true" identically. The header claimed a renamed
field or a retyped one would be caught; only half of that was true. It now
prints `json.dumps(v)`, so a string carries its quotes into the comparison and
the type is part of the assertion — at the cost of ~15 call sites spelling
their string expectations as '"marks"'. Measured with a muxa wrapper that
emits `"exit_code":"0"`: the committed version reported 9 passed / exit 0, this
one reports three failures and exit 1, the first reading
`exit_code="0", want 0`.

Setup failures skipped when they should have failed. Everything after the
environment probes — keygen refusing, the key not landing where keygen said,
the QUIC daemon not coming up, the relay not binding — is a DEFECT, and a
skip made it invisible. Measured with a muxd wrapper whose `keygen` exits 1:
the committed version reported `4 passed, 0 failed, 5 skipped` and exit 0,
which is CI saying yes to a `muxd --quic` that no longer works. It now reports
`4 passed, 5 failed` and exit 1. python3 and `timeout` move to the top of the
file as hard prerequisites, since every scenario needs them and nine SKIP
lines would be this suite reporting success for work it did not do; what is
left at the QUIC gate has no legitimate excuse, so the gate has one outcome.
The TUI block splits the same way but keeps both: no vi and no less on the box
is the environment's business and skips, a TUI that is present whose daemon
did not answer is a defect and fails.

Scenario 6 accepted `"connection lost"*`, which is a prefix of all three
endings muxa has for a lost connection. Only one of them is honest in that
setup — this client redialled and the redial could not complete — so the glob
is now the composed narrative. Measured with a muxa wrapper that rewrites the
detail to a bare `connection lost`, the regression where nothing tried to
redial: the committed version passed it, this one fails with the expectation
quoted.

Five minors alongside. cleanup's comment described a `|| true` that is not
there and missed the load-bearing part, which is the `return 0` keeping a
cleanup on a PASSING run from failing the suite. The TUI wrapper single-quotes
$TMP in both the HOME line and the exec, so a TMPDIR with a space in it cannot
split either. D_TUI keeps its pid when the daemon is up but unresponsive
instead of being blanked, so cleanup can still reach it — a separate flag
carries "usable". And scenario 8's coupling to 5 and 6 is named where it
lives: it is the third scenario to drive that one bash session, scenario 6
leaves a `sleep 20` running there that killing the client does not kill, and
what covers it is the 20s scenario 7 spends in between. Its bound goes to 12s
so a fast failure in 7 narrows the margin without reaching it.

Two consecutive runs green at 51s each, unchanged from before. No stray
daemons or relays, no tmpdirs left, and ~/.config/mux/key still carries its
mtime from four days ago after ten runs of this suite — hermetic, and now
checked by mtime rather than by an `ls | head -3` that had been hiding the
file behind the directory entries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
test/agent.sh
Old New
@@ -19,6 +19,17 @@ MUXA="${2:-$ROOT/zig-out/bin/muxa}"
19 [ -x "$MUXD" ] || { echo "agent FAIL: no muxd at $MUXD (run: zig build)"; exit 1; } 19 [ -x "$MUXD" ] || { echo "agent FAIL: no muxd at $MUXD (run: zig build)"; exit 1; }
20 [ -x "$MUXA" ] || { echo "agent FAIL: no muxa at $MUXA (run: zig build)"; exit 1; } 20 [ -x "$MUXA" ] || { echo "agent FAIL: no muxa at $MUXA (run: zig build)"; exit 1; }
21 21
22 # The two tools every scenario needs, checked here rather than per-scenario
23 # because a box without them cannot run ANY of this: python3 parses every
24 # assertion (see jget) and builds the relay, and `timeout` is what makes a hung
25 # muxa a failure instead of a wedged suite. Missing either is a refusal to run,
26 # not a skip — nine SKIP lines and exit 0 would be this suite reporting success
27 # for work it did not do.
28 for _tool in python3 timeout; do
29 command -v "$_tool" >/dev/null 2>&1 ||
30 { echo "agent FAIL: no $_tool — this suite cannot assert or bound anything without it"; exit 1; }
31 done
32
22 # One directory for everything this run writes: sockets, keys, captures, the 33 # One directory for everything this run writes: sockets, keys, captures, the
23 # relay, the TUI's HOME. Removed by the trap, so a failing run leaves nothing 34 # relay, the TUI's HOME. Removed by the trap, so a failing run leaves nothing
24 # behind but its output. 35 # behind but its output.
@@ -81,8 +92,11 @@ SINK_PID=""
81 CLI_PID="" 92 CLI_PID=""
82 93
83 cleanup() { 94 cleanup() {
84 # `|| true` on every kill: without `set -e` here it is belt and braces, but 95 # Every pid this run started, including the ones already dead: a kill that
85 # this trap also runs on INT, and a half-run trap leaves a live daemon. 96 # finds nothing is not a problem here, which is why the status of each one
97 # is discarded rather than tested. The `return 0` at the bottom is the load-
98 # bearing part — without it the trap would exit with the status of whatever
99 # ran last, and a cleanup that fired on a PASSING run could fail the suite.
86 for p in "$D_MARKS" "$D_TUI" "$D_SETTLE" "$D_TEAR" "$D_QUIET" \ 100 for p in "$D_MARKS" "$D_TUI" "$D_SETTLE" "$D_TEAR" "$D_QUIET" \
87 "$RELAY_PID" "$SINK_PID" "$CLI_PID"; do 101 "$RELAY_PID" "$SINK_PID" "$CLI_PID"; do
88 [ -n "$p" ] && kill "$p" 2>/dev/null 102 [ -n "$p" ] && kill "$p" 2>/dev/null
@@ -138,10 +152,17 @@ run_scenario() {
138 esac 152 esac
139 } 153 }
140 154
141 # One field out of a JSON object, printed flat: `null` for JSON null, `true` 155 # One field out of a JSON object, re-encoded as JSON: `null`, `true`, `0`, and
142 # /`false` for booleans (so an assertion reads as the wire spelling), and a 156 # `"marks"` WITH its quotes. The quotes are the point. muxa's contract is a
143 # loud sentinel for a missing key or a body that is not JSON at all. A muxa 157 # typed one — exit_code is a number, alt_screen a boolean, mechanism a string —
144 # that printed a stack trace fails here as `<unparseable>`, not as a mismatch. 158 # and a `str(v)` here would print all three the same way, so a daemon that
159 # started spelling exit_code as "0" or alt_screen as "true" would sail past
160 # every assertion below. Re-encoding makes the type part of the comparison, and
161 # the cost is that string expectations at the call sites carry their quotes too.
162 #
163 # The three sentinels cannot collide with any of that: a field whose value were
164 # literally the text `<missing>` re-encodes to `"<missing>"`, quotes and all. A
165 # muxa that printed a stack trace fails as `<unparseable>`, not as a mismatch.
145 jget() { 166 jget() {
146 python3 - "$1" "$2" <<'PY' 167 python3 - "$1" "$2" <<'PY'
147 import json, sys 168 import json, sys
@@ -153,13 +174,13 @@ if not isinstance(obj, dict):
153 print("<not-an-object>"); raise SystemExit(0) 174 print("<not-an-object>"); raise SystemExit(0)
154 if sys.argv[2] not in obj: 175 if sys.argv[2] not in obj:
155 print("<missing>"); raise SystemExit(0) 176 print("<missing>"); raise SystemExit(0)
156 v = obj[sys.argv[2]] 177 print(json.dumps(obj[sys.argv[2]]))
157 print("null" if v is None else ("true" if v is True else "false" if v is False else str(v)))
158 PY 178 PY
159 } 179 }
160 180
161 # want FILE FIELD VALUE — assert one field, naming the whole body on a miss so 181 # want FILE FIELD VALUE — assert one field, naming the whole body on a miss so
162 # a wrong answer is read in context rather than alone. 182 # a wrong answer is read in context rather than alone. VALUE is JSON: bare for
183 # null/true/false/numbers, quoted for strings.
163 want() { 184 want() {
164 _got=$(jget "$1" "$2") 185 _got=$(jget "$1" "$2")
165 [ "$_got" = "$3" ] && return 0 186 [ "$_got" = "$3" ] && return 0
@@ -218,13 +239,13 @@ scen_marks() {
218 timeout 20 "$MUXA" run --sock "$SOCK_MARKS" --timeout 8000 'true' >"$TMP/m1" 2>&1 239 timeout 20 "$MUXA" run --sock "$SOCK_MARKS" --timeout 8000 'true' >"$TMP/m1" 2>&1
219 _rc=$? 240 _rc=$?
220 [ "$_rc" -eq 0 ] || why "run 'true' exited $_rc [$(tr -d '\n' < "$TMP/m1")]" || return 1 241 [ "$_rc" -eq 0 ] || why "run 'true' exited $_rc [$(tr -d '\n' < "$TMP/m1")]" || return 1
221 want "$TMP/m1" reason returned || return 1 242 want "$TMP/m1" reason '"returned"' || return 1
222 want "$TMP/m1" mechanism marks || return 1 243 want "$TMP/m1" mechanism '"marks"' || return 1
223 want "$TMP/m1" exit_code 0 || return 1 244 want "$TMP/m1" exit_code 0 || return 1
224 245
225 timeout 20 "$MUXA" run --sock "$SOCK_MARKS" --timeout 8000 'false' >"$TMP/m2" 2>&1 246 timeout 20 "$MUXA" run --sock "$SOCK_MARKS" --timeout 8000 'false' >"$TMP/m2" 2>&1
226 want "$TMP/m2" reason returned || return 1 247 want "$TMP/m2" reason '"returned"' || return 1
227 want "$TMP/m2" mechanism marks || return 1 248 want "$TMP/m2" mechanism '"marks"' || return 1
228 # The command failed; muxa did not. A nonzero exit_code is an ANSWER, and 249 # The command failed; muxa did not. A nonzero exit_code is an ANSWER, and
229 # an agent that branches on muxa's own status must not see it as an error. 250 # an agent that branches on muxa's own status must not see it as an error.
230 want "$TMP/m2" exit_code 1 || return 1 251 want "$TMP/m2" exit_code 1 || return 1
@@ -233,13 +254,13 @@ scen_marks() {
233 # and one that is unique per run so a stale grid can never satisfy it. 254 # and one that is unique per run so a stale grid can never satisfy it.
234 _mark="out-$$" 255 _mark="out-$$"
235 timeout 20 "$MUXA" run --sock "$SOCK_MARKS" --timeout 8000 "echo $_mark" >"$TMP/m3" 2>&1 256 timeout 20 "$MUXA" run --sock "$SOCK_MARKS" --timeout 8000 "echo $_mark" >"$TMP/m3" 2>&1
236 want "$TMP/m3" reason returned || return 1 257 want "$TMP/m3" reason '"returned"' || return 1
237 want "$TMP/m3" mechanism marks || return 1 258 want "$TMP/m3" mechanism '"marks"' || return 1
238 want "$TMP/m3" exit_code 0 || return 1 259 want "$TMP/m3" exit_code 0 || return 1
239 # Exactly the output, not "contains": the span between the two marks is 260 # Exactly the output, not "contains": the span between the two marks is
240 # the command's transcript, and a prompt or an echoed command line leaking 261 # the command's transcript, and a prompt or an echoed command line leaking
241 # into it is the bug this equality is here to catch. 262 # into it is the bug this equality is here to catch.
242 want "$TMP/m3" output "$_mark" || return 1 263 want "$TMP/m3" output "\"$_mark\"" || return 1
243 264
244 "$MUXD" stop --sock "$SOCK_MARKS" >/dev/null 2>&1 265 "$MUXD" stop --sock "$SOCK_MARKS" >/dev/null 2>&1
245 D_MARKS="" 266 D_MARKS=""
@@ -253,34 +274,61 @@ run_scenario "marks: exit codes and output come back from a bash session" scen_m
253 # execs whatever path it is given, so a one-line wrapper carries the argument. 274 # execs whatever path it is given, so a one-line wrapper carries the argument.
254 # HOME points into the tmpdir for e2e.sh's $SHELL reason: ~/.lesskey and 275 # HOME points into the tmpdir for e2e.sh's $SHELL reason: ~/.lesskey and
255 # ~/.vimrc are arbitrary code on the session under test. 276 # ~/.vimrc are arbitrary code on the session under test.
256 TUI_KIND=none 277 #
257 TUI_CMD="" 278 # The outcome splits three ways, and the split is the point: no TUI on the box
279 # is the environment's business and skips, but a TUI that is here and whose
280 # daemon did not come up is a DEFECT and must fail. Blanking D_TUI for both
281 # would report a broken muxd as a skip — and would also lose the pid, orphaning
282 # a daemon that is merely unresponsive rather than dead.
283 TUI_BIN=""
258 TUI_QUIT="" 284 TUI_QUIT=""
285 TUI_SKIP=""
286 TUI_FAIL=""
287 TUI_OK=""
259 if command -v vi >/dev/null 2>&1; then 288 if command -v vi >/dev/null 2>&1; then
260 TUI_KIND=vi 289 TUI_BIN=$(command -v vi)
261 TUI_CMD="$(command -v vi) /etc/hostname"
262 TUI_QUIT=':q!\n' 290 TUI_QUIT=':q!\n'
263 elif command -v less >/dev/null 2>&1; then 291 elif command -v less >/dev/null 2>&1; then
264 TUI_KIND=less 292 TUI_BIN=$(command -v less)
265 TUI_CMD="$(command -v less) /etc/hostname"
266 TUI_QUIT='q' 293 TUI_QUIT='q'
267 fi 294 fi
268 if [ "$TUI_KIND" != none ]; then 295 if [ -z "$TUI_BIN" ]; then
296 TUI_SKIP="neither vi nor less on this box"
297 else
269 mkdir -p "$TMP/home" 298 mkdir -p "$TMP/home"
299 # Single-quoted in the generated script, both of them: $TMP contains $$ and
300 # is usually tame, but a TMPDIR with a space in it would otherwise split
301 # HOME in half and hand `exec` an argument it never meant to have.
270 { 302 {
271 echo '#!/bin/sh' 303 echo '#!/bin/sh'
272 echo "HOME=$TMP/home; export HOME" 304 echo "HOME='$TMP/home'; export HOME"
273 # less reads its own switches out of the environment; a developer with 305 # less reads its own switches out of the environment; a developer with
274 # -F exported would make the session exit before it was ever driven. 306 # -F exported would make the session exit before it was ever driven.
275 echo 'LESS=; export LESS' 307 echo 'LESS=; export LESS'
276 echo 'unset LESSOPEN LESSCLOSE' 308 echo 'unset LESSOPEN LESSCLOSE'
277 echo "exec $TUI_CMD" 309 echo "exec '$TUI_BIN' /etc/hostname"
278 } > "$TUISH" 310 } > "$TUISH"
279 chmod +x "$TUISH" 311 chmod +x "$TUISH"
312 # D_TUI keeps the pid whatever happens next, so cleanup can always reach a
313 # daemon that came up but never answered.
280 D_TUI=$(start_daemon "$TMP/tui.log" --sock "$SOCK_TUI" --shell "$TUISH") 314 D_TUI=$(start_daemon "$TMP/tui.log" --sock "$SOCK_TUI" --shell "$TUISH")
281 wait_ready "$SOCK_TUI" || D_TUI="" 315 if [ -z "$D_TUI" ]; then
316 TUI_FAIL="the TUI daemon printed no up-line [$(tr -d '\n' < "$TMP/tui.log")]"
317 elif wait_ready "$SOCK_TUI"; then
318 TUI_OK=1
319 else
320 TUI_FAIL="the TUI daemon (pid $D_TUI) never answered on $SOCK_TUI"
321 fi
282 fi 322 fi
283 323
324 # The gate both TUI scenarios open with: 1 for a defect, 2 for a box that has
325 # no TUI to drive. `return $?` propagates whichever it was.
326 tui_gate() {
327 [ -z "$TUI_FAIL" ] || { WHY="$TUI_FAIL"; return 1; }
328 [ -n "$TUI_OK" ] || { WHY="$TUI_SKIP"; return 2; }
329 return 0
330 }
331
284 # --- 2: the alt-screen guard ------------------------------------------------ 332 # --- 2: the alt-screen guard ------------------------------------------------
285 # Ordered before the drive below because that one ENDS this session. Nothing 333 # Ordered before the drive below because that one ENDS this session. Nothing
286 # on a full-screen program's grid can mean "the command returned" — there are 334 # on a full-screen program's grid can mean "the command returned" — there are
@@ -289,10 +337,10 @@ fi
289 # worst failure in the surface: an agent would read an exit code that no 337 # worst failure in the surface: an agent would read an exit code that no
290 # command ever produced. 338 # command ever produced.
291 scen_altguard() { 339 scen_altguard() {
292 [ -n "$D_TUI" ] || { WHY="no TUI to drive (vi/less both absent or the daemon died)"; return 2; } 340 tui_gate || return $?
293 timeout 20 "$MUXA" run --sock "$SOCK_TUI" --timeout 1500 'true' >"$TMP/g1" 2>&1 341 timeout 20 "$MUXA" run --sock "$SOCK_TUI" --timeout 1500 'true' >"$TMP/g1" 2>&1
294 _rc=$? 342 _rc=$?
295 want "$TMP/g1" reason timeout || return 1 343 want "$TMP/g1" reason '"timeout"' || return 1
296 # Exit 3 is the whole point of having a code for it: `returned` and 344 # Exit 3 is the whole point of having a code for it: `returned` and
297 # `settled` are answers and exit 0, a timeout is a question still open. 345 # `settled` are answers and exit 0, a timeout is a question still open.
298 [ "$_rc" -eq 3 ] || why "exit $_rc, want 3 [$(tr -d '\n' < "$TMP/g1")]" || return 1 346 [ "$_rc" -eq 3 ] || why "exit $_rc, want 3 [$(tr -d '\n' < "$TMP/g1")]" || return 1
@@ -302,13 +350,13 @@ run_scenario "alt-screen: run times out rather than fabricating a return" scen_a
302 350
303 # --- 3: the ephemeral TUI, driven and quit --------------------------------- 351 # --- 3: the ephemeral TUI, driven and quit ---------------------------------
304 scen_tui() { 352 scen_tui() {
305 [ -n "$D_TUI" ] || { WHY="no TUI to drive (vi/less both absent or the daemon died)"; return 2; } 353 tui_gate || return $?
306 timeout 20 "$MUXA" status --sock "$SOCK_TUI" --timeout 5000 >"$TMP/t1" 2>&1 354 timeout 20 "$MUXA" status --sock "$SOCK_TUI" --timeout 5000 >"$TMP/t1" 2>&1
307 want "$TMP/t1" alt_screen true || return 1 355 want "$TMP/t1" alt_screen true || return 1
308 # Marks are a shell's doing. A program that is not a shell cannot have 356 # Marks are a shell's doing. A program that is not a shell cannot have
309 # them, and claiming otherwise is what scenario 2 would then read. 357 # them, and claiming otherwise is what scenario 2 would then read.
310 _mech=$(jget "$TMP/t1" mechanism) 358 _mech=$(jget "$TMP/t1" mechanism)
311 [ "$_mech" != marks ] || why "mechanism=marks on a TUI that no shell started" || return 1 359 [ "$_mech" != '"marks"' ] || why "mechanism=marks on a TUI that no shell started" || return 1
312 360
313 timeout 10 "$MUXA" send --sock "$SOCK_TUI" -- "$TUI_QUIT" >"$TMP/t2" 2>&1 361 timeout 10 "$MUXA" send --sock "$SOCK_TUI" -- "$TUI_QUIT" >"$TMP/t2" 2>&1
314 want "$TMP/t2" sent true || return 1 362 want "$TMP/t2" sent true || return 1
@@ -360,8 +408,8 @@ scen_settle() {
360 [ "$_rc" -eq 0 ] || why "run exited $_rc [$(tr -d '\n' < "$TMP/s1")]" || return 1 408 [ "$_rc" -eq 0 ] || why "run exited $_rc [$(tr -d '\n' < "$TMP/s1")]" || return 1
361 _reason=$(jget "$TMP/s1" reason) 409 _reason=$(jget "$TMP/s1" reason)
362 case "$_reason" in 410 case "$_reason" in
363 settled|returned) ;; 411 '"settled"'|'"returned"') ;;
364 *) why "reason=$_reason, want settled or returned [$(tr -d '\n' < "$TMP/s1")]" || return 1 ;; 412 *) why "reason=$_reason, want \"settled\" or \"returned\" [$(tr -d '\n' < "$TMP/s1")]" || return 1 ;;
365 esac 413 esac
366 # No marks means no exit code, and muxa says so with a null rather than a 414 # No marks means no exit code, and muxa says so with a null rather than a
367 # zero — an agent must never read "it worked" out of a mechanism that 415 # zero — an agent must never read "it worked" out of a mechanism that
@@ -430,18 +478,23 @@ while True:
430 front.sendto(data, addr) 478 front.sendto(data, addr)
431 PY 479 PY
432 480
433 QUIC_WHY="" 481 # Every way this setup can go wrong is a DEFECT, and every one of them fails
434 if ! command -v python3 >/dev/null 2>&1; then 482 # rather than skips. There is no environmental escape hatch left at this point:
435 QUIC_WHY="no python3 to run the UDP relay, and the tear cannot be made without one" 483 # python3 and `timeout` were made hard prerequisites at the top of the file, so
436 elif ! command -v timeout >/dev/null 2>&1; then 484 # what remains — keygen refusing, the key not landing where keygen said it did,
437 QUIC_WHY="no coreutils timeout to bound the backgrounded clients" 485 # the QUIC daemon not coming up, the relay not binding — is either muxd
438 elif ! "$MUXD" keygen >"$TMP/keygen.log" 2>&1; then 486 # misbehaving or this box handing out a port twice. A skip here would be the
439 QUIC_WHY="muxd keygen failed [$(tr -d '\n' < "$TMP/keygen.log")]" 487 # one door in the suite wide enough for a real regression to walk through
440 else 488 # wearing green: `muxd --quic` breaking outright would have reported "4 passed,
441 cp "$XDG_CONFIG_HOME/mux/key" "$KEY" 2>/dev/null || QUIC_WHY="keygen wrote no key where it said it did" 489 # 5 skipped" and exited 0, which is CI saying yes to a broken binary.
490 QUIC_FAIL=""
491 if ! "$MUXD" keygen >"$TMP/keygen.log" 2>&1; then
492 QUIC_FAIL="muxd keygen failed [$(tr -d '\n' < "$TMP/keygen.log")]"
493 elif ! cp "$XDG_CONFIG_HOME/mux/key" "$KEY" 2>/dev/null; then
494 QUIC_FAIL="keygen wrote no key where it said it did [$(tr -d '\n' < "$TMP/keygen.log")]"
442 fi 495 fi
443 496
444 if [ -z "$QUIC_WHY" ]; then 497 if [ -z "$QUIC_FAIL" ]; then
445 # The reduced idle is the schedule the tear scenarios wait for. It is a 498 # The reduced idle is the schedule the tear scenarios wait for. It is a
446 # transport parameter, so the NEGOTIATED value is the min of the two ends 499 # transport parameter, so the NEGOTIATED value is the min of the two ends
447 # and this daemon's 4s governs both — muxa has no idle flag of its own, 500 # and this daemon's 4s governs both — muxa has no idle flag of its own,
@@ -449,14 +502,25 @@ if [ -z "$QUIC_WHY" ]; then
449 # runtime. The quiet-await scenario below deliberately does not use it. 502 # runtime. The quiet-await scenario below deliberately does not use it.
450 D_TEAR=$(start_daemon "$TMP/tear.log" --sock "$SOCK_TEAR" --shell /bin/bash \ 503 D_TEAR=$(start_daemon "$TMP/tear.log" --sock "$SOCK_TEAR" --shell /bin/bash \
451 --quic "127.0.0.1:$PORT_TEAR" --key "$KEY" --quic-idle-ms 4000) 504 --quic "127.0.0.1:$PORT_TEAR" --key "$KEY" --quic-idle-ms 4000)
452 [ -n "$D_TEAR" ] && wait_ready "$SOCK_TEAR" || QUIC_WHY="the QUIC daemon never came up [$(cat "$TMP/tear.log")]" 505 if [ -z "$D_TEAR" ]; then
506 QUIC_FAIL="the QUIC daemon printed no up-line [$(tr -d '\n' < "$TMP/tear.log")]"
507 elif ! wait_ready "$SOCK_TEAR"; then
508 QUIC_FAIL="the QUIC daemon (pid $D_TEAR) never answered on $SOCK_TEAR"
509 fi
453 fi 510 fi
454 if [ -z "$QUIC_WHY" ]; then 511 if [ -z "$QUIC_FAIL" ]; then
455 python3 "$RELAY" "$PORT_RELAY" "$PORT_TEAR" "$CTL_FLOW" "$CTL_ALL" >"$RELAY_LOG" 2>&1 & 512 python3 "$RELAY" "$PORT_RELAY" "$PORT_TEAR" "$CTL_FLOW" "$CTL_ALL" >"$RELAY_LOG" 2>&1 &
456 RELAY_PID=$! 513 RELAY_PID=$!
457 wait_for "$RELAY_LOG" "relay up" 5 || QUIC_WHY="the relay never bound $PORT_RELAY [$(cat "$RELAY_LOG")]" 514 wait_for "$RELAY_LOG" "relay up" 5 || QUIC_FAIL="the relay never bound $PORT_RELAY [$(tr -d '\n' < "$RELAY_LOG")]"
458 fi 515 fi
459 516
517 # The gate every QUIC scenario opens with. One outcome only — there is nothing
518 # left here that a box could legitimately be excused from.
519 quic_gate() {
520 [ -z "$QUIC_FAIL" ] || { WHY="$QUIC_FAIL"; return 1; }
521 return 0
522 }
523
460 # --- 5: a tear mid-await heals, and the command still ran exactly once ------ 524 # --- 5: a tear mid-await heals, and the command still ran exactly once ------
461 # The two claims an agent's whole reconnect story rests on. The reply that 525 # The two claims an agent's whole reconnect story rests on. The reply that
462 # arrives after the heal carries the ORIGINAL command's return — muxa re-issued 526 # arrives after the heal carries the ORIGINAL command's return — muxa re-issued
@@ -466,7 +530,7 @@ fi
466 # reconnect would have run it twice, and on anything but `sleep` that is a 530 # reconnect would have run it twice, and on anything but `sleep` that is a
467 # second deploy, not a second read. 531 # second deploy, not a second read.
468 scen_tear_heal() { 532 scen_tear_heal() {
469 [ -z "$QUIC_WHY" ] || { WHY="$QUIC_WHY"; return 2; } 533 quic_gate || return $?
470 _tally="$TMP/tally" 534 _tally="$TMP/tally"
471 rm -f "$_tally" 535 rm -f "$_tally"
472 # Counted in the FILESYSTEM, not in the grid: a grid count would also see 536 # Counted in the FILESYSTEM, not in the grid: a grid count would also see
@@ -485,8 +549,8 @@ scen_tear_heal() {
485 _rc=$? 549 _rc=$?
486 CLI_PID="" 550 CLI_PID=""
487 [ "$_rc" -eq 0 ] || why "run exited $_rc after the heal [$(tr -d '\n' < "$TMP/q1")]" || return 1 551 [ "$_rc" -eq 0 ] || why "run exited $_rc after the heal [$(tr -d '\n' < "$TMP/q1")]" || return 1
488 want "$TMP/q1" reason returned || return 1 552 want "$TMP/q1" reason '"returned"' || return 1
489 want "$TMP/q1" mechanism marks || return 1 553 want "$TMP/q1" mechanism '"marks"' || return 1
490 want "$TMP/q1" exit_code 0 || return 1 554 want "$TMP/q1" exit_code 0 || return 1
491 555
492 # A second flow through the relay is the reconnect, observed from outside 556 # A second flow through the relay is the reconnect, observed from outside
@@ -508,7 +572,7 @@ run_scenario "quic: a tear mid-await heals, and the command ran exactly once" sc
508 # `Timeout` goes and checks its own command; an agent told `connection lost; 572 # `Timeout` goes and checks its own command; an agent told `connection lost;
509 # reconnect failed: Timeout` knows to check the network. 573 # reconnect failed: Timeout` knows to check the network.
510 scen_tear_fatal() { 574 scen_tear_fatal() {
511 [ -z "$QUIC_WHY" ] || { WHY="$QUIC_WHY"; return 2; } 575 quic_gate || return $?
512 # A hang is the failure mode here, so the deadline is asserted twice: the 576 # A hang is the failure mode here, so the deadline is asserted twice: the
513 # outer `timeout` makes one impossible to sit through, and the wall clock 577 # outer `timeout` makes one impossible to sit through, and the wall clock
514 # below makes one impossible to pass with. 578 # below makes one impossible to pass with.
@@ -535,10 +599,16 @@ scen_tear_fatal() {
535 # --timeout plus muxa's 2s grace over the daemon's own window, plus room 599 # --timeout plus muxa's 2s grace over the daemon's own window, plus room
536 # for the box. Anything near 30s means the deadline was not honoured. 600 # for the box. Anything near 30s means the deadline was not honoured.
537 [ "$_spent" -lt 20000 ] || why "took ${_spent}ms for a 12000ms timeout" || return 1 601 [ "$_spent" -lt 20000 ] || why "took ${_spent}ms for a 12000ms timeout" || return 1
602 # The COMPOSED narrative, not merely a prefix of it. muxa has three endings
603 # for a lost connection and only one of them is honest here: this client
604 # redialled and the redial could not complete. A bare `connection lost`
605 # means nothing tried to redial, and `connection lost again, after the one
606 # reconnect` means the redial was already spent — both would be regressions
607 # in this setup, and a `"connection lost"*` glob would pass for either.
538 _detail=$(jget "$TMP/q2" detail) 608 _detail=$(jget "$TMP/q2" detail)
539 case "$_detail" in 609 case "$_detail" in
540 "connection lost"*) ;; 610 '"connection lost; reconnect failed: '*) ;;
541 *) why "detail=$_detail, want 'connection lost...' [$(tr -d '\n' < "$TMP/q2")]" || return 1 ;; 611 *) why "detail=$_detail, want '\"connection lost; reconnect failed: ...' [$(tr -d '\n' < "$TMP/q2")]" || return 1 ;;
542 esac 612 esac
543 return 0 613 return 0
544 } 614 }
@@ -551,7 +621,7 @@ run_scenario "quic: a tear with no path back fails with the whole story" scen_te
551 # connection — an agent would go looking for a network fault that never 621 # connection — an agent would go looking for a network fault that never
552 # happened, and the honest answer (still running) would have been one field. 622 # happened, and the honest answer (still running) would have been one field.
553 scen_keepalive() { 623 scen_keepalive() {
554 [ -z "$QUIC_WHY" ] || { WHY="$QUIC_WHY"; return 2; } 624 quic_gate || return $?
555 D_QUIET=$(start_daemon "$TMP/quiet.log" --sock "$SOCK_QUIET" --shell /bin/bash \ 625 D_QUIET=$(start_daemon "$TMP/quiet.log" --sock "$SOCK_QUIET" --shell /bin/bash \
556 --quic "127.0.0.1:$PORT_QUIET" --key "$KEY") 626 --quic "127.0.0.1:$PORT_QUIET" --key "$KEY")
557 [ -n "$D_QUIET" ] || why "daemon never printed an up-line [$(cat "$TMP/quiet.log")]" || return 1 627 [ -n "$D_QUIET" ] || why "daemon never printed an up-line [$(cat "$TMP/quiet.log")]" || return 1
@@ -560,7 +630,7 @@ scen_keepalive() {
560 timeout 40 "$MUXA" await --quic "127.0.0.1:$PORT_QUIET" --key "$KEY" --timeout 20000 >"$TMP/q3" 2>&1 630 timeout 40 "$MUXA" await --quic "127.0.0.1:$PORT_QUIET" --key "$KEY" --timeout 20000 >"$TMP/q3" 2>&1
561 _rc=$? 631 _rc=$?
562 [ "$_rc" -eq 3 ] || why "await exited $_rc, want 3 [$(tr -d '\n' < "$TMP/q3")]" || return 1 632 [ "$_rc" -eq 3 ] || why "await exited $_rc, want 3 [$(tr -d '\n' < "$TMP/q3")]" || return 1
563 want "$TMP/q3" reason timeout || return 1 633 want "$TMP/q3" reason '"timeout"' || return 1
564 # The number is the assertion: 15000 would be the idle timeout wearing a 634 # The number is the assertion: 15000 would be the idle timeout wearing a
565 # timeout's clothes, and only a duration past it proves the keepalives ran. 635 # timeout's clothes, and only a duration past it proves the keepalives ran.
566 _dur=$(jget "$TMP/q3" duration_ms) 636 _dur=$(jget "$TMP/q3" duration_ms)
@@ -578,14 +648,27 @@ run_scenario "quic: a quiet 20s await outlives the 15s idle timeout" scen_keepal
578 # shell exited 5; the connection then closed BECAUSE it did. Reporting the 648 # shell exited 5; the connection then closed BECAUSE it did. Reporting the
579 # close is reporting the consequence and losing the cause, and the exit code 649 # close is reporting the consequence and losing the cause, and the exit code
580 # is the one thing the agent came for. 650 # is the one thing the agent came for.
651 #
652 # Coupling worth naming: this dials the tear daemon DIRECTLY, but it is the
653 # third scenario to drive that one bash session — 5 and 6 reached it through
654 # the relay. Scenario 6 leaves a `sleep 20` running there whether it passes or
655 # fails: killing the client does not kill what the session was already typed.
656 # What covers it is scenario 7, which spends 20s of its own in between, so the
657 # sleep is long finished before `exit 5` is ever sent. If 7 itself fails fast
658 # that margin narrows, hence the 12s bound rather than a snug one — the queued
659 # `exit 5` still lands, just late. Never a hang either way: the outer `timeout`
660 # is the backstop.
661 #
662 # Not given its own daemon because the session's death IS the assertion — this
663 # scenario destroys what it runs on, so it goes last among the three regardless.
581 scen_session_exit() { 664 scen_session_exit() {
582 [ -z "$QUIC_WHY" ] || { WHY="$QUIC_WHY"; return 2; } 665 quic_gate || return $?
583 timeout 20 "$MUXA" run --quic "127.0.0.1:$PORT_TEAR" --key "$KEY" --timeout 8000 \ 666 timeout 30 "$MUXA" run --quic "127.0.0.1:$PORT_TEAR" --key "$KEY" --timeout 12000 \
584 'exit 5' >"$TMP/q4" 2>&1 667 'exit 5' >"$TMP/q4" 2>&1
585 _rc=$? 668 _rc=$?
586 # An ANSWER, not a failure: the command is over and this is how. 669 # An ANSWER, not a failure: the command is over and this is how.
587 [ "$_rc" -eq 0 ] || why "run exited $_rc [$(tr -d '\n' < "$TMP/q4")]" || return 1 670 [ "$_rc" -eq 0 ] || why "run exited $_rc [$(tr -d '\n' < "$TMP/q4")]" || return 1
588 want "$TMP/q4" reason session_ended || return 1 671 want "$TMP/q4" reason '"session_ended"' || return 1
589 want "$TMP/q4" exit_code 5 || return 1 672 want "$TMP/q4" exit_code 5 || return 1
590 return 0 673 return 0
591 } 674 }
@@ -600,7 +683,7 @@ D_TEAR=""
600 # and never replies makes the HANDSHAKE hang, and the only thing that can end 683 # and never replies makes the HANDSHAKE hang, and the only thing that can end
601 # it is muxa's own clock. 684 # it is muxa's own clock.
602 scen_blackhole() { 685 scen_blackhole() {
603 [ -z "$QUIC_WHY" ] || { WHY="$QUIC_WHY"; return 2; } 686 quic_gate || return $?
604 # The relay, blackholing from birth: its control file exists before it 687 # The relay, blackholing from birth: its control file exists before it
605 # starts, so it swallows the first packet it ever sees. 688 # starts, so it swallows the first packet it ever sees.
606 : > "$TMP/sink.all" 689 : > "$TMP/sink.all"