a73x

af1908af

test: e2e — prediction confirmed, suppressed, demoted, and flushed, by counter

a73x   2026-08-08 16:31

Commit message
test: e2e — prediction confirmed, suppressed, demoted, and flushed, by counter

Five scenarios, all asserting counters, because the screen cannot tell a
predicted glyph from an echoed one — that is what the feature is for — so a
rendering test would pass just as happily with prediction switched off.

The session shell is /bin/cat, not /bin/sh: an interactive bash prompt runs
at icanon=0/echo=0 because readline echoes for itself, which is the adaptive
tier and not the always-predict one. cat is a genuinely canonical reader and
the only way to reach `.always` here.

Line mode types one character into a 600ms round trip and snapshots the
client's output a quarter second later: the underlined glyph is there and
the pty's own echo is not, which is the milestone's claim as an effect
rather than a number. The burst types five characters at 200ms into that
same round trip and asserts all five confirm with nothing contradicted —
under the old judgment rule every character after the first was refuted by a
frame built before it was typed. The password scenario runs a session that
is canonical with echo off from its first instruction, and asserts nothing
was even attempted, that the secret appears nowhere in the emitted bytes,
and that the shell nonetheless received all seven characters. Raw mode drives
the rawmode helper through promotion and then a swallowed keystroke, pinning
every counter and the identity. The reconnect tears the transport down with a
prediction outstanding.

Each scenario was checked against a mutation of the thing it exists to
protect, and two of them did not fail on the first attempt:

The password scenario passed with echo-off canonical mapped to the
always-predict tier, because `printf hunter2` is ONE write and reached the
client as a multi-byte chunk — refused for being multi-byte long before the
tier was consulted. It now types one character per write, and the mutation
fails it with made=7.

The reconnect scenario passed with the reconnect's own flush deleted,
because the resync that follows also flushes and the outcome is identical
either way. It now asserts expired=0: with the flush the outstanding
prediction is abandoned by it, and without, it survives to be caught by the
expiry bound a second later. The mutation fails it with expired=1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
build.zig
Old New
@@ -248,6 +248,11 @@ pub fn build(b: *std.Build) void {
248 const e2e = b.addSystemCommand(&.{"test/e2e.sh"}); 248 const e2e = b.addSystemCommand(&.{"test/e2e.sh"});
249 e2e.addArtifactArg(exe); 249 e2e.addArtifactArg(exe);
250 e2e.addArtifactArg(mux_exe); 250 e2e.addArtifactArg(mux_exe);
251 // The prediction scenarios need a deterministic editor and a slow path;
252 // passed as artifacts so the suite runs against the binaries this build
253 // just produced rather than whatever is installed.
254 e2e.addArtifactArg(rawmode_exe);
255 e2e.addArtifactArg(delaypipe_exe);
251 const e2e_step = b.step("e2e", "Run end-to-end test"); 256 const e2e_step = b.step("e2e", "Run end-to-end test");
252 e2e_step.dependOn(&e2e.step); 257 e2e_step.dependOn(&e2e.step);
253 258
test/e2e.sh
Old New
@@ -4,6 +4,10 @@
4 set -eu 4 set -eu
5 MUXD="$1" 5 MUXD="$1"
6 MUX="$2" 6 MUX="$2"
7 # M9 prediction helpers: a deterministic stand-in for an editor, and a pipe
8 # that makes a slow round trip without netem or root.
9 RAWMODE="$3"
10 DELAYPIPE="$4"
7 SOCK="${TMPDIR:-/tmp}/muxd-e2e-$$.sock" 11 SOCK="${TMPDIR:-/tmp}/muxd-e2e-$$.sock"
8 OUT="${TMPDIR:-/tmp}/mux-e2e-out-$$" 12 OUT="${TMPDIR:-/tmp}/mux-e2e-out-$$"
9 # Second daemon, used only by the M7 abort scenario; declared here so the 13 # Second daemon, used only by the M7 abort scenario; declared here so the
@@ -22,6 +26,37 @@ D4PID=""
22 # A port out of the way of the ephemeral range, made per-run so two suites can 26 # A port out of the way of the ephemeral range, made per-run so two suites can
23 # overlap. Collisions surface as a loud bind failure, never as a silent pass. 27 # overlap. Collisions surface as a loud bind failure, never as a silent pass.
24 QPORT=$(( 21000 + ($$ % 4000) )) 28 QPORT=$(( 21000 + ($$ % 4000) ))
29 # M9 prediction: each scenario needs a session whose LINE DISCIPLINE it
30 # controls, so they cannot share the long-lived /bin/sh daemon.
31 SOCK5="${TMPDIR:-/tmp}/muxd-e2e-pred-$$.sock"
32 D5PID=""
33 SOCK6="${TMPDIR:-/tmp}/muxd-e2e-pw-$$.sock"
34 D6PID=""
35 SOCK7="${TMPDIR:-/tmp}/muxd-e2e-raw-$$.sock"
36 D7PID=""
37 PWSH="${TMPDIR:-/tmp}/mux-e2e-pw-$$.sh"
38
39 # One counter out of a MUX_PREDICT_STATS line. The client prints exactly one
40 # such line on exit; every field is a key=value pair, so a rename or reorder
41 # in the client shows up here as an empty read rather than a wrong number.
42 predict_stat() {
43 sed -n "s/.*predict .*$2=\([0-9]*\).*/\1/p" "$1" | head -1
44 }
45
46 # Assert one counter, with the whole line in the failure so a wrong number is
47 # read in context rather than alone.
48 want_stat() {
49 _got=$(predict_stat "$1" "$2")
50 [ -n "$_got" ] || {
51 echo "e2e FAIL: $4: no predict stats line (wanted $2=$3); got:"
52 cat "$1"; exit 1;
53 }
54 [ "$_got" = "$3" ] || {
55 echo "e2e FAIL: $4: $2=$_got, want $3"
56 grep "^predict " "$1" || true
57 exit 1;
58 }
59 }
25 60
26 # Wait until PATTERN shows up in FILE (default 15s). Timing that keys off the 61 # Wait until PATTERN shows up in FILE (default 15s). Timing that keys off the
27 # session's own output instead of a fixed sleep: the marker is proof the 62 # session's own output instead of a fixed sleep: the marker is proof the
@@ -51,7 +86,12 @@ cleanup() {
51 [ -n "$D2PID" ] && kill "$D2PID" 2>/dev/null || true 86 [ -n "$D2PID" ] && kill "$D2PID" 2>/dev/null || true
52 [ -n "$D3PID" ] && kill "$D3PID" 2>/dev/null || true 87 [ -n "$D3PID" ] && kill "$D3PID" 2>/dev/null || true
53 [ -n "$D4PID" ] && kill "$D4PID" 2>/dev/null || true 88 [ -n "$D4PID" ] && kill "$D4PID" 2>/dev/null || true
54 rm -f "$SOCK" "$SOCK2" "$SOCK3" "$SOCK4" "$SOCK4.second" "$QKEY" "$QKEY.bad" \ 89 [ -n "$D5PID" ] && kill "$D5PID" 2>/dev/null || true
90 [ -n "$D6PID" ] && kill "$D6PID" 2>/dev/null || true
91 [ -n "$D7PID" ] && kill "$D7PID" 2>/dev/null || true
92 rm -f "$SOCK5" "$SOCK6" "$SOCK7" "$PWSH" \
93 "$OUT.p1" "$OUT.p1.early" "$OUT.pb" "$OUT.pw" "$OUT.rw" "$OUT.pr" \
94 "$SOCK" "$SOCK2" "$SOCK3" "$SOCK4" "$SOCK4.second" "$QKEY" "$QKEY.bad" \
55 "$OUT" "$OUT.kill" "$OUT.re" "$OUT.a" \ 95 "$OUT" "$OUT.kill" "$OUT.re" "$OUT.a" \
56 "$OUT.b" "$OUT.via" "$OUT.dead" "$OUT.abort" "$OUT.m7" "$OUT.m7b" \ 96 "$OUT.b" "$OUT.via" "$OUT.dead" "$OUT.abort" "$OUT.m7" "$OUT.m7b" \
57 "$OUT.q" "$OUT.qc" "$OUT.qr" "$OUT.qa" "$OUT.qk" "$QKEY.wrong" 97 "$OUT.q" "$OUT.qc" "$OUT.qr" "$OUT.qa" "$OUT.qk" "$QKEY.wrong"
@@ -633,4 +673,269 @@ kill "$D4PID" 2>/dev/null || true
633 D4PID="" 673 D4PID=""
634 rm -f "$OUT.q" "$OUT.qc" "$OUT.qr" "$OUT.qa" "$OUT.qk" "$QKEY" "$QKEY.bad" "$QKEY.wrong" 674 rm -f "$OUT.q" "$OUT.qc" "$OUT.qr" "$OUT.qa" "$OUT.qk" "$QKEY" "$QKEY.bad" "$QKEY.wrong"
635 675
676 # ---- M9: prediction ---------------------------------------------------
677 #
678 # Every scenario here asserts COUNTERS, because the screen cannot tell a
679 # predicted glyph from an echoed one — that is the entire point of the
680 # feature — so a rendering test would pass just as happily with prediction
681 # switched off.
682 #
683 # The session shell is /bin/cat, not /bin/sh. An interactive bash prompt is
684 # icanon=0/echo=0 (readline echoes for itself), which is the ADAPTIVE tier,
685 # not the always-predict one; cat is a genuinely canonical reader and the
686 # only way to exercise `.always` here. That distinction cost a milestone's
687 # worth of confusion to learn and is why it is written down twice.
688
689 # Delay per direction. The round trip is twice this, and every assertion
690 # below about "before the daemon could have answered" is measured against
691 # it. Deliberately larger than the plan's 150ms: the margin between "the
692 # prediction is painted" and "the echo could have arrived" is what keeps
693 # this scenario from being a race, and 300ms each way makes that margin
694 # half a second rather than a tenth.
695 PDELAY=300
696
697 "$MUXD" run --sock "$SOCK5" --shell /bin/cat > "$OUT.p1" 2>&1 &
698 D5PID=$!
699 i=0
700 while [ ! -S "$SOCK5" ] && [ "$i" -lt 50 ]; do sleep 0.1; i=$((i+1)); done
701 [ -S "$SOCK5" ] || { echo "e2e FAIL: prediction daemon never bound"; cat "$OUT.p1"; exit 1; }
702
703 # 1. Line mode: the glyph is on screen before the round trip could have
704 # delivered it.
705 set +e
706 { sleep 2; printf 'z'; sleep 4; printf '\034'; } | \
707 DELAY_MS="$PDELAY" MUX_PREDICT_STATS=1 timeout 40 "$MUX" \
708 --via "$DELAYPIPE | $MUXD proxy --sock $SOCK5 | $DELAYPIPE" \
709 > "$OUT.p1" 2>&1 &
710 P1PID=$!
711 set -e
712 # Snapshot the client's output a quarter second after the keystroke — long
713 # before the 600ms round trip could bring the pty's own echo back.
714 sleep 2.25
715 cp "$OUT.p1" "$OUT.p1.early" 2>/dev/null || true
716 set +e
717 wait "$P1PID"
718 RC=$?
719 set -e
720 [ "$RC" -eq 0 ] || { echo "e2e FAIL: line-mode prediction client exited $RC"; cat "$OUT.p1"; exit 1; }
721
722 # The underlined glyph, in the snapshot taken before the echo could arrive.
723 # This is the whole claim of the milestone, as an effect rather than a
724 # counter: the character was on the screen while it was still in flight.
725 grep -q "$(printf '\033\[4mz')" "$OUT.p1.early" || {
726 echo "e2e FAIL: no predicted glyph 250ms after the keystroke (RTT is $((PDELAY * 2))ms)"
727 echo "--- early snapshot ---"; cat -v "$OUT.p1.early"; exit 1;
728 }
729 # ...and the daemon's own answer was NOT there yet, which is what makes the
730 # line above mean anything. A delta paints a row with EL(2) before its
731 # content; the prediction never does.
732 if grep -q "$(printf '\033\[2K\033\[0mz')" "$OUT.p1.early"; then
733 echo "e2e FAIL: the pty's echo arrived within 250ms; the delay pipe is not delaying,"
734 echo " so the assertion above proves nothing about prediction"
735 exit 1
736 fi
737 want_stat "$OUT.p1" contradicted 0 "line mode"
738 CONF=$(predict_stat "$OUT.p1" confirmed)
739 [ -n "$CONF" ] && [ "$CONF" -ge 1 ] || {
740 echo "e2e FAIL: line mode: confirmed=$CONF, want >=1"; grep "^predict " "$OUT.p1"; exit 1;
741 }
742 # The session really received the keystroke: cat echoed it into the grid.
743 "$MUXD" dump --sock "$SOCK5" | grep -q "z" || {
744 echo "e2e FAIL: line mode: daemon grid never saw the keystroke"; exit 1;
745 }
746
747 # 2. A burst outrunning the round trip. This is reconcile v2's reason to
748 # exist: under the old rule every character after the first was judged by
749 # a frame built before it was typed, read as a contradiction, and the
750 # whole queue was flushed once per round trip. If anyone regresses the
751 # three-way judgment, this scenario is what says so.
752 set +e
753 { sleep 2; for c in b u r s t; do printf '%s' "$c"; sleep 0.2; done; \
754 sleep 4; printf '\034'; } | \
755 DELAY_MS="$PDELAY" MUX_PREDICT_STATS=1 timeout 40 "$MUX" \
756 --via "$DELAYPIPE | $MUXD proxy --sock $SOCK5 | $DELAYPIPE" \
757 > "$OUT.pb" 2>&1
758 RC=$?
759 set -e
760 [ "$RC" -eq 0 ] || { echo "e2e FAIL: burst client exited $RC"; cat "$OUT.pb"; exit 1; }
761
762 # Five keystrokes typed at 200ms into a 600ms round trip: three are always
763 # outstanding at once. `made` is asserted too, so two keystrokes arriving in
764 # one read (which would be suppressed as a multi-byte chunk, not predicted)
765 # fails loudly instead of quietly weakening the test.
766 want_stat "$OUT.pb" made 5 "burst"
767 want_stat "$OUT.pb" confirmed 5 "burst"
768 want_stat "$OUT.pb" contradicted 0 "burst"
769 want_stat "$OUT.pb" expired 0 "burst"
770 "$MUXD" dump --sock "$SOCK5" | grep -q "burst" || {
771 echo "e2e FAIL: burst: daemon grid does not hold the typed text"; exit 1;
772 }
773
774 kill "$D5PID" 2>/dev/null || true
775 D5PID=""
776
777 # 3. A password prompt. The session is canonical with echo OFF from the
778 # moment it starts, so the whole run sits in the tier that predicts
779 # nothing at all.
780 cat > "$PWSH" <<'PWEOF'
781 #!/bin/sh
782 stty -echo
783 printf 'pw-ready\n'
784 read secret
785 printf 'pw-len-%s\n' "${#secret}"
786 # Blocks forever rather than exiting: the shell exiting would end the
787 # session, and the daemon would be gone before its grid could be read.
788 read _hold
789 PWEOF
790 chmod +x "$PWSH"
791
792 "$MUXD" run --sock "$SOCK6" --shell "$PWSH" > "$OUT.pw" 2>&1 &
793 D6PID=$!
794 i=0
795 while [ ! -S "$SOCK6" ] && [ "$i" -lt 50 ]; do sleep 0.1; i=$((i+1)); done
796 [ -S "$SOCK6" ] || { echo "e2e FAIL: password daemon never bound"; cat "$OUT.pw"; exit 1; }
797
798 # One character per write, and NOT as a single `printf hunter2`. A whole
799 # word in one write reaches the client as a multi-byte chunk, which is
800 # refused for being multi-byte before the tier is ever consulted — so the
801 # scenario would pass with the password tier completely broken. It did:
802 # mapping echo-off canonical to the always-predict tier survived this
803 # scenario until the typing was split up.
804 set +e
805 { sleep 2; for c in h u n t e r 2; do printf '%s' "$c"; sleep 0.2; done; \
806 printf '\n'; sleep 2; printf '\034'; } | \
807 MUX_PREDICT_STATS=1 timeout 40 "$MUX" --sock "$SOCK6" > "$OUT.pw" 2>&1
808 RC=$?
809 set -e
810 [ "$RC" -eq 0 ] || { echo "e2e FAIL: password client exited $RC"; cat "$OUT.pw"; exit 1; }
811
812 # Nothing was shown, and nothing was even attempted: echo-off canonical is
813 # the tier where a prediction must never be MADE, not merely never painted,
814 # because a made prediction sits in a buffer the overlay paints from.
815 want_stat "$OUT.pw" made 0 "password"
816 want_stat "$OUT.pw" displayed 0 "password"
817 # The effect, not the counter: the secret appears nowhere in the bytes the
818 # client wrote to the terminal.
819 if grep -q "hunter2" "$OUT.pw"; then
820 echo "e2e FAIL: the password appears in the client's terminal output"; exit 1
821 fi
822 # ...and the run was not vacuous: the shell really received all seven
823 # characters, so the absence above is prediction declining rather than
824 # nothing having been typed.
825 "$MUXD" dump --sock "$SOCK6" | grep -q "pw-len-7" || {
826 echo "e2e FAIL: the password never reached the shell; the absence above proves nothing"
827 "$MUXD" dump --sock "$SOCK6"; exit 1;
828 }
829
830 kill "$D6PID" 2>/dev/null || true
831 D6PID=""
832
833 # 4. Raw mode: display is earned, then lost to a keystroke the application
834 # swallows. rawmode echoes like an editor in insert mode until it is sent
835 # 0x00, after which it consumes input and prints nothing.
836 "$MUXD" run --sock "$SOCK7" --shell "$RAWMODE" > "$OUT.rw" 2>&1 &
837 D7PID=$!
838 i=0
839 while [ ! -S "$SOCK7" ] && [ "$i" -lt 50 ]; do sleep 0.1; i=$((i+1)); done
840 [ -S "$SOCK7" ] || { echo "e2e FAIL: rawmode daemon never bound"; cat "$OUT.rw"; exit 1; }
841
842 set +e
843 { sleep 2; for c in a b c; do printf '%s' "$c"; sleep 0.5; done; \
844 printf '\000'; sleep 0.5; printf 'j'; sleep 2.5; printf '\034'; } | \
845 MUX_PREDICT_STATS=1 timeout 40 "$MUX" --sock "$SOCK7" > "$OUT.rw" 2>&1
846 RC=$?
847 set -e
848 [ "$RC" -eq 0 ] || { echo "e2e FAIL: rawmode client exited $RC"; cat "$OUT.rw"; exit 1; }
849
850 # Four printable keystrokes made predictions; the 0x00 that switched modes
851 # is not printable and was refused.
852 want_stat "$OUT.rw" made 4 "raw mode"
853 want_stat "$OUT.rw" suppressed 1 "raw mode"
854 # a and b are invisible while promotion is earned; c is the first painted,
855 # and j is painted because by then display had been earned.
856 want_stat "$OUT.rw" displayed 2 "raw mode"
857 want_stat "$OUT.rw" confirmed 3 "raw mode"
858 # j was swallowed: no frame ever answered the cell it was drawn in, so the
859 # expiry bound retired it rather than leaving a phantom glyph on screen for
860 # the rest of the session.
861 want_stat "$OUT.rw" expired 1 "raw mode"
862 want_stat "$OUT.rw" contradicted 1 "raw mode"
863 want_stat "$OUT.rw" abandoned 1 "raw mode"
864
865 # Every prediction accounted for: made = confirmed + abandoned + pending,
866 # and the queue is empty at exit, so nothing survived unexplained.
867 RW_MADE=$(predict_stat "$OUT.rw" made)
868 RW_CONF=$(predict_stat "$OUT.rw" confirmed)
869 RW_ABND=$(predict_stat "$OUT.rw" abandoned)
870 [ "$((RW_MADE - RW_CONF - RW_ABND))" -eq 0 ] || {
871 echo "e2e FAIL: raw mode: $((RW_MADE - RW_CONF - RW_ABND)) predictions unaccounted for"
872 grep "^predict " "$OUT.rw"; exit 1;
873 }
874
875 kill "$D7PID" 2>/dev/null || true
876 D7PID=""
877
878 # 5. A transport torn down with predictions outstanding. The overlay must
879 # come back empty: what was queued was predicted against a connection
880 # that no longer exists.
881 "$MUXD" run --sock "$SOCK5" --shell /bin/cat > "$OUT.pr" 2>&1 &
882 D5PID=$!
883 i=0
884 while [ ! -S "$SOCK5" ] && [ "$i" -lt 50 ]; do sleep 0.1; i=$((i+1)); done
885 [ -S "$SOCK5" ] || { echo "e2e FAIL: reconnect daemon never bound"; cat "$OUT.pr"; exit 1; }
886
887 set +e
888 { sleep 2; printf 'p'; sleep 5; printf 'q'; sleep 4; printf '\034'; } | \
889 DELAY_MS="$PDELAY" MUX_PREDICT_STATS=1 timeout 60 "$MUX" \
890 --via "$DELAYPIPE | $MUXD proxy --sock $SOCK5 | $DELAYPIPE" \
891 > "$OUT.pr" 2>&1 &
892 PRPID=$!
893 set -e
894 # Kill the transport a tenth of a second after the keystroke, while its
895 # prediction is still outstanding — the round trip is 600ms, so it cannot
896 # have been judged yet. By comm+pid: the client's own argv contains the
897 # --via string, so a pattern kill would take out the client under test.
898 sleep 2.1
899 PP=$(proxy_pid "$SOCK5")
900 [ -n "$PP" ] || { echo "e2e FAIL: no proxy to tear down"; cat "$OUT.pr"; exit 1; }
901 kill -9 "$PP"
902
903 set +e
904 wait "$PRPID"
905 RC=$?
906 set -e
907 [ "$RC" -eq 0 ] || { echo "e2e FAIL: reconnect client exited $RC"; cat "$OUT.pr"; exit 1; }
908
909 # The queue is empty at exit, by attribution rather than by inspection:
910 # nothing predicted is still outstanding, and the flush that dropped the
911 # in-flight prediction was not counted as anybody being wrong.
912 PR_MADE=$(predict_stat "$OUT.pr" made)
913 PR_CONF=$(predict_stat "$OUT.pr" confirmed)
914 PR_ABND=$(predict_stat "$OUT.pr" abandoned)
915 [ -n "$PR_MADE" ] || { echo "e2e FAIL: reconnect: no predict stats"; cat "$OUT.pr"; exit 1; }
916 [ "$((PR_MADE - PR_CONF - PR_ABND))" -eq 0 ] || {
917 echo "e2e FAIL: reconnect: $((PR_MADE - PR_CONF - PR_ABND)) predictions outlived the tear"
918 grep "^predict " "$OUT.pr"; exit 1;
919 }
920 # And it was the reconnect that dropped it, not the expiry bound catching it
921 # a second later. Without this the scenario cannot tell the two apart: the
922 # resync after a reconnect flushes as well, so the outcome looks identical
923 # either way and removing the reconnect's own flush passes unnoticed. It
924 # did, until this line.
925 want_stat "$OUT.pr" expired 0 "reconnect"
926 [ -n "$PR_ABND" ] && [ "$PR_ABND" -ge 1 ] || {
927 echo "e2e FAIL: reconnect: nothing was abandoned ($PR_ABND), so no prediction was"
928 echo " outstanding when the transport died and the tear proves nothing"
929 grep "^predict " "$OUT.pr"; exit 1;
930 }
931 # The session survived and the client resumed into it: the keystroke typed
932 # after the tear reached the shell.
933 "$MUXD" dump --sock "$SOCK5" | grep -q "q" || {
934 echo "e2e FAIL: reconnect: the post-tear keystroke never reached the session"
935 "$MUXD" dump --sock "$SOCK5"; exit 1;
936 }
937
938 kill "$D5PID" 2>/dev/null || true
939 D5PID=""
940
636 echo "e2e OK" 941 echo "e2e OK"