a73x

6a4b3a79

chore: remove obsolete demos and QUIC spike

a73x   2026-09-06 14:18

Commit message
chore: remove obsolete demos and QUIC spike
spike/quic/README.md
Old New
@@ -1,216 +0,0 @@
1 # M8 Task 1 spike — can our toolchain ship a QUIC stack?
2
3 **Throwaway.** Nothing here is wired into `make test`, `make build`, `make e2e`
4 or `make bench`; the main gates are untouched by construction. This directory
5 exists to answer one question and to record the answer.
6
7 ## Verdict
8
9 **Candidate (a), ngtcp2 1.25.0 + wolfSSL 5.9.2-stable, clears the bar.**
10 Candidate (b) (picoquic + picotls) was not reached and not needed.
11
12 Every requirement the plan set for the spike, and what happened:
13
14 | Bar | Result |
15 |---|---|
16 | Built by our **pinned Zig 0.15.2** (as `zig cc`/`ar`/`ranlib`) | yes — no gcc, no system clang |
17 | **Native** build | yes |
18 | **x86_64-linux-musl static** build | yes — `ELF 64-bit LSB executable, statically linked` |
19 | Reproducible from **pinned sources** | yes — `build.sh`, URLs + sha256, fetch-not-vendor (see below) |
20 | **TLS 1.3 external PSK** | yes — no certificates, no CA, anywhere in the handshake |
21 | **fd-drivable** (no imposed event loop) | yes — the echo pair is one `poll()` over one UDP fd |
22 | Working **echo pair** | yes — `quicecho.c`, both roles, verified locally and on the box |
23
24 `build.sh` reruns the whole thing from nothing in a few minutes.
25
26 ## What was NOT proven, and why — read this before quoting the numbers
27
28 **The plan's "measure over the real WAN" step could not be run, for an
29 environmental reason that matters to the whole milestone: the box has no
30 inbound UDP path.** It sits behind NAT on `10.78.3.3/24`, reachable only by
31 ssh through a jump host. Verified rather than assumed: a UDP listener on the
32 box, a packet sent to its egress address `86.153.165.146`, and nothing
33 arrived.
34
35 So the "client here, server there" topology QUIC needs is impossible against
36 this box today, and **that is a finding about the milestone, not about the
37 spike**. The plan's out-of-scope list says "NAT traversal / hole punching
38 (VMs have reachable addresses; banked)" — for the one VM we have, that
39 assumption is false. A direct QUIC transport reaches a VM only if something
40 can send UDP *to* it. This needs a controller decision before Task 2:
41 reachable VMs, a relay, or NAT traversal stops being out of scope.
42
43 What was measured instead, and labelled as such below: the static binary
44 running **on the box**, with the real link's round trip (16.5ms, measured
45 that same day) emulated on the box's loopback via `netem`. Handshake cost in
46 *round trips* is exactly what that measures well, and round trips are what
47 the ssh comparison is about. Loss, queueing, and PMTU on the real path are
48 **not** exercised, and no number here should be quoted as if they were.
49
50 ## Numbers
51
52 Emulated 16.55ms RTT (`netem delay 8250us` on the box's `lo`; `ping` confirmed
53 16.547/16.552/16.561ms), static musl binary, 6 reps:
54
55 ```
56 #REP 0 handshake_ms=18.50 echo_ms=35.51 resumed=0
57 #REP 1 handshake_ms=18.65 echo_ms=35.68 resumed=1
58 #REP 5 handshake_ms=18.48 echo_ms=35.53 resumed=1
59 ```
60
61 - **Cold handshake: 18.5ms ≈ 1.1 × RTT.** A TLS 1.3 external-PSK handshake is
62 one round trip, and that is what the wire shows.
63 - **Handshake → first echoed byte: 35.5ms ≈ 2.1 × RTT.**
64 - **Resumption bought nothing (18.5 cold vs 18.5 resumed), and that is
65 expected, not a defect.** With an external PSK the *cold* handshake is
66 already 1-RTT; there is no certificate exchange to skip. The only thing
67 left to win is 0-RTT early data, which this spike did not wire up. Anyone
68 planning Task 3 around "resumption makes reconnects cheap" should read this
69 line twice: the win is already in the cold path, and the next win requires
70 0-RTT specifically.
71
72 **Against ssh, from the same day's `wan.sh` run on the same link** (baseline
73 RTT 16.4-16.8ms): ssh `viafloor` — channel open + remote exec, before one
74 protocol byte moves — measured **36.4-38.1ms, or 2.2-2.3 × RTT**. QUIC's
75 entire cold handshake is **1.1 × RTT**, and QUIC handshake-to-echoed-byte
76 (2.1 × RTT) is about what ssh spends on channel setup *alone*. That is the
77 milestone's thesis surviving its first contact with numbers — with the
78 caveat above about emulated versus real path.
79
80 Local loopback (~0 RTT), for the CPU-cost floor: cold 1.3-1.6ms, resumed
81 0.8-1.0ms.
82
83 ## Integration assessment (plan Step 4)
84
85 **How it wants to be driven.** ngtcp2 imposes no event loop and owns no
86 socket — it is a state machine over buffers. The daemon integration is the
87 shape muxd already has:
88
89 - one UDP fd added to the existing `poll()` set; readable → `recvfrom` →
90 `ngtcp2_conn_read_pkt(conn, path, pi, buf, len, now)`;
91 - after *any* event, drain egress: `ngtcp2_conn_writev_stream(...)` in a loop
92 until it returns 0, `sendto` each packet;
93 - one timer: `ngtcp2_conn_get_expiry(conn)` gives an absolute deadline, which
94 becomes the `poll()` timeout (`pumpOnce` already takes one); on expiry,
95 `ngtcp2_conn_handle_expiry(conn, now)` then drain egress again.
96
97 No timerfd is required — the existing poll timeout is enough, which is
98 convenient because `pumpOnce(timeout_ms)` is already the daemon's shape. Each
99 connection is one `ngtcp2_conn*` plus one wolfSSL object; a client slot binds
100 to one bidi stream id.
101
102 **Task 2 requirements this spike does NOT demonstrate.** Named here because
103 the echo pair's success is misleading about each of them — it exchanges a
104 few dozen bytes per connection and then exits, which is precisely the
105 workload under which all four stay invisible:
106
107 1. **Flow control is mandatory and its absence looks like a hang.** ngtcp2
108 does not self-extend windows: as data is consumed the receiver must call
109 `ngtcp2_conn_extend_max_stream_offset` (stream level — the header says so
110 explicitly) *and* `ngtcp2_conn_extend_max_offset` (connection level).
111 Omit them and the peer stalls forever once it has sent the initial window
112 — on the order of 64KB, which a muxd session of snapshots plus deltas
113 reaches in seconds. It presents as a silent mid-session freeze with no
114 error anywhere, which is about the worst failure shape available.
115 `quicecho.c` never calls either, and never sends enough to notice.
116 2. **The daemon is fd-centric and a QUIC connection is not an fd.**
117 `clients[i].fd` feeds the poll array directly, and `queueFrame` /
118 `flushClient` / `drainPending` all reach `std.posix.send` on that fd. A
119 QUIC client shares ONE UDP fd with every other QUIC client and is
120 identified by connection ID, not by descriptor. Task 2 therefore needs a
121 send/recv indirection over "client i" — a small tagged union or vtable at
122 the three or four call sites that touch `.fd` — before a QUIC connection
123 can present as an ordinary client slot. Sized work, not a line change,
124 and worth doing as its own step so the socket path stays provably intact.
125 3. **`get_expiry` becomes a minimum across connections.** One connection's
126 deadline is the poll timeout; N connections means folding
127 `ngtcp2_conn_get_expiry` over all of them and taking the earliest, then
128 servicing every connection whose deadline has passed. The spike has
129 exactly one connection and so never had to.
130 4. **Retry / address validation.** On a public UDP port a server that
131 answers unvalidated Initials is an amplification reflector.
132 `quicecho.c` deliberately has no Retry or token path; the daemon needs
133 one. This moves from "not in the spike" to a Task 2 requirement.
134
135 **PSK API.** wolfSSL TLS 1.3 external PSK is two callbacks, one per side:
136 `wolfSSL_CTX_set_psk_client_tls13_callback` (fills identity, key, and picks
137 the ciphersuite) and `wolfSSL_CTX_set_psk_server_tls13_callback` (maps
138 identity → key). Both are ordinary C function pointers with no global state
139 beyond what we give them. Note the server callback's last parameter is
140 `const char **ciphersuite` (out), not `const char *` — the compiler catches
141 it, but the wolfSSL headers carry both shapes for different callback
142 families and it is easy to pick the wrong one. `wolfSSL_CTX_set_cipher_list`
143 pins the suite. No certificate calls are needed on either side: possession of
144 the key *is* the authentication, which is exactly the deploy model.
145
146 **Stream API.** One bidi stream carries everything:
147 `ngtcp2_conn_open_bidi_stream` (client, after
148 `extend_max_local_streams_bidi`), then data goes out as `ngtcp2_vec`s through
149 `writev_stream` and arrives in the `recv_stream_data` callback as a plain
150 `(data, datalen)`. That is byte-stream semantics — the existing frame
151 protocol needs no change to ride it, which is the "zero protocol changes"
152 requirement met in the API rather than by hope.
153
154 **Binary size.** Static musl `quicecho`: 8.7MB unstripped, **1.68MB
155 stripped** — that is the whole QUIC + TLS stack plus a small program. Today's
156 `muxd` static musl is 15.5MB unstripped. So QUIC costs on the order of
157 **+1.7MB stripped**, against an artifact that is already 15MB because of
158 ghostty-vt.
159
160 ## Gotchas Task 2 will hit (both cost a link failure here)
161
162 1. **`zig cc` instruments C with UBSan by default.** A static musl link then
163 fails on undefined `__ubsan_handle_*`. Fix: `-fno-sanitize=undefined` (or
164 accept the runtime). This will bite again the moment these sources are
165 compiled via `addCSourceFiles` in `build.zig`.
166 2. **wolfSSL omits AES-ECB by default**, and ngtcp2's wolfSSL backend needs
167 it for QUIC header protection (undefined `wolfSSL_EVP_aes_128_ecb`). Fix:
168 `-DWOLFSSL_AESECB=yes`.
169 3. **Build ngtcp2's library only** (`ENABLE_LIB_ONLY`), not its examples.
170 The examples are what want brotli, and cmake would otherwise resolve it
171 to the host's shared libraries and break the static musl link. An earlier
172 version of this file credited the fix to
173 `CMAKE_DISABLE_FIND_PACKAGE_Libbrotli*`; those flags were inert —
174 ngtcp2's `find_package(Libbrotli*)` calls sit inside
175 `if(NOT ENABLE_LIB_ONLY)`, so with the library-only build they never run,
176 and cmake duly reported the flags as unused-cli. The lesson that carries
177 into Task 2 is the general one, and it survives the move away from cmake:
178 **compile the library sources, not the example sources.**
179 4. **`WOLFSSL_KEYLOG_EXPORT` writes TLS secrets to disk with nothing asking
180 it to — it is now OFF in `build.sh`, and this is the corrected record.**
181 Built with that flag, these binaries drop an `sslkeylog.log` of
182 `CLIENT_HANDSHAKE_TRAFFIC_SECRET` lines into the current directory on
183 every run: no env var, no flag, no message. Re-verified in an empty temp
184 directory after the fact — one run, one secrets file.
185
186 The first version of this README said the file "is deleted". That was
187 wrong. Three copies existed and one was deleted: the repo root. **A
188 second sat in the local scratch tree, and a third — 12,640 bytes,
189 world-readable, 20 handshakes — sat in `/home/ubuntu` on the shared box**,
190 left there by the spike runs. Review found both; the box copy was removed
191 in review, the scratch copy when this correction was written. All keys
192 involved were throwaway loopback/emulated-RTT test keys, and no real
193 session was ever carried over this stack — but "I cleaned it up" was a
194 claim I had not checked, on exactly the class of artifact where checking
195 is the whole job.
196
197 The flag is now `no` in the committed script, so anyone who runs it gets
198 no keylog by default. That is structure rather than advice, which is the
199 only reason to trust it. Task 2 keeps it off.
200
201 ## On "vendored" vs fetched
202
203 The plan said vendor pinned snapshots; `build.sh` fetches them against
204 sha256 pins instead. These are ~120MB of third-party C, this directory is
205 explicitly throwaway, and for the question the spike asks — *can our
206 toolchain build it* — a pinned fetch is the same evidence as a checked-in
207 copy. Vendoring is the right call at Task 2, when the answer is "yes, and we
208 are keeping it"; doing it now would put 120MB in the history for an
209 experiment we might have deleted.
210
211 ## Files
212
213 - `quicecho.c` — the echo pair, both roles, ~600 lines. Deliberately not a
214 design sketch: no retry/token, no migration, one connection at a time.
215 - `build.sh` — pinned fetch + both builds, pinned Zig only. `./build.sh` and
216 it produces `work/quicecho-native` and `work/quicecho-musl`.
spike/quic/build.sh
Old New
@@ -1,128 +0,0 @@
1 #!/bin/sh
2 # M8 Task 1 spike: build a QUIC echo pair from pinned sources using ONLY the
3 # repo's pinned Zig 0.15.2 as the C toolchain (zig cc / zig ar / zig ranlib).
4 #
5 # The point is not the echo pair. The point is that every line below runs
6 # under the toolchain muxd already ships with, for native AND
7 # x86_64-linux-musl static — which is the question Task 1 exists to answer.
8 #
9 # ./build.sh [workdir] (default: ./work, ~700MB, throwaway)
10 #
11 # Sources are fetched, not vendored into git: they are ~120MB of C, this is a
12 # throwaway spike, and a URL + sha256 pin answers "can our toolchain build
13 # it" exactly as well as a checked-in copy would. Vendoring is a Task 2
14 # decision, when the answer is "yes, and we are keeping it".
15 set -eu
16
17 NGTCP2_VER=1.25.0
18 WOLFSSL_VER=5.9.2-stable
19 NGTCP2_SHA=1c0843076528a87b65e9a9d455100941f4cb65d44f96c5da6ae56df146043955
20 WOLFSSL_SHA=2f4ef3d4fd387a9b3191d36a6316d69116c46ff69bb9583b6c82b36d7b8ca114
21
22 ZIG="${ZIG:-$HOME/Downloads/zig-x86_64-linux-0.15.2/zig}"
23 SELF="$(cd "$(dirname "$0")" && pwd)"
24 # An absolute workdir must be taken as given. Prepending the script's
25 # directory to one silently built a 274MB tree inside the repo during
26 # review, where .gitignore's `work/` then hid it.
27 case "${1:-work}" in
28 /*) W="$1" ;;
29 *) W="$SELF/${1:-work}" ;;
30 esac
31
32 [ -x "$ZIG" ] || { echo "no zig at $ZIG (set ZIG=...)" >&2; exit 1; }
33 mkdir -p "$W/src" "$W/bin"
34
35 # --- zig as the C toolchain -------------------------------------------------
36 cat > "$W/bin/zigcc" <<EOF
37 #!/bin/sh
38 exec $ZIG cc "\$@"
39 EOF
40 cat > "$W/bin/zigcc-musl" <<EOF
41 #!/bin/sh
42 exec $ZIG cc -target x86_64-linux-musl "\$@"
43 EOF
44 cat > "$W/bin/zigar" <<EOF
45 #!/bin/sh
46 exec $ZIG ar "\$@"
47 EOF
48 cat > "$W/bin/zigranlib" <<EOF
49 #!/bin/sh
50 exec $ZIG ranlib "\$@"
51 EOF
52 chmod +x "$W"/bin/*
53
54 # --- pinned sources ---------------------------------------------------------
55 fetch() { # url sha file
56 [ -f "$W/src/$3" ] || curl -sSL -o "$W/src/$3" "$1"
57 echo "$2 $W/src/$3" | sha256sum -c - >/dev/null || {
58 echo "checksum mismatch for $3" >&2; exit 1; }
59 }
60 fetch "https://github.com/ngtcp2/ngtcp2/releases/download/v$NGTCP2_VER/ngtcp2-$NGTCP2_VER.tar.gz" \
61 "$NGTCP2_SHA" "ngtcp2-$NGTCP2_VER.tar.gz"
62 fetch "https://github.com/wolfSSL/wolfssl/archive/refs/tags/v$WOLFSSL_VER.tar.gz" \
63 "$WOLFSSL_SHA" "wolfssl-$WOLFSSL_VER.tar.gz"
64 [ -d "$W/src/ngtcp2-$NGTCP2_VER" ] || tar -C "$W/src" -xzf "$W/src/ngtcp2-$NGTCP2_VER.tar.gz"
65 [ -d "$W/src/wolfssl-$WOLFSSL_VER" ] || tar -C "$W/src" -xzf "$W/src/wolfssl-$WOLFSSL_VER.tar.gz"
66
67 for T in native musl; do
68 CC="$W/bin/zigcc"; XTRA=""
69 if [ "$T" = musl ]; then
70 CC="$W/bin/zigcc-musl"
71 XTRA="-DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=x86_64"
72 fi
73
74 # wolfSSL. Two flags here are load-bearing and were both found the hard
75 # way, by link failure:
76 # -fno-sanitize=undefined zig cc instruments C by default, and the
77 # ubsan runtime is not in a static musl link
78 # (undefined __ubsan_handle_*).
79 # WOLFSSL_AESECB ngtcp2's wolfSSL backend needs AES-ECB for
80 # QUIC header protection; wolfSSL omits it by
81 # default (undefined wolfSSL_EVP_aes_128_ecb).
82 #
83 # WOLFSSL_KEYLOG_EXPORT is deliberately OFF. Built with it on, these
84 # binaries write every handshake's secrets to ./sslkeylog.log with
85 # nothing asking them to — no env var, no flag, no message. Flip it to
86 # `yes` locally when you need to decrypt your own capture, and do not
87 # commit that. It is off here so that running this script cannot leak
88 # keys by default, which is a stronger guarantee than a warning.
89 cmake -S "$W/src/wolfssl-$WOLFSSL_VER" -B "$W/build/wolfssl-$T" $XTRA \
90 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS="-fno-sanitize=undefined -O2" \
91 -DCMAKE_C_COMPILER="$CC" -DCMAKE_AR="$W/bin/zigar" -DCMAKE_RANLIB="$W/bin/zigranlib" \
92 -DBUILD_SHARED_LIBS=OFF -DWOLFSSL_QUIC=yes -DWOLFSSL_PSK=yes \
93 -DWOLFSSL_AESECB=yes -DWOLFSSL_AESCTR=yes -DWOLFSSL_KEYLOG_EXPORT=no \
94 -DWOLFSSL_SESSION_TICKET=yes -DWOLFSSL_EXAMPLES=no -DWOLFSSL_CRYPT_TESTS=no \
95 -DWOLFSSL_INSTALL=yes -DCMAKE_INSTALL_PREFIX="$W/out/$T" >/dev/null
96 cmake --build "$W/build/wolfssl-$T" -j"$(nproc)" >/dev/null
97 cmake --install "$W/build/wolfssl-$T" >/dev/null
98
99 # ngtcp2 + its wolfSSL crypto backend. ENABLE_LIB_ONLY is what keeps the
100 # static musl link clean: ngtcp2's find_package(Libbrotli*) calls live
101 # inside `if(NOT ENABLE_LIB_ONLY)`, so building the library alone never
102 # goes looking for the host's shared brotli in the first place. (An
103 # earlier version of this script also passed
104 # CMAKE_DISABLE_FIND_PACKAGE_Libbrotli* and credited them with the fix;
105 # cmake reported them unused-cli, because they were.)
106 PKG_CONFIG_PATH="$W/out/$T/lib/pkgconfig" \
107 cmake -S "$W/src/ngtcp2-$NGTCP2_VER" -B "$W/build/ngtcp2-$T" $XTRA \
108 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS="-fno-sanitize=undefined -O2" \
109 -DCMAKE_C_COMPILER="$CC" -DCMAKE_AR="$W/bin/zigar" -DCMAKE_RANLIB="$W/bin/zigranlib" \
110 -DENABLE_STATIC_LIB=ON -DENABLE_SHARED_LIB=OFF -DENABLE_WOLFSSL=ON \
111 -DENABLE_OPENSSL=OFF -DENABLE_GNUTLS=OFF -DENABLE_BORINGSSL=OFF \
112 -DENABLE_LIB_ONLY=ON \
113 -DCMAKE_INSTALL_PREFIX="$W/out/$T" >/dev/null
114 cmake --build "$W/build/ngtcp2-$T" -j"$(nproc)" >/dev/null
115 cmake --install "$W/build/ngtcp2-$T" >/dev/null
116 done
117
118 L() { echo "$W/out/$1/lib/libngtcp2_crypto_wolfssl.a $W/out/$1/lib/libngtcp2.a $W/out/$1/lib/libwolfssl.a"; }
119 # shellcheck disable=SC2046
120 "$W/bin/zigcc" -O2 -o "$W/quicecho-native" "$SELF/quicecho.c" \
121 -I"$W/out/native/include" $(L native) -lm -pthread
122 # shellcheck disable=SC2046
123 "$W/bin/zigcc-musl" -O2 -static -o "$W/quicecho-musl" "$SELF/quicecho.c" \
124 -I"$W/out/musl/include" $(L musl) -lm
125
126 echo "built:"
127 ls -la "$W/quicecho-native" "$W/quicecho-musl"
128 file "$W/quicecho-musl" | cut -d, -f1-3
spike/quic/quicecho.c
Old New
@@ -1,598 +0,0 @@
1 /* quicecho — M8 Task 1 spike.
2 *
3 * A QUIC echo pair (one binary, two roles) over ngtcp2 + wolfSSL, existing
4 * only to answer one question: can our pinned toolchain ship this? It is
5 * NOT a design sketch for the daemon — no retry/token, no migration, one
6 * connection at a time, minimal error recovery.
7 *
8 * quicecho server <port> <psk-hex>
9 * quicecho client <host> <port> <psk-hex> [reps]
10 *
11 * Authentication is TLS 1.3 external PSK: both sides hold the same 32-byte
12 * key, nobody holds a certificate, and there is no CA anywhere. That is the
13 * property the milestone's deploy model needs ("scp a binary, scp a key").
14 *
15 * The client reports, per rep: handshake completion and first echoed byte,
16 * both measured from the first packet sent. Rep 0 is cold; later reps reuse
17 * the wolfSSL session, which is what the resumed number measures.
18 */
19 #include <arpa/inet.h>
20 #include <assert.h>
21 #include <errno.h>
22 #include <netdb.h>
23 #include <poll.h>
24 #include <stdio.h>
25 #include <stdlib.h>
26 #include <string.h>
27 #include <sys/socket.h>
28 #include <sys/types.h>
29 #include <time.h>
30 #include <unistd.h>
31
32 #include <ngtcp2/ngtcp2.h>
33 #include <ngtcp2/ngtcp2_crypto.h>
34 #include <ngtcp2/ngtcp2_crypto_wolfssl.h>
35
36 #include <wolfssl/options.h>
37 #include <wolfssl/ssl.h>
38
39 #define ALPN "\x08""quicecho"
40 #define PSK_IDENTITY "mux"
41 #define PSK_CIPHERSUITE "TLS13-AES128-GCM-SHA256"
42 #define MAX_PSK 64
43 #define SERVER_SCIDLEN 8
44
45 static uint8_t g_psk[MAX_PSK];
46 static size_t g_psklen;
47
48 static uint64_t timestamp_ns(void) {
49 struct timespec tp;
50 clock_gettime(CLOCK_MONOTONIC, &tp);
51 return (uint64_t)tp.tv_sec * NGTCP2_SECONDS + (uint64_t)tp.tv_nsec;
52 }
53
54 static int hex2bin(const char *hex, uint8_t *out, size_t outcap, size_t *outlen) {
55 size_t n = strlen(hex), i;
56 if (n % 2 || n / 2 > outcap) return -1;
57 for (i = 0; i < n; i += 2) {
58 unsigned v;
59 if (sscanf(hex + i, "%2x", &v) != 1) return -1;
60 out[i / 2] = (uint8_t)v;
61 }
62 *outlen = n / 2;
63 return 0;
64 }
65
66 /* ---- PSK callbacks -------------------------------------------------- */
67
68 static unsigned int psk_client_cb(WOLFSSL *ssl, const char *hint, char *identity,
69 unsigned int id_max, unsigned char *key,
70 unsigned int key_max, const char **ciphersuite) {
71 (void)ssl;
72 (void)hint;
73 if (id_max < sizeof(PSK_IDENTITY) || key_max < g_psklen) return 0;
74 strcpy(identity, PSK_IDENTITY);
75 memcpy(key, g_psk, g_psklen);
76 *ciphersuite = PSK_CIPHERSUITE;
77 return (unsigned int)g_psklen;
78 }
79
80 static unsigned int psk_server_cb(WOLFSSL *ssl, const char *identity,
81 unsigned char *key, unsigned int key_max,
82 const char **ciphersuite) {
83 (void)ssl;
84 if (ciphersuite) *ciphersuite = PSK_CIPHERSUITE;
85 /* Possession of the key IS the authentication; the identity only selects
86 * which key, and this spike has exactly one. */
87 if (!identity || strcmp(identity, PSK_IDENTITY) != 0) return 0;
88 if (key_max < g_psklen) return 0;
89 memcpy(key, g_psk, g_psklen);
90 return (unsigned int)g_psklen;
91 }
92
93 /* ---- shared connection state ---------------------------------------- */
94
95 struct peer {
96 ngtcp2_crypto_conn_ref conn_ref;
97 WOLFSSL_CTX *ssl_ctx;
98 WOLFSSL *ssl;
99 ngtcp2_conn *conn;
100 int fd;
101 struct sockaddr_storage local_addr;
102 socklen_t local_addrlen;
103 struct sockaddr_storage remote_addr;
104 socklen_t remote_addrlen;
105 int64_t stream_id;
106 int handshake_done;
107 int got_echo;
108 int closed;
109 uint64_t t0;
110 uint64_t t_handshake;
111 uint64_t t_echo;
112 const char *payload;
113 size_t payload_len;
114 size_t payload_sent;
115 int is_server;
116 };
117
118 static ngtcp2_conn *get_conn(ngtcp2_crypto_conn_ref *ref) {
119 return ((struct peer *)ref->user_data)->conn;
120 }
121
122 static void rand_cb(uint8_t *dest, size_t destlen, const ngtcp2_rand_ctx *ctx) {
123 (void)ctx;
124 if (wolfSSL_RAND_bytes(dest, (int)destlen) != 1) abort();
125 }
126
127 static int get_new_connection_id_cb(ngtcp2_conn *conn, ngtcp2_cid *cid,
128 ngtcp2_stateless_reset_token *token,
129 size_t cidlen, void *user_data) {
130 (void)conn;
131 (void)user_data;
132 if (wolfSSL_RAND_bytes(cid->data, (int)cidlen) != 1) return NGTCP2_ERR_CALLBACK_FAILURE;
133 cid->datalen = cidlen;
134 if (wolfSSL_RAND_bytes(token->data, NGTCP2_STATELESS_RESET_TOKENLEN) != 1)
135 return NGTCP2_ERR_CALLBACK_FAILURE;
136 return 0;
137 }
138
139 static int handshake_completed_cb(ngtcp2_conn *conn, void *user_data) {
140 struct peer *p = user_data;
141 (void)conn;
142 p->handshake_done = 1;
143 p->t_handshake = timestamp_ns();
144 return 0;
145 }
146
147 static int recv_stream_data_cb(ngtcp2_conn *conn, uint32_t flags,
148 int64_t stream_id, uint64_t offset,
149 const uint8_t *data, size_t datalen,
150 void *user_data, void *stream_user_data) {
151 struct peer *p = user_data;
152 (void)flags;
153 (void)offset;
154 (void)stream_user_data;
155
156 if (p->is_server) {
157 /* Echo: the whole point. Bounce it back on the same stream. */
158 ngtcp2_vec vec = {.base = (uint8_t *)data, .len = datalen};
159 ngtcp2_ssize wcont;
160 uint8_t buf[1500];
161 ngtcp2_path_storage ps;
162 ngtcp2_pkt_info pi;
163 ngtcp2_path_storage_zero(&ps);
164 for (;;) {
165 ngtcp2_ssize n = ngtcp2_conn_writev_stream(
166 conn, &ps.path, &pi, buf, sizeof(buf), &wcont, NGTCP2_WRITE_STREAM_FLAG_NONE,
167 stream_id, &vec, 1, timestamp_ns());
168 if (n <= 0) break;
169 sendto(p->fd, buf, (size_t)n, 0, (struct sockaddr *)&p->remote_addr,
170 p->remote_addrlen);
171 if (wcont > 0) break;
172 }
173 } else {
174 p->got_echo = 1;
175 p->t_echo = timestamp_ns();
176 }
177 return 0;
178 }
179
180 static int extend_max_local_streams_bidi_cb(ngtcp2_conn *conn, uint64_t max_streams,
181 void *user_data) {
182 struct peer *p = user_data;
183 (void)max_streams;
184 if (!p->is_server && p->stream_id == -1) {
185 if (ngtcp2_conn_open_bidi_stream(conn, &p->stream_id, NULL) != 0) p->stream_id = -1;
186 }
187 return 0;
188 }
189
190 /* ---- TLS setup ------------------------------------------------------ */
191
192 static int tls_init(struct peer *p, int is_server) {
193 p->ssl_ctx = wolfSSL_CTX_new(is_server ? wolfTLSv1_3_server_method()
194 : wolfTLSv1_3_client_method());
195 if (!p->ssl_ctx) return -1;
196
197 if (is_server) {
198 if (ngtcp2_crypto_wolfssl_configure_server_context(p->ssl_ctx) != 0) return -1;
199 wolfSSL_CTX_set_psk_server_tls13_callback(p->ssl_ctx, psk_server_cb);
200 wolfSSL_CTX_use_psk_identity_hint(p->ssl_ctx, "");
201 } else {
202 if (ngtcp2_crypto_wolfssl_configure_client_context(p->ssl_ctx) != 0) return -1;
203 wolfSSL_CTX_set_psk_client_tls13_callback(p->ssl_ctx, psk_client_cb);
204 }
205 /* No certificates anywhere: possession of the PSK is the authentication. */
206 wolfSSL_CTX_set_cipher_list(p->ssl_ctx, PSK_CIPHERSUITE);
207
208 p->ssl = wolfSSL_new(p->ssl_ctx);
209 if (!p->ssl) return -1;
210 wolfSSL_set_app_data(p->ssl, &p->conn_ref);
211 wolfSSL_set_quic_use_legacy_codepoint(p->ssl, 0);
212 if (!is_server) {
213 wolfSSL_UseALPN(p->ssl, (char *)ALPN + 1, sizeof(ALPN) - 2, WOLFSSL_ALPN_FAILED_ON_MISMATCH);
214 wolfSSL_UseSNI(p->ssl, WOLFSSL_SNI_HOST_NAME, "quicecho", 8);
215 }
216 return 0;
217 }
218
219 /* ---- write/flush ---------------------------------------------------- */
220
221 static int flush_egress(struct peer *p) {
222 uint8_t buf[1500];
223 ngtcp2_path_storage ps;
224 ngtcp2_pkt_info pi;
225 ngtcp2_ssize wcont;
226 uint64_t ts = timestamp_ns();
227
228 ngtcp2_path_storage_zero(&ps);
229 for (;;) {
230 ngtcp2_vec vec;
231 ngtcp2_vec *pvec = NULL;
232 size_t vcnt = 0;
233 int64_t sid = -1;
234 uint32_t flags = NGTCP2_WRITE_STREAM_FLAG_NONE;
235
236 if (!p->is_server && p->stream_id != -1 && p->payload_sent < p->payload_len) {
237 vec.base = (uint8_t *)p->payload + p->payload_sent;
238 vec.len = p->payload_len - p->payload_sent;
239 pvec = &vec;
240 vcnt = 1;
241 sid = p->stream_id;
242 }
243
244 ngtcp2_ssize n = ngtcp2_conn_writev_stream(p->conn, &ps.path, &pi, buf,
245 sizeof(buf), &wcont, flags, sid,
246 pvec, vcnt, ts);
247 if (n < 0) {
248 if (n == NGTCP2_ERR_WRITE_MORE) {
249 if (wcont > 0) p->payload_sent += (size_t)wcont;
250 continue;
251 }
252 fprintf(stderr, "writev_stream: %s\n", ngtcp2_strerror((int)n));
253 return -1;
254 }
255 if (n == 0) return 0;
256 if (wcont > 0) p->payload_sent += (size_t)wcont;
257
258 if (sendto(p->fd, buf, (size_t)n, 0, (struct sockaddr *)&p->remote_addr,
259 p->remote_addrlen) < 0) {
260 fprintf(stderr, "sendto: %s\n", strerror(errno));
261 return -1;
262 }
263 }
264 }
265
266 static int feed_packet(struct peer *p, const uint8_t *data, size_t datalen,
267 struct sockaddr *from, socklen_t fromlen) {
268 ngtcp2_path path = {
269 .local = {.addr = (struct sockaddr *)&p->local_addr, .addrlen = p->local_addrlen},
270 .remote = {.addr = from, .addrlen = fromlen},
271 };
272 ngtcp2_pkt_info pi = {0};
273 int rv = ngtcp2_conn_read_pkt(p->conn, &path, &pi, data, datalen, timestamp_ns());
274 if (rv != 0) {
275 if (rv != NGTCP2_ERR_DRAINING && rv != NGTCP2_ERR_CLOSING)
276 fprintf(stderr, "read_pkt: %s\n", ngtcp2_strerror(rv));
277 return -1;
278 }
279 return 0;
280 }
281
282 /* Returns milliseconds until ngtcp2 wants attention, capped. */
283 static int poll_timeout_ms(struct peer *p) {
284 ngtcp2_tstamp expiry = ngtcp2_conn_get_expiry(p->conn);
285 uint64_t now = timestamp_ns();
286 if (expiry == UINT64_MAX) return 200;
287 if (expiry <= now) return 0;
288 uint64_t d = (expiry - now) / NGTCP2_MILLISECONDS;
289 return d > 200 ? 200 : (int)d;
290 }
291
292 /* ---- client --------------------------------------------------------- */
293
294 static int client_once(const char *host, const char *port, int rep,
295 WOLFSSL_SESSION *resume, WOLFSSL_SESSION **out_session) {
296 struct addrinfo hints = {.ai_family = AF_UNSPEC, .ai_socktype = SOCK_DGRAM};
297 struct addrinfo *res, *ai;
298 struct peer p;
299 char msg[64];
300 int rv = -1;
301
302 memset(&p, 0, sizeof(p));
303 p.stream_id = -1;
304 p.is_server = 0;
305 p.conn_ref.get_conn = get_conn;
306 p.conn_ref.user_data = &p;
307
308 if (getaddrinfo(host, port, &hints, &res) != 0) return -1;
309 for (ai = res; ai; ai = ai->ai_next) {
310 p.fd = socket(ai->ai_family, ai->ai_socktype, 0);
311 if (p.fd < 0) continue;
312 if (connect(p.fd, ai->ai_addr, ai->ai_addrlen) == 0) break;
313 close(p.fd);
314 p.fd = -1;
315 }
316 if (!ai) { freeaddrinfo(res); return -1; }
317 memcpy(&p.remote_addr, ai->ai_addr, ai->ai_addrlen);
318 p.remote_addrlen = ai->ai_addrlen;
319 freeaddrinfo(res);
320
321 p.local_addrlen = sizeof(p.local_addr);
322 getsockname(p.fd, (struct sockaddr *)&p.local_addr, &p.local_addrlen);
323
324 if (tls_init(&p, 0) != 0) goto out;
325 if (resume) wolfSSL_set_session(p.ssl, resume);
326
327 {
328 ngtcp2_path path = {
329 .local = {.addr = (struct sockaddr *)&p.local_addr, .addrlen = p.local_addrlen},
330 .remote = {.addr = (struct sockaddr *)&p.remote_addr, .addrlen = p.remote_addrlen},
331 };
332 ngtcp2_callbacks cb = {
333 .client_initial = ngtcp2_crypto_client_initial_cb,
334 .recv_crypto_data = ngtcp2_crypto_recv_crypto_data_cb,
335 .encrypt = ngtcp2_crypto_encrypt_cb,
336 .decrypt = ngtcp2_crypto_decrypt_cb,
337 .hp_mask = ngtcp2_crypto_hp_mask_cb,
338 .recv_retry = ngtcp2_crypto_recv_retry_cb,
339 .update_key = ngtcp2_crypto_update_key_cb,
340 .delete_crypto_aead_ctx = ngtcp2_crypto_delete_crypto_aead_ctx_cb,
341 .delete_crypto_cipher_ctx = ngtcp2_crypto_delete_crypto_cipher_ctx_cb,
342 .version_negotiation = ngtcp2_crypto_version_negotiation_cb,
343 .get_path_challenge_data2 = ngtcp2_crypto_get_path_challenge_data2_cb,
344 .rand = rand_cb,
345 .get_new_connection_id2 = get_new_connection_id_cb,
346 .handshake_completed = handshake_completed_cb,
347 .recv_stream_data = recv_stream_data_cb,
348 .extend_max_local_streams_bidi = extend_max_local_streams_bidi_cb,
349 };
350 ngtcp2_cid dcid, scid;
351 ngtcp2_settings settings;
352 ngtcp2_transport_params params;
353
354 dcid.datalen = NGTCP2_MIN_INITIAL_DCIDLEN;
355 scid.datalen = 8;
356 if (wolfSSL_RAND_bytes(dcid.data, (int)dcid.datalen) != 1) goto out;
357 if (wolfSSL_RAND_bytes(scid.data, (int)scid.datalen) != 1) goto out;
358
359 ngtcp2_settings_default(&settings);
360 settings.initial_ts = timestamp_ns();
361 ngtcp2_transport_params_default(&params);
362 params.initial_max_streams_bidi = 8;
363 params.initial_max_stream_data_bidi_local = 256 * 1024;
364 params.initial_max_stream_data_bidi_remote = 256 * 1024;
365 params.initial_max_data = 1024 * 1024;
366
367 if (ngtcp2_conn_client_new(&p.conn, &dcid, &scid, &path, NGTCP2_PROTO_VER_V1,
368 &cb, &settings, &params, NULL, &p) != 0)
369 goto out;
370 ngtcp2_conn_set_tls_native_handle(p.conn, p.ssl);
371 }
372
373 snprintf(msg, sizeof(msg), "quicecho-rep-%d\n", rep);
374 p.payload = msg;
375 p.payload_len = strlen(msg);
376 p.t0 = timestamp_ns();
377
378 if (flush_egress(&p) != 0) goto out;
379
380 while (!p.got_echo) {
381 struct pollfd pfd = {.fd = p.fd, .events = POLLIN};
382 int ready = poll(&pfd, 1, poll_timeout_ms(&p));
383 if (ready < 0) goto out;
384 if (ready > 0) {
385 uint8_t buf[65536];
386 struct sockaddr_storage from;
387 socklen_t fromlen = sizeof(from);
388 ssize_t n = recvfrom(p.fd, buf, sizeof(buf), 0, (struct sockaddr *)&from, &fromlen);
389 if (n > 0 && feed_packet(&p, buf, (size_t)n, (struct sockaddr *)&from, fromlen) != 0)
390 goto out;
391 } else if (ngtcp2_conn_handle_expiry(p.conn, timestamp_ns()) != 0) {
392 goto out;
393 }
394 if (flush_egress(&p) != 0) goto out;
395 if (timestamp_ns() - p.t0 > 10 * NGTCP2_SECONDS) {
396 fprintf(stderr, "rep %d: timed out (handshake_done=%d)\n", rep, p.handshake_done);
397 goto out;
398 }
399 }
400
401 /* Tell the server we are done. Without this it sits in its one-connection
402 * loop and the next rep's Initial never reaches a fresh conn — which is
403 * exactly how the resumed rep failed the first time this ran. */
404 {
405 uint8_t cbuf[1500];
406 ngtcp2_path_storage cps;
407 ngtcp2_pkt_info cpi;
408 ngtcp2_ccerr ccerr;
409 ngtcp2_ssize cn;
410 ngtcp2_path_storage_zero(&cps);
411 ngtcp2_ccerr_default(&ccerr);
412 cn = ngtcp2_conn_write_connection_close(p.conn, &cps.path, &cpi, cbuf,
413 sizeof(cbuf), &ccerr, timestamp_ns());
414 if (cn > 0)
415 sendto(p.fd, cbuf, (size_t)cn, 0, (struct sockaddr *)&p.remote_addr,
416 p.remote_addrlen);
417 }
418
419 printf("#REP %d handshake_ms=%.2f echo_ms=%.2f resumed=%d\n", rep,
420 (double)(p.t_handshake - p.t0) / 1e6, (double)(p.t_echo - p.t0) / 1e6,
421 resume ? 1 : 0);
422 fflush(stdout);
423
424 if (out_session) {
425 WOLFSSL_SESSION *s = wolfSSL_get1_session(p.ssl);
426 if (s) *out_session = s;
427 }
428 rv = 0;
429
430 out:
431 if (p.conn) ngtcp2_conn_del(p.conn);
432 if (p.ssl) wolfSSL_free(p.ssl);
433 if (p.ssl_ctx) wolfSSL_CTX_free(p.ssl_ctx);
434 if (p.fd >= 0) close(p.fd);
435 return rv;
436 }
437
438 /* ---- server --------------------------------------------------------- */
439
440 static int server_run(const char *port) {
441 struct addrinfo hints = {.ai_family = AF_INET, .ai_socktype = SOCK_DGRAM,
442 .ai_flags = AI_PASSIVE};
443 struct addrinfo *res;
444 int fd;
445
446 if (getaddrinfo(NULL, port, &hints, &res) != 0) return -1;
447 fd = socket(res->ai_family, res->ai_socktype, 0);
448 if (fd < 0) { freeaddrinfo(res); return -1; }
449 {
450 int on = 1;
451 setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on));
452 }
453 if (bind(fd, res->ai_addr, res->ai_addrlen) != 0) {
454 fprintf(stderr, "bind: %s\n", strerror(errno));
455 freeaddrinfo(res);
456 close(fd);
457 return -1;
458 }
459 freeaddrinfo(res);
460 fprintf(stderr, "quicecho server listening on udp/%s\n", port);
461
462 for (;;) {
463 /* One connection at a time: spike, not a server. */
464 struct peer p;
465 uint8_t buf[65536];
466 struct sockaddr_storage from;
467 socklen_t fromlen = sizeof(from);
468 ssize_t n;
469 ngtcp2_pkt_hd hd;
470 int alive = 1;
471
472 n = recvfrom(fd, buf, sizeof(buf), 0, (struct sockaddr *)&from, &fromlen);
473 if (n <= 0) continue;
474 if (ngtcp2_accept(&hd, buf, (size_t)n) != 0) continue;
475
476 memset(&p, 0, sizeof(p));
477 p.fd = fd;
478 p.is_server = 1;
479 p.stream_id = -1;
480 p.conn_ref.get_conn = get_conn;
481 p.conn_ref.user_data = &p;
482 memcpy(&p.remote_addr, &from, fromlen);
483 p.remote_addrlen = fromlen;
484 p.local_addrlen = sizeof(p.local_addr);
485 getsockname(fd, (struct sockaddr *)&p.local_addr, &p.local_addrlen);
486
487 if (tls_init(&p, 1) != 0) continue;
488
489 {
490 ngtcp2_path path = {
491 .local = {.addr = (struct sockaddr *)&p.local_addr, .addrlen = p.local_addrlen},
492 .remote = {.addr = (struct sockaddr *)&p.remote_addr, .addrlen = p.remote_addrlen},
493 };
494 ngtcp2_callbacks cb = {
495 .recv_client_initial = ngtcp2_crypto_recv_client_initial_cb,
496 .recv_crypto_data = ngtcp2_crypto_recv_crypto_data_cb,
497 .encrypt = ngtcp2_crypto_encrypt_cb,
498 .decrypt = ngtcp2_crypto_decrypt_cb,
499 .hp_mask = ngtcp2_crypto_hp_mask_cb,
500 .update_key = ngtcp2_crypto_update_key_cb,
501 .delete_crypto_aead_ctx = ngtcp2_crypto_delete_crypto_aead_ctx_cb,
502 .delete_crypto_cipher_ctx = ngtcp2_crypto_delete_crypto_cipher_ctx_cb,
503 .version_negotiation = ngtcp2_crypto_version_negotiation_cb,
504 .get_path_challenge_data2 = ngtcp2_crypto_get_path_challenge_data2_cb,
505 .rand = rand_cb,
506 .get_new_connection_id2 = get_new_connection_id_cb,
507 .handshake_completed = handshake_completed_cb,
508 .recv_stream_data = recv_stream_data_cb,
509 };
510 ngtcp2_cid scid;
511 ngtcp2_settings settings;
512 ngtcp2_transport_params params;
513
514 scid.datalen = SERVER_SCIDLEN;
515 if (wolfSSL_RAND_bytes(scid.data, SERVER_SCIDLEN) != 1) goto next;
516
517 ngtcp2_settings_default(&settings);
518 settings.initial_ts = timestamp_ns();
519 ngtcp2_transport_params_default(&params);
520 params.initial_max_streams_bidi = 8;
521 params.initial_max_stream_data_bidi_local = 256 * 1024;
522 params.initial_max_stream_data_bidi_remote = 256 * 1024;
523 params.initial_max_data = 1024 * 1024;
524 params.original_dcid = hd.dcid;
525 params.original_dcid_present = 1;
526
527 if (ngtcp2_conn_server_new(&p.conn, &hd.scid, &scid, &path, hd.version, &cb,
528 &settings, &params, NULL, &p) != 0)
529 goto next;
530 ngtcp2_conn_set_tls_native_handle(p.conn, p.ssl);
531 }
532
533 if (feed_packet(&p, buf, (size_t)n, (struct sockaddr *)&from, fromlen) != 0) goto next;
534 if (flush_egress(&p) != 0) goto next;
535
536 while (alive) {
537 struct pollfd pfd = {.fd = fd, .events = POLLIN};
538 int ready = poll(&pfd, 1, poll_timeout_ms(&p));
539 if (ready < 0) break;
540 if (ready > 0) {
541 fromlen = sizeof(from);
542 n = recvfrom(fd, buf, sizeof(buf), 0, (struct sockaddr *)&from, &fromlen);
543 if (n > 0 && feed_packet(&p, buf, (size_t)n, (struct sockaddr *)&from, fromlen) != 0)
544 alive = 0;
545 } else if (ngtcp2_conn_handle_expiry(p.conn, timestamp_ns()) != 0) {
546 alive = 0;
547 }
548 if (alive && flush_egress(&p) != 0) alive = 0;
549 }
550
551 next:
552 if (p.conn) ngtcp2_conn_del(p.conn);
553 if (p.ssl) wolfSSL_free(p.ssl);
554 if (p.ssl_ctx) wolfSSL_CTX_free(p.ssl_ctx);
555 fprintf(stderr, "connection over; waiting for the next\n");
556 }
557 }
558
559 int main(int argc, char **argv) {
560 wolfSSL_Init();
561
562 if (argc >= 4 && strcmp(argv[1], "server") == 0) {
563 if (hex2bin(argv[3], g_psk, sizeof(g_psk), &g_psklen) != 0) {
564 fprintf(stderr, "bad psk hex\n");
565 return 2;
566 }
567 return server_run(argv[2]) == 0 ? 0 : 1;
568 }
569
570 if (argc >= 5 && strcmp(argv[1], "client") == 0) {
571 int reps = argc >= 6 ? atoi(argv[5]) : 2;
572 WOLFSSL_SESSION *sess = NULL;
573 int i, failed = 0;
574 if (hex2bin(argv[4], g_psk, sizeof(g_psk), &g_psklen) != 0) {
575 fprintf(stderr, "bad psk hex\n");
576 return 2;
577 }
578 for (i = 0; i < reps; i++) {
579 WOLFSSL_SESSION *next = NULL;
580 if (client_once(argv[2], argv[3], i, sess, &next) != 0) {
581 failed = 1;
582 if (next) wolfSSL_SESSION_free(next);
583 break;
584 }
585 if (next) {
586 if (sess) wolfSSL_SESSION_free(sess);
587 sess = next;
588 }
589 }
590 if (sess) wolfSSL_SESSION_free(sess);
591 return failed;
592 }
593
594 fprintf(stderr,
595 "usage: quicecho server <port> <psk-hex>\n"
596 " quicecho client <host> <port> <psk-hex> [reps]\n");
597 return 2;
598 }
try-bar.sh
Old New
@@ -1,20 +0,0 @@
1 #!/bin/sh
2 # Try the always-on label bar by hand: a scratch daemon, an isolated
3 # XDG_STATE_HOME (so nothing lands on your real wall), and a zoomed
4 # attach. The top row is the tile's label bar — session name and state —
5 # which used to vanish the moment only one tile was visible.
6 #
7 # deps/zig/zig build && ./try-bar.sh
8 #
9 # Detach with Ctrl-\ d. The scratch daemon is stopped for you.
10 set -eu
11 cd "$(dirname "$0")"
12 SCRATCH=$(mktemp -d /tmp/mux-try-bar-XXXX)
13 export XDG_STATE_HOME="$SCRATCH/state" XDG_RUNTIME_DIR="$SCRATCH/run"
14 mkdir -p "$XDG_STATE_HOME" "$XDG_RUNTIME_DIR"
15 chmod 700 "$XDG_RUNTIME_DIR"
16 MUX=$PWD/zig-out/bin/mux
17 "$MUX" d start -d --sock "$SCRATCH/d.sock"
18 "$MUX" --sock "$SCRATCH/d.sock" || true
19 "$MUX" d stop --sock "$SCRATCH/d.sock" || true
20 rm -rf "$SCRATCH"
try-push.sh
Old New
@@ -1,77 +0,0 @@
1 #!/bin/sh
2 # try-push.sh — watch `mux d upgrade HOST` work against a sandboxed "remote".
3 #
4 # The "remote" is a scratch HOME on this box behind a fake ssh, exactly the
5 # e2e 16_push rig with narration: it has its own ~/.local/bin/mux (this
6 # build, with a marker byte tail so replacement is visible), its own runtime
7 # dir, and a daemon started from that installed binary. Everything is
8 # isolated — nothing touches your real daemon, hosts file, or state.
9 #
10 # Run from the repo root after `make build` (or `deps/zig/zig build`).
11 set -eu
12
13 MUX="$(pwd)/zig-out/bin/mux"
14 [ -x "$MUX" ] || { echo "try-push: build first ($MUX missing)"; exit 1; }
15
16 SCRATCH=$(mktemp -d /tmp/mux-try-push.XXXXXX)
17 REMOTE="$SCRATCH/remote-home"
18 BARE="$SCRATCH/bare-home"
19 INST="$REMOTE/.local/bin/mux"
20 DPID=""
21 cleanup() {
22 [ -n "$DPID" ] && kill "$DPID" 2>/dev/null
23 rm -rf "$SCRATCH"
24 }
25 trap cleanup EXIT INT TERM
26
27 # The pusher side is isolated too: a hand rig that dials default sockets
28 # with the real HOME leaves dead lines on the real wall (CLAUDE.md says so).
29 XDG_STATE_HOME="$SCRATCH/state" XDG_RUNTIME_DIR="$SCRATCH/run"
30 XDG_CONFIG_HOME="$SCRATCH/cfg" XDG_CACHE_HOME="$SCRATCH/cache"
31 export XDG_STATE_HOME XDG_RUNTIME_DIR XDG_CONFIG_HOME XDG_CACHE_HOME
32 mkdir -p "$SCRATCH/run" "$REMOTE/.local/bin" "$REMOTE/run" "$REMOTE/state" "$BARE"
33
34 # The fake ssh: drops options and the host word, swaps in the remote's HOME
35 # and a bare PATH, and hands the command word to a shell — what sshd does.
36 mkdir -p "$SCRATCH/bin"
37 cat > "$SCRATCH/bin/ssh" <<SHIM
38 #!/bin/sh
39 while [ \$# -gt 0 ]; do case "\$1" in -o) shift 2;; -*) shift;; *) break;; esac; done
40 shift
41 HOME="\${FAKE_HOME:-$REMOTE}" \\
42 XDG_RUNTIME_DIR="\${FAKE_HOME:-$REMOTE}/run" \\
43 XDG_STATE_HOME="\${FAKE_HOME:-$REMOTE}/state" \\
44 XDG_CONFIG_HOME="\${FAKE_HOME:-$REMOTE}/cfg" \\
45 XDG_CACHE_HOME="\${FAKE_HOME:-$REMOTE}/cache" \\
46 SHELL=/bin/sh PATH=/usr/bin:/bin exec /bin/sh -c "\$*"
47 SHIM
48 chmod +x "$SCRATCH/bin/ssh"
49 PATH="$SCRATCH/bin:$PATH"
50
51 # The remote's installed mux: this build plus a marker tail, so "replaced"
52 # is a fact you can see and not two identical files agreeing.
53 cp "$MUX" "$INST"
54 printf 'OLDIMAGE' >> "$INST"
55 chmod 755 "$INST"
56
57 echo "== 1. a box with no mux is refused (nothing gets invented) =="
58 FAKE_HOME="$BARE" "$MUX" d upgrade demobox || true
59 echo
60
61 echo "== 2. mux installed, no daemon: the push lands, nothing starts =="
62 echo " installed tail before: $(tail -c 8 "$INST")"
63 "$MUX" d upgrade demobox
64 cmp -s "$INST" "$MUX" && echo " installed mux is now byte-for-byte this build"
65 echo
66
67 echo "== 3. live daemon: push + in-place exec, same pid =="
68 printf 'OLDIMAGE' >> "$INST" # marked again so this push provably replaces
69 env HOME="$REMOTE" XDG_RUNTIME_DIR="$REMOTE/run" XDG_STATE_HOME="$REMOTE/state" \
70 XDG_CONFIG_HOME="$REMOTE/cfg" XDG_CACHE_HOME="$REMOTE/cache" SHELL=/bin/sh \
71 "$INST" d start > "$SCRATCH/daemon.log" 2>&1 &
72 DPID=$!
73 for _ in $(seq 50); do [ -S "$REMOTE/run/muxd.sock" ] && break; sleep 0.1; done
74 echo " remote daemon pid $DPID, exe: $(readlink "/proc/$DPID/exe")"
75 "$MUX" d upgrade demobox --allow-same-version
76 echo " remote daemon pid $DPID, exe: $(readlink "/proc/$DPID/exe")"
77 cmp -s "$INST" "$MUX" && echo " installed mux replaced; same pid kept serving"
try.sh
Old New
@@ -1,144 +0,0 @@
1 #!/usr/bin/env bash
2 # Demo for "the wall is the layout": two scratch daemons standing in for two
3 # machines, three sessions each, and two DEVICES (two state dirs) looking at
4 # them. Device one's layout names three of the six sessions; device two has
5 # never connected. Nothing here touches your real daemon, hosts file or
6 # layout: every path is under one scratch directory.
7 #
8 # Run from the repo root after `deps/zig/zig build`:
9 # ./try.sh drives you through it on your own terminal
10 # ./try.sh --auto the same on a hidden pty; prints what each wall showed
11 #
12 # Keys once a wall is up: Ctrl-\ s picks a host, Enter lists its sessions,
13 # Enter on one adds it as a pane, c starts a new session there, x ends one
14 # (asks first when someone else holds it), Esc backs out. On the wall,
15 # Ctrl-\ x takes the focused pane off THIS wall and ends nothing; Ctrl-\ d
16 # detaches.
17 set -u
18
19 cd "$(dirname "$0")" || exit 1
20 MUX=$PWD/zig-out/bin/mux
21 PTY=$PWD/zig-out/bin/ptyclient
22 [ -x "$MUX" ] || { echo "try: build first (deps/zig/zig build); $MUX missing"; exit 1; }
23 AUTO=0
24 [ "${1:-}" = "--auto" ] && AUTO=1
25
26 SCRATCH=$(mktemp -d /tmp/try.XXXX)
27 RUN=$SCRATCH/run
28 mkdir -p "$RUN"
29 chmod 700 "$RUN"
30 export XDG_RUNTIME_DIR=$RUN XDG_CONFIG_HOME=$SCRATCH/cfg XDG_CACHE_HOME=$SCRATCH/cache HOME=$SCRATCH/home
31 mkdir -p "$XDG_CONFIG_HOME" "$XDG_CACHE_HOME" "$HOME"
32 SOCK_A=$RUN/laptop.sock
33 SOCK_B=$RUN/desk.sock
34 DEV1=$SCRATCH/device-one
35 DEV2=$SCRATCH/device-two
36 FILL=$SCRATCH/fill-state
37
38 cleanup() {
39 "$MUX" d stop --sock "$SOCK_A" >/dev/null 2>&1
40 "$MUX" d stop --sock "$SOCK_B" >/dev/null 2>&1
41 rm -rf "$SCRATCH"
42 }
43 trap cleanup EXIT INT TERM
44
45 say() { printf '\n== %s\n' "$*"; }
46
47 # A daemon starts with session 0; two more are born by a piped attach that
48 # detaches at once (Ctrl-\ twice), the way the e2e suite fills a daemon.
49 fill() {
50 for name in 1 2; do
51 { sleep 1.5; printf '\034\034'; } | XDG_STATE_HOME=$FILL timeout 20 \
52 "$MUX" --sock "$1" --session "$name" >/dev/null 2>&1
53 done
54 }
55
56 say "two daemons, three sessions each"
57 "$MUX" d start -d --sock "$SOCK_A" >/dev/null 2>&1 || { echo "laptop daemon did not start"; exit 1; }
58 "$MUX" d start -d --sock "$SOCK_B" >/dev/null 2>&1 || { echo "desk daemon did not start"; exit 1; }
59 fill "$SOCK_A"; fill "$SOCK_B"
60 "$MUX" d stats --sock "$SOCK_A" | grep -o 'sessions=[0-9]*' | sed 's/^/laptop: /'
61 "$MUX" d stats --sock "$SOCK_B" | grep -o 'sessions=[0-9]*' | sed 's/^/desk: /'
62
63 say "device one knows both daemons and has arranged three panes"
64 mkdir -p "$DEV1/mux" "$DEV2/mux"
65 printf -- '--sock %s\n--sock %s\n' "$SOCK_A" "$SOCK_B" | tee "$DEV1/mux/hosts" > "$DEV2/mux/hosts"
66 {
67 printf 'mux-layout 1\nbeside 0\n'
68 printf ' leaf 1 --sock %s#0\n leaf 1 --sock %s#1\n leaf 1 --sock %s#0\n' "$SOCK_A" "$SOCK_A" "$SOCK_B"
69 printf 'focus 1\n'
70 } > "$DEV1/mux/layout"
71 sed 's/^/ /' "$DEV1/mux/layout"
72 echo "device two has the same hosts file and no layout: it has never connected."
73
74 show_layout() {
75 say "$1's layout file now"
76 if [ -f "$2/mux/layout" ]; then sed 's/^/ /' "$2/mux/layout"; else echo " (none)"; fi
77 }
78
79 if [ "$AUTO" -eq 0 ]; then
80 say "device one: mux opens the three panes; the other three sessions are nowhere."
81 echo " Try: Ctrl-\\ x on the focused pane (it leaves this wall, the session lives on),"
82 echo " Ctrl-\\ s, 2, Enter to see the desk's sessions, Enter on one to add it."
83 echo " Ctrl-\\ d when done."
84 read -r -p " [Enter to open device one] " _
85 XDG_STATE_HOME=$DEV1 "$MUX" || true
86 show_layout "device one" "$DEV1"
87 say "device two: mux opens the picker on an empty wall; add what you want."
88 echo " Try: 2, Enter, then Enter on a session; Ctrl-\\ d when done."
89 read -r -p " [Enter to open device two] " _
90 XDG_STATE_HOME=$DEV2 "$MUX" || true
91 show_layout "device two" "$DEV2"
92 show_layout "device one (unchanged by device two)" "$DEV1"
93 say "who holds what, asked of the daemons"
94 "$MUX" d stats --sock "$SOCK_A" | tr ' ' '\n' | grep -E '^(clients|sessions)=|^session' | paste -sd' ' | sed 's/^/laptop: /'
95 "$MUX" d stats --sock "$SOCK_B" | tr ' ' '\n' | grep -E '^(clients|sessions)=|^session' | paste -sd' ' | sed 's/^/desk: /'
96 exit 0
97 fi
98
99 # --auto: the same journey on a hidden pty, narrated from the captures.
100 [ -x "$PTY" ] || { echo "try --auto: $PTY missing"; exit 1; }
101 A=${SOCK_A##*/}; B=${SOCK_B##*/}
102 said() { grep -aoE "$2" "$1" | sort -u | sed 's/^/ saw: /'; }
103
104 say "device one: a wall of exactly its three panes"
105 XDG_STATE_HOME=$DEV1 timeout 60 "$PTY" --cols 160 --rows 40 --out "$SCRATCH/d1.cap" --err "$SCRATCH/d1.err" -- "$MUX" > "$SCRATCH/d1.pc" 2>&1 <<EOF
106 expect $B#0 15000
107 expect $A#1 15000
108 settle 800 10000
109 send \x1cx
110 expect [pane removed - the session is still on its daemon] 10000
111 settle 800 10000
112 send \x1cd
113 EOF
114 [ "$?" -eq 0 ] || { echo " device one's script FAILED:"; sed 's/^/ /' "$SCRATCH/d1.pc"; echo " stderr:"; sed 's/^/ /' "$SCRATCH/d1.err"; }
115 said "$SCRATCH/d1.cap" "$A#[0-9]|$B#[0-9]|\[pane removed[^]]*\]"
116 for absent in "$A#2" "$B#1" "$B#2"; do
117 if grep -aq "$absent" "$SCRATCH/d1.cap"; then echo " UNEXPECTED: $absent was on device one's wall"; else echo " absent, as it should be: $absent"; fi
118 done
119 show_layout "device one (Ctrl-\\ x took A#1 off it)" "$DEV1"
120
121 say "device two: an empty wall, the picker, and one session added from the desk"
122 XDG_STATE_HOME=$DEV2 timeout 60 "$PTY" --cols 160 --rows 40 --out "$SCRATCH/d2.cap" --err "$SCRATCH/d2.err" -- "$MUX" > "$SCRATCH/d2.pc" 2>&1 <<EOF
123 expect Ctrl-\\\\ s to pick a host 15000
124 send \x1cs
125 settle 600 10000
126 send 2
127 send \r
128 expect client 15000
129 settle 400 10000
130 send j
131 send j
132 send \r
133 expect $B#2 15000
134 settle 1200 10000
135 send \x1cd
136 EOF
137 [ "$?" -eq 0 ] || { echo " device two's script FAILED:"; sed 's/^/ /' "$SCRATCH/d2.pc"; echo " stderr:"; sed 's/^/ /' "$SCRATCH/d2.err"; }
138 said "$SCRATCH/d2.cap" "$A#[0-9]|$B#[0-9]|[0-9]+ clients?|on this wall"
139 show_layout "device two (only what it added)" "$DEV2"
140 show_layout "device one (untouched by device two)" "$DEV1"
141
142 say "the daemons' own account: every session still runs, held only by whoever added it"
143 "$MUX" d stats --sock "$SOCK_A" | tr ' ' '\n' | grep -E '^(clients|sessions)=' | paste -sd' ' | sed 's/^/laptop: /'
144 "$MUX" d stats --sock "$SOCK_B" | tr ' ' '\n' | grep -E '^(clients|sessions)=' | paste -sd' ' | sed 's/^/desk: /'