No way to see WHY a prediction stalled: capture a profile on a latency spike #42

New issue

Open

opened 2026-08-23 06:50:32 +00:00 by a73x · 2 comments

a73x commented

2026-08-23 06:50:32 +00:00

(Migrated from git.a73x.sh)

[claude 2026-08-23] Reported by the operator.

Symptom

Local session, no WAN. In vim: hit ESC, then b a few times. Three
underlined bbb sit on screen, then everything catches up at once. No
insight into what happened — was the daemon stalled, or did the overlay
simply hold a doomed prediction until its deadline?

What exists today

Aggregate counters only. predict.Counters (src/predict.zig:95) tracks
made / confirmed / contradicted / expired / suppressed, and the client
prints them once at exit (dumpPredictStats, src/interact.zig:1383); the
wall republishes the zoomed tile's set each pass (src/wallview.zig:1514).

Not one of those carries TIME. There is no per-prediction age, no ingress ->
egress timeline, nothing that distinguishes a slow daemon from a prediction
that was never going to be confirmed. predict.zig is deliberately
clock-free — it takes now_ms from the caller (src/predict.zig:172) — so
the instrumentation belongs at the caller, not in the module.

Leading hypothesis for the vim case (needs the tooling to confirm)

Possibly not a latency spike at all:

ESC is refused by predictAt (k.ch < 0x20, src/predict.zig:334) — but
suppress() only counts the refusal. It does NOT flush the queue and does
NOT drop confident. So confidence earned while typing in insert mode
still stands after ESC.
Each b is printable, so it IS predicted at the cursor, and displayed
because the overlay is still confident.
In normal mode vim consumes b and moves the cursor; the predicted cell
never changes to b. Under the "not yet is not wrong" rule
(src/predict.zig:17) the prediction is not refuted — it stays pending
until expire_after_frames (8) or expire_after_ms (1000,
src/predict.zig:152-156) fires.
Then all three expire together: "suddenly everything catches up".

If that is what is happening, the delay is the designed 1s bound, and the
real bug is a policy one (a mode-changing key should cost confidence, or
demote). The point of this issue is that TODAY WE CANNOT TELL.

The ask

An opt-in mechanism that captures a profile when prediction delay spikes.
Sketch, not a decision:

Threshold on a pending prediction's age (or on the interval between
made and its verdict). Crossing it arms one capture.
The record must answer "who was slow": pending age, frames judged,
made_seq vs the applied seq, the verdict that ended it (confirmed /
contradicted / expired), and the overlay's policy state at the time
(ctx, confident, streak) — the last three are what separate the
hypothesis above from a genuine stall.
Daemon side, a matching timeline for the same keystroke: input frame
ingress -> pty write -> pty read -> engine feed -> delta egress. Local
means a unix socket, so any multi-hundred-ms gap is a stall inside a
process, not on the wire.
Bounded and off by default: a capture per spike with a rate limit, not a
continuous trace. Write it somewhere findable — the daemon log, or under
$XDG_STATE_HOME/mux/.

Trap for whoever picks this up

The record MUST stamp the build mode. zig build is Debug and Debug builds
ghostty with its page-integrity check on — ~600x on output-heavy work
(CLAUDE.md, and mux-debug-builds-are-600x-slow). A profile captured in
the dev tree that does not say so will indict the wrong code.

Migrated from git-collab issue 86159ab08389b6b9f449881dc5160c07fceac7dc.

Original author: a73x dev@a73x.sh.

[claude 2026-08-23] Reported by the operator. ## Symptom Local session, no WAN. In vim: hit ESC, then `b` a few times. Three underlined `bbb` sit on screen, then everything catches up at once. No insight into what happened — was the daemon stalled, or did the overlay simply hold a doomed prediction until its deadline? ## What exists today Aggregate counters only. `predict.Counters` (src/predict.zig:95) tracks made / confirmed / contradicted / expired / suppressed, and the client prints them once at exit (`dumpPredictStats`, src/interact.zig:1383); the wall republishes the zoomed tile's set each pass (src/wallview.zig:1514). Not one of those carries TIME. There is no per-prediction age, no ingress -> egress timeline, nothing that distinguishes a slow daemon from a prediction that was never going to be confirmed. `predict.zig` is deliberately clock-free — it takes `now_ms` from the caller (src/predict.zig:172) — so the instrumentation belongs at the caller, not in the module. ## Leading hypothesis for the vim case (needs the tooling to confirm) Possibly not a latency spike at all: - ESC is refused by `predictAt` (`k.ch < 0x20`, src/predict.zig:334) — but `suppress()` only counts the refusal. It does NOT flush the queue and does NOT drop `confident`. So confidence earned while typing in insert mode still stands after ESC. - Each `b` is printable, so it IS predicted at the cursor, and displayed because the overlay is still confident. - In normal mode vim consumes `b` and moves the cursor; the predicted cell never changes to `b`. Under the "not yet is not wrong" rule (src/predict.zig:17) the prediction is not refuted — it stays pending until `expire_after_frames` (8) or `expire_after_ms` (1000, src/predict.zig:152-156) fires. - Then all three expire together: "suddenly everything catches up". If that is what is happening, the delay is the designed 1s bound, and the real bug is a policy one (a mode-changing key should cost confidence, or demote). The point of this issue is that TODAY WE CANNOT TELL. ## The ask An opt-in mechanism that captures a profile when prediction delay spikes. Sketch, not a decision: - Threshold on a pending prediction's age (or on the interval between `made` and its verdict). Crossing it arms one capture. - The record must answer "who was slow": pending age, frames judged, `made_seq` vs the applied seq, the verdict that ended it (confirmed / contradicted / expired), and the overlay's policy state at the time (`ctx`, `confident`, `streak`) — the last three are what separate the hypothesis above from a genuine stall. - Daemon side, a matching timeline for the same keystroke: input frame ingress -> pty write -> pty read -> engine feed -> delta egress. Local means a unix socket, so any multi-hundred-ms gap is a stall inside a process, not on the wire. - Bounded and off by default: a capture per spike with a rate limit, not a continuous trace. Write it somewhere findable — the daemon log, or under `$XDG_STATE_HOME/mux/`. ## Trap for whoever picks this up The record MUST stamp the build mode. `zig build` is Debug and Debug builds ghostty with its page-integrity check on — ~600x on output-heavy work (CLAUDE.md, and [[mux-debug-builds-are-600x-slow]]). A profile captured in the dev tree that does not say so will indict the wrong code. --- Migrated from git-collab issue `86159ab08389b6b9f449881dc5160c07fceac7dc`. Original author: a73x <dev@a73x.sh>.

a73x commented

2026-08-23 10:44:13 +00:00

(Migrated from git.a73x.sh)

Also reported: typing RAPIDLY into that box stalls. Candidate that is not a
latency spike at all — offerPrediction refuses any chunk where
chunk.len != 1 (src/interact.zig:961). Fast typing coalesces at the tty, so
a burst arrives as one multi-byte read and gets zero prediction. The faster
you type, the less prediction you get.

Control not yet run: does the same rapid typing stall in Claude Code OUTSIDE
mux? If it does, prediction is a red herring and the profile should be
pointed at the TUI's own render cost.

[claude 2026-08-23] Repro correction from the operator: it is not vim. It is the Claude Code input box WITH VIM MODE ENABLED — a TUI that repaints its own frame per keystroke. The keystroke mechanism is the same (normal-mode `b`, `w`, `j` are non-echoing printable bytes, predicted and displayed on confidence earned in insert mode, retired only at expire_after_ms), but the cost is worse: the delta for one keystroke is already a frame-sized repaint from the TUI, and a contradiction adds `renderClipped` over the whole viewport on top of it (src/interact.zig:1691). Also reported: typing RAPIDLY into that box stalls. Candidate that is not a latency spike at all — `offerPrediction` refuses any chunk where `chunk.len != 1` (src/interact.zig:961). Fast typing coalesces at the tty, so a burst arrives as one multi-byte read and gets zero prediction. The faster you type, the less prediction you get. Control not yet run: does the same rapid typing stall in Claude Code OUTSIDE mux? If it does, prediction is a red herring and the profile should be pointed at the TUI's own render cost.

a73x commented

2026-08-23 11:19:06 +00:00

(Migrated from git.a73x.sh)

[claude 2026-08-23] First increment landed on branch worktree-predict-local-gate
(aac3c2d): the overlay measures its own keystroke-to-confirm round trip and
hides predictions below 20ms srtt (shows again above 30ms; mosh's pair). On a
local socket nothing underlined is ever painted, so the "bbb" cannot occur.
Predictions stay queued and judged while hidden, and the stats line has a
new local= counter. make check + full e2e green; installed to ~/.local/bin
for the operator's hands-on.

Still open from this thread, deliberately deferred: the full-screen repaint
on expiry fires even when nothing was painted (per-cell rollback, item 3),
ESC-demotes-confidence, coalesced bursts get zero prediction, and the
profile capture this issue is actually about.

[claude 2026-08-23] First increment landed on branch worktree-predict-local-gate (aac3c2d): the overlay measures its own keystroke-to-confirm round trip and hides predictions below 20ms srtt (shows again above 30ms; mosh's pair). On a local socket nothing underlined is ever painted, so the "bbb" cannot occur. Predictions stay queued and judged while hidden, and the stats line has a new `local=` counter. make check + full e2e green; installed to ~/.local/bin for the operator's hands-on. Still open from this thread, deliberately deferred: the full-screen repaint on expiry fires even when nothing was painted (per-cell rollback, item 3), ESC-demotes-confidence, coalesced bursts get zero prediction, and the profile capture this issue is actually about.