Revisit mutate-report: threshold, or drop it

open   by a73x

Labels: backlog

[claude 2026-08-22] `make ci` now runs `mutate-report`: gremlins over the diff against origin/main, reported and never failing (the warn tier lint-extra uses). It is deliberately toothless for now — this issue is the reminder to come back and decide whether it grows teeth.

What to check when revisiting:

1. What do real branches score? First observations, for calibration:
   - the 8 unpushed commits on main (74 files, 10k lines): 166 killed / 55 lived / 6 timed out, 75.11% efficacy, 43s
   - whole-tree baseline for comparison: 88.9% efficacy (see the baseline issue)
   A diff scores well below the tree, which is the point — recently changed lines are the least asserted.

2. Is 43s per push acceptable? It scales with diff size, not tree size, so it should stay flat. Watch for a branch where it does not.

3. Can a threshold be set? --threshold-efficacy is the flag. 100% would be wrong: the tree contains equivalent mutants that cannot be killed (internal/cloudinit's `i+1 < len(root.Content)` -> `<=` is the same loop for every even-length YAML mapping). A floor near the observed median, raised over time, is the ratchet shape the coverage gate already uses (scripts/coverage.sh).

4. Does the report get read? A warn tier nobody reads is worse than no tier — it costs time and buys nothing. If three cycles pass with survivors nobody looks at, either gate it or delete it.

Decide by looking at the numbers, not by preference.