a73x

Teardown observability: a quarantined VM reports power_state=running with all actual-state fields blank

open   by a73x

Labels: backlog

[claude 2026-08-21] Found while tearing down the demo VMs on onyx. A tombstoned
VM spends 5 minutes (`-tombstone-grace`, internal/agent/run/cli.go:138) in
quarantine before its disk is destroyed. During that window the API reports the
VM in a way that actively misleads:

- `power_state` still reads `running` — that is stale DESIRED state. The agent
  has already called `Prov.Shutdown` (internal/agent/reconcile/reconcile.go:481);
  the guest is off within seconds of the DELETE.
- `phase`, `actual_power` and `status_detail` all go EMPTY. reconcile.go:350-352
  puts quarantined VMs in `Quarantined[]` rather than `Vms[]`, so the server has
  no ActualVM for them and every actual-state field blanks — precisely during the
  one window where the operator most wants to know what teardown is doing.

Net effect: for 5 minutes the console and API say "running", with no phase and no
detail, for a guest that is already stopped and counting down to disk destruction.

## Repro

    DELETE /api/v1/vms/{id}
    GET /api/v1/vms      # power_state=running, phase="", actual_power="", status_detail=""
                         # lifecycle=deleting, destroy_at=<now+5min>

## Why it matters

I misread this as a stalled teardown and went looking for agent logs. The only
in-band signal that the agent had actually acted was the host's `allocated`
dropping (releaseCompute runs at quarantine) — which requires knowing to check a
different endpoint and knowing each VM's shape to do the arithmetic.

`destroy_at` is present and correct, so the countdown IS on the wire; the
surrounding fields just contradict it.

## Suggested shape (not decided)

A `phase=quarantined` (or `stopping`/`draining`) with `status_detail` carrying the
countdown would answer the question directly. Whether that means synthesising an
ActualVM for quarantined VMs server-side, or widening the Quarantined[] entry, is
the design question. Note the wire has a field-NUMBER lock (#32) — a new field
needs a fresh number.