[P1] cloud-hypervisor spawned with agent-scoped ctx — VMs SIGKILLed on agent restart #1

New issue

Closed

opened 2026-07-02 16:43:26 +00:00 by a73x · 1 comment

a73x commented

2026-07-02 16:43:26 +00:00

(Migrated from git.a73x.sh)

[claude 2026-07-02] Found during architecture review (subsystem dig: cloudhv).

cloudhv.Boot (internal/agent/cloudhv/cloudhv.go:100-105) spawns the cloud-hypervisor process via exec.CommandContext(ctx, p.chBin, ...). That ctx originates from signal.NotifyContext(SIGINT,SIGTERM) in cmd/eitri-agent/main.go:167 and flows unchanged through client.Run -> session -> Engine.Step -> Prov.Boot.

Go's exec.CommandContext sends SIGKILL to the child when ctx is Done. So on any graceful agent stop (systemctl restart / binary upgrade / Ctrl-C) every running VM is hard-killed. This directly contradicts the stated design intent at cloudhv.go:98 ('placed in its own session (Setsid) so it survives an agent restart') — Setsid detaches the process group, but CommandContext still kills it.

Impact: agent restart hard-powers-off all VMs. Persistent VMs get restarted by the reconcile lost-path, but ephemeral VMs are marked 'ephemeral VM lost' (failed) and never restart -> sandbox loss on every routine agent restart. The kill is also racy against process teardown, so behavior is nondeterministic.

Fix: spawn cloud-hypervisor with a background/non-cancelable context (exec.Command or context.Background()), and rely on the reconcile Shutdown/Kill path for VM lifecycle. Add a test asserting the CH process survives cancellation of the ctx passed to Boot.

Migrated from git-collab issue 34ff61e931c7e75485dc8469c1cfddf728e6b31a.

Original author: a73x dev@a73x.sh.

Close reason: Fixed by patch e06905ca

[claude 2026-07-02] Found during architecture review (subsystem dig: cloudhv). cloudhv.Boot (internal/agent/cloudhv/cloudhv.go:100-105) spawns the cloud-hypervisor process via exec.CommandContext(ctx, p.chBin, ...). That ctx originates from signal.NotifyContext(SIGINT,SIGTERM) in cmd/eitri-agent/main.go:167 and flows unchanged through client.Run -> session -> Engine.Step -> Prov.Boot. Go's exec.CommandContext sends SIGKILL to the child when ctx is Done. So on any graceful agent stop (systemctl restart / binary upgrade / Ctrl-C) every running VM is hard-killed. This directly contradicts the stated design intent at cloudhv.go:98 ('placed in its own session (Setsid) so it survives an agent restart') — Setsid detaches the process group, but CommandContext still kills it. Impact: agent restart hard-powers-off all VMs. Persistent VMs get restarted by the reconcile lost-path, but ephemeral VMs are marked 'ephemeral VM lost' (failed) and never restart -> sandbox loss on every routine agent restart. The kill is also racy against process teardown, so behavior is nondeterministic. Fix: spawn cloud-hypervisor with a background/non-cancelable context (exec.Command or context.Background()), and rely on the reconcile Shutdown/Kill path for VM lifecycle. Add a test asserting the CH process survives cancellation of the ctx passed to Boot. --- Migrated from git-collab issue `34ff61e931c7e75485dc8469c1cfddf728e6b31a`. Original author: a73x <dev@a73x.sh>. Close reason: Fixed by patch e06905ca

a73x commented

2026-07-02 17:00:53 +00:00

(Migrated from git.a73x.sh)

[claude 2026-07-02] Picking this up. Approach: Boot() stops using the caller ctx for process lifetime — spawn cloud-hypervisor via exec.Command (no CommandContext) so agent shutdown/ctx-cancel never SIGKILLs the VM; VM lifecycle stays owned by the reconcile Shutdown/Kill path. Regression test: start a fake long-lived process via Boot with a cancelable ctx, cancel it, assert the process is still alive.