VMs cannot move between hosts #73

New issue

Open

opened 2026-08-21 18:31:10 +00:00 by a73x · 0 comments

a73x commented

2026-08-21 18:31:10 +00:00

(Migrated from git.a73x.sh)

[claude 2026-08-21] Noticed while tearing down the demo VMs on onyx. A VM is
pinned to the host it was created on for life: host_id is set at create and
nothing in the API or the agent can change it. The only way to "move" a workload
today is to delete it and rebuild it by hand on another host, which loses the
disk.

This blocked nothing today (the onyx VMs were disposable) but it is the reason
teardown is the only option: there is no way to say "keep this, put it on
charizard".

Open question — the shape is undecided

Three candidate mechanisms, materially different in cost and in what they
require of the backend:

Live migrate — cloud-hypervisor supports it, but it needs shared or
replicated storage between hosts and a migration channel. Highest value,
by far the most infrastructure. Nothing in the fleet has shared storage.
Stop → copy disk → recreate on target — cold migration. Works with what
exists today (each agent owns local disks under AGENT_STATE_DIR); needs a
host-to-host copy path and a way to re-point a VM row at a new host. Downtime
is the copy time.
Export / import as an image — the loosest coupling: snapshot a VM's disk
to an image, create a fresh VM from it anywhere. Not migration proper, but
it covers "I want this workload on a different box" and composes with the
existing image_url create path.

(3) is closest to what already exists; (2) is the one people usually mean by
"migration"; (1) is a real project. Needs a decision before any of it is worth
planning.

Note: VM snapshots are already SHELVED pending a desired-state model, and (3)
overlaps with them — resolving that overlap is part of the decision.

Migrated from git-collab issue e698610ccb62e5ac419bed2dc6d3953dd3732680.

Original author: a73x dev@a73x.sh.

[claude 2026-08-21] Noticed while tearing down the demo VMs on onyx. A VM is pinned to the host it was created on for life: `host_id` is set at create and nothing in the API or the agent can change it. The only way to "move" a workload today is to delete it and rebuild it by hand on another host, which loses the disk. This blocked nothing today (the onyx VMs were disposable) but it is the reason teardown is the only option: there is no way to say "keep this, put it on charizard". ## Open question — the shape is undecided Three candidate mechanisms, materially different in cost and in what they require of the backend: 1. **Live migrate** — cloud-hypervisor supports it, but it needs shared or replicated storage between hosts and a migration channel. Highest value, by far the most infrastructure. Nothing in the fleet has shared storage. 2. **Stop → copy disk → recreate on target** — cold migration. Works with what exists today (each agent owns local disks under AGENT_STATE_DIR); needs a host-to-host copy path and a way to re-point a VM row at a new host. Downtime is the copy time. 3. **Export / import as an image** — the loosest coupling: snapshot a VM's disk to an image, create a fresh VM from it anywhere. Not migration proper, but it covers "I want this workload on a different box" and composes with the existing image_url create path. (3) is closest to what already exists; (2) is the one people usually mean by "migration"; (1) is a real project. Needs a decision before any of it is worth planning. Note: VM snapshots are already SHELVED pending a desired-state model, and (3) overlaps with them — resolving that overlap is part of the decision. --- Migrated from git-collab issue `e698610ccb62e5ac419bed2dc6d3953dd3732680`. Original author: a73x <dev@a73x.sh>.