a73x

707f1a44

feat: a host tells the fleet which subnet its guests are on

a73x   2026-08-06 09:12

Commit message
feat: a host tells the fleet which subnet its guests are on

The subnet a host's guests live on is a fact about that host, and the host is
the only thing that can observe it. A Linux host builds the bridge, so it knows;
a Mac's guests live wherever vmnet puts them, which no fleet allocation will
ever contain. So the direction reverses: ActualStateReport carries guest_cidr,
the fleet records what it is told, and Hello's bridge_cidr — the host echoing
back an allocation made for it, which no server ever read — is reserved.

It is polled, once per report, rather than resolved at startup and announced.
That is the difference between a fact and a constant: a host's guest network can
change while its agent stays connected, and on a platform whose OS owns that
network it may not be knowable at connect time at all. Resolving it once would
mean a host that could not answer at startup answers nothing for the life of the
process — the same permanent silence this exists to end, one level up. Capacity
already sits in both messages for this reason: an opening value, then the
ongoing truth.

Empty is "no answer", never "no network", at every layer that handles it. A host
that cannot see its own network leaves the fleet's record alone rather than
erasing it, exactly as an unreported guest address leaves assigned_ip alone.

The agent resolves its own subnet through one function that join and serve both
call, and persists the answer. They are separate processes that sourced it
differently — join never had one, serve read the identity — so
`eitri-agent join --bridge-cidr X` followed by a plain `eitri-agent` could
re-home a host on its next start, moving the bridge under live guests. The
durable record outranks the flag for the same reason: a deliberate re-home is an
edit to identity.json, not something a flag does by accident.

The last-resort subnet lives with the Linux backend rather than in the resolver.
A fabricated network is only ever right where something is about to create the
network it names; a host whose OS already owns one must answer "" and go and
look.

Server-side, the value is sanity-checked and not second-guessed: it must parse
as an IPv4 prefix, the same check netenv already makes before handing one to the
kernel, applied on the receiving side so a garbage string cannot reach the
console. It is never checked against the pool — a host's guest subnet is that
host's business.

The write is guarded per host in memory. Every host reports every tick forever
and the store runs on a single connection, so the SQL's own WHERE clause, which
makes a concurrent reconnect harmless, would still spend a round trip on that
connection for every host for the life of the fleet.
internal/agent/run/cli.go
Old New
@@ -47,6 +47,7 @@ func hostRunner(ctx context.Context, name string, args ...string) (string, error
47 type Config struct { 47 type Config struct {
48 StateDir, CHBin, Firmware string 48 StateDir, CHBin, Firmware string
49 VfkitBin string 49 VfkitBin string
50 BridgeCIDR string
50 BootstrapURL string 51 BootstrapURL string
51 TombstoneGrace, VanishGrace time.Duration 52 TombstoneGrace, VanishGrace time.Duration
52 VMTimeout time.Duration 53 VMTimeout time.Duration
@@ -102,6 +103,7 @@ func parseConfig(args []string) (Config, []string, error) {
102 // Silicon and /usr/local/bin on Intel, so $PATH is the only answer that is 103 // Silicon and /usr/local/bin on Intel, so $PATH is the only answer that is
103 // right on both. 104 // right on both.
104 vfkitBin := fs.String("vfkit-bin", "vfkit", "path to the vfkit binary (macOS hosts)") 105 vfkitBin := fs.String("vfkit-bin", "vfkit", "path to the vfkit binary (macOS hosts)")
106 bridgeCIDR := fs.String("bridge-cidr", "", "the subnet this host's guests are on (Linux); empty adopts the control plane's suggestion at enrollment. Ignored where the host OS owns the guest network")
105 bootstrapURL := fs.String("bootstrap-url", "https://eitri.sh/dl/latest/manifest.json", "eitri.sh release manifest to fetch cloud-hypervisor/firmware from if missing at startup (empty disables bootstrap)") 107 bootstrapURL := fs.String("bootstrap-url", "https://eitri.sh/dl/latest/manifest.json", "eitri.sh release manifest to fetch cloud-hypervisor/firmware from if missing at startup (empty disables bootstrap)")
106 tombstoneGrace := fs.Duration("tombstone-grace", 5*time.Minute, "quarantine period for tombstoned VMs before destroy") 108 tombstoneGrace := fs.Duration("tombstone-grace", 5*time.Minute, "quarantine period for tombstoned VMs before destroy")
107 vanishGrace := fs.Duration("vanish-grace", time.Hour, "quarantine period for VMs that vanished without a tombstone") 109 vanishGrace := fs.Duration("vanish-grace", time.Hour, "quarantine period for VMs that vanished without a tombstone")
@@ -135,6 +137,7 @@ func parseConfig(args []string) (Config, []string, error) {
135 CHBin: *chBin, 137 CHBin: *chBin,
136 Firmware: *firmware, 138 Firmware: *firmware,
137 VfkitBin: *vfkitBin, 139 VfkitBin: *vfkitBin,
140 BridgeCIDR: *bridgeCIDR,
138 BootstrapURL: *bootstrapURL, 141 BootstrapURL: *bootstrapURL,
139 TombstoneGrace: *tombstoneGrace, 142 TombstoneGrace: *tombstoneGrace,
140 VanishGrace: *vanishGrace, 143 VanishGrace: *vanishGrace,
@@ -209,7 +212,7 @@ func serve(st *state.Store, cfg Config) error {
209 // from the live agent without bouncing the process. 212 // from the live agent without bouncing the process.
210 covsnap.Install(ctx) 213 covsnap.Install(ctx)
211 214
212 plat, err := newPlatform(ctx, cfg, st, id.BridgeCIDR) 215 plat, err := newPlatform(ctx, cfg, st)
213 if err != nil { 216 if err != nil {
214 return err 217 return err
215 } 218 }
@@ -276,6 +279,7 @@ func serve(st *state.Store, cfg Config) error {
276 Identity: id, 279 Identity: id,
277 StateDir: cfg.StateDir, 280 StateDir: cfg.StateDir,
278 Provisioner: platformProvisioner, 281 Provisioner: platformProvisioner,
282 GuestCIDR: plat.GuestCIDR,
279 Runner: hostRunner, 283 Runner: hostRunner,
280 Console: pumps, 284 Console: pumps,
281 MaxVCPUs: cfg.MaxVCPUs, 285 MaxVCPUs: cfg.MaxVCPUs,
@@ -283,7 +287,7 @@ func serve(st *state.Store, cfg Config) error {
283 MaxDiskGB: cfg.MaxDiskGB, 287 MaxDiskGB: cfg.MaxDiskGB,
284 } 288 }
285 289
286 slog.Info("agent started", "host_id", id.HostID, "bridge_cidr", id.BridgeCIDR) 290 slog.Info("agent started", "host_id", id.HostID, "guest_cidr", plat.GuestCIDR())
287 client.Run(ctx) 291 client.Run(ctx)
288 292
289 // client.Run blocks until ctx is cancelled (SIGINT/SIGTERM) and only then 293 // client.Run blocks until ctx is cancelled (SIGINT/SIGTERM) and only then
internal/agent/run/guestcidr.go
Old New
@@ -0,0 +1,60 @@
1 package run
2
3 import (
4 "log/slog"
5
6 "github.com/a73x/eitri/internal/agent/state"
7 )
8
9 // resolveGuestCIDR decides which subnet this host's guests are on, and persists
10 // the answer the first time it produces one the identity lacks.
11 //
12 // Order, fixed:
13 //
14 // 1. Identity.BridgeCIDR — the durable record, and it wins whenever present.
15 // 2. --bridge-cidr — the operator's explicit opinion at first start.
16 // 3. suggestion — what the control plane offered at enrollment (empty at any
17 // other time; the suggestion only exists in the enroll response).
18 //
19 // Rule 1 outranking the flag is the whole of "re-homing an existing host is not
20 // something this makes easy". Letting a flag override a persisted identity
21 // would move the bridge under live guests, and every one of them would lose its
22 // address on the next restart. A deliberate re-home is an edit to identity.json.
23 //
24 // join and serve both call this, which is the point: they are separate
25 // processes that used to source the subnet differently — join never had one and
26 // serve read the identity — so `eitri-agent join --bridge-cidr X` followed by a
27 // plain `eitri-agent` could silently re-home the host on the next start.
28 //
29 // It returns "" when nothing has an opinion. That is a real answer on a
30 // platform whose OS owns the guest network and takes no suggestion; a platform
31 // that must BUILD one supplies its own last resort, because a fabricated subnet
32 // is only ever right where something is going to create it.
33 func resolveGuestCIDR(st *state.Store, cfg Config, suggestion string) string {
34 if id, ok := st.Identity(); ok && id.BridgeCIDR != "" {
35 return id.BridgeCIDR
36 }
37 for _, candidate := range []string{cfg.BridgeCIDR, suggestion} {
38 if candidate == "" {
39 continue
40 }
41 persistGuestCIDR(st, candidate)
42 return candidate
43 }
44 return ""
45 }
46
47 // persistGuestCIDR writes the resolved subnet into the durable identity so the
48 // next start reads it from rule 1 rather than re-deriving it. Best-effort: a
49 // failed write leaves the agent running on the value it resolved, and the next
50 // start resolves the same way from the same inputs.
51 func persistGuestCIDR(st *state.Store, cidr string) {
52 id, ok := st.Identity()
53 if !ok || id.BridgeCIDR == cidr {
54 return
55 }
56 id.BridgeCIDR = cidr
57 if err := st.SaveIdentity(id); err != nil {
58 slog.Warn("persist guest cidr", "cidr", cidr, "err", err)
59 }
60 }
internal/agent/run/guestcidr_test.go
Old New
@@ -0,0 +1,64 @@
1 package run
2
3 import (
4 "testing"
5
6 "github.com/a73x/eitri/internal/agent/state"
7 "github.com/stretchr/testify/assert"
8 "github.com/stretchr/testify/require"
9 )
10
11 func storeWithIdentity(t *testing.T, cidr string) *state.Store {
12 t.Helper()
13 st, err := state.Open(t.TempDir())
14 require.NoError(t, err)
15 require.NoError(t, st.SaveIdentity(state.Identity{HostID: "h1", BridgeCIDR: cidr}))
16 return st
17 }
18
19 // TestResolveGuestCIDRPrecedence pins D2's ladder, and in particular that a
20 // persisted identity outranks the flag. Letting a flag win would move the
21 // bridge under live guests, and every one of them would lose its address on the
22 // next restart — so a deliberate re-home is an edit to identity.json, not a
23 // flag.
24 func TestResolveGuestCIDRPrecedence(t *testing.T) {
25 t.Run("the durable record wins over everything", func(t *testing.T) {
26 st := storeWithIdentity(t, "10.20.0.0/24")
27 got := resolveGuestCIDR(st, Config{BridgeCIDR: "10.30.0.0/24"}, "10.40.0.0/24")
28 assert.Equal(t, "10.20.0.0/24", got)
29 })
30
31 t.Run("the flag wins over the suggestion", func(t *testing.T) {
32 st := storeWithIdentity(t, "")
33 got := resolveGuestCIDR(st, Config{BridgeCIDR: "10.30.0.0/24"}, "10.40.0.0/24")
34 assert.Equal(t, "10.30.0.0/24", got)
35 })
36
37 t.Run("the suggestion is taken when nothing else has an opinion", func(t *testing.T) {
38 st := storeWithIdentity(t, "")
39 assert.Equal(t, "10.40.0.0/24", resolveGuestCIDR(st, Config{}, "10.40.0.0/24"))
40 })
41
42 // "" is a real answer, not a failure: a platform whose OS owns the guest
43 // network takes no suggestion and goes and looks instead.
44 t.Run("nothing has an opinion", func(t *testing.T) {
45 st := storeWithIdentity(t, "")
46 assert.Empty(t, resolveGuestCIDR(st, Config{}, ""))
47 })
48 }
49
50 // TestResolveGuestCIDRPersists pins that join and serve cannot disagree. They
51 // are separate processes that used to source the subnet differently, so
52 // `eitri-agent join --bridge-cidr X` followed by a plain `eitri-agent` could
53 // re-home the host on its next start.
54 func TestResolveGuestCIDRPersists(t *testing.T) {
55 st := storeWithIdentity(t, "")
56 require.Equal(t, "10.30.0.0/24", resolveGuestCIDR(st, Config{BridgeCIDR: "10.30.0.0/24"}, ""))
57
58 id, ok := st.Identity()
59 require.True(t, ok)
60 assert.Equal(t, "10.30.0.0/24", id.BridgeCIDR, "the resolved subnet must be durable")
61
62 // A later start with no flag at all resolves the same, from rule 1.
63 assert.Equal(t, "10.30.0.0/24", resolveGuestCIDR(st, Config{}, ""))
64 }
internal/agent/run/wire_darwin.go
Old New
@@ -21,6 +21,9 @@ const platformProvisioner = "vfkit"
21 type platform struct { 21 type platform struct {
22 Prov reconcile.Provisioner 22 Prov reconcile.Provisioner
23 Pumps *serialpump.Manager 23 Pumps *serialpump.Manager
24 // GuestCIDR reports the subnet this host's guests are on, asked once per
25 // report rather than resolved at startup. See the sync client's field.
26 GuestCIDR func() string
24 } 27 }
25 28
26 // newPlatform builds the macOS backend: vfkit over Apple's 29 // newPlatform builds the macOS backend: vfkit over Apple's
@@ -29,17 +32,20 @@ type platform struct {
29 // It does nothing before returning, and the two things it does not do are the 32 // It does nothing before returning, and the two things it does not do are the
30 // whole difference from Linux. There is no host networking to set up: vmnet's 33 // whole difference from Linux. There is no host networking to set up: vmnet's
31 // NAT and its bootpd are already running, own the subnet, and hand out the 34 // NAT and its bootpd are already running, own the subnet, and hand out the
32 // addresses — which is why bridgeCIDR, the CIDR the server assigned this host, 35 // addresses — so this host observes its guest network rather than building one.
33 // is accepted for signature parity and then ignored. And there is no runtime to 36 // And there is no runtime to bootstrap: cloud-hypervisor is a binary the agent
34 // bootstrap: cloud-hypervisor is a binary the agent can fetch and install, 37 // can fetch and install, while vfkit works only if it carries Apple's
35 // while vfkit works only if it carries Apple's virtualization entitlement, and 38 // virtualization entitlement, and an entitlement lives in a signature we cannot
36 // an entitlement lives in a signature we cannot produce. So a Mac without vfkit 39 // produce. So a Mac without vfkit is not one the agent can fix at startup — it
37 // is not one the agent can fix at startup — it is one whose provisioner refuses 40 // is one whose provisioner refuses at Preflight, in a sentence naming the
38 // at Preflight, in a sentence naming the install. 41 // install.
39 func newPlatform(_ context.Context, cfg Config, st *state.Store, _ string) (platform, error) { 42 func newPlatform(_ context.Context, cfg Config, st *state.Store) (platform, error) {
40 prov := vfkit.New(st, cfg.VfkitBin, hostRunner) 43 prov := vfkit.New(st, cfg.VfkitBin, hostRunner)
41 pumps := serialpump.NewManager(vfkit.NewConsoleSource(prov.SocketPath), st.SerialLogPath) 44 pumps := serialpump.NewManager(vfkit.NewConsoleSource(prov.SocketPath), st.SerialLogPath)
42 prov.Pumps = pumps 45 prov.Pumps = pumps
43 46
44 return platform{Prov: prov, Pumps: pumps}, nil 47 // Reading vmnet's subnet lands in a later commit; until then this host has
48 // no answer, which by the empty-means-no-answer rule leaves the fleet's
49 // record alone rather than erasing it.
50 return platform{Prov: prov, Pumps: pumps, GuestCIDR: func() string { return "" }}, nil
45 } 51 }
internal/agent/run/wire_linux.go
Old New
@@ -40,15 +40,33 @@ const platformProvisioner = "cloudhv"
40 type platform struct { 40 type platform struct {
41 Prov reconcile.Provisioner 41 Prov reconcile.Provisioner
42 Pumps *serialpump.Manager 42 Pumps *serialpump.Manager
43 // GuestCIDR reports the subnet this host's guests are on, asked once per
44 // report. Linux genuinely knows at startup — it builds the bridge — so this
45 // returns a constant; the poll shape costs it nothing and lets a platform
46 // whose OS owns the network answer late, or not at all.
47 GuestCIDR func() string
43 } 48 }
44 49
50 // lastResortGuestCIDR is used only when nothing else produced one: no persisted
51 // identity, no flag, and no suggestion from the control plane. It is a last
52 // resort and not "the default" — a host that reaches it is a host the fleet
53 // never advised, and two of them on one network would collide.
54 //
55 // It lives here, not in the resolver, because a fabricated subnet is only ever
56 // right on a platform that is about to CREATE the network it names. A host
57 // whose OS already owns the guest network must answer "" and go and look.
58 const lastResortGuestCIDR = "10.77.1.0/24"
59
45 // newPlatform builds the Linux backend: cloud-hypervisor over a bridge and tap, 60 // newPlatform builds the Linux backend: cloud-hypervisor over a bridge and tap,
46 // with the guest console on cloud-hypervisor's serial socket. bridgeCIDR is the 61 // with the guest console on cloud-hypervisor's serial socket. A bare host that
47 // server-assigned CIDR from the enrolled identity (state.Store.Identity), not a 62 // just joined has neither the hypervisor nor its UEFI firmware, so newPlatform
48 // Config flag; a second platform's newPlatform must source it the same way. A 63 // installs them before anything can try to launch a VM.
49 // bare host that just joined has neither the hypervisor nor its UEFI firmware, 64 func newPlatform(ctx context.Context, cfg Config, st *state.Store) (platform, error) {
50 // so newPlatform installs them before anything can try to launch a VM. 65 bridgeCIDR := resolveGuestCIDR(st, cfg, "")
51 func newPlatform(ctx context.Context, cfg Config, st *state.Store, bridgeCIDR string) (platform, error) { 66 if bridgeCIDR == "" {
67 bridgeCIDR = lastResortGuestCIDR
68 persistGuestCIDR(st, bridgeCIDR)
69 }
52 net, err := netenv.New(hostRunner, bridgeCIDR) 70 net, err := netenv.New(hostRunner, bridgeCIDR)
53 if err != nil { 71 if err != nil {
54 return platform{}, fmt.Errorf("netenv init: %w", err) 72 return platform{}, fmt.Errorf("netenv init: %w", err)
@@ -79,5 +97,6 @@ func newPlatform(ctx context.Context, cfg Config, st *state.Store, bridgeCIDR st
79 pumps := serialpump.NewManager(cloudhv.ConsoleSource(st.SerialSocketPath), st.SerialLogPath) 97 pumps := serialpump.NewManager(cloudhv.ConsoleSource(st.SerialSocketPath), st.SerialLogPath)
80 prov.Pumps = pumps 98 prov.Pumps = pumps
81 99
82 return platform{Prov: prov, Pumps: pumps}, nil 100 return platform{Prov: prov, Pumps: pumps,
101 GuestCIDR: func() string { return bridgeCIDR }}, nil
83 } 102 }
internal/agent/syncclient/client.go
Old New
@@ -64,6 +64,10 @@ type Client struct {
64 // server stores comes from enroll, not from here. 64 // server stores comes from enroll, not from here.
65 Provisioner string 65 Provisioner string
66 66
67 // GuestCIDR reports the subnet this host's guests are on, asked once per
68 // report. Nil, or an empty return, means "no answer" — never "no network".
69 GuestCIDR func() string
70
67 // Runner executes host-introspection subprocesses (on Linux, 71 // Runner executes host-introspection subprocesses (on Linux,
68 // systemd-detect-virt via hostinfo). Injected so this data-plane package 72 // systemd-detect-virt via hostinfo). Injected so this data-plane package
69 // never imports os/exec (R6); nil is tolerated (virt reported as unknown). 73 // never imports os/exec (R6); nil is tolerated (virt reported as unknown).
@@ -293,7 +297,7 @@ func (c *Client) session(ctx context.Context) error {
293 hostname, _ := os.Hostname() 297 hostname, _ := os.Hostname()
294 hello := &pb.AgentMessage{Msg: &pb.AgentMessage_Hello{Hello: &pb.Hello{ 298 hello := &pb.AgentMessage{Msg: &pb.AgentMessage_Hello{Hello: &pb.Hello{
295 HostId: c.Identity.HostID, Hostname: hostname, Os: runtime.GOOS, Arch: runtime.GOARCH, 299 HostId: c.Identity.HostID, Hostname: hostname, Os: runtime.GOOS, Arch: runtime.GOARCH,
296 Provisioner: c.Provisioner, BridgeCidr: c.Identity.BridgeCIDR, 300 Provisioner: c.Provisioner,
297 LastSeenEpoch: c.St.Epoch(), Capacity: c.advertisedCapacity(stateDir), 301 LastSeenEpoch: c.St.Epoch(), Capacity: c.advertisedCapacity(stateDir),
298 Facts: helloFacts(ctx, c.Runner), 302 Facts: helloFacts(ctx, c.Runner),
299 Credential: c.Identity.Credential, 303 Credential: c.Identity.Credential,
@@ -351,6 +355,13 @@ func (c *Client) session(ctx context.Context) error {
351 rep := c.Engine.Step(ctx, snap) 355 rep := c.Engine.Step(ctx, snap)
352 rep.Capacity = c.advertisedCapacity(stateDir) 356 rep.Capacity = c.advertisedCapacity(stateDir)
353 rep.Metrics = hostinfo.Metrics(stateDir) 357 rep.Metrics = hostinfo.Metrics(stateDir)
358 // Polled, not resolved once: a host's guest subnet can change while the
359 // agent stays connected, and on a platform whose OS owns the network it
360 // may not be knowable at connect time at all. Empty is "no answer" and
361 // leaves the fleet's record alone.
362 if c.GuestCIDR != nil {
363 rep.GuestCidr = c.GuestCIDR()
364 }
354 return transport.WriteMsg(up, &pb.AgentMessage{Msg: &pb.AgentMessage_Report{Report: rep}}) 365 return transport.WriteMsg(up, &pb.AgentMessage{Msg: &pb.AgentMessage_Report{Report: rep}})
355 } 366 }
356 367
internal/pb/sync.pb.go
Old New
@@ -234,17 +234,17 @@ func (*ServerMessage_ConsoleOpen) isServerMessage_Msg() {}
234 func (*ServerMessage_TcpOpen) isServerMessage_Msg() {} 234 func (*ServerMessage_TcpOpen) isServerMessage_Msg() {}
235 235
236 type Hello struct { 236 type Hello struct {
237 state protoimpl.MessageState `protogen:"open.v1"` 237 state protoimpl.MessageState `protogen:"open.v1"`
238 HostId string `protobuf:"bytes,1,opt,name=host_id,json=hostId,proto3" json:"host_id,omitempty"` 238 HostId string `protobuf:"bytes,1,opt,name=host_id,json=hostId,proto3" json:"host_id,omitempty"`
239 Hostname string `protobuf:"bytes,2,opt,name=hostname,proto3" json:"hostname,omitempty"` 239 Hostname string `protobuf:"bytes,2,opt,name=hostname,proto3" json:"hostname,omitempty"`
240 Os string `protobuf:"bytes,3,opt,name=os,proto3" json:"os,omitempty"` 240 Os string `protobuf:"bytes,3,opt,name=os,proto3" json:"os,omitempty"`
241 Arch string `protobuf:"bytes,4,opt,name=arch,proto3" json:"arch,omitempty"` 241 Arch string `protobuf:"bytes,4,opt,name=arch,proto3" json:"arch,omitempty"`
242 Provisioner string `protobuf:"bytes,5,opt,name=provisioner,proto3" json:"provisioner,omitempty"` // "cloudhv" 242 Provisioner string `protobuf:"bytes,5,opt,name=provisioner,proto3" json:"provisioner,omitempty"` // "cloudhv"
243 BridgeCidr string `protobuf:"bytes,6,opt,name=bridge_cidr,json=bridgeCidr,proto3" json:"bridge_cidr,omitempty"` // echo of server-assigned CIDR 243 // ActualStateReport.guest_cidr.
244 LastSeenEpoch uint64 `protobuf:"varint,7,opt,name=last_seen_epoch,json=lastSeenEpoch,proto3" json:"last_seen_epoch,omitempty"` // for the restore runbook 244 LastSeenEpoch uint64 `protobuf:"varint,7,opt,name=last_seen_epoch,json=lastSeenEpoch,proto3" json:"last_seen_epoch,omitempty"` // for the restore runbook
245 Capacity *Capacity `protobuf:"bytes,8,opt,name=capacity,proto3" json:"capacity,omitempty"` 245 Capacity *Capacity `protobuf:"bytes,8,opt,name=capacity,proto3" json:"capacity,omitempty"`
246 Credential string `protobuf:"bytes,9,opt,name=credential,proto3" json:"credential,omitempty"` // Bearer host credential, verified in first frame 246 Credential string `protobuf:"bytes,9,opt,name=credential,proto3" json:"credential,omitempty"` // Bearer host credential, verified in first frame
247 Facts *HostFacts `protobuf:"bytes,10,opt,name=facts,proto3" json:"facts,omitempty"` // best-effort static host identity; refreshed each Hello 247 Facts *HostFacts `protobuf:"bytes,10,opt,name=facts,proto3" json:"facts,omitempty"` // best-effort static host identity; refreshed each Hello
248 unknownFields protoimpl.UnknownFields 248 unknownFields protoimpl.UnknownFields
249 sizeCache protoimpl.SizeCache 249 sizeCache protoimpl.SizeCache
250 } 250 }
@@ -314,13 +314,6 @@ func (x *Hello) GetProvisioner() string {
314 return "" 314 return ""
315 } 315 }
316 316
317 func (x *Hello) GetBridgeCidr() string {
318 if x != nil {
319 return x.BridgeCidr
320 }
321 return ""
322 }
323
324 func (x *Hello) GetLastSeenEpoch() uint64 { 317 func (x *Hello) GetLastSeenEpoch() uint64 {
325 if x != nil { 318 if x != nil {
326 return x.LastSeenEpoch 319 return x.LastSeenEpoch
@@ -610,7 +603,7 @@ type ActualVM struct {
610 VmId string `protobuf:"bytes,1,opt,name=vm_id,json=vmId,proto3" json:"vm_id,omitempty"` 603 VmId string `protobuf:"bytes,1,opt,name=vm_id,json=vmId,proto3" json:"vm_id,omitempty"`
611 Power string `protobuf:"bytes,2,opt,name=power,proto3" json:"power,omitempty"` // "running"|"stopped" 604 Power string `protobuf:"bytes,2,opt,name=power,proto3" json:"power,omitempty"` // "running"|"stopped"
612 Phase string `protobuf:"bytes,3,opt,name=phase,proto3" json:"phase,omitempty"` // "creating"|"ready"|"failed"|"quarantined" 605 Phase string `protobuf:"bytes,3,opt,name=phase,proto3" json:"phase,omitempty"` // "creating"|"ready"|"failed"|"quarantined"
613 Ip string `protobuf:"bytes,4,opt,name=ip,proto3" json:"ip,omitempty"` // agent-allocated; server validates within bridge_cidr 606 Ip string `protobuf:"bytes,4,opt,name=ip,proto3" json:"ip,omitempty"` // the address this guest has, however its host came by it
614 LastError string `protobuf:"bytes,5,opt,name=last_error,json=lastError,proto3" json:"last_error,omitempty"` 607 LastError string `protobuf:"bytes,5,opt,name=last_error,json=lastError,proto3" json:"last_error,omitempty"`
615 unknownFields protoimpl.UnknownFields 608 unknownFields protoimpl.UnknownFields
616 sizeCache protoimpl.SizeCache 609 sizeCache protoimpl.SizeCache
@@ -760,8 +753,17 @@ type ActualStateReport struct {
760 FenceViolation bool `protobuf:"varint,5,opt,name=fence_violation,json=fenceViolation,proto3" json:"fence_violation,omitempty"` 753 FenceViolation bool `protobuf:"varint,5,opt,name=fence_violation,json=fenceViolation,proto3" json:"fence_violation,omitempty"`
761 LastSeenEpoch uint64 `protobuf:"varint,6,opt,name=last_seen_epoch,json=lastSeenEpoch,proto3" json:"last_seen_epoch,omitempty"` 754 LastSeenEpoch uint64 `protobuf:"varint,6,opt,name=last_seen_epoch,json=lastSeenEpoch,proto3" json:"last_seen_epoch,omitempty"`
762 Metrics *HostMetrics `protobuf:"bytes,7,opt,name=metrics,proto3" json:"metrics,omitempty"` // live measured host utilization (heartbeat) 755 Metrics *HostMetrics `protobuf:"bytes,7,opt,name=metrics,proto3" json:"metrics,omitempty"` // live measured host utilization (heartbeat)
763 unknownFields protoimpl.UnknownFields 756 // The subnet this host's guests are on, as the HOST observes it. Empty means
764 sizeCache protoimpl.SizeCache 757 // "not yet known", never "no network" — a host that cannot answer leaves the
758 // fleet's record alone rather than erasing it.
759 //
760 // In the report rather than Hello because it can change while an agent stays
761 // connected (an admin edits the host's network), and on some platforms is not
762 // knowable at connect time at all. Capacity is in both messages for the same
763 // reason: an opening value, then the ongoing truth.
764 GuestCidr string `protobuf:"bytes,8,opt,name=guest_cidr,json=guestCidr,proto3" json:"guest_cidr,omitempty"`
765 unknownFields protoimpl.UnknownFields
766 sizeCache protoimpl.SizeCache
765 } 767 }
766 768
767 func (x *ActualStateReport) Reset() { 769 func (x *ActualStateReport) Reset() {
@@ -843,6 +845,13 @@ func (x *ActualStateReport) GetMetrics() *HostMetrics {
843 return nil 845 return nil
844 } 846 }
845 847
848 func (x *ActualStateReport) GetGuestCidr() string {
849 if x != nil {
850 return x.GuestCidr
851 }
852 return ""
853 }
854
846 type VMDesired struct { 855 type VMDesired struct {
847 state protoimpl.MessageState `protogen:"open.v1"` 856 state protoimpl.MessageState `protogen:"open.v1"`
848 VmId string `protobuf:"bytes,1,opt,name=vm_id,json=vmId,proto3" json:"vm_id,omitempty"` 857 VmId string `protobuf:"bytes,1,opt,name=vm_id,json=vmId,proto3" json:"vm_id,omitempty"`
@@ -1351,22 +1360,20 @@ const file_proto_eitri_v1_sync_proto_rawDesc = "" +
1351 "\bsnapshot\x18\x01 \x01(\v2\x1e.eitri.v1.DesiredStateSnapshotH\x00R\bsnapshot\x12:\n" + 1360 "\bsnapshot\x18\x01 \x01(\v2\x1e.eitri.v1.DesiredStateSnapshotH\x00R\bsnapshot\x12:\n" +
1352 "\fconsole_open\x18\x02 \x01(\v2\x15.eitri.v1.ConsoleOpenH\x00R\vconsoleOpen\x12.\n" + 1361 "\fconsole_open\x18\x02 \x01(\v2\x15.eitri.v1.ConsoleOpenH\x00R\vconsoleOpen\x12.\n" +
1353 "\btcp_open\x18\x03 \x01(\v2\x11.eitri.v1.TCPOpenH\x00R\atcpOpenB\x05\n" + 1362 "\btcp_open\x18\x03 \x01(\v2\x11.eitri.v1.TCPOpenH\x00R\atcpOpenB\x05\n" +
1354 "\x03msg\"\xc6\x02\n" + 1363 "\x03msg\"\xb8\x02\n" +
1355 "\x05Hello\x12\x17\n" + 1364 "\x05Hello\x12\x17\n" +
1356 "\ahost_id\x18\x01 \x01(\tR\x06hostId\x12\x1a\n" + 1365 "\ahost_id\x18\x01 \x01(\tR\x06hostId\x12\x1a\n" +
1357 "\bhostname\x18\x02 \x01(\tR\bhostname\x12\x0e\n" + 1366 "\bhostname\x18\x02 \x01(\tR\bhostname\x12\x0e\n" +
1358 "\x02os\x18\x03 \x01(\tR\x02os\x12\x12\n" + 1367 "\x02os\x18\x03 \x01(\tR\x02os\x12\x12\n" +
1359 "\x04arch\x18\x04 \x01(\tR\x04arch\x12 \n" + 1368 "\x04arch\x18\x04 \x01(\tR\x04arch\x12 \n" +
1360 "\vprovisioner\x18\x05 \x01(\tR\vprovisioner\x12\x1f\n" + 1369 "\vprovisioner\x18\x05 \x01(\tR\vprovisioner\x12&\n" +
1361 "\vbridge_cidr\x18\x06 \x01(\tR\n" +
1362 "bridgeCidr\x12&\n" +
1363 "\x0flast_seen_epoch\x18\a \x01(\x04R\rlastSeenEpoch\x12.\n" + 1370 "\x0flast_seen_epoch\x18\a \x01(\x04R\rlastSeenEpoch\x12.\n" +
1364 "\bcapacity\x18\b \x01(\v2\x12.eitri.v1.CapacityR\bcapacity\x12\x1e\n" + 1371 "\bcapacity\x18\b \x01(\v2\x12.eitri.v1.CapacityR\bcapacity\x12\x1e\n" +
1365 "\n" + 1372 "\n" +
1366 "credential\x18\t \x01(\tR\n" + 1373 "credential\x18\t \x01(\tR\n" +
1367 "credential\x12)\n" + 1374 "credential\x12)\n" +
1368 "\x05facts\x18\n" + 1375 "\x05facts\x18\n" +
1369 " \x01(\v2\x13.eitri.v1.HostFactsR\x05facts\"P\n" + 1376 " \x01(\v2\x13.eitri.v1.HostFactsR\x05factsJ\x04\b\x06\x10\aR\vbridge_cidr\"P\n" +
1370 "\bCapacity\x12\x14\n" + 1377 "\bCapacity\x12\x14\n" +
1371 "\x05vcpus\x18\x01 \x01(\x03R\x05vcpus\x12\x15\n" + 1378 "\x05vcpus\x18\x01 \x01(\x03R\x05vcpus\x12\x15\n" +
1372 "\x06mem_mb\x18\x02 \x01(\x03R\x05memMb\x12\x17\n" + 1379 "\x06mem_mb\x18\x02 \x01(\x03R\x05memMb\x12\x17\n" +
@@ -1403,7 +1410,7 @@ const file_proto_eitri_v1_sync_proto_rawDesc = "" +
1403 "\x04name\x18\x02 \x01(\tR\x04name\x12\x1f\n" + 1410 "\x04name\x18\x02 \x01(\tR\x04name\x12\x1f\n" +
1404 "\vvmspec_json\x18\x03 \x01(\fR\n" + 1411 "\vvmspec_json\x18\x03 \x01(\fR\n" +
1405 "vmspecJson\x12&\n" + 1412 "vmspecJson\x12&\n" +
1406 "\x0fdestroy_at_unix\x18\x04 \x01(\x03R\rdestroyAtUnix\"\xc4\x02\n" + 1413 "\x0fdestroy_at_unix\x18\x04 \x01(\x03R\rdestroyAtUnix\"\xe3\x02\n" +
1407 "\x11ActualStateReport\x12$\n" + 1414 "\x11ActualStateReport\x12$\n" +
1408 "\x03vms\x18\x01 \x03(\v2\x12.eitri.v1.ActualVMR\x03vms\x12\x1c\n" + 1415 "\x03vms\x18\x01 \x03(\v2\x12.eitri.v1.ActualVMR\x03vms\x12\x1c\n" +
1409 "\tdestroyed\x18\x02 \x03(\tR\tdestroyed\x129\n" + 1416 "\tdestroyed\x18\x02 \x03(\tR\tdestroyed\x129\n" +
@@ -1411,7 +1418,9 @@ const file_proto_eitri_v1_sync_proto_rawDesc = "" +
1411 "\bcapacity\x18\x04 \x01(\v2\x12.eitri.v1.CapacityR\bcapacity\x12'\n" + 1418 "\bcapacity\x18\x04 \x01(\v2\x12.eitri.v1.CapacityR\bcapacity\x12'\n" +
1412 "\x0ffence_violation\x18\x05 \x01(\bR\x0efenceViolation\x12&\n" + 1419 "\x0ffence_violation\x18\x05 \x01(\bR\x0efenceViolation\x12&\n" +
1413 "\x0flast_seen_epoch\x18\x06 \x01(\x04R\rlastSeenEpoch\x12/\n" + 1420 "\x0flast_seen_epoch\x18\x06 \x01(\x04R\rlastSeenEpoch\x12/\n" +
1414 "\ametrics\x18\a \x01(\v2\x15.eitri.v1.HostMetricsR\ametrics\"\x85\x04\n" + 1421 "\ametrics\x18\a \x01(\v2\x15.eitri.v1.HostMetricsR\ametrics\x12\x1d\n" +
1422 "\n" +
1423 "guest_cidr\x18\b \x01(\tR\tguestCidr\"\x85\x04\n" +
1415 "\tVMDesired\x12\x13\n" + 1424 "\tVMDesired\x12\x13\n" +
1416 "\x05vm_id\x18\x01 \x01(\tR\x04vmId\x12\x12\n" + 1425 "\x05vm_id\x18\x01 \x01(\tR\x04vmId\x12\x12\n" +
1417 "\x04name\x18\x02 \x01(\tR\x04name\x12\x1b\n" + 1426 "\x04name\x18\x02 \x01(\tR\x04name\x12\x1b\n" +
internal/server/store/store.go
Old New
@@ -498,10 +498,30 @@ func (s *Store) GetHost(id string) (Host, error) {
498 return h, nil 498 return h, nil
499 } 499 }
500 500
501 // UpdateHostFacts overwrites the host's OS facts from its Hello. Callers treat 501 // RecordHostNetwork stores the subnet a host says its guests are on. The value
502 // failure as non-fatal (it must not drop the connection). Empty fields are 502 // is free text supplied by whoever claims to be that host, so it is validated
503 // written as-is: an agent that can no longer read a source clears the stale 503 // the same way netenv validates it before handing it to the kernel — parses as
504 // value rather than freezing it. 504 // a prefix, and IPv4 — applied on the receiving side so a garbage string cannot
505 // reach the console, an operator's eyes, or anything that reads the row.
506 //
507 // This is sanity, not topology: it asks whether the value is a network, never
508 // which network it ought to be. A host's guest subnet is that host's business.
509 //
510 // The WHERE clause makes a concurrent reconnect harmless and a repeat free at
511 // the storage layer; the caller still guards the round trip, because reports
512 // arrive every tick forever and the store runs on one connection.
513 func (s *Store) RecordHostNetwork(id, cidr string) error {
514 p, err := netip.ParsePrefix(cidr)
515 if err != nil {
516 return fmt.Errorf("guest cidr %q: %w", cidr, err)
517 }
518 if !p.Addr().Is4() {
519 return fmt.Errorf("guest cidr %q must be IPv4", cidr)
520 }
521 _, err = s.db.Exec(`UPDATE hosts SET bridge_cidr=? WHERE id=? AND bridge_cidr<>?`, cidr, id, cidr)
522 return err
523 }
524
505 // UpdateHostFacts refreshes what a host reports about itself. Every column 525 // UpdateHostFacts refreshes what a host reports about itself. Every column
506 // keeps its prior value when the reported one is EMPTY, because silence is not 526 // keeps its prior value when the reported one is EMPTY, because silence is not
507 // a statement: an agent too old to send a field, or a Hello carrying no facts 527 // a statement: an agent too old to send a field, or a Hello carrying no facts
internal/server/store/store_test.go
Old New
@@ -844,3 +844,43 @@ func TestUpdateHostFactsEmptyNeverErases(t *testing.T) {
844 assert.Equal(t, "7.0.1-generic", got.Kernel) 844 assert.Equal(t, "7.0.1-generic", got.Kernel)
845 assert.Equal(t, "vfkit", got.Provisioner, "an unmentioned field keeps its value") 845 assert.Equal(t, "vfkit", got.Provisioner, "an unmentioned field keeps its value")
846 } 846 }
847
848 // TestRecordHostNetworkSanityChecksWithoutJudgingTopology pins what the fleet
849 // asks of a subnet a host reports: that it is a network, never which network it
850 // ought to be. A host's guest subnet is that host's business — a Mac's is
851 // vmnet's, and no fleet allocation will ever contain it.
852 func TestRecordHostNetworkSanityChecksWithoutJudgingTopology(t *testing.T) {
853 s := newStore(t)
854 h := enrollHost(t, s) // allocated 10.77.1.0/24 from the pool
855
856 for _, tc := range []struct {
857 name, cidr string
858 wantErr bool
859 }{
860 {"the fleet's own allocation", "10.77.1.0/24", false},
861 {"a subnet no allocation contains", "192.168.64.0/24", false},
862 {"unparseable", "not-a-network", true},
863 {"a bare address", "192.168.64.1", true},
864 {"IPv6", "fd00::/64", true},
865 {"empty", "", true},
866 } {
867 t.Run(tc.name, func(t *testing.T) {
868 err := s.RecordHostNetwork(h.ID, tc.cidr)
869 if tc.wantErr {
870 require.Error(t, err)
871 return
872 }
873 require.NoError(t, err)
874 got, err := s.GetHost(h.ID)
875 require.NoError(t, err)
876 assert.Equal(t, tc.cidr, got.BridgeCIDR)
877 })
878 }
879
880 // A rejected value must leave the last good one standing.
881 require.NoError(t, s.RecordHostNetwork(h.ID, "192.168.64.0/24"))
882 require.Error(t, s.RecordHostNetwork(h.ID, "rubbish"))
883 got, err := s.GetHost(h.ID)
884 require.NoError(t, err)
885 assert.Equal(t, "192.168.64.0/24", got.BridgeCIDR)
886 }
internal/server/syncsvc/syncsvc.go
Old New
@@ -47,6 +47,7 @@ type Service struct {
47 // changed. 47 // changed.
48 recorder vmStatusRecorder 48 recorder vmStatusRecorder
49 tracker *statusTracker 49 tracker *statusTracker
50 netTrack *netTracker
50 // maxCredAge, when non-zero, rejects credentials whose issued-at is older. 51 // maxCredAge, when non-zero, rejects credentials whose issued-at is older.
51 // Zero disables the age check (default: expiry without an auto-renewal 52 // Zero disables the age check (default: expiry without an auto-renewal
52 // channel would force periodic re-enrolls; per-host generation revocation 53 // channel would force periodic re-enrolls; per-host generation revocation
@@ -81,7 +82,7 @@ func newWithWriteTimeout(st *store.Store, reg *registry.Registry, h *hub.Hub, se
81 writeTimeout = defaultWriteTimeout 82 writeTimeout = defaultWriteTimeout
82 } 83 }
83 return &Service{st: st, reg: reg, hub: h, secret: secret, maxCredAge: maxCredAge, writeTimeout: writeTimeout, 84 return &Service{st: st, reg: reg, hub: h, secret: secret, maxCredAge: maxCredAge, writeTimeout: writeTimeout,
84 conns: map[string]quic.Connection{}, recorder: st, tracker: newStatusTracker(), 85 conns: map[string]quic.Connection{}, recorder: st, tracker: newStatusTracker(), netTrack: newNetTracker(),
85 offers: map[string]*pb.AgentUpgrade{}} 86 offers: map[string]*pb.AgentUpgrade{}}
86 } 87 }
87 88
@@ -201,6 +202,11 @@ func (s *Service) handleConn(ctx context.Context, conn quic.Connection) {
201 delete(s.conns, hostID) 202 delete(s.conns, hostID)
202 } 203 }
203 s.consoleMu.Unlock() 204 s.consoleMu.Unlock()
205 // Forget this host's cached guest subnet, so the next connection writes
206 // it once rather than trusting a memory of a row that may have been
207 // removed, re-enrolled or edited while the host was away. Bounds the map
208 // to connected hosts, and costs one write per reconnect.
209 s.netTrack.forget(hostID)
204 }() 210 }()
205 211
206 // Single writer for the down-stream: the poke goroutine. 212 // Single writer for the down-stream: the poke goroutine.
@@ -394,6 +400,18 @@ func (s *Service) applyReport(hostID string, rep *pb.ActualStateReport) {
394 } 400 }
395 } 401 }
396 402
403 // The subnet this host says its guests are on. Empty is "no answer", never
404 // "no network" — the same rule noteAddress follows for a VM's address, and
405 // for the same reason: a host that cannot see its own network yet must not
406 // erase what it told us when it could.
407 if cidr := rep.GetGuestCidr(); cidr != "" {
408 if err := s.netTrack.writeThrough(hostID, cidr, func() error {
409 return s.st.RecordHostNetwork(hostID, cidr)
410 }); err != nil {
411 slog.Warn("record host network", "host", hostID, "cidr", cidr, "err", err)
412 }
413 }
414
397 // Fence violation: log ERROR and point at the restore runbook. 415 // Fence violation: log ERROR and point at the restore runbook.
398 if rep.GetFenceViolation() { 416 if rep.GetFenceViolation() {
399 slog.Error("agent refused snapshot: epoch fence violation — see restore runbook", 417 slog.Error("agent refused snapshot: epoch fence violation — see restore runbook",
internal/server/syncsvc/syncsvc_test.go
Old New
@@ -875,3 +875,33 @@ func TestHelloRefreshesProvisioner(t *testing.T) {
875 return err == nil && h.Provisioner == "vfkit" 875 return err == nil && h.Provisioner == "vfkit"
876 }, 2*time.Second, 20*time.Millisecond, "the host row must follow what the agent reports") 876 }, 2*time.Second, 20*time.Millisecond, "the host row must follow what the agent reports")
877 } 877 }
878
879 // TestReportRecordsTheHostsGuestSubnet pins the inversion end to end: the fleet
880 // records the subnet the HOST says its guests are on, rather than the host
881 // echoing back an allocation the fleet made for it. A Mac's guests live on
882 // vmnet's subnet, which no fleet allocation will ever contain.
883 func TestReportRecordsTheHostsGuestSubnet(t *testing.T) {
884 f := setup(t)
885 before, err := f.st.GetHost(f.host.ID)
886 require.NoError(t, err)
887 require.NotEqual(t, "192.168.64.0/24", before.BridgeCIDR)
888
889 c := mustDial(t, f)
890 c.recv(t)
891
892 c.send(t, &pb.AgentMessage{Msg: &pb.AgentMessage_Report{
893 Report: &pb.ActualStateReport{GuestCidr: "192.168.64.0/24"}}})
894 require.Eventually(t, func() bool {
895 h, err := f.st.GetHost(f.host.ID)
896 return err == nil && h.BridgeCIDR == "192.168.64.0/24"
897 }, 2*time.Second, 20*time.Millisecond, "the fleet must record what the host reports")
898
899 // Empty is "no answer", never "no network": a host that cannot see its own
900 // network yet must not erase what it told us when it could.
901 c.send(t, &pb.AgentMessage{Msg: &pb.AgentMessage_Report{
902 Report: &pb.ActualStateReport{GuestCidr: ""}}})
903 require.Never(t, func() bool {
904 h, err := f.st.GetHost(f.host.ID)
905 return err == nil && h.BridgeCIDR != "192.168.64.0/24"
906 }, 500*time.Millisecond, 25*time.Millisecond, "an empty report must leave the record alone")
907 }
internal/server/syncsvc/tracker.go
Old New
@@ -77,3 +77,44 @@ func (t *statusTracker) forget(vmID string) {
77 defer t.mu.Unlock() 77 defer t.mu.Unlock()
78 delete(t.last, vmID) 78 delete(t.last, vmID)
79 } 79 }
80
81 // netTracker remembers the guest subnet each host last had WRITTEN, so a report
82 // that repeats it costs nothing. The guard is not premature: every host reports
83 // every tick — ten seconds by default — forever, and the store runs on a single
84 // connection, so an unguarded write spends a round trip on that connection for
85 // every host for the life of the fleet. The SQL's own `WHERE bridge_cidr<>?`
86 // makes a concurrent reconnect harmless; this makes the steady state free.
87 //
88 // Keyed by hostID. Only a SUCCESSFUL write is remembered, so a rejected one is
89 // retried by the next report rather than suppressed — the same rule the status
90 // tracker follows, for the same reason.
91 type netTracker struct {
92 mu sync.Mutex
93 last map[string]string
94 }
95
96 func newNetTracker() *netTracker { return &netTracker{last: map[string]string{}} }
97
98 // writeThrough runs decide→write→commit for one host under a single lock, so
99 // two connections for the same host cannot interleave and leave the cache
100 // disagreeing with the row.
101 func (t *netTracker) writeThrough(hostID, cidr string, write func() error) error {
102 t.mu.Lock()
103 defer t.mu.Unlock()
104 if t.last[hostID] == cidr {
105 return nil
106 }
107 if err := write(); err != nil {
108 return err
109 }
110 t.last[hostID] = cidr
111 return nil
112 }
113
114 // forget drops a host's cached subnet, so a host that leaves and returns is
115 // written afresh rather than trusted to a memory of a row that may be gone.
116 func (t *netTracker) forget(hostID string) {
117 t.mu.Lock()
118 defer t.mu.Unlock()
119 delete(t.last, hostID)
120 }
internal/server/syncsvc/tracker_test.go
Old New
@@ -194,3 +194,40 @@ func TestStatusTrackerPerVMKeys(t *testing.T) {
194 t.Fatal("vm1 unchanged must not write") 194 t.Fatal("vm1 unchanged must not write")
195 } 195 }
196 } 196 }
197
198 // TestNetTrackerWritesOnceThenStaysQuiet pins the guard that makes the steady
199 // state free: every host reports its subnet every tick forever, and the store
200 // runs on one connection, so an unguarded write spends a round trip per host
201 // per tick for the life of the fleet.
202 func TestNetTrackerWritesOnceThenStaysQuiet(t *testing.T) {
203 tr := newNetTracker()
204 writes := 0
205 report := func(host, cidr string) error {
206 return tr.writeThrough(host, cidr, func() error { writes++; return nil })
207 }
208
209 require.NoError(t, report("h1", "10.77.1.0/24"))
210 require.Equal(t, 1, writes, "the first report writes")
211 require.NoError(t, report("h1", "10.77.1.0/24"))
212 require.Equal(t, 1, writes, "a repeat writes nothing")
213 require.NoError(t, report("h1", "192.168.64.0/24"))
214 require.Equal(t, 2, writes, "a changed subnet writes")
215
216 // Hosts are independent.
217 require.NoError(t, report("h2", "192.168.64.0/24"))
218 require.Equal(t, 3, writes, "a different host is a different key")
219
220 // A REJECTED write is not remembered, so the next report retries it rather
221 // than being suppressed by a value that never reached the row.
222 sentinel := errors.New("rejected")
223 require.ErrorIs(t, tr.writeThrough("h3", "junk", func() error { return sentinel }), sentinel)
224 attempted := false
225 require.NoError(t, tr.writeThrough("h3", "junk", func() error { attempted = true; return nil }))
226 require.True(t, attempted, "a failed write must not suppress the retry")
227
228 // Forgetting a host writes it afresh: the row may have changed while it was
229 // disconnected.
230 tr.forget("h1")
231 require.NoError(t, report("h1", "192.168.64.0/24"))
232 require.Equal(t, 4, writes)
233 }
proto/eitri/v1/sync.proto
Old New
@@ -25,7 +25,9 @@ message Hello {
25 string os = 3; 25 string os = 3;
26 string arch = 4; 26 string arch = 4;
27 string provisioner = 5; // "cloudhv" 27 string provisioner = 5; // "cloudhv"
28 string bridge_cidr = 6; // echo of server-assigned CIDR 28 reserved 6; // was bridge_cidr: the fleet told the host its
29 reserved "bridge_cidr"; // subnet. The host reports it now — see
30 // ActualStateReport.guest_cidr.
29 uint64 last_seen_epoch = 7; // for the restore runbook 31 uint64 last_seen_epoch = 7; // for the restore runbook
30 Capacity capacity = 8; 32 Capacity capacity = 8;
31 string credential = 9; // Bearer host credential, verified in first frame 33 string credential = 9; // Bearer host credential, verified in first frame
@@ -67,7 +69,7 @@ message ActualVM {
67 string vm_id = 1; 69 string vm_id = 1;
68 string power = 2; // "running"|"stopped" 70 string power = 2; // "running"|"stopped"
69 string phase = 3; // "creating"|"ready"|"failed"|"quarantined" 71 string phase = 3; // "creating"|"ready"|"failed"|"quarantined"
70 string ip = 4; // agent-allocated; server validates within bridge_cidr 72 string ip = 4; // the address this guest has, however its host came by it
71 string last_error = 5; 73 string last_error = 5;
72 } 74 }
73 75
@@ -88,6 +90,15 @@ message ActualStateReport {
88 bool fence_violation = 5; 90 bool fence_violation = 5;
89 uint64 last_seen_epoch = 6; 91 uint64 last_seen_epoch = 6;
90 HostMetrics metrics = 7; // live measured host utilization (heartbeat) 92 HostMetrics metrics = 7; // live measured host utilization (heartbeat)
93 // The subnet this host's guests are on, as the HOST observes it. Empty means
94 // "not yet known", never "no network" — a host that cannot answer leaves the
95 // fleet's record alone rather than erasing it.
96 //
97 // In the report rather than Hello because it can change while an agent stays
98 // connected (an admin edits the host's network), and on some platforms is not
99 // knowable at connect time at all. Capacity is in both messages for the same
100 // reason: an opening value, then the ongoing truth.
101 string guest_cidr = 8;
91 } 102 }
92 103
93 message VMDesired { 104 message VMDesired {