a73x

a1533a4f

feat: a host proposes its guest subnet, and the fleet only suggests one

a73x   2026-08-06 09:12

Commit message
feat: a host proposes its guest subnet, and the fleet only suggests one

The pool becomes advice. A joining host may state the subnet its guests are on,
and the fleet records it as given; a host with no opinion is offered an
allocation as before. Which is which is carried by a pointer, because absent and
empty are different answers: nil is "no opinion" — an older agent, or a Linux
host started without --bridge-cidr — while an explicit "" is "none of my own",
which is what a host whose OS owns the guest network says, and which is left
alone to report what it observes later rather than handed a fabricated one.

A proposal is sanity-checked and never second-guessed. It must parse as an IPv4
prefix, the same check netenv makes before handing one to the kernel and the
same one a reported subnet already passes — now one function, used by both write
sites. It is never checked against the pool: a host's guest subnet is that
host's business.

CIDR recycling is deleted, and this is the commit that makes it unsafe rather
than merely unnecessary. hosts.bridge_cidr now holds what the HOST claims, so
returning a departing host's value to the pool would offer a Mac's
192.168.64.0/24 to the next Linux host as the subnet to build. The monotonic
index alone is sufficient — a /16 is 65k /24s — and leaking indices across host
churn is cheaper than re-issuing a value the fleet never chose. freed_cidrs is
dropped from existing databases through a new evolve.go step, which carries
ensureColumn's identifier-interpolation warning verbatim.

join stops copying the response's bridge_cidr into the identity. That field
belongs to the resolver now, and copying it back would reinstate server-wins on
every re-enroll — the exact behaviour being removed. The response is still
returned and still read: as the suggestion the resolver considers, ranked below
the operator's flag and above nothing at all.

Two tests reverse rather than die, and say so: a departing host's subnet is now
proven NOT to be re-issued. The pool itself, its config knob, its allocator and
its exhaustion behaviour are all untouched.

docs/architecture.md
Old New
@@ -26,8 +26,11 @@ live in the agent.
26 Real SSH into a guest goes through the **SSH jump gate** 26 Real SSH into a guest goes through the **SSH jump gate**
27 (`internal/server/sshgate` + `internal/server/sshca`): the server holds an SSH 27 (`internal/server/sshgate` + `internal/server/sshca`): the server holds an SSH
28 CA, mints short-lived user certs, and tunnels TCP:22 to the VM over the existing 28 CA, mints short-lived user certs, and tunnels TCP:22 to the VM over the existing
29 server↔agent sync channel. There is no user network—VMs are reachable at their 29 server↔agent sync channel. There is no user network—a VM is reachable at the
30 bridge IP (`assigned_ip`) via the agent. 30 guest address its host reports (`assigned_ip`), dialled by the agent on that
31 host. How the guest came by that address is the host's business: a Linux host
32 allocates it from a bridge it builds, a Mac reads what vmnet's own DHCP handed
33 out.
31 34
32 ## Invariants 35 ## Invariants
33 36
@@ -37,7 +40,7 @@ bridge IP (`assigned_ip`) via the agent.
37 | **R2** | No `internal/server` package shells out—the server is pure control plane. Checked transitively: an internal wrapper around `os/exec` cannot smuggle a shell-out in. | `internal/arch` `TestServerNeverShellsOut` (transitive) + `depguard` `server-no-exec` (direct, fast in-editor). | 40 | **R2** | No `internal/server` package shells out—the server is pure control plane. Checked transitively: an internal wrapper around `os/exec` cannot smuggle a shell-out in. | `internal/arch` `TestServerNeverShellsOut` (transitive) + `depguard` `server-no-exec` (direct, fast in-editor). |
38 | **R3** | The wire contract (`pb`, `transport`) imports no other internal package, so a heavy dependency can't leak across the boundary into both binaries. | `internal/arch` `TestWireContractIsLeaf`. Behavior pinned by `transport` round-trip contract tests. | 41 | **R3** | The wire contract (`pb`, `transport`) imports no other internal package, so a heavy dependency can't leak across the boundary into both binaries. | `internal/arch` `TestWireContractIsLeaf`. Behavior pinned by `transport` round-trip contract tests. |
39 | **R4** | Pure domain packages (`agent/state`, `agent/seed`, `agent/ipalloc`, `server/registry`) don't depend on the transport stack (HTTP/QUIC/`transport`). `server/store` may use `transport` (cert helpers) but not HTTP/QUIC. | `internal/arch` `TestDomainDoesNotImportTransportStack` + `depguard` `domain-no-transport`. | 42 | **R4** | Pure domain packages (`agent/state`, `agent/seed`, `agent/ipalloc`, `server/registry`) don't depend on the transport stack (HTTP/QUIC/`transport`). `server/store` may use `transport` (cert helpers) but not HTTP/QUIC. | `internal/arch` `TestDomainDoesNotImportTransportStack` + `depguard` `domain-no-transport`. |
40 | **R5** | The reconcile boundary interface (`Provisioner`) stays consumer-owned and small. A VM's network attachment lives inside it, because on every backend we ship the NIC is a launch argument rather than a separately-lifecycled resource; what crosses the seam is data (the VM's address) and not mechanism (taps, reservations). | Convention (below) + `ireturn` allow-list keeps the seam's interface returns honest. | 43 | **R5** | The reconcile boundary interface (`Provisioner`) stays consumer-owned and small. A VM's network attachment lives inside it, because on every backend we ship the NIC is a launch argument rather than a separately-lifecycled resource; what crosses the seam is data (the VM's address) and not mechanism (taps, reservations). The host-wide counterpart crosses the *sync* seam the same way: a host reports the subnet its guests are on (`guest_cidr`) rather than being told which one to build, so the fleet records an observation and never infers a mechanism from it. | Convention (below) + `ireturn` allow-list keeps the seam's interface returns honest. |
41 | **R6** | All external process execution in the data plane funnels through `agent/exec.Runner`. The exceptions are the provisioners — `agent/cloudhv` and `agent/vfkit` — which launch the long-lived VMM process directly, and the composition root `agent/run`, which builds the Runner it injects. Checked transitively (reaching `os/exec` via a sanctioned package is fine). | `internal/arch` `TestOnlyProvisionersAndRootImportOsExecInDataPlane` (transitive). | 44 | **R6** | All external process execution in the data plane funnels through `agent/exec.Runner`. The exceptions are the provisioners — `agent/cloudhv` and `agent/vfkit` — which launch the long-lived VMM process directly, and the composition root `agent/run`, which builds the Runner it injects. Checked transitively (reaching `os/exec` via a sanctioned package is fine). | `internal/arch` `TestOnlyProvisionersAndRootImportOsExecInDataPlane` (transitive). |
42 | **R13** | The OIDC issuer and the relying party stay separate binaries. `eitri-server` is a pure relying party: the bundled issuer (`internal/oidcprovider`) is importable only by its own binary `cmd/eitri-oidc`, and the `go-oidc` verifier module only by `internal/server/api` (the RP). A server import of the issuer would silently rebuild the embedded-IdP coupling; `go-oidc` anywhere but the RP means a second relying party is being hand-rolled. | `internal/arch` `TestIssuerAndRelyingPartyAreSeparate`. | 45 | **R13** | The OIDC issuer and the relying party stay separate binaries. `eitri-server` is a pure relying party: the bundled issuer (`internal/oidcprovider`) is importable only by its own binary `cmd/eitri-oidc`, and the `go-oidc` verifier module only by `internal/server/api` (the RP). A server import of the issuer would silently rebuild the embedded-IdP coupling; `go-oidc` anywhere but the RP means a second relying party is being hand-rolled. | `internal/arch` `TestIssuerAndRelyingPartyAreSeparate`. |
43 46
docs/decisions.md
Old New
@@ -49,6 +49,15 @@ injection via cloud-init or an external DHCP daemon. Addressing is a host
49 resource like memory; owning it host-locally keeps images untouched and needs 49 resource like memory; owning it host-locally keeps images untouched and needs
50 no external service. 50 no external service.
51 51
52 ### The host reports its guest network
53
54 A host tells the fleet which subnet its guests are on and the fleet records it;
55 the pool becomes a suggestion for hosts that have no opinion. Instead of the
56 fleet allocating every host a subnet and validating reported addresses against
57 it. A host whose OS owns the guest network sits on a subnet no allocation of
58 ours can contain—so the fleet can only be told, which is also why a value it did
59 not choose is never recycled back into the pool.
60
52 ### No superuser principal 61 ### No superuser principal
53 62
54 Every API principal is `{Tenant, Fleet}`; fleet-level endpoints are gated 63 Every API principal is `{Tenant, Fleet}`; fleet-level endpoints are gated
docs/openapi.json
Old New
@@ -161,6 +161,12 @@
161 "arch": { 161 "arch": {
162 "type": "string" 162 "type": "string"
163 }, 163 },
164 "bridge_cidr": {
165 "type": [
166 "string",
167 "null"
168 ]
169 },
164 "name": { 170 "name": {
165 "type": "string" 171 "type": "string"
166 }, 172 },
internal/agent/enrollclient/enrollclient.go
Old New
@@ -25,6 +25,10 @@ type Request struct {
25 OS string `json:"os"` 25 OS string `json:"os"`
26 Arch string `json:"arch"` 26 Arch string `json:"arch"`
27 Provisioner string `json:"provisioner"` 27 Provisioner string `json:"provisioner"`
28 // BridgeCIDR is this host's proposal, if it has one. Nil means "no
29 // opinion" and takes the fleet's suggestion; a non-nil "" means "none of my
30 // own", which is what a host whose OS owns the guest network says.
31 BridgeCIDR *string `json:"bridge_cidr,omitempty"`
28 } 32 }
29 33
30 // Response carries the fields the agent consumes from a successful enroll. The 34 // Response carries the fields the agent consumes from a successful enroll. The
internal/agent/run/cli.go
Old New
@@ -80,7 +80,7 @@ func RunCLI(args []string) error {
80 if len(rest) > 1 { 80 if len(rest) > 1 {
81 blob = rest[1] 81 blob = rest[1]
82 } 82 }
83 return join(st, blob) 83 return join(st, cfg, blob)
84 } 84 }
85 85
86 return serve(st, cfg) 86 return serve(st, cfg)
@@ -161,7 +161,7 @@ func parseConfig(args []string) (Config, []string, error) {
161 // join handles the "join <blob>" subcommand: decode the join blob, enroll, 161 // join handles the "join <blob>" subcommand: decode the join blob, enroll,
162 // and persist identity — pinning the server cert from the blob (the enroll 162 // and persist identity — pinning the server cert from the blob (the enroll
163 // response's fingerprint is ignored, so the blob is the sole trust root). 163 // response's fingerprint is ignored, so the blob is the sole trust root).
164 func join(st *state.Store, blob string) error { 164 func join(st *state.Store, cfg Config, blob string) error {
165 if blob == "" { 165 if blob == "" {
166 return errors.New("usage: eitri-agent join <join-blob>") 166 return errors.New("usage: eitri-agent join <join-blob>")
167 } 167 }
@@ -182,6 +182,7 @@ func join(st *state.Store, blob string) error {
182 OS: runtime.GOOS, 182 OS: runtime.GOOS,
183 Arch: runtime.GOARCH, 183 Arch: runtime.GOARCH,
184 Provisioner: platformProvisioner, 184 Provisioner: platformProvisioner,
185 BridgeCIDR: proposeGuestCIDR(cfg),
185 }) 186 })
186 if errors.Is(err, enrollclient.ErrTokenRejected) { 187 if errors.Is(err, enrollclient.ErrTokenRejected) {
187 return errors.New("enroll rejected: token already used or expired — mint a new join token") 188 return errors.New("enroll rejected: token already used or expired — mint a new join token")
@@ -190,17 +191,20 @@ func join(st *state.Store, blob string) error {
190 return fmt.Errorf("enroll failed: %w", err) 191 return fmt.Errorf("enroll failed: %w", err)
191 } 192 }
192 193
194 // The response's bridge_cidr is a SUGGESTION, not a value to store: the
195 // resolver owns that field. Copying it here would reinstate server-wins on
196 // every re-enroll, which is the whole thing being removed.
193 id := state.Identity{ 197 id := state.Identity{
194 HostID: result.HostID, 198 HostID: result.HostID,
195 Credential: result.Credential, 199 Credential: result.Credential,
196 BridgeCIDR: result.BridgeCIDR,
197 ServerQUICAddr: f.QUICAddr, 200 ServerQUICAddr: f.QUICAddr,
198 ServerCertSHA256: f.CertFP, // authoritative; response fingerprint ignored 201 ServerCertSHA256: f.CertFP, // authoritative; response fingerprint ignored
199 } 202 }
200 if err := st.SaveIdentity(id); err != nil { 203 if err := st.SaveIdentity(id); err != nil {
201 return fmt.Errorf("save identity: %w", err) 204 return fmt.Errorf("save identity: %w", err)
202 } 205 }
203 fmt.Printf("Enrolled: host_id=%s bridge_cidr=%s\n", result.HostID, result.BridgeCIDR) 206 guestCIDR := resolveGuestCIDR(st, cfg, result.BridgeCIDR)
207 fmt.Printf("Enrolled: host_id=%s guest_cidr=%s\n", result.HostID, guestCIDR)
204 return nil 208 return nil
205 } 209 }
206 210
internal/agent/run/cli_test.go
Old New
@@ -111,7 +111,7 @@ func TestParseConfigBadFlag(t *testing.T) {
111 // TestJoinEmptyBlob rejects a missing blob before any decode or network call; 111 // TestJoinEmptyBlob rejects a missing blob before any decode or network call;
112 // st is never touched, so nil is safe. 112 // st is never touched, so nil is safe.
113 func TestJoinEmptyBlob(t *testing.T) { 113 func TestJoinEmptyBlob(t *testing.T) {
114 err := join(nil, "") 114 err := join(nil, Config{}, "")
115 require.Error(t, err) 115 require.Error(t, err)
116 assert.Contains(t, err.Error(), "usage: eitri-agent join") 116 assert.Contains(t, err.Error(), "usage: eitri-agent join")
117 } 117 }
@@ -120,7 +120,7 @@ func TestJoinEmptyBlob(t *testing.T) {
120 // itself (it carries a bearer token). Decode fails before st is used. 120 // itself (it carries a bearer token). Decode fails before st is used.
121 func TestJoinInvalidBlob(t *testing.T) { 121 func TestJoinInvalidBlob(t *testing.T) {
122 const secret = "not-a-valid-join-blob-with-secret" 122 const secret = "not-a-valid-join-blob-with-secret"
123 err := join(nil, secret) 123 err := join(nil, Config{}, secret)
124 require.Error(t, err) 124 require.Error(t, err)
125 assert.Contains(t, err.Error(), "invalid join blob") 125 assert.Contains(t, err.Error(), "invalid join blob")
126 assert.NotContains(t, err.Error(), secret, "the blob carries a bearer token and must never appear in errors") 126 assert.NotContains(t, err.Error(), secret, "the blob carries a bearer token and must never appear in errors")
internal/agent/run/wire_darwin.go
Old New
@@ -18,6 +18,12 @@ import (
18 // time — an opaque label the server stores and never interprets. 18 // time — an opaque label the server stores and never interprets.
19 const platformProvisioner = "vfkit" 19 const platformProvisioner = "vfkit"
20 20
21 // proposeGuestCIDR is what this platform offers the fleet at enrollment. A Mac
22 // proposes an explicit NOTHING rather than no opinion: vmnet owns the guest
23 // network, so this host has neither a subnet of its own to state nor any use
24 // for one the fleet would allocate. It reports what it observes later instead.
25 func proposeGuestCIDR(Config) *string { none := ""; return &none }
26
21 // defaultStateDir is where a Mac agent keeps its state when nothing says 27 // defaultStateDir is where a Mac agent keeps its state when nothing says
22 // otherwise: a dotdir in the running account's home, which is what the tools a 28 // otherwise: a dotdir in the running account's home, which is what the tools a
23 // Mac user already runs do (colima keeps ~/.colima, lima ~/.lima). Not 29 // Mac user already runs do (colima keeps ~/.colima, lima ~/.lima). Not
internal/agent/run/wire_linux.go
Old New
@@ -47,6 +47,21 @@ type platform struct {
47 GuestCIDR func() string 47 GuestCIDR func() string
48 } 48 }
49 49
50 // proposeGuestCIDR is what this platform offers the fleet at enrollment: the
51 // operator's flag, or no opinion at all. Nil takes the fleet's suggestion,
52 // which is what a Linux host without --bridge-cidr wants.
53 //
54 // Per-platform because parseConfig is deliberately platform-neutral — every
55 // flag is accepted everywhere and ignored by hosts that do not run them — so a
56 // shared reader of cfg.BridgeCIDR would have a Mac propose a fabricated Linux
57 // subnet.
58 func proposeGuestCIDR(cfg Config) *string {
59 if cfg.BridgeCIDR == "" {
60 return nil
61 }
62 return &cfg.BridgeCIDR
63 }
64
50 // defaultStateDir is where a Linux agent keeps its state when nothing says 65 // defaultStateDir is where a Linux agent keeps its state when nothing says
51 // otherwise. The unit runs as root and the agent creates the directory itself, 66 // otherwise. The unit runs as root and the agent creates the directory itself,
52 // so there is nothing for an operator to pre-create. 67 // so there is nothing for an operator to pre-create.
internal/agent/syncclient/client_test.go
Old New
@@ -201,7 +201,7 @@ func (h *serverHarness) stop() {
201 // enroll creates a host and returns a valid credential for it. 201 // enroll creates a host and returns a valid credential for it.
202 func (h *serverHarness) enroll() (hostID, cred string) { 202 func (h *serverHarness) enroll() (hostID, cred string) {
203 tok, _ := h.st.CreateEnrollmentToken(testTenant) 203 tok, _ := h.st.CreateEnrollmentToken(testTenant)
204 host, err := h.st.RedeemEnrollmentToken(tok, "h", "linux", "amd64", "cloudhv", "") 204 host, err := h.st.RedeemEnrollmentToken(tok, store.EnrollFacts{Name: "h", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
205 require.NoError(h.t, err) 205 require.NoError(h.t, err)
206 return host.ID, hosttoken.Mint(h.secret, host.ID, host.CredGeneration, time.Now()) 206 return host.ID, hosttoken.Mint(h.secret, host.ID, host.CredGeneration, time.Now())
207 } 207 }
internal/server/api/api.go
Old New
@@ -338,7 +338,10 @@ func (a *API) handleEnroll(w http.ResponseWriter, r *http.Request) {
338 if !decodeJSON(w, r, &req) { 338 if !decodeJSON(w, r, &req) {
339 return 339 return
340 } 340 }
341 host, err := a.st.RedeemEnrollmentToken(req.Token, req.Name, req.OS, req.Arch, req.Provisioner, clientIP(r)) 341 host, err := a.st.RedeemEnrollmentToken(req.Token, store.EnrollFacts{
342 Name: req.Name, OS: req.OS, Arch: req.Arch, Provisioner: req.Provisioner,
343 Remote: clientIP(r), BridgeCIDR: req.BridgeCIDR,
344 })
342 if err != nil { 345 if err != nil {
343 // Unauthenticated enroll attempt: no tenant is resolvable (the token was 346 // Unauthenticated enroll attempt: no tenant is resolvable (the token was
344 // rejected), so the denied row is filed under the system audit scope. 347 // rejected), so the denied row is filed under the system audit scope.
internal/server/api/testdata/enroll-request.golden.json
Old New
@@ -3,5 +3,6 @@
3 "name": "host-nine", 3 "name": "host-nine",
4 "os": "linux", 4 "os": "linux",
5 "arch": "arm64", 5 "arch": "arm64",
6 "provisioner": "cloudhypervisor" 6 "provisioner": "cloudhypervisor",
7 "bridge_cidr": "10.42.0.0/24"
7 } 8 }
internal/server/api/types/types.go
Old New
@@ -170,6 +170,13 @@ type EnrollRequest struct {
170 OS string `json:"os"` 170 OS string `json:"os"`
171 Arch string `json:"arch"` 171 Arch string `json:"arch"`
172 Provisioner string `json:"provisioner"` 172 Provisioner string `json:"provisioner"`
173 // BridgeCIDR is the subnet this host says its guests are on. A POINTER
174 // because absent and empty differ: absent is "no opinion" — an older agent,
175 // or a host that wants the fleet's suggestion — while an explicit "" is "no
176 // subnet of my own", which is what a host whose OS owns the guest network
177 // says. The first gets an allocation; the second is left alone to report its
178 // own later. RevokeSSHCertRequest.Serial is the same trick.
179 BridgeCIDR *string `json:"bridge_cidr"`
173 } 180 }
174 181
175 // CreateVMRequest is the POST /api/v1/vms body. Every field except host_id is 182 // CreateVMRequest is the POST /api/v1/vms body. Every field except host_id is
internal/server/api/wire_golden_test.go
Old New
@@ -134,7 +134,12 @@ func TestWireGolden(t *testing.T) {
134 Reason: "key compromised", 134 Reason: "key compromised",
135 }}) 135 }})
136 136
137 // bridge_cidr carries a proposal here so the golden pins the field's
138 // presence and its pointer shape; nil and "" are separately meaningful and
139 // covered by the store's own tests.
140 proposal := "10.42.0.0/24"
137 goldenCheck(t, "enroll-request", types.EnrollRequest{ 141 goldenCheck(t, "enroll-request", types.EnrollRequest{
142 BridgeCIDR: &proposal,
138 Token: "tok-secret-01", 143 Token: "tok-secret-01",
139 Name: "host-nine", 144 Name: "host-nine",
140 OS: "linux", 145 OS: "linux",
internal/server/boot/sshgate_test.go
Old New
@@ -31,7 +31,7 @@ func makeTenantHost(t *testing.T, s *store.Store, subject, email string) (string
31 require.NoError(t, err) 31 require.NoError(t, err)
32 tok, err := s.CreateEnrollmentToken(tn.ID) 32 tok, err := s.CreateEnrollmentToken(tn.ID)
33 require.NoError(t, err) 33 require.NoError(t, err)
34 h, err := s.RedeemEnrollmentToken(tok, "host-"+tn.ID, "linux", "amd64", "cloudhv", "") 34 h, err := s.RedeemEnrollmentToken(tok, store.EnrollFacts{Name: "host-" + tn.ID, OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
35 require.NoError(t, err) 35 require.NoError(t, err)
36 return tn.ID, h 36 return tn.ID, h
37 } 37 }
internal/server/store/allocation_test.go
Old New
@@ -49,7 +49,7 @@ func TestAllocatedByHostSeparatesHosts(t *testing.T) {
49 s := newStore(t) 49 s := newStore(t)
50 h1 := enrollHost(t, s) 50 h1 := enrollHost(t, s)
51 tok, _ := s.CreateEnrollmentToken(testTenant) 51 tok, _ := s.CreateEnrollmentToken(testTenant)
52 h2, _ := s.RedeemEnrollmentToken(tok, "h2", "linux", "amd64", "cloudhv", "") 52 h2, _ := s.RedeemEnrollmentToken(tok, EnrollFacts{Name: "h2", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
53 vmWithResources(t, s, h1, "a", 2, 2048, 10) 53 vmWithResources(t, s, h1, "a", 2, 2048, 10)
54 vmWithResources(t, s, h2, "b", 8, 8192, 40) 54 vmWithResources(t, s, h2, "b", 8, 8192, 40)
55 55
@@ -64,3 +64,62 @@ func (s *Store) mustAllocated(t *testing.T) map[string]Alloc {
64 require.NoError(t, err) 64 require.NoError(t, err)
65 return alloc 65 return alloc
66 } 66 }
67
68 // TestEnrollHostProposesTheFleetSuggests pins the inversion at the moment a
69 // host joins. The three states are deliberately distinct, which is why the
70 // field is a pointer: no opinion takes the fleet's suggestion, a stated subnet
71 // is recorded as given, and an explicit none is left alone for a host that will
72 // report its own once it can see it.
73 func TestEnrollHostProposesTheFleetSuggests(t *testing.T) {
74 cidr := func(s string) *string { return &s }
75
76 for _, tc := range []struct {
77 name string
78 proposal *string
79 want string // "" = expect the pool's allocation
80 wantErr string
81 }{
82 {name: "no opinion takes the suggestion", proposal: nil, want: "10.77.1.0/24"},
83 {name: "a stated subnet is recorded as given", proposal: cidr("192.168.64.0/24"), want: "192.168.64.0/24"},
84 {name: "an explicit none is left alone", proposal: cidr(""), want: ""},
85 {name: "unparseable is refused", proposal: cidr("not-a-network"), wantErr: "guest cidr"},
86 {name: "a bare address is refused", proposal: cidr("192.168.64.1"), wantErr: "guest cidr"},
87 {name: "IPv6 is refused", proposal: cidr("fd00::/64"), wantErr: "must be IPv4"},
88 } {
89 t.Run(tc.name, func(t *testing.T) {
90 s := newStore(t)
91 tok, err := s.CreateEnrollmentToken(testTenant)
92 require.NoError(t, err)
93 h, err := s.RedeemEnrollmentToken(tok, EnrollFacts{
94 Name: "h", OS: "linux", Arch: "amd64", Provisioner: "cloudhv",
95 BridgeCIDR: tc.proposal,
96 })
97 if tc.wantErr != "" {
98 require.Error(t, err)
99 assert.Contains(t, err.Error(), tc.wantErr)
100 return
101 }
102 require.NoError(t, err)
103 assert.Equal(t, tc.want, h.BridgeCIDR)
104 })
105 }
106 }
107
108 // TestEnrollWithAProposalDoesNotSpendAnAllocation pins that a host stating its
109 // own subnet leaves the pool where it was — the fleet allocates for hosts that
110 // want advice, not for every host that joins.
111 func TestEnrollWithAProposalDoesNotSpendAnAllocation(t *testing.T) {
112 s := newStore(t)
113 own := "192.168.64.0/24"
114 tok, _ := s.CreateEnrollmentToken(testTenant)
115 _, err := s.RedeemEnrollmentToken(tok, EnrollFacts{
116 Name: "mac", OS: "darwin", Arch: "arm64", Provisioner: "vfkit", BridgeCIDR: &own})
117 require.NoError(t, err)
118
119 tok2, _ := s.CreateEnrollmentToken(testTenant)
120 linux, err := s.RedeemEnrollmentToken(tok2, EnrollFacts{
121 Name: "linux", OS: "linux", Arch: "amd64", Provisioner: "cloudhv"})
122 require.NoError(t, err)
123 assert.Equal(t, "10.77.1.0/24", linux.BridgeCIDR,
124 "the first allocation is still available: the Mac consumed none")
125 }
internal/server/store/decommission_test.go
Old New
@@ -45,29 +45,36 @@ func TestDecommissionHostTombstonesVMsAndSetsStatus(t *testing.T) {
45 assert.Greater(t, epochAfter, epochBefore, "decommission must bump epoch") 45 assert.Greater(t, epochAfter, epochBefore, "decommission must bump epoch")
46 } 46 }
47 47
48 func TestRemoveHostFreesCIDRForReuse(t *testing.T) { 48 // TestRemoveHostDoesNotReissueItsSubnet pins the reversal of what this test
49 // used to assert. A departing host's bridge_cidr is NOT returned to the pool,
50 // because post-inversion that column holds what the HOST claimed — so
51 // re-issuing it would hand one host's subnet to the next as a suggestion, and a
52 // Mac's 192.168.64.0/24 is not something to advise a Linux host to build.
53 //
54 // The monotonic index alone is sufficient: a /16 pool is 65k /24s, and leaking
55 // indices across host churn is cheaper than re-issuing a value the fleet never
56 // chose.
57 func TestRemoveHostDoesNotReissueItsSubnet(t *testing.T) {
49 s := newStore(t) 58 s := newStore(t)
50 h1 := enrollHost(t, s) 59 h1 := enrollHost(t, s)
51 assert.Equal(t, "10.77.1.0/24", h1.BridgeCIDR) 60 assert.Equal(t, "10.77.1.0/24", h1.BridgeCIDR)
52 tok2, _ := s.CreateEnrollmentToken(testTenant) 61 tok2, _ := s.CreateEnrollmentToken(testTenant)
53 h2, _ := s.RedeemEnrollmentToken(tok2, "b", "linux", "amd64", "cloudhv", "") 62 h2, _ := s.RedeemEnrollmentToken(tok2, EnrollFacts{Name: "b", OS: "linux", Arch: "amd64", Provisioner: "cloudhv"})
54 assert.Equal(t, "10.77.2.0/24", h2.BridgeCIDR) 63 assert.Equal(t, "10.77.2.0/24", h2.BridgeCIDR)
55 64
56 // Decommission h1 and simulate the agent reaping its VMs (hard-delete).
57 vm := makeVM(t, s, h1, "vm-a") 65 vm := makeVM(t, s, h1, "vm-a")
58 require.NoError(t, s.DecommissionHost(h1.ID)) 66 require.NoError(t, s.DecommissionHost(h1.ID))
59 require.NoError(t, s.HardDeleteVM(vm.ID)) 67 require.NoError(t, s.HardDeleteVM(vm.ID))
60
61 require.NoError(t, s.RemoveHost(h1.ID)) 68 require.NoError(t, s.RemoveHost(h1.ID))
62 69
63 _, err := s.GetHost(h1.ID) 70 _, err := s.GetHost(h1.ID)
64 assert.Error(t, err, "removed host should be gone") 71 assert.Error(t, err, "removed host should be gone")
65 72
66 // A new enrollment reuses h1's freed CIDR rather than allocating a fresh one.
67 tok3, _ := s.CreateEnrollmentToken(testTenant) 73 tok3, _ := s.CreateEnrollmentToken(testTenant)
68 h3, err := s.RedeemEnrollmentToken(tok3, "c", "linux", "amd64", "cloudhv", "") 74 h3, err := s.RedeemEnrollmentToken(tok3, EnrollFacts{Name: "c", OS: "linux", Arch: "amd64", Provisioner: "cloudhv"})
69 require.NoError(t, err) 75 require.NoError(t, err)
70 assert.Equal(t, "10.77.1.0/24", h3.BridgeCIDR, "freed CIDR should be reused") 76 assert.Equal(t, "10.77.3.0/24", h3.BridgeCIDR,
77 "the allocator moves on; a departed host's subnet is never re-issued")
71 } 78 }
72 79
73 func TestRemoveHostRefusesWhileVMsRemain(t *testing.T) { 80 func TestRemoveHostRefusesWhileVMsRemain(t *testing.T) {
internal/server/store/evolve.go
Old New
@@ -28,3 +28,17 @@ func ensureColumn(db *sql.DB, table, column, decl string) error {
28 } 28 }
29 return nil 29 return nil
30 } 30 }
31
32 // dropTable removes a table that is no longer part of the schema, so an
33 // existing database stops carrying it. The schema const only creates tables;
34 // removing one has to go through here for the same reason adding a column does.
35 //
36 // table is interpolated verbatim (SQLite cannot bind identifiers): pass trusted
37 // compile-time constants only, never caller- or request-derived strings — the
38 // same rule ensureColumn states, and for the same reason.
39 func dropTable(db *sql.DB, table string) error {
40 if _, err := db.Exec(fmt.Sprintf(`DROP TABLE IF EXISTS %s`, table)); err != nil {
41 return fmt.Errorf("drop %s: %w", table, err)
42 }
43 return nil
44 }
internal/server/store/store.go
Old New
@@ -166,12 +166,6 @@ CREATE TABLE IF NOT EXISTS vms (
166 -- vms name uniqueness is per-tenant. 166 -- vms name uniqueness is per-tenant.
167 CREATE UNIQUE INDEX IF NOT EXISTS vms_tenant_name ON vms(tenant, name) WHERE deleted_at IS NULL; 167 CREATE UNIQUE INDEX IF NOT EXISTS vms_tenant_name ON vms(tenant, name) WHERE deleted_at IS NULL;
168 168
169 -- bridge CIDRs returned to the pool by host decommission, available for reuse
170 -- before the monotonic next_cidr_index allocator is consulted.
171 CREATE TABLE IF NOT EXISTS freed_cidrs (
172 bridge_cidr TEXT PRIMARY KEY
173 );
174
175 -- revoked SSH user certs: a tenant can revoke a specific user cert by its serial 169 -- revoked SSH user certs: a tenant can revoke a specific user cert by its serial
176 -- (crypto-random uint64, set at mint) so it is rejected at the jump gate before 170 -- (crypto-random uint64, set at mint) so it is rejected at the jump gate before
177 -- its short TTL expires. serial is stored as the int64 bit-pattern of the uint64 171 -- its short TTL expires. serial is stored as the int64 bit-pattern of the uint64
@@ -294,6 +288,15 @@ func Open(path, cidrPool string) (*Store, error) {
294 return nil, err 288 return nil, err
295 } 289 }
296 } 290 }
291 // freed_cidrs held bridge CIDRs returned by a departing host, to be re-issued
292 // before the monotonic allocator was consulted. Nothing is recycled now: the
293 // column holds what the HOST claims, so re-issuing it would hand one host's
294 // subnet to another as a suggestion.
295 if err := dropTable(db, "freed_cidrs"); err != nil {
296 db.Close()
297 return nil, err
298 }
299
297 // One identity binds at most one tenant (per issuer). Partial index so 300 // One identity binds at most one tenant (per issuer). Partial index so
298 // unbound rows (empty issuer+subject) don't collide. 301 // unbound rows (empty issuer+subject) don't collide.
299 if _, err := db.Exec(`CREATE UNIQUE INDEX IF NOT EXISTS tenants_identity 302 if _, err := db.Exec(`CREATE UNIQUE INDEX IF NOT EXISTS tenants_identity
@@ -378,12 +381,30 @@ func (s *Store) CreateEnrollmentToken(tenant string) (string, error) {
378 return tok, nil 381 return tok, nil
379 } 382 }
380 383
384 // EnrollFacts is what a joining host says about itself. It is a struct rather
385 // than more positional strings because the enrollment transaction already
386 // carried six of them, and because BridgeCIDR needs three states that a plain
387 // string cannot express — see its field.
388 type EnrollFacts struct {
389 Name, OS, Arch, Provisioner string
390 // Remote is the client address, recorded in the audit row.
391 Remote string
392 // BridgeCIDR is the subnet this host says its guests are on, and it is a
393 // POINTER because absent and empty mean different things. Nil is "this host
394 // has no opinion" — an older agent, or a Linux host started without
395 // --bridge-cidr — and takes the pool's suggestion. A non-nil empty string is
396 // "this host deliberately has none", which is what a Mac says: its OS owns
397 // the guest network and will report the subnet later, so allocating one for
398 // it would invent a fact. A non-nil value is stored as given.
399 BridgeCIDR *string
400 }
401
381 // RedeemEnrollmentToken atomically consumes tok and creates the host row. The 402 // RedeemEnrollmentToken atomically consumes tok and creates the host row. The
382 // created host inherits the tenant bound to the consumed token. remote (the 403 // created host inherits the tenant bound to the consumed token. remote (the
383 // enrolling client's IP) is recorded in a host.enroll audit row written in 404 // enrolling client's IP) is recorded in a host.enroll audit row written in
384 // the SAME transaction, so an enrolled host can never exist without its 405 // the SAME transaction, so an enrolled host can never exist without its
385 // durable audit record. 406 // durable audit record.
386 func (s *Store) RedeemEnrollmentToken(tok, name, osName, arch, provisioner, remote string) (Host, error) { 407 func (s *Store) RedeemEnrollmentToken(tok string, f EnrollFacts) (Host, error) {
387 h := sha256.Sum256([]byte(tok)) 408 h := sha256.Sum256([]byte(tok))
388 hash := hex.EncodeToString(h[:]) 409 hash := hex.EncodeToString(h[:])
389 410
@@ -427,14 +448,29 @@ func (s *Store) RedeemEnrollmentToken(tok, name, osName, arch, provisioner, remo
427 return Host{}, fmt.Errorf("parse cidr_pool: %w", err) 448 return Host{}, fmt.Errorf("parse cidr_pool: %w", err)
428 } 449 }
429 450
430 // Reuse a CIDR freed by a prior decommission before extending the monotonic 451 // The host proposes; the fleet only suggests. A host that states its subnet
431 // allocator, so the pool doesn't leak across host churn. 452 // is recorded as given — validated as a network, never checked against the
453 // pool, because a host whose OS owns the guest network is on a subnet no
454 // allocation of ours will ever contain.
455 //
456 // Nothing is recycled back into the pool. Post-inversion this column holds
457 // what the HOST claims, so returning it would hand a Mac's 192.168.64.0/24
458 // to the next Linux host as its suggested bridge. A /16 pool is 65k /24s and
459 // the monotonic index alone is sufficient: leaking indices is cheaper than
460 // re-issuing a value the fleet did not choose.
432 var bridgeCIDR string 461 var bridgeCIDR string
433 if err := tx.QueryRow(`SELECT bridge_cidr FROM freed_cidrs ORDER BY bridge_cidr LIMIT 1`).Scan(&bridgeCIDR); err == nil { 462 switch {
434 if _, err := tx.Exec(`DELETE FROM freed_cidrs WHERE bridge_cidr=?`, bridgeCIDR); err != nil { 463 case f.BridgeCIDR != nil && *f.BridgeCIDR != "":
435 return Host{}, fmt.Errorf("consume freed cidr: %w", err) 464 if err := validGuestCIDR(*f.BridgeCIDR); err != nil {
465 return Host{}, err
436 } 466 }
437 } else if err == sql.ErrNoRows { 467 bridgeCIDR = *f.BridgeCIDR
468 case f.BridgeCIDR != nil:
469 // Deliberately none: this host will report its subnet once it can see
470 // it. Allocating one here would invent a fact and put it on the row.
471 bridgeCIDR = ""
472 default:
473 // No opinion — an older agent, or a host that wants the fleet's advice.
438 bridgeCIDR, err = subnetForIndex(prefix, nextIdx) 474 bridgeCIDR, err = subnetForIndex(prefix, nextIdx)
439 if err != nil { 475 if err != nil {
440 return Host{}, err 476 return Host{}, err
@@ -442,15 +478,13 @@ func (s *Store) RedeemEnrollmentToken(tok, name, osName, arch, provisioner, remo
442 if _, err := tx.Exec(`UPDATE meta SET value=? WHERE key='next_cidr_index'`, nextIdx+1); err != nil { 478 if _, err := tx.Exec(`UPDATE meta SET value=? WHERE key='next_cidr_index'`, nextIdx+1); err != nil {
443 return Host{}, fmt.Errorf("increment next_cidr_index: %w", err) 479 return Host{}, fmt.Errorf("increment next_cidr_index: %w", err)
444 } 480 }
445 } else {
446 return Host{}, fmt.Errorf("read freed_cidrs: %w", err)
447 } 481 }
448 482
449 id := random.Hex(16) 483 id := random.Hex(16)
450 484
451 if _, err := tx.Exec( 485 if _, err := tx.Exec(
452 `INSERT INTO hosts(id, name, os, arch, provisioner, bridge_cidr, enrolled_at, tenant) VALUES (?,?,?,?,?,?,?,?)`, 486 `INSERT INTO hosts(id, name, os, arch, provisioner, bridge_cidr, enrolled_at, tenant) VALUES (?,?,?,?,?,?,?,?)`,
453 id, name, osName, arch, provisioner, bridgeCIDR, now.Format(time.RFC3339), tenant, 487 id, f.Name, f.OS, f.Arch, f.Provisioner, bridgeCIDR, now.Format(time.RFC3339), tenant,
454 ); err != nil { 488 ); err != nil {
455 return Host{}, fmt.Errorf("insert host: %w", err) 489 return Host{}, fmt.Errorf("insert host: %w", err)
456 } 490 }
@@ -459,8 +493,8 @@ func (s *Store) RedeemEnrollmentToken(tok, name, osName, arch, provisioner, remo
459 // once the enrollment itself commits. The token appears only as a hash 493 // once the enrollment itself commits. The token appears only as a hash
460 // prefix (matches the enrollment_tokens.token_hash the mint row logs). 494 // prefix (matches the enrollment_tokens.token_hash the mint row logs).
461 detail, _ := json.Marshal(map[string]string{ 495 detail, _ := json.Marshal(map[string]string{
462 "host_id": id, "name": name, "os": osName, "arch": arch, 496 "host_id": id, "name": f.Name, "os": f.OS, "arch": f.Arch,
463 "remote": remote, "token_hash_prefix": hash[:8], 497 "remote": f.Remote, "token_hash_prefix": hash[:8],
464 }) 498 })
465 if _, err := tx.Exec(`INSERT INTO audit_log(at, tenant, action, detail) VALUES (?, ?, ?, ?)`, 499 if _, err := tx.Exec(`INSERT INTO audit_log(at, tenant, action, detail) VALUES (?, ?, ?, ?)`,
466 now.Format(time.RFC3339), tenant, "host.enroll", string(detail)); err != nil { 500 now.Format(time.RFC3339), tenant, "host.enroll", string(detail)); err != nil {
@@ -473,10 +507,10 @@ func (s *Store) RedeemEnrollmentToken(tok, name, osName, arch, provisioner, remo
473 507
474 return Host{ 508 return Host{
475 ID: id, 509 ID: id,
476 Name: name, 510 Name: f.Name,
477 OS: osName, 511 OS: f.OS,
478 Arch: arch, 512 Arch: f.Arch,
479 Provisioner: provisioner, 513 Provisioner: f.Provisioner,
480 BridgeCIDR: bridgeCIDR, 514 BridgeCIDR: bridgeCIDR,
481 Status: "enrolled", 515 Status: "enrolled",
482 CredGeneration: 1, 516 CredGeneration: 1,
@@ -498,6 +532,26 @@ func (s *Store) GetHost(id string) (Host, error) {
498 return h, nil 532 return h, nil
499 } 533 }
500 534
535 // validGuestCIDR is the sanity check applied to every subnet a host claims,
536 // wherever it arrives — proposed at enrollment, or reported in a tick. It is
537 // the same check netenv makes before handing the value to the kernel, applied
538 // on the receiving side so a garbage string cannot reach the console, an
539 // operator's eyes, or anything that reads the row.
540 //
541 // It asks whether the value is a network. It never asks WHICH network: a host's
542 // guest subnet is that host's business, and applying sanity to one of a row's
543 // two network columns and not the other would be sanity by coincidence.
544 func validGuestCIDR(cidr string) error {
545 p, err := netip.ParsePrefix(cidr)
546 if err != nil {
547 return fmt.Errorf("guest cidr %q: %w", cidr, err)
548 }
549 if !p.Addr().Is4() {
550 return fmt.Errorf("guest cidr %q must be IPv4", cidr)
551 }
552 return nil
553 }
554
501 // RecordHostNetwork stores the subnet a host says its guests are on. The value 555 // RecordHostNetwork stores the subnet a host says its guests are on. The value
502 // is free text supplied by whoever claims to be that host, so it is validated 556 // is free text supplied by whoever claims to be that host, so it is validated
503 // the same way netenv validates it before handing it to the kernel — parses as 557 // the same way netenv validates it before handing it to the kernel — parses as
@@ -511,14 +565,10 @@ func (s *Store) GetHost(id string) (Host, error) {
511 // the storage layer; the caller still guards the round trip, because reports 565 // the storage layer; the caller still guards the round trip, because reports
512 // arrive every tick forever and the store runs on one connection. 566 // arrive every tick forever and the store runs on one connection.
513 func (s *Store) RecordHostNetwork(id, cidr string) error { 567 func (s *Store) RecordHostNetwork(id, cidr string) error {
514 p, err := netip.ParsePrefix(cidr) 568 if err := validGuestCIDR(cidr); err != nil {
515 if err != nil { 569 return err
516 return fmt.Errorf("guest cidr %q: %w", cidr, err)
517 }
518 if !p.Addr().Is4() {
519 return fmt.Errorf("guest cidr %q must be IPv4", cidr)
520 } 570 }
521 _, err = s.db.Exec(`UPDATE hosts SET bridge_cidr=? WHERE id=? AND bridge_cidr<>?`, cidr, id, cidr) 571 _, err := s.db.Exec(`UPDATE hosts SET bridge_cidr=? WHERE id=? AND bridge_cidr<>?`, cidr, id, cidr)
522 return err 572 return err
523 } 573 }
524 574
@@ -1014,13 +1064,6 @@ func (s *Store) RemoveHost(id string) error {
1014 return fmt.Errorf("host %s still has %d VM(s); not drained", id, n) 1064 return fmt.Errorf("host %s still has %d VM(s); not drained", id, n)
1015 } 1065 }
1016 1066
1017 var bridgeCIDR string
1018 if err := tx.QueryRow(`SELECT bridge_cidr FROM hosts WHERE id=?`, id).Scan(&bridgeCIDR); err != nil {
1019 return fmt.Errorf("lookup host cidr: %w", err)
1020 }
1021 if _, err := tx.Exec(`INSERT INTO freed_cidrs(bridge_cidr) VALUES(?) ON CONFLICT DO NOTHING`, bridgeCIDR); err != nil {
1022 return fmt.Errorf("free cidr: %w", err)
1023 }
1024 res, err := tx.Exec(`DELETE FROM hosts WHERE id=?`, id) 1067 res, err := tx.Exec(`DELETE FROM hosts WHERE id=?`, id)
1025 if err != nil { 1068 if err != nil {
1026 return fmt.Errorf("delete host: %w", err) 1069 return fmt.Errorf("delete host: %w", err)
@@ -1047,12 +1090,14 @@ func (s *Store) ForceRemoveHost(id string) (int, error) {
1047 } 1090 }
1048 defer tx.Rollback() 1091 defer tx.Rollback()
1049 1092
1050 var bridgeCIDR string 1093 // Existence check only: nothing is reclaimed from a departing host, so the
1051 switch err := tx.QueryRow(`SELECT bridge_cidr FROM hosts WHERE id=?`, id).Scan(&bridgeCIDR); { 1094 // row's contents no longer matter here.
1095 var exists int
1096 switch err := tx.QueryRow(`SELECT 1 FROM hosts WHERE id=?`, id).Scan(&exists); {
1052 case errors.Is(err, sql.ErrNoRows): 1097 case errors.Is(err, sql.ErrNoRows):
1053 return 0, sql.ErrNoRows 1098 return 0, sql.ErrNoRows
1054 case err != nil: 1099 case err != nil:
1055 return 0, fmt.Errorf("lookup host cidr: %w", err) 1100 return 0, fmt.Errorf("lookup host: %w", err)
1056 } 1101 }
1057 1102
1058 res, err := tx.Exec(`DELETE FROM vms WHERE host_id=?`, id) 1103 res, err := tx.Exec(`DELETE FROM vms WHERE host_id=?`, id)
@@ -1061,9 +1106,6 @@ func (s *Store) ForceRemoveHost(id string) (int, error) {
1061 } 1106 }
1062 purged, _ := res.RowsAffected() 1107 purged, _ := res.RowsAffected()
1063 1108
1064 if _, err := tx.Exec(`INSERT INTO freed_cidrs(bridge_cidr) VALUES(?) ON CONFLICT DO NOTHING`, bridgeCIDR); err != nil {
1065 return 0, fmt.Errorf("free cidr: %w", err)
1066 }
1067 if _, err := tx.Exec(`DELETE FROM hosts WHERE id=?`, id); err != nil { 1109 if _, err := tx.Exec(`DELETE FROM hosts WHERE id=?`, id); err != nil {
1068 return 0, fmt.Errorf("delete host: %w", err) 1110 return 0, fmt.Errorf("delete host: %w", err)
1069 } 1111 }
internal/server/store/store_test.go
Old New
@@ -39,7 +39,7 @@ func enrollHost(t *testing.T, s *Store) Host {
39 t.Helper() 39 t.Helper()
40 tok, err := s.CreateEnrollmentToken(testTenant) 40 tok, err := s.CreateEnrollmentToken(testTenant)
41 require.NoError(t, err) 41 require.NoError(t, err)
42 h, err := s.RedeemEnrollmentToken(tok, "host-a", "linux", "amd64", "cloudhv", "") 42 h, err := s.RedeemEnrollmentToken(tok, EnrollFacts{Name: "host-a", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
43 require.NoError(t, err) 43 require.NoError(t, err)
44 return h 44 return h
45 } 45 }
@@ -244,15 +244,15 @@ func TestTenantUserCAs(t *testing.T) {
244 func TestEnrollmentAssignsSequentialCIDRsAndIsOneTimeUse(t *testing.T) { 244 func TestEnrollmentAssignsSequentialCIDRsAndIsOneTimeUse(t *testing.T) {
245 s := newStore(t) 245 s := newStore(t)
246 tok1, _ := s.CreateEnrollmentToken(testTenant) 246 tok1, _ := s.CreateEnrollmentToken(testTenant)
247 h1, err := s.RedeemEnrollmentToken(tok1, "a", "linux", "amd64", "cloudhv", "") 247 h1, err := s.RedeemEnrollmentToken(tok1, EnrollFacts{Name: "a", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
248 require.NoError(t, err) 248 require.NoError(t, err)
249 assert.Equal(t, "10.77.1.0/24", h1.BridgeCIDR) 249 assert.Equal(t, "10.77.1.0/24", h1.BridgeCIDR)
250 250
251 _, err = s.RedeemEnrollmentToken(tok1, "b", "linux", "amd64", "cloudhv", "") 251 _, err = s.RedeemEnrollmentToken(tok1, EnrollFacts{Name: "b", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
252 assert.Error(t, err, "token must be one-time use") 252 assert.Error(t, err, "token must be one-time use")
253 253
254 tok2, _ := s.CreateEnrollmentToken(testTenant) 254 tok2, _ := s.CreateEnrollmentToken(testTenant)
255 h2, _ := s.RedeemEnrollmentToken(tok2, "b", "linux", "amd64", "cloudhv", "") 255 h2, _ := s.RedeemEnrollmentToken(tok2, EnrollFacts{Name: "b", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
256 assert.Equal(t, "10.77.2.0/24", h2.BridgeCIDR) 256 assert.Equal(t, "10.77.2.0/24", h2.BridgeCIDR)
257 } 257 }
258 258
@@ -368,7 +368,7 @@ func TestEnrollmentCIDRWorksForNonSlash16Pools(t *testing.T) {
368 _, err = s.CreateTenantForIdentity("https://test-issuer", "test-subject", testTenant+"@test.local") 368 _, err = s.CreateTenantForIdentity("https://test-issuer", "test-subject", testTenant+"@test.local")
369 require.NoError(t, err) 369 require.NoError(t, err)
370 tok, _ := s.CreateEnrollmentToken(testTenant) 370 tok, _ := s.CreateEnrollmentToken(testTenant)
371 h, err := s.RedeemEnrollmentToken(tok, "a", "linux", "amd64", "cloudhv", "") 371 h, err := s.RedeemEnrollmentToken(tok, EnrollFacts{Name: "a", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
372 require.NoError(t, err) 372 require.NoError(t, err)
373 assert.Equal(t, "192.168.5.0/24", h.BridgeCIDR, "1st /24 within the pool, 0th reserved") 373 assert.Equal(t, "192.168.5.0/24", h.BridgeCIDR, "1st /24 within the pool, 0th reserved")
374 } 374 }
@@ -380,11 +380,11 @@ func TestEnrollmentFailsWhenPoolExhausted(t *testing.T) {
380 _, err = s.CreateTenantForIdentity("https://test-issuer", "test-subject", testTenant+"@test.local") 380 _, err = s.CreateTenantForIdentity("https://test-issuer", "test-subject", testTenant+"@test.local")
381 require.NoError(t, err) 381 require.NoError(t, err)
382 tok1, _ := s.CreateEnrollmentToken(testTenant) 382 tok1, _ := s.CreateEnrollmentToken(testTenant)
383 h, err := s.RedeemEnrollmentToken(tok1, "a", "linux", "amd64", "cloudhv", "") 383 h, err := s.RedeemEnrollmentToken(tok1, EnrollFacts{Name: "a", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
384 require.NoError(t, err) 384 require.NoError(t, err)
385 assert.Equal(t, "10.9.9.0/24", h.BridgeCIDR) 385 assert.Equal(t, "10.9.9.0/24", h.BridgeCIDR)
386 tok2, _ := s.CreateEnrollmentToken(testTenant) 386 tok2, _ := s.CreateEnrollmentToken(testTenant)
387 _, err = s.RedeemEnrollmentToken(tok2, "b", "linux", "amd64", "cloudhv", "") 387 _, err = s.RedeemEnrollmentToken(tok2, EnrollFacts{Name: "b", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
388 assert.ErrorContains(t, err, "exhausted") 388 assert.ErrorContains(t, err, "exhausted")
389 } 389 }
390 390
@@ -394,7 +394,7 @@ func TestRecordVMStatusUnknownVMErrors(t *testing.T) {
394 assert.Error(t, err) 394 assert.Error(t, err)
395 } 395 }
396 396
397 func TestForceRemoveHostPurgesVMsAndFreesCIDR(t *testing.T) { 397 func TestForceRemoveHostPurgesVMs(t *testing.T) {
398 s := newStore(t) 398 s := newStore(t)
399 h := enrollHost(t, s) 399 h := enrollHost(t, s)
400 makeVM(t, s, h, "vm-a") 400 makeVM(t, s, h, "vm-a")
@@ -419,13 +419,14 @@ func TestForceRemoveHostPurgesVMsAndFreesCIDR(t *testing.T) {
419 assert.NotEqual(t, h.ID, vm.HostID, "no VM rows should remain for the force-removed host") 419 assert.NotEqual(t, h.ID, vm.HostID, "no VM rows should remain for the force-removed host")
420 } 420 }
421 421
422 // The freed CIDR is consulted before the monotonic allocator, so a fresh 422 // Its subnet is not returned to the pool: the column holds what the HOST
423 // enrollment reuses it. 423 // claimed, and re-issuing that to the next host would advise it to build a
424 // network somebody else chose.
424 tok, err := s.CreateEnrollmentToken(testTenant) 425 tok, err := s.CreateEnrollmentToken(testTenant)
425 require.NoError(t, err) 426 require.NoError(t, err)
426 h2, err := s.RedeemEnrollmentToken(tok, "host-b", "linux", "amd64", "cloudhv", "") 427 h2, err := s.RedeemEnrollmentToken(tok, EnrollFacts{Name: "host-b", OS: "linux", Arch: "amd64", Provisioner: "cloudhv"})
427 require.NoError(t, err) 428 require.NoError(t, err)
428 assert.Equal(t, h.BridgeCIDR, h2.BridgeCIDR, "force-removed host's CIDR must return to the pool") 429 assert.NotEqual(t, h.BridgeCIDR, h2.BridgeCIDR, "a force-removed host's subnet is never re-issued")
429 } 430 }
430 431
431 func TestForceRemoveHostUnknownIsNoRows(t *testing.T) { 432 func TestForceRemoveHostUnknownIsNoRows(t *testing.T) {
@@ -595,7 +596,7 @@ func TestRedeemWritesAuditRowAtomically(t *testing.T) {
595 s := newStore(t) 596 s := newStore(t)
596 tok, err := s.CreateEnrollmentToken(testTenant) 597 tok, err := s.CreateEnrollmentToken(testTenant)
597 require.NoError(t, err) 598 require.NoError(t, err)
598 h, err := s.RedeemEnrollmentToken(tok, "host-a", "linux", "amd64", "cloudhv", "192.0.2.9") 599 h, err := s.RedeemEnrollmentToken(tok, EnrollFacts{Name: "host-a", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: "192.0.2.9"})
599 require.NoError(t, err) 600 require.NoError(t, err)
600 601
601 rows, err := s.ListAudit(testTenant, 5) 602 rows, err := s.ListAudit(testTenant, 5)
@@ -756,7 +757,7 @@ func TestEnrollmentCarriesTenantToHost(t *testing.T) {
756 757
757 tok, err := s.CreateEnrollmentToken("t2") 758 tok, err := s.CreateEnrollmentToken("t2")
758 require.NoError(t, err) 759 require.NoError(t, err)
759 h, err := s.RedeemEnrollmentToken(tok, "box", "linux", "amd64", "cloudhv", "203.0.113.9") 760 h, err := s.RedeemEnrollmentToken(tok, EnrollFacts{Name: "box", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: "203.0.113.9"})
760 require.NoError(t, err) 761 require.NoError(t, err)
761 assert.Equal(t, "t2", h.Tenant, "host must inherit the token's tenant") 762 assert.Equal(t, "t2", h.Tenant, "host must inherit the token's tenant")
762 got, err := s.GetHost(h.ID) 763 got, err := s.GetHost(h.ID)
internal/server/syncsvc/syncsvc_test.go
Old New
@@ -56,7 +56,7 @@ func setupWithWriteTimeout(t *testing.T, writeTimeout time.Duration) *fixture {
56 t.Cleanup(func() { st.Close() }) 56 t.Cleanup(func() { st.Close() })
57 seedTestTenant(t, st) 57 seedTestTenant(t, st)
58 tok, _ := st.CreateEnrollmentToken(testTenant) 58 tok, _ := st.CreateEnrollmentToken(testTenant)
59 host, err := st.RedeemEnrollmentToken(tok, "h", "linux", "amd64", "cloudhv", "") 59 host, err := st.RedeemEnrollmentToken(tok, store.EnrollFacts{Name: "h", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
60 require.NoError(t, err) 60 require.NoError(t, err)
61 61
62 reg := registry.New(time.Now) 62 reg := registry.New(time.Now)
@@ -543,7 +543,7 @@ func TestExpiredCredentialRejectedWhenMaxAgeSet(t *testing.T) {
543 t.Cleanup(func() { st.Close() }) 543 t.Cleanup(func() { st.Close() })
544 seedTestTenant(t, st) 544 seedTestTenant(t, st)
545 tok, _ := st.CreateEnrollmentToken(testTenant) 545 tok, _ := st.CreateEnrollmentToken(testTenant)
546 host, err := st.RedeemEnrollmentToken(tok, "h", "linux", "amd64", "cloudhv", "") 546 host, err := st.RedeemEnrollmentToken(tok, store.EnrollFacts{Name: "h", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
547 require.NoError(t, err) 547 require.NoError(t, err)
548 548
549 reg := registry.New(time.Now) 549 reg := registry.New(time.Now)
@@ -586,7 +586,7 @@ func TestMaxAgeEnforcedMidSession(t *testing.T) {
586 t.Cleanup(func() { st.Close() }) 586 t.Cleanup(func() { st.Close() })
587 seedTestTenant(t, st) 587 seedTestTenant(t, st)
588 tok, _ := st.CreateEnrollmentToken(testTenant) 588 tok, _ := st.CreateEnrollmentToken(testTenant)
589 host, err := st.RedeemEnrollmentToken(tok, "h", "linux", "amd64", "cloudhv", "") 589 host, err := st.RedeemEnrollmentToken(tok, store.EnrollFacts{Name: "h", OS: "linux", Arch: "amd64", Provisioner: "cloudhv", Remote: ""})
590 require.NoError(t, err) 590 require.NoError(t, err)
591 591
592 reg := registry.New(time.Now) 592 reg := registry.New(time.Now)
web/src/lib/api-types.ts
Old New
@@ -1271,6 +1271,7 @@ export interface components {
1271 }; 1271 };
1272 EnrollRequest: { 1272 EnrollRequest: {
1273 arch?: string; 1273 arch?: string;
1274 bridge_cidr?: string | null;
1274 name?: string; 1275 name?: string;
1275 os?: string; 1276 os?: string;
1276 provisioner?: string; 1277 provisioner?: string;
web/src/routes/+page.svelte
Old New
@@ -212,7 +212,7 @@
212 {#if fleet.hosts.length > 0 && shownHosts.length > 0} 212 {#if fleet.hosts.length > 0 && shownHosts.length > 0}
213 <table> 213 <table>
214 <thead> 214 <thead>
215 <tr><th>Name</th><th>Status</th><th>OS</th><th>Version</th><th>Load</th><th>VMs</th><th>CIDR</th><th>Used / total (vCPU · mem · disk)</th><th></th></tr> 215 <tr><th>Name</th><th>Status</th><th>OS</th><th>Version</th><th>Load</th><th>VMs</th><th>Guest network</th><th>Used / total (vCPU · mem · disk)</th><th></th></tr>
216 </thead> 216 </thead>
217 <tbody> 217 <tbody>
218 {#each shownHosts as h (h.id)} 218 {#each shownHosts as h (h.id)}
@@ -246,7 +246,7 @@
246 {/if} 246 {/if}
247 </td> 247 </td>
248 <td>{vmCountByHost.get(h.id) ?? 0}</td> 248 <td>{vmCountByHost.get(h.id) ?? 0}</td>
249 <td>{h.bridge_cidr}</td> 249 <td>{h.bridge_cidr || '—'}</td>
250 <td> 250 <td>
251 {h.allocated.vcpus}/{h.capacity.vcpus || '?'}c · 251 {h.allocated.vcpus}/{h.capacity.vcpus || '?'}c ·
252 {h.allocated.mem_mb}/{h.capacity.mem_mb || '?'}MB · 252 {h.allocated.mem_mb}/{h.capacity.mem_mb || '?'}MB ·
web/src/routes/hosts/[id]/+page.svelte
Old New
@@ -63,7 +63,16 @@
63 <tr><th>CPU</th><td>{host.cpu_model || '—'}</td></tr> 63 <tr><th>CPU</th><td>{host.cpu_model || '—'}</td></tr>
64 <tr><th>Virtualization</th><td>{host.virt || '—'}</td></tr> 64 <tr><th>Virtualization</th><td>{host.virt || '—'}</td></tr>
65 <tr><th>Provisioner</th><td>{host.provisioner}</td></tr> 65 <tr><th>Provisioner</th><td>{host.provisioner}</td></tr>
66 <tr><th>Bridge CIDR</th><td>{host.bridge_cidr}</td></tr> 66 <tr>
67 <th>Guest network</th>
68 <td>
69 {#if host.bridge_cidr}
70 {host.bridge_cidr}
71 {:else}
72 <span class="hint">not yet reported</span>
73 {/if}
74 </td>
75 </tr>
67 <tr><th>Capacity</th><td>{host.capacity.vcpus}c / {host.capacity.mem_mb}MB / {host.capacity.disk_gb}GB</td></tr> 76 <tr><th>Capacity</th><td>{host.capacity.vcpus}c / {host.capacity.mem_mb}MB / {host.capacity.disk_gb}GB</td></tr>
68 <tr><th>Enrolled</th><td>{host.enrolled_at}</td></tr> 77 <tr><th>Enrolled</th><td>{host.enrolled_at}</td></tr>
69 </tbody> 78 </tbody>