a73x

7dd7ae3c

feat(exposures): a published port can carry datagrams

a73x   2026-08-09 18:21

Commit message
feat(exposures): a published port can carry datagrams

An exposure names its protocol. `tcp` is the default and everything a caller
already wrote means the same thing it did; `udp` publishes a host port that
forwards datagrams to the guest, and the API takes those two values and no
others — a third would become a grant no host ever binds.

A host port is claimed per protocol, so 30080/tcp and 30080/udp are two
services. The unique index carries the pair, and a database that predates it
drops the index that spanned only (host, port) when it opens. Allocation stays
protocol-blind: a caller that names no port gets a number nothing on that host
holds either way, because the number is the thing it hands to someone else, and
the reserved range is deep enough that the ports this leaves on the table cost
nothing that reasoning about half-taken numbers would cost more.

The agent forwards datagrams beside its TCP splices. A session is one client
address and one socket connected to the guest, whose address is resolved when
the session opens and pinned there — never re-resolved per packet, because a
proxy that re-resolves mid-conversation can quietly start sending a guest's
traffic to whatever took its address over. A converge drops a pin that no
longer matches, an error on the guest socket ends the session (which is where
an ICMP port-unreachable arrives), and idle ends it too: thirty seconds for a
session the guest has never answered, two minutes once it has, so a port scan
cannot fill the table with conversations that were never real. The table is
capped like the connection table beside it, and a full one refuses new clients
rather than sacrificing a live conversation for a stranger's first packet.

That is the whole posture, stated in the docs beside the auth answer: UDP
proves nothing about where a datagram came from, so the host answers the
address the datagram claimed, and a published UDP port can be aimed at a third
party. It is published onto a network the tenant is content to serve, which is
what a published port has always meant here.

Two things the connection cap needed while the file was open. A spec change now
rebinds inside the exposure entry instead of replacing it, so connections
draining under the old listener still count against the cap they were accepted
under. And a descriptor shortage says so in what the exposure reports: the port
stays bound, which is what "active" means, and an operator looking at a
published port nothing is getting through gets the sentence that explains it.

The boot gate proves it end to end. The MCP leg starts a UDP echo inside its
own guest, publishes the port over the same tool a model would use, and
requires its datagram back.

docs/assumptions.md
Old New
@@ -358,13 +358,24 @@ error and retried on every converge.
358 358
359 ### A published port may be bound on every interface 359 ### A published port may be bound on every interface
360 360
361 The listener binds 0.0.0.0—every interface, IPv4 only. An exposure is 361 The socket binds 0.0.0.0—every interface, IPv4 only. An exposure is
362 reachable on any IPv4 address the host has, and on none of its IPv6 ones. 362 reachable on any IPv4 address the host has, and on none of its IPv6 ones.
363 Underpins there being one exposure rather than one per network. 363 Underpins there being one exposure rather than one per network.
364 **Proven** in code; the posture is deliberate—see the auth answer in 364 **Proven** in code; the posture is deliberate—see the auth answer in
365 [faq.md](faq.md). A host whose networks are not equally trusted gets more 365 [faq.md](faq.md). A host whose networks are not equally trusted gets more
366 reach than it asked for. 366 reach than it asked for.
367 367
368 ### A published UDP port may be used as a reflector
369
370 UDP has no handshake, so a datagram's source address is a claim and nothing
371 more. The host sends the guest's reply to whatever address that claim names,
372 which means a published UDP port can be aimed at a third party. Underpins
373 publishing UDP at all, on the same terms as TCP: reaching the host is reaching
374 the service, and a port is published onto a network the tenant is content to
375 serve. **True by construction**, not mitigated—there is no rate limit and no
376 source check in the proxy. A guest whose service amplifies (answers larger than
377 it is asked) makes the host a better reflector than a bare port would.
378
368 ### eitri holds no signing key for anyone 379 ### eitri holds no signing key for anyone
369 380
370 A caller lends eitri a credential instead: eitri offers an ephemeral public key, 381 A caller lends eitri a credential instead: eitri offers an ephemeral public key,
docs/decisions.md
Old New
@@ -83,7 +83,9 @@ not choose is never recycled back into the pool.
83 ### Networking stays minimal: just enough to be usable 83 ### Networking stays minimal: just enough to be usable
84 84
85 A guest gets an address, a way out, and the ports its tenant chose to publish— 85 A guest gets an address, a way out, and the ports its tenant chose to publish—
86 embedded DHCP, the host as NAT gateway, and a userspace port-forward table—any 86 embedded DHCP, the host as NAT gateway, and a userspace port-forward table for
87 TCP and UDP alike (a spliced connection per caller, a session per client
88 address; both capped, both deletable, neither routing anything else)—any
87 of which could be deleted with the VM still booting, syncing, and taking SSH. 89 of which could be deleted with the VM still booting, syncing, and taking SSH.
88 Instead of growing the agent toward a virtual router (guest DNS, ACLs, 90 Instead of growing the agent toward a virtual router (guest DNS, ACLs,
89 east-west, in-path policy). Network machinery that basic lifecycle would come 91 east-west, in-path policy). Network machinery that basic lifecycle would come
docs/faq.md
Old New
@@ -19,11 +19,18 @@ your tenant's CA. A published port has nothing in front of it: whoever can
19 reach the host on that port reaches the service, exactly as if the service were 19 reach the host on that port reaches the service, exactly as if the service were
20 running on the host itself. 20 running on the host itself.
21 21
22 That is the LAN trust posture, deliberately. The listener binds every interface 22 That is the LAN trust posture, deliberately. The socket binds every interface
23 on the host, IPv4 only. Publish what you are content to serve to everything 23 on the host, IPv4 only. Publish what you are content to serve to everything
24 that can reach it; put anything else behind the gate, or behind the service's 24 that can reach it; put anything else behind the gate, or behind the service's
25 own authentication. 25 own authentication.
26 26
27 A published UDP port is the same posture carried one step further. Nothing in
28 UDP proves where a datagram came from, so the host answers the address the
29 datagram claimed—someone else's, if that is what it said. A UDP exposure can
30 therefore be pointed at a third party, and a guest service that answers larger
31 than it is asked makes that worse. Publish UDP the way you would publish
32 anything else here: onto a network you are content to serve.
33
27 ## VMs boot and SSH works, but have no outbound network—why? 34 ## VMs boot and SSH works, but have no outbound network—why?
28 35
29 Docker. Installing (or starting) Docker on a host sets the kernel's iptables 36 Docker. Installing (or starting) Docker on a host sets the kernel's iptables
docs/mcp.md
Old New
@@ -22,9 +22,9 @@ Both serve the same tools.
22 | `vm_exec` | Run a shell command in a VM over SSH; returns stdout, stderr, exit code. | 22 | `vm_exec` | Run a shell command in a VM over SSH; returns stdout, stderr, exit code. |
23 | `vm_write_file` | Write content to a path in a VM over SFTP (parent dirs created). | 23 | `vm_write_file` | Write content to a path in a VM over SFTP (parent dirs created). |
24 | `vm_read_file` | Read a file from a VM over SFTP (capped at 1 MiB, truncation flagged). | 24 | `vm_read_file` | Read a file from a VM over SFTP (capped at 1 MiB, truncation flagged). |
25 | `vm_expose` | Publish a guest TCP port on the VM's host; returns the exposure id, both ports, the address to dial and the listener's state. Omit `host_port` to allocate one from 30000–32767. | 25 | `vm_expose` | Publish a guest port on the VM's host, `protocol` `tcp` (the default) or `udp`; returns the exposure id, both ports, the protocol, the address to dial and the socket's state. Omit `host_port` to allocate one from 30000–32767. |
26 | `vm_exposures` | List a VM's published ports, same shape. | 26 | `vm_exposures` | List a VM's published ports, same shape. |
27 | `vm_unexpose` | Stop publishing a guest port (`host_port` disambiguates when one guest port is published twice). | 27 | `vm_unexpose` | Stop publishing a guest port (`host_port` or `protocol` disambiguates when one guest port is published twice). |
28 | `vm_destroy` | Destroy a VM by id or exact name. Explicit-only—never called automatically. | 28 | `vm_destroy` | Destroy a VM by id or exact name. Explicit-only—never called automatically. |
29 | `ca_upload` | Register your SSH user CA's public key with your tenant, with an optional label. Only the public half is sent. A guest trusts the CA set it was created with, so VMs that already exist will not accept certificates from a CA uploaded now. | 29 | `ca_upload` | Register your SSH user CA's public key with your tenant, with an optional label. Only the public half is sent. A guest trusts the CA set it was created with, so VMs that already exist will not accept certificates from a CA uploaded now. |
30 | `tenant_info` | Show this tenant's setup: registered CAs (fingerprint and label), whether a delegation is live and when it expires, and the gate address. Read-only. | 30 | `tenant_info` | Show this tenant's setup: registered CAs (fingerprint and label), whether a delegation is live and when it expires, and the gate address. Read-only. |
@@ -174,9 +174,11 @@ result or error.
174 table above. 174 table above.
175 - A published port is **unauthenticated**. `vm_expose` binds a host port and 175 - A published port is **unauthenticated**. `vm_expose` binds a host port and
176 pipes it to the guest; whoever can reach the host on that port reaches the 176 pipes it to the guest; whoever can reach the host on that port reaches the
177 service. The tool descriptions say so, so the model treats publishing as a 177 service. A UDP one goes further: with no handshake to prove where a datagram
178 deliberate act. DNS, TLS certs and routing remain out of scope—the tools 178 came from, the host answers the address the datagram claimed, so the port can
179 hand back a host address and a port. 179 be aimed at a third party. The tool descriptions say so, so the model treats
180 publishing as a deliberate act. DNS, TLS certs and routing remain out of
181 scope—the tools hand back a host address and a port.
180 - The remote endpoint authenticates with a bearer PAT. Browser connectors 182 - The remote endpoint authenticates with a bearer PAT. Browser connectors
181 (claude.ai) need OAuth, which the endpoint does not speak. 183 (claude.ai) need OAuth, which the endpoint does not speak.
182 184
@@ -198,4 +200,5 @@ cover the tools and the HTTP transport against fake API and SSH seams. The
198 deploy boot-gate (`make deploy`) drives a whole VM life through the remote 200 deploy boot-gate (`make deploy`) drives a whole VM life through the remote
199 endpoint with a bearer PAT: register a CA, `delegate_begin`, sign, refuse a 201 endpoint with a bearer PAT: register a CA, `delegate_begin`, sign, refuse a
200 certificate from an unregistered CA, `delegate_complete`, `vm_create`, 202 certificate from an unregistered CA, `delegate_complete`, `vm_create`,
201 `vm_exec`, `vm_expose`, dial the published port, `vm_destroy`. 203 `vm_exec`, `vm_expose`, dial the published port, publish a UDP port and echo a
204 datagram through it, `vm_destroy`.
docs/openapi.json
Old New
@@ -110,6 +110,9 @@
110 }, 110 },
111 "host_port": { 111 "host_port": {
112 "type": "integer" 112 "type": "integer"
113 },
114 "protocol": {
115 "type": "string"
113 } 116 }
114 }, 117 },
115 "type": "object" 118 "type": "object"
@@ -2061,7 +2064,7 @@
2061 "patToken": [] 2064 "patToken": []
2062 } 2065 }
2063 ], 2066 ],
2064 "summary": "Publish a guest TCP port on the VM's host. Omit host_port to allocate one from the reserved range 30000-32767; a named port must be \u003e= 1024 and is honored or refused." 2067 "summary": "Publish a guest port on the VM's host, protocol \"tcp\" (the default) or \"udp\". Omit host_port to allocate one from the reserved range 30000-32767; a named port must be \u003e= 1024 and is honored or refused, and is taken only by another exposure of the same protocol."
2065 } 2068 }
2066 }, 2069 },
2067 "/api/v1/vms/{id}/restore": { 2070 "/api/v1/vms/{id}/restore": {
docs/quickstart.md
Old New
@@ -149,20 +149,31 @@ The gate reaches a guest over SSH. Anything else a guest serves needs a
149 published port: the fleet binds one on the VM's host and pipes it to the guest. 149 published port: the fleet binds one on the VM's host and pipes it to the guest.
150 150
151 Open the VM's page in the console, find **Exposures**, and enter the port your 151 Open the VM's page in the console, find **Exposures**, and enter the port your
152 service listens on inside the guest—8080, say. Leave the host port blank and 152 service listens on inside the guest—8080, say. Pick `tcp` or `udp`. Leave the
153 eitri allocates one from 30000–32767, the range it reserves on every host. The 153 host port blank and eitri allocates one from 30000–32767, the range it reserves
154 row then reads: 154 on every host. The row then reads:
155 155
156 guest :8080 → 192.168.0.190:30080 ● active 156 guest :8080/tcp → 192.168.0.190:30080 ● active
157 157
158 That address is the host's. Anything that can reach the host on that port 158 That address is the host's. Anything that can reach the host on that port
159 reaches the service—there is no authentication in front of a published port, so 159 reaches the service—there is no authentication in front of a published port, so
160 publish what you are content to serve to everything on that network, and leave 160 publish what you are content to serve to everything on that network, and leave
161 the rest to the gate. 161 the rest to the gate.
162 162
163 `active` means the host's listener is bound. Whether anything answers on the 163 A host port is claimed per protocol, so `30080/tcp` and `30080/udp` can be two
164 different services.
165
166 `active` means the host's socket is bound. Whether anything answers on the
164 guest's 8080 is the guest's business. 167 guest's 8080 is the guest's business.
165 168
169 A UDP exposure asks for one thing more of you than a TCP one. UDP has no
170 handshake, so nothing proves a datagram came from the address it claims: the
171 host sends the guest's reply wherever that source header says, which makes a
172 published UDP port something an outsider can aim at a third party. That is the
173 same bargain the rest of a published port already is—reaching the host is
174 reaching the service—so it is accepted rather than papered over, and it is one
175 more reason to publish only onto a network you are content to serve.
176
166 Remove the row to take the port down. Deleting the VM takes its exposures with 177 Remove the row to take the port down. Deleting the VM takes its exposures with
167 it. 178 it.
168 179
internal/agent/exposeproxy/exposeproxy.go
Old New
@@ -1,10 +1,15 @@
1 // Package exposeproxy publishes guest ports on their host. A Manager owns one 1 // Package exposeproxy publishes guest ports on their host. A Manager owns one
2 // listener and one goroutine per exposure id — the same manager shape as the 2 // socket and one goroutine per exposure id — the same manager shape as the
3 // serial pumps — and Converge drives that set toward the exposures the fleet 3 // serial pumps — and Converge drives that set toward the exposures the fleet
4 // says this host should be serving: a new id binds, a vanished id closes, a 4 // says this host should be serving: a new id binds, a vanished id closes, a
5 // changed spec closes and rebinds. 5 // changed spec closes and rebinds.
6 // 6 //
7 // The listener binds 0.0.0.0. All interfaces is the point: the LAN address is 7 // A TCP exposure is a listener and a spliced connection per caller; a UDP one
8 // is a single bound socket and a session per client address (udp.go). The
9 // protocol is part of an exposure's spec, so changing it is a rebind like any
10 // other change.
11 //
12 // The socket binds 0.0.0.0. All interfaces is the point: the LAN address is
8 // the feature, the tailnet address is how an operator reaches it from 13 // the feature, the tailnet address is how an operator reaches it from
9 // elsewhere, and the guest-side bridge seeing it is the same trust domain the 14 // elsewhere, and the guest-side bridge seeing it is the same trust domain the
10 // LAN posture already accepts. There is no authentication in front of a 15 // LAN posture already accepts. There is no authentication in front of a
@@ -12,10 +17,10 @@
12 // 17 //
13 // Each accepted connection asks for the VM's address AT THAT MOMENT and dials 18 // Each accepted connection asks for the VM's address AT THAT MOMENT and dials
14 // the guest fresh. That per-connection lookup is what makes an exposure created 19 // the guest fresh. That per-connection lookup is what makes an exposure created
15 // before its guest has booted simply work: the listener binds now, and 20 // before its guest has booted simply work: the socket binds now, and
16 // connections start succeeding when the guest does. 21 // connections start succeeding when the guest does.
17 // 22 //
18 // Nothing here is persisted. Listeners are rebuilt from the first snapshot 23 // Nothing here is persisted. Sockets are rebuilt from the first snapshot
19 // after the agent starts, the same way the consoles are; connections in flight 24 // after the agent starts, the same way the consoles are; connections in flight
20 // across an agent restart drop, and reconnecting works. 25 // across an agent restart drop, and reconnecting works.
21 // 26 //
@@ -57,7 +62,7 @@ const (
57 acceptRetryCeiling = time.Second 62 acceptRetryCeiling = time.Second
58 ) 63 )
59 64
60 // Manager runs one listener per active exposure. 65 // Manager runs one socket per active exposure.
61 type Manager struct { 66 type Manager struct {
62 // addr answers a VM's current guest address, or "" when this host does not 67 // addr answers a VM's current guest address, or "" when this host does not
63 // know one. Injected so the package stays a leaf: it never learns what a VM 68 // know one. Injected so the package stays a leaf: it never learns what a VM
@@ -65,10 +70,13 @@ type Manager struct {
65 // goroutines; it must be safe for concurrent use. 70 // goroutines; it must be safe for concurrent use.
66 addr func(vmID string) string 71 addr func(vmID string) string
67 72
68 // maxConns is maxConnsPerExposure, held per-Manager so a test can serve the 73 // maxConns, maxSessions and the two idle windows are the package constants,
69 // same behaviour with a handful of connections instead of hundreds. Written 74 // held per-Manager so a test can serve the same behaviour with a handful of
70 // once at construction and only read after, so it needs no lock. 75 // callers instead of hundreds, and with windows it can outlast. Written
71 maxConns int64 76 // once at construction and only read after, so they need no lock.
77 maxConns int64
78 maxSessions int
79 unrepliedIdle, repliedIdle time.Duration
72 80
73 mu sync.Mutex 81 mu sync.Mutex
74 live map[string]*exposure 82 live map[string]*exposure
@@ -77,47 +85,77 @@ type Manager struct {
77 // NewManager returns a Manager that resolves each connection's destination 85 // NewManager returns a Manager that resolves each connection's destination
78 // through addr. 86 // through addr.
79 func NewManager(addr func(vmID string) string) *Manager { 87 func NewManager(addr func(vmID string) string) *Manager {
80 return &Manager{addr: addr, maxConns: maxConnsPerExposure, live: map[string]*exposure{}} 88 return &Manager{
89 addr: addr,
90 maxConns: maxConnsPerExposure,
91 maxSessions: maxSessionsPerExposure,
92 unrepliedIdle: udpUnrepliedIdle,
93 repliedIdle: udpRepliedIdle,
94 live: map[string]*exposure{},
95 }
81 } 96 }
82 97
83 // exposure is one published port's running state: the spec key it was bound 98 // exposure is one published port's running state: the spec key it is bound for,
84 // for, its listener (nil when the bind failed), what the OS said if it did, and 99 // its socket — a listener for TCP, a packet socket for UDP, both nil when the
85 // how many connections it is holding open right now. 100 // bind failed — what the OS said if it did, and what it is holding open right
101 // now: connections for TCP, sessions for UDP.
86 // 102 //
87 // The count lives here rather than with the listener so that an exposure whose 103 // One entry lives per exposure id for as long as the fleet wants that id, and a
88 // accept loop died and was rebound keeps counting what is still draining 104 // changed spec closes and rebinds INSIDE it rather than replacing it. That is
89 // through it: the descriptors those connections hold are still spent. 105 // what keeps the count honest: connections spliced under the old spec drain
106 // through this same struct, and the descriptors they hold are still spent.
90 type exposure struct { 107 type exposure struct {
91 key string 108 key string
92 ln net.Listener 109 ln net.Listener
110 pc *net.UDPConn
93 reason string 111 reason string
94 conns atomic.Int64 112 conns atomic.Int64
113
114 // sessions is the UDP session table, guarded by its own mutex because the
115 // packet loop touches it on every datagram while the manager lock is held
116 // across whole converges. nil until the first UDP bind.
117 smu sync.Mutex
118 sessions map[string]*udpSession
95 } 119 }
96 120
97 // close tears down the listener, which is what ends its accept goroutine. 121 // bound reports whether this exposure currently holds a socket.
122 func (e *exposure) bound() bool { return e.ln != nil || e.pc != nil }
123
124 // close tears down the socket, which is what ends the goroutine serving it, and
125 // drops every UDP session with it: the sessions belong to the socket they were
126 // created on, and there is nothing left to answer them through.
98 func (e *exposure) close() { 127 func (e *exposure) close() {
99 if e.ln != nil { 128 if e.ln != nil {
100 e.ln.Close() 129 e.ln.Close()
101 e.ln = nil 130 e.ln = nil
102 } 131 }
132 if e.pc != nil {
133 e.pc.Close()
134 e.pc = nil
135 }
136 e.closeSessions()
103 } 137 }
104 138
105 // specKey renders everything about a desired exposure that a bound listener 139 // specKey renders everything about a desired exposure that a bound socket
106 // depends on. A change to any of it is a close and a rebind rather than an 140 // depends on. A change to any of it is a close and a rebind rather than an
107 // edit — the listener's own address is part of what changed. 141 // edit — the socket's own address, or the protocol it speaks, is part of what
142 // changed.
108 func specKey(d *pb.ExposureDesired) string { 143 func specKey(d *pb.ExposureDesired) string {
109 return fmt.Sprintf("%s/%d/%d/%s", d.GetVmId(), d.GetGuestPort(), d.GetHostPort(), d.GetProtocol()) 144 return fmt.Sprintf("%s/%d/%d/%s", d.GetVmId(), d.GetGuestPort(), d.GetHostPort(), d.GetProtocol())
110 } 145 }
111 146
112 // Converge drives the running listeners toward desired and reports what each 147 // Converge drives the running sockets toward desired and reports what each
113 // exposure is doing, in desired order. It is level-triggered: every call 148 // exposure is doing, in desired order. It is level-triggered: every call
114 // re-examines the whole set, so an exposure whose listener is gone — a failed 149 // re-examines the whole set, so an exposure whose socket is gone — a failed
115 // bind, or an accept loop that died — is retried here and heals the moment the 150 // bind, or a serving loop that died — is retried here and heals the moment the
116 // port frees. 151 // port frees.
117 // 152 //
118 // "active" means the listener is bound. Eitri owns the host half of the pipe; 153 // "active" means the socket is bound. Eitri owns the host half of the pipe;
119 // whether anything answers inside the guest is the guest's half, and this does 154 // whether anything answers inside the guest is the guest's half, and this does
120 // not pretend otherwise. 155 // not pretend otherwise. A bound exposure still carries a reason when something
156 // about serving it is going wrong — a descriptor shortage the accept loop is
157 // riding out — because a port that is bound and struggling is not the same
158 // thing as a port that is bound and fine.
121 func (m *Manager) Converge(desired []*pb.ExposureDesired) []*pb.ExposureActual { 159 func (m *Manager) Converge(desired []*pb.ExposureDesired) []*pb.ExposureActual {
122 m.mu.Lock() 160 m.mu.Lock()
123 defer m.mu.Unlock() 161 defer m.mu.Unlock()
@@ -127,7 +165,7 @@ func (m *Manager) Converge(desired []*pb.ExposureDesired) []*pb.ExposureActual {
127 want[d.GetId()] = d 165 want[d.GetId()] = d
128 } 166 }
129 for id, ex := range m.live { 167 for id, ex := range m.live {
130 if d, ok := want[id]; ok && ex.key == specKey(d) { 168 if _, ok := want[id]; ok {
131 continue 169 continue
132 } 170 }
133 ex.close() 171 ex.close()
@@ -137,26 +175,65 @@ func (m *Manager) Converge(desired []*pb.ExposureDesired) []*pb.ExposureActual {
137 out := make([]*pb.ExposureActual, 0, len(desired)) 175 out := make([]*pb.ExposureActual, 0, len(desired))
138 for _, d := range desired { 176 for _, d := range desired {
139 ex, ok := m.live[d.GetId()] 177 ex, ok := m.live[d.GetId()]
140 if !ok { 178 switch {
179 case !ok:
141 ex = &exposure{key: specKey(d)} 180 ex = &exposure{key: specKey(d)}
142 m.live[d.GetId()] = ex 181 m.live[d.GetId()] = ex
182 case ex.key != specKey(d):
183 // A rebind in place, not a fresh entry: the id is what the fleet
184 // named and what callers are still draining through.
185 ex.close()
186 ex.key, ex.reason = specKey(d), ""
143 } 187 }
144 if ex.ln == nil { 188 if !ex.bound() {
145 ln, err := net.Listen("tcp", net.JoinHostPort("0.0.0.0", strconv.Itoa(int(d.GetHostPort())))) 189 if err := m.bind(d, ex); err != nil {
146 if err != nil {
147 ex.reason = err.Error() 190 ex.reason = err.Error()
148 out = append(out, &pb.ExposureActual{Id: d.GetId(), State: "failed", Reason: ex.reason}) 191 out = append(out, &pb.ExposureActual{Id: d.GetId(), State: "failed", Reason: ex.reason})
149 continue 192 continue
150 } 193 }
151 ex.ln, ex.reason = ln, ""
152 go m.accept(d.GetId(), ex, ln, d.GetVmId(), d.GetGuestPort())
153 } 194 }
154 out = append(out, &pb.ExposureActual{Id: d.GetId(), State: "active"}) 195 // Where the guest is can change under a live session — a VM re-imaged,
196 // re-addressed, or gone. A session pins the address it was created for,
197 // so this is where a pin that no longer matches is dropped: the next
198 // datagram from that client starts a session pointing at wherever the
199 // guest is now. Only UDP exposures ask, because only they hold anything
200 // pinned, and asking is a read of the VM's record on disk.
201 if ex.pc != nil {
202 ex.evictMovedSessions(m.addr(d.GetVmId()))
203 }
204 out = append(out, &pb.ExposureActual{Id: d.GetId(), State: "active", Reason: ex.reason})
155 } 205 }
156 return out 206 return out
157 } 207 }
158 208
159 // StopAll closes every listener (agent shutdown, tests). 209 // bind opens the socket one desired exposure calls for and starts the loop that
210 // serves it. UDP is a bound packet socket; everything else is a TCP listener —
211 // a server that names no protocol at all means the one this proxy started with.
212 func (m *Manager) bind(d *pb.ExposureDesired, ex *exposure) error {
213 hostAddr := net.JoinHostPort("0.0.0.0", strconv.Itoa(int(d.GetHostPort())))
214 if d.GetProtocol() == "udp" {
215 ua, err := net.ResolveUDPAddr("udp4", hostAddr)
216 if err != nil {
217 return err
218 }
219 pc, err := net.ListenUDP("udp4", ua)
220 if err != nil {
221 return err
222 }
223 ex.pc, ex.reason = pc, ""
224 go m.serve(d.GetId(), ex, pc, d.GetVmId(), d.GetGuestPort())
225 return nil
226 }
227 ln, err := net.Listen("tcp", hostAddr)
228 if err != nil {
229 return err
230 }
231 ex.ln, ex.reason = ln, ""
232 go m.accept(d.GetId(), ex, ln, d.GetVmId(), d.GetGuestPort())
233 return nil
234 }
235
236 // StopAll closes every socket (agent shutdown, tests).
160 func (m *Manager) StopAll() { 237 func (m *Manager) StopAll() {
161 m.mu.Lock() 238 m.mu.Lock()
162 defer m.mu.Unlock() 239 defer m.mu.Unlock()
@@ -181,8 +258,11 @@ func (m *Manager) StopAll() {
181 // the listener back would unbind a working port and ask the next converge to 258 // the listener back would unbind a working port and ask the next converge to
182 // find a descriptor for a fresh one — exactly what the process has none of — 259 // find a descriptor for a fresh one — exactly what the process has none of —
183 // so the loop pauses and keeps the port, and the backlog keeps callers waiting 260 // so the loop pauses and keeps the port, and the backlog keeps callers waiting
184 // rather than refusing them. Everything else is a broken listener and heals the 261 // rather than refusing them. It says so in the exposure's reason for as long as
185 // only way a broken listener can, by being rebound. 262 // the streak lasts: the port is bound, which is what "active" means, and an
263 // operator looking at a published port nothing is getting through deserves the
264 // one sentence that explains it. Everything else is a broken listener and heals
265 // the only way a broken listener can, by being rebound.
186 func (m *Manager) accept(id string, ex *exposure, ln net.Listener, vmID string, guestPort uint32) { 266 func (m *Manager) accept(id string, ex *exposure, ln net.Listener, vmID string, guestPort uint32) {
187 var pause time.Duration 267 var pause time.Duration
188 capped := false 268 capped := false
@@ -193,6 +273,7 @@ func (m *Manager) accept(id string, ex *exposure, ln net.Listener, vmID string,
193 if pause == 0 { 273 if pause == 0 {
194 pause = acceptRetryFloor 274 pause = acceptRetryFloor
195 slog.Warn("exposure accept out of descriptors, retrying", "exposure", id, "err", err) 275 slog.Warn("exposure accept out of descriptors, retrying", "exposure", id, "err", err)
276 m.setReason(id, ln, "accept: out of descriptors, retrying: "+err.Error())
196 } else { 277 } else {
197 pause = min(pause*2, acceptRetryCeiling) 278 pause = min(pause*2, acceptRetryCeiling)
198 } 279 }
@@ -211,7 +292,11 @@ func (m *Manager) accept(id string, ex *exposure, ln net.Listener, vmID string,
211 m.mu.Unlock() 292 m.mu.Unlock()
212 return 293 return
213 } 294 }
214 pause = 0 295 if pause != 0 {
296 pause = 0
297 slog.Info("exposure accept has descriptors again", "exposure", id)
298 m.setReason(id, ln, "")
299 }
215 300
216 // Increment first and give it back if the cap says no: counting only 301 // Increment first and give it back if the cap says no: counting only
217 // after the decision would let a burst of simultaneous accepts each read 302 // after the decision would let a burst of simultaneous accepts each read
@@ -233,6 +318,20 @@ func (m *Manager) accept(id string, ex *exposure, ln net.Listener, vmID string,
233 } 318 }
234 } 319 }
235 320
321 // setReason records what an accept loop wants the next report to say about a
322 // still-bound exposure. It writes under the manager lock, which is what makes
323 // the reason a converge reads a whole one rather than a torn one, and only when
324 // the entry still holds THIS loop's listener — the same identity guard the
325 // loop's exit takes, because a loop whose exposure was rebound underneath it
326 // has nothing to say about the listener that replaced its own.
327 func (m *Manager) setReason(id string, ln net.Listener, reason string) {
328 m.mu.Lock()
329 defer m.mu.Unlock()
330 if cur, ok := m.live[id]; ok && cur.ln == ln {
331 cur.reason = reason
332 }
333 }
334
236 // outOfDescriptors reports whether an accept failed because the process (EMFILE) 335 // outOfDescriptors reports whether an accept failed because the process (EMFILE)
237 // or the machine (ENFILE) has no descriptor to give it. The runtime already 336 // or the machine (ENFILE) has no descriptor to give it. The runtime already
238 // retries the other transient accept errors — an interrupted call, a caller that 337 // retries the other transient accept errors — an interrupted call, a caller that
internal/agent/exposeproxy/exposeproxy_test.go
Old New
@@ -4,6 +4,7 @@ import (
4 "io" 4 "io"
5 "net" 5 "net"
6 "strconv" 6 "strconv"
7 "strings"
7 "syscall" 8 "syscall"
8 "testing" 9 "testing"
9 "time" 10 "time"
@@ -82,6 +83,10 @@ func desired(id, vmID string, guestPort, hostPort uint32) *pb.ExposureDesired {
82 return &pb.ExposureDesired{Id: id, VmId: vmID, GuestPort: guestPort, HostPort: hostPort, Protocol: "tcp"} 83 return &pb.ExposureDesired{Id: id, VmId: vmID, GuestPort: guestPort, HostPort: hostPort, Protocol: "tcp"}
83 } 84 }
84 85
86 func desiredUDP(id, vmID string, guestPort, hostPort uint32) *pb.ExposureDesired {
87 return &pb.ExposureDesired{Id: id, VmId: vmID, GuestPort: guestPort, HostPort: hostPort, Protocol: "udp"}
88 }
89
85 // speak dials addr, sends msg, and returns what came back. 90 // speak dials addr, sends msg, and returns what came back.
86 func speak(t *testing.T, addr, msg string) string { 91 func speak(t *testing.T, addr, msg string) string {
87 t.Helper() 92 t.Helper()
@@ -422,6 +427,92 @@ func TestTheCapReleasesWhenAConnectionEnds(t *testing.T) {
422 assert.Equal(t, "after", exchange(t, hold(t, addr), "after")) 427 assert.Equal(t, "after", exchange(t, hold(t, addr), "after"))
423 } 428 }
424 429
430 func TestTheCapCountsConnectionsDrainingThroughARebind(t *testing.T) {
431 first, second := newHoldingGuest(t), newHoldingGuest(t)
432 m := newTestManager(t, map[string]string{"vm1": first.addr})
433 m.maxConns = 1
434 m.Converge([]*pb.ExposureDesired{desired("e1", "vm1", first.port, 0)})
435
436 held := hold(t, boundPort(t, m, "e1"))
437
438 // The spec changes and the exposure rebinds. The connection under the old
439 // listener is still open and still spending its two descriptors, so it is
440 // still what the cap is counting — an exposure that forgot it on the rebind
441 // would let this port hold twice what it is allowed.
442 m.Converge([]*pb.ExposureDesired{desired("e1", "vm1", second.port, 0)})
443 waitConns(t, m, "e1", 1)
444
445 over, err := net.Dial("tcp", boundPort(t, m, "e1"))
446 require.NoError(t, err)
447 defer over.Close()
448 require.NoError(t, over.SetDeadline(time.Now().Add(5*time.Second)))
449 out, err := io.ReadAll(over)
450 require.NoError(t, err)
451 assert.Empty(t, out, "the rebound listener is at its cap, because the draining connection still counts")
452
453 // The old connection drains rather than being cut, and the slot it holds is
454 // the one the next caller gets when it ends.
455 assert.Equal(t, "drain", exchange(t, held, "drain"))
456 require.NoError(t, held.Close())
457 waitConns(t, m, "e1", 0)
458 assert.Equal(t, "after", exchange(t, hold(t, boundPort(t, m, "e1")), "after"))
459 }
460
461 // waitReason waits for an exposure's reported reason to satisfy ok, reading it
462 // under the lock the accept loop writes it under.
463 func waitReason(t *testing.T, m *Manager, id string, ok func(string) bool, what string) string {
464 t.Helper()
465 var got string
466 for i := 0; i < 200; i++ {
467 m.mu.Lock()
468 ex, live := m.live[id]
469 if live {
470 got = ex.reason
471 }
472 m.mu.Unlock()
473 if live && ok(got) {
474 return got
475 }
476 time.Sleep(10 * time.Millisecond)
477 }
478 t.Fatalf("exposure %q reason %q: %s", id, got, what)
479 return ""
480 }
481
482 func TestADescriptorShortageIsReportedWhileThePortStaysBound(t *testing.T) {
483 g := newFakeGuest(t)
484 m := newTestManager(t, map[string]string{"vm1": g.addr})
485
486 d := desired("e1", "vm1", g.port, 0)
487 ln := &scriptedListener{next: make(chan acceptResult), addr: g.ln.Addr()}
488 ex := &exposure{key: specKey(d), ln: ln}
489 m.mu.Lock()
490 m.live["e1"] = ex
491 m.mu.Unlock()
492 go m.accept("e1", ex, ln, "vm1", g.port)
493
494 ln.next <- acceptResult{err: syscall.EMFILE}
495 waitReason(t, m, "e1", func(r string) bool { return strings.Contains(r, "out of descriptors") },
496 "a port nothing can get through must say why")
497
498 // The port is still bound, so it is still active — with the streak beside
499 // it, which is the whole point: "active" alone would read as healthy.
500 got := m.Converge([]*pb.ExposureDesired{d})
501 require.Len(t, got, 1)
502 assert.Equal(t, "active", got[0].GetState())
503 assert.Contains(t, got[0].GetReason(), "out of descriptors")
504
505 // Descriptors come back. The next accept clears the streak, and the report
506 // stops saying something is wrong.
507 client, accepted := net.Pipe()
508 defer client.Close()
509 ln.next <- acceptResult{conn: accepted}
510 waitReason(t, m, "e1", func(r string) bool { return r == "" }, "a recovered port must stop reporting a shortage")
511 got = m.Converge([]*pb.ExposureDesired{d})
512 assert.Equal(t, "active", got[0].GetState())
513 assert.Empty(t, got[0].GetReason())
514 }
515
425 // scriptedListener hands an accept loop exactly the sequence a test writes to 516 // scriptedListener hands an accept loop exactly the sequence a test writes to
426 // it — the errors a real listener only produces under a load a test cannot 517 // it — the errors a real listener only produces under a load a test cannot
427 // stage. 518 // stage.
internal/agent/exposeproxy/udp.go
Old New
@@ -0,0 +1,283 @@
1 package exposeproxy
2
3 import (
4 "errors"
5 "log/slog"
6 "net"
7 "os"
8 "strconv"
9 "sync"
10 "sync/atomic"
11 "time"
12 )
13
14 // A UDP exposure is one bound socket on the host and one session per client
15 // address behind it. The session is the whole design: UDP has no connection to
16 // follow, so the proxy invents the smallest thing that lets a reply find its
17 // way home — a connected socket toward the guest, whose local port is what the
18 // guest answers to, held for as long as the conversation looks live.
19 //
20 // The guest's address is resolved ONCE, when the session is created, and pinned
21 // there. A datagram that arrives before this host knows where the guest is is
22 // dropped and starts nothing: a session pinned to nowhere would have to
23 // re-resolve on some later packet, and a proxy that re-resolves mid-conversation
24 // is one that can quietly start sending a guest's traffic to whatever took its
25 // address over. Converge is where a stale pin is dropped instead.
26 const (
27 // maxSessionsPerExposure is how many client conversations one published UDP
28 // port may hold at once. Each is one descriptor — the socket toward the
29 // guest; the socket callers reach is shared by all of them — so the sizing
30 // rationale is the TCP cap's, one descriptor per session instead of two,
31 // against the same 65536 the shipped unit grants the agent. A table at its
32 // cap refuses new clients and keeps the ones it has: eviction would trade a
33 // conversation that is working for one that might not be, and a UDP client
34 // has no close to notice.
35 maxSessionsPerExposure = 256
36
37 // udpUnrepliedIdle and udpRepliedIdle are how long a session outlives its
38 // last datagram. A session the guest has never answered is a guess — a
39 // scanner's single packet, or a service that was not listening — and thirty
40 // seconds is long enough for a slow first reply and short enough that a
41 // sweep of the port space does not fill the table. Once traffic has flowed
42 // back the session is a real conversation, and two minutes carries the
43 // quiet stretches those have (a DNS client between queries, a game tick
44 // paused, an NTP poll interval).
45 udpUnrepliedIdle = 30 * time.Second
46 udpRepliedIdle = 120 * time.Second
47
48 // udpDatagramMax is the read buffer on both halves — larger than the
49 // largest datagram either side can send, so what goes in one end comes out
50 // the other whole. One datagram in is one datagram out: nothing here
51 // batches, coalesces, or reassembles.
52 udpDatagramMax = 64 * 1024
53 )
54
55 // udpSession is one client address's conversation with one guest port: the
56 // connected socket the guest sees, the client to send its answers back to, and
57 // the two facts that decide when the session ends.
58 type udpSession struct {
59 // key is the client address as the session table indexes it.
60 key string
61 client *net.UDPAddr
62
63 // guest is CONNECTED to the pinned address, which is what makes a reply
64 // readable here rather than on the shared socket — and what surfaces an
65 // ICMP port-unreachable as a read error instead of silence.
66 guest *net.UDPConn
67
68 // pinned is the guest address this session was created for. Converge
69 // compares it against where the VM is now.
70 pinned string
71
72 // unrepliedIdle and repliedIdle are the manager's two windows, copied here
73 // so a session decides its own expiry without reaching back for them.
74 unrepliedIdle, repliedIdle time.Duration
75
76 // last is the unix-nano time of the last datagram in either direction, and
77 // replied records that the guest has answered at least once. Both are
78 // touched by the packet loop and the relay goroutine, so both are atomic.
79 last atomic.Int64
80 replied atomic.Bool
81
82 closeOnce sync.Once
83 }
84
85 func newUDPSession(client *net.UDPAddr, guest *net.UDPConn, pinned string, unreplied, replied time.Duration) *udpSession {
86 s := &udpSession{
87 key: client.String(), client: client, guest: guest, pinned: pinned,
88 unrepliedIdle: unreplied, repliedIdle: replied,
89 }
90 s.touch()
91 return s
92 }
93
94 // touch records that a datagram just moved.
95 func (s *udpSession) touch() { s.last.Store(time.Now().UnixNano()) }
96
97 // expiry is when the session ends if nothing else moves: its last datagram plus
98 // the window its promotion earns it.
99 func (s *udpSession) expiry() time.Time {
100 idle := s.unrepliedIdle
101 if s.replied.Load() {
102 idle = s.repliedIdle
103 }
104 return time.Unix(0, s.last.Load()).Add(idle)
105 }
106
107 // close releases the session's descriptor, which is also what ends its relay.
108 func (s *udpSession) close() {
109 s.closeOnce.Do(func() { s.guest.Close() })
110 }
111
112 // session returns the session serving client, or nil when there is none.
113 func (e *exposure) session(key string) *udpSession {
114 e.smu.Lock()
115 defer e.smu.Unlock()
116 return e.sessions[key]
117 }
118
119 // addSession puts s in the table unless it is full, in which case the caller
120 // closes s and the datagram that would have started it is dropped.
121 func (e *exposure) addSession(s *udpSession, max int) bool {
122 e.smu.Lock()
123 defer e.smu.Unlock()
124 if len(e.sessions) >= max {
125 return false
126 }
127 if e.sessions == nil {
128 e.sessions = map[string]*udpSession{}
129 }
130 e.sessions[s.key] = s
131 return true
132 }
133
134 // dropSession removes s and closes it. Idempotent, and identity-checked: a
135 // session that has already been replaced under its key is not the one to
136 // remove.
137 func (e *exposure) dropSession(s *udpSession) {
138 e.smu.Lock()
139 if cur, ok := e.sessions[s.key]; ok && cur == s {
140 delete(e.sessions, s.key)
141 }
142 e.smu.Unlock()
143 s.close()
144 }
145
146 // closeSessions empties the table, ending every relay with it.
147 func (e *exposure) closeSessions() {
148 e.smu.Lock()
149 held := e.sessions
150 e.sessions = nil
151 e.smu.Unlock()
152 for _, s := range held {
153 s.close()
154 }
155 }
156
157 // evictMovedSessions drops every session whose pinned guest address is not
158 // where the VM is now — including when the host no longer knows where that is,
159 // which is not an address to keep sending to either.
160 func (e *exposure) evictMovedSessions(guestIP string) {
161 e.smu.Lock()
162 var moved []*udpSession
163 for key, s := range e.sessions {
164 if s.pinned != guestIP {
165 delete(e.sessions, key)
166 moved = append(moved, s)
167 }
168 }
169 e.smu.Unlock()
170 for _, s := range moved {
171 s.close()
172 }
173 }
174
175 // serve reads one exposure's published UDP socket until it ends: a converge
176 // dropped it, StopAll closed it, or the socket itself broke. Like the accept
177 // loop, a broken socket is handed back so the next converge rebinds it — a
178 // bound port nothing is reading is worse than no port at all, because it
179 // swallows datagrams instead of refusing them.
180 //
181 // Every datagram either continues a session or starts one. Nothing here waits:
182 // a client with nowhere to send, a guest that will not take a connected socket,
183 // and a table at its cap all drop the datagram, because a datagram held is a
184 // datagram late, and UDP callers retry.
185 func (m *Manager) serve(id string, ex *exposure, pc *net.UDPConn, vmID string, guestPort uint32) {
186 buf := make([]byte, udpDatagramMax)
187 full := false
188 for {
189 n, client, err := pc.ReadFromUDP(buf)
190 if err != nil {
191 m.mu.Lock()
192 if cur, ok := m.live[id]; ok && cur.pc == pc {
193 cur.close()
194 cur.reason = "read: " + err.Error()
195 // The reason travels in the next report, which nobody watching
196 // the host sees; a broken socket belongs in the agent's log too.
197 slog.Warn("exposure packet loop died", "exposure", id, "err", err)
198 }
199 m.mu.Unlock()
200 return
201 }
202
203 s := ex.session(client.String())
204 if s == nil {
205 s = m.openSession(ex, pc, client, vmID, guestPort, &full)
206 if s == nil {
207 continue
208 }
209 }
210 if _, err := s.guest.Write(buf[:n]); err != nil {
211 ex.dropSession(s)
212 continue
213 }
214 s.touch()
215 }
216 }
217
218 // openSession resolves where the guest is right now, pins a connected socket
219 // there, and files it under the client's address. It returns nil when the
220 // datagram that asked for it is to be dropped — no guest address, a socket the
221 // OS would not give, or a table with no room — with full carrying the
222 // at-capacity streak so a saturated port logs once rather than per packet.
223 func (m *Manager) openSession(ex *exposure, pc *net.UDPConn, client *net.UDPAddr, vmID string, guestPort uint32, full *bool) *udpSession {
224 ip := m.addr(vmID)
225 if ip == "" {
226 return nil
227 }
228 guestAddr, err := net.ResolveUDPAddr("udp4", net.JoinHostPort(ip, strconv.Itoa(int(guestPort))))
229 if err != nil {
230 return nil
231 }
232 guest, err := net.DialUDP("udp4", nil, guestAddr)
233 if err != nil {
234 return nil
235 }
236 s := newUDPSession(client, guest, ip, m.unrepliedIdle, m.repliedIdle)
237 if !ex.addSession(s, m.maxSessions) {
238 s.close()
239 if !*full {
240 *full = true
241 slog.Warn("exposure at its UDP session cap, dropping", "cap", m.maxSessions, "client", client.String())
242 }
243 return nil
244 }
245 *full = false
246 go m.relay(ex, pc, s)
247 return s
248 }
249
250 // relay carries one session's replies back to its client until the session
251 // ends. It ends on the guest socket going quiet for longer than the session's
252 // window, on any error reading it — which is where an ICMP port-unreachable
253 // from a guest with nothing listening arrives — and on a failure to answer the
254 // client.
255 //
256 // The read deadline IS the idle timer: there is no sweeper, because the only
257 // thing that needs to notice an expired session is the goroutine already
258 // waiting on it. A deadline that fires early because the forward direction
259 // moved the session on is simply re-armed against the newer expiry.
260 func (m *Manager) relay(ex *exposure, pc *net.UDPConn, s *udpSession) {
261 defer ex.dropSession(s)
262 buf := make([]byte, udpDatagramMax)
263 for {
264 if err := s.guest.SetReadDeadline(s.expiry()); err != nil {
265 return
266 }
267 n, err := s.guest.Read(buf)
268 if err != nil {
269 if errors.Is(err, os.ErrDeadlineExceeded) && time.Now().Before(s.expiry()) {
270 continue
271 }
272 return
273 }
274 if _, err := pc.WriteToUDP(buf[:n], s.client); err != nil {
275 return
276 }
277 s.touch()
278 // Promotion, and it only happens here: traffic that has come BACK from
279 // the guest is what tells a real conversation apart from a stray packet
280 // at a port.
281 s.replied.Store(true)
282 }
283 }
internal/agent/exposeproxy/udp_test.go
Old New
@@ -0,0 +1,386 @@
1 package exposeproxy
2
3 import (
4 "bytes"
5 "net"
6 "testing"
7 "time"
8
9 "github.com/a73x/eitri/internal/pb"
10 "github.com/stretchr/testify/assert"
11 "github.com/stretchr/testify/require"
12 )
13
14 // fakeUDPGuest is a bound UDP socket standing in for a service inside a guest.
15 // It echoes every datagram back to whoever sent it, prefixed, so a test can
16 // prove a datagram went both ways through the proxy and came back whole.
17 type fakeUDPGuest struct {
18 pc *net.UDPConn
19 addr string
20 port uint32
21 }
22
23 func newFakeUDPGuest(t *testing.T) *fakeUDPGuest {
24 t.Helper()
25 pc, err := net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)})
26 require.NoError(t, err)
27 t.Cleanup(func() { pc.Close() })
28 g := &fakeUDPGuest{pc: pc, addr: "127.0.0.1", port: uint32(pc.LocalAddr().(*net.UDPAddr).Port)}
29 go func() {
30 buf := make([]byte, 64*1024)
31 for {
32 n, from, err := pc.ReadFromUDP(buf)
33 if err != nil {
34 return
35 }
36 pc.WriteToUDP(append([]byte("echo:"), buf[:n]...), from) //nolint:errcheck
37 }
38 }()
39 return g
40 }
41
42 // boundUDPPort reads the address the manager's packet socket for id bound.
43 func boundUDPPort(t *testing.T, m *Manager, id string) string {
44 t.Helper()
45 m.mu.Lock()
46 defer m.mu.Unlock()
47 ex, ok := m.live[id]
48 require.True(t, ok, "no exposure %q", id)
49 require.NotNil(t, ex.pc, "exposure %q has no packet socket", id)
50 return ex.pc.LocalAddr().String()
51 }
52
53 // udpClient is one caller of a published UDP port, held open across datagrams
54 // so the proxy sees the same source address each time — which is what makes it
55 // one session rather than several.
56 func udpClient(t *testing.T, addr string) *net.UDPConn {
57 t.Helper()
58 ua, err := net.ResolveUDPAddr("udp4", addr)
59 require.NoError(t, err)
60 c, err := net.DialUDP("udp4", nil, ua)
61 require.NoError(t, err)
62 t.Cleanup(func() { c.Close() })
63 require.NoError(t, c.SetDeadline(time.Now().Add(5*time.Second)))
64 return c
65 }
66
67 // say sends one datagram and returns the one that comes back.
68 func say(t *testing.T, c *net.UDPConn, msg string) string {
69 t.Helper()
70 _, err := c.Write([]byte(msg))
71 require.NoError(t, err)
72 buf := make([]byte, 64*1024)
73 n, err := c.Read(buf)
74 require.NoError(t, err)
75 return string(buf[:n])
76 }
77
78 // sessions is how many the exposure is holding right now.
79 func sessions(t *testing.T, m *Manager, id string) int {
80 t.Helper()
81 m.mu.Lock()
82 ex, ok := m.live[id]
83 m.mu.Unlock()
84 require.True(t, ok, "no exposure %q", id)
85 ex.smu.Lock()
86 defer ex.smu.Unlock()
87 return len(ex.sessions)
88 }
89
90 // waitSessions waits for the session count to reach want. A session is filed
91 // synchronously with the datagram that starts it, but it is torn down by a
92 // goroutine finishing on its own time.
93 func waitSessions(t *testing.T, m *Manager, id string, want int) {
94 t.Helper()
95 var got int
96 for i := 0; i < 200; i++ {
97 if got = sessions(t, m, id); got == want {
98 return
99 }
100 time.Sleep(10 * time.Millisecond)
101 }
102 t.Fatalf("exposure %q holds %d sessions, want %d", id, got, want)
103 }
104
105 // oneSession returns the exposure's single session, for tests that reach past
106 // the wire to what the proxy is holding.
107 func oneSession(t *testing.T, m *Manager, id string) *udpSession {
108 t.Helper()
109 m.mu.Lock()
110 ex, ok := m.live[id]
111 m.mu.Unlock()
112 require.True(t, ok, "no exposure %q", id)
113 ex.smu.Lock()
114 defer ex.smu.Unlock()
115 require.Len(t, ex.sessions, 1, "want exactly one session")
116 for _, s := range ex.sessions {
117 return s
118 }
119 return nil
120 }
121
122 func TestUDPConvergeBindsAndForwardsBothWays(t *testing.T) {
123 g := newFakeUDPGuest(t)
124 m := newTestManager(t, map[string]string{"vm1": g.addr})
125
126 got := m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", g.port, 0)})
127 require.Len(t, got, 1)
128 assert.Equal(t, "active", got[0].GetState(), "active means the packet socket is bound")
129
130 c := udpClient(t, boundUDPPort(t, m, "e1"))
131 assert.Equal(t, "echo:hello", say(t, c, "hello"))
132 assert.Equal(t, 1, sessions(t, m, "e1"), "one client address is one session")
133 }
134
135 func TestUDPKeepsDatagramBoundaries(t *testing.T) {
136 g := newFakeUDPGuest(t)
137 m := newTestManager(t, map[string]string{"vm1": g.addr})
138 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", g.port, 0)})
139 c := udpClient(t, boundUDPPort(t, m, "e1"))
140
141 // Two datagrams in, two out, in order and whole — nothing here coalesces a
142 // stream out of them.
143 assert.Equal(t, "echo:one", say(t, c, "one"))
144 assert.Equal(t, "echo:two", say(t, c, "two"))
145
146 // And a big one survives intact: the read buffers are sized past the
147 // largest datagram either side can send.
148 big := string(bytes.Repeat([]byte("x"), 8000))
149 assert.Equal(t, "echo:"+big, say(t, c, big))
150 }
151
152 func TestUDPSessionIsPerClientAddress(t *testing.T) {
153 g := newFakeUDPGuest(t)
154 m := newTestManager(t, map[string]string{"vm1": g.addr})
155 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", g.port, 0)})
156 addr := boundUDPPort(t, m, "e1")
157
158 first, second := udpClient(t, addr), udpClient(t, addr)
159 assert.Equal(t, "echo:a", say(t, first, "a"))
160 assert.Equal(t, "echo:b", say(t, second, "b"))
161 assert.Equal(t, 2, sessions(t, m, "e1"), "two callers are two conversations")
162
163 // The same caller again is the same session, not a third.
164 assert.Equal(t, "echo:again", say(t, first, "again"))
165 assert.Equal(t, 2, sessions(t, m, "e1"))
166 }
167
168 func TestUDPSessionIsPromotedOnlyByAGuestReply(t *testing.T) {
169 // A guest that takes datagrams and never answers: the session stays on the
170 // short window, however much the client says.
171 silent, err := net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)})
172 require.NoError(t, err)
173 t.Cleanup(func() { silent.Close() })
174 quietPort := uint32(silent.LocalAddr().(*net.UDPAddr).Port)
175
176 m := newTestManager(t, map[string]string{"vm1": "127.0.0.1"})
177 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", quietPort, 0)})
178 c := udpClient(t, boundUDPPort(t, m, "e1"))
179 _, err = c.Write([]byte("anyone there"))
180 require.NoError(t, err)
181 waitSessions(t, m, "e1", 1)
182
183 s := oneSession(t, m, "e1")
184 assert.False(t, s.replied.Load(), "nothing has come back; the session is still a guess")
185 assert.WithinDuration(t, time.Now().Add(m.unrepliedIdle), s.expiry(), time.Second)
186
187 // Now a guest that answers. The reply is the promotion.
188 g := newFakeUDPGuest(t)
189 m.Converge([]*pb.ExposureDesired{desiredUDP("e2", "vm1", g.port, 0)})
190 talking := udpClient(t, boundUDPPort(t, m, "e2"))
191 require.Equal(t, "echo:hi", say(t, talking, "hi"))
192
193 promoted := oneSession(t, m, "e2")
194 assert.True(t, promoted.replied.Load())
195 assert.WithinDuration(t, time.Now().Add(m.repliedIdle), promoted.expiry(), time.Second)
196 }
197
198 func TestUDPSessionExpiresOnBothWindows(t *testing.T) {
199 g := newFakeUDPGuest(t)
200 m := newTestManager(t, map[string]string{"vm1": g.addr})
201 // Windows a test can outlast. Everything about them is what the shipped
202 // constants do, at a scale that fits in a test.
203 m.unrepliedIdle, m.repliedIdle = 250*time.Millisecond, 10*time.Second
204
205 // Never answered: gone at the short window.
206 silent, err := net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)})
207 require.NoError(t, err)
208 t.Cleanup(func() { silent.Close() })
209 m.Converge([]*pb.ExposureDesired{
210 desiredUDP("quiet", "vm1", uint32(silent.LocalAddr().(*net.UDPAddr).Port), 0),
211 desiredUDP("live", "vm1", g.port, 0),
212 })
213
214 unanswered := udpClient(t, boundUDPPort(t, m, "quiet"))
215 _, err = unanswered.Write([]byte("hello?"))
216 require.NoError(t, err)
217 waitSessions(t, m, "quiet", 1)
218 waitSessions(t, m, "quiet", 0)
219
220 // Answered once: the same stretch of quiet does not end it, because the
221 // promotion bought it the longer window.
222 c := udpClient(t, boundUDPPort(t, m, "live"))
223 require.Equal(t, "echo:hi", say(t, c, "hi"))
224 waitSessions(t, m, "live", 1)
225 time.Sleep(600 * time.Millisecond)
226 assert.Equal(t, 1, sessions(t, m, "live"), "a conversation that has answered outlives the short window")
227 assert.Equal(t, "echo:still here", say(t, c, "still here"), "and it is the same session, still carrying traffic")
228 }
229
230 func TestUDPRefusesNewSessionsAtItsCapWithoutEvicting(t *testing.T) {
231 g := newFakeUDPGuest(t)
232 m := newTestManager(t, map[string]string{"vm1": g.addr})
233 m.maxSessions = 1
234 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", g.port, 0)})
235 addr := boundUDPPort(t, m, "e1")
236
237 held := udpClient(t, addr)
238 require.Equal(t, "echo:mine", say(t, held, "mine"), "the first caller has the only slot, and has been answered")
239
240 // The table is full. The second caller's datagram is dropped where it
241 // arrives: no session, and nothing comes back.
242 over := udpClient(t, addr)
243 _, err := over.Write([]byte("me too"))
244 require.NoError(t, err)
245 require.NoError(t, over.SetReadDeadline(time.Now().Add(300*time.Millisecond)))
246 buf := make([]byte, 64)
247 _, err = over.Read(buf)
248 assert.Error(t, err, "a caller past the cap is answered by nothing")
249 assert.Equal(t, 1, sessions(t, m, "e1"))
250
251 // And the conversation that was already working is untouched — a full table
252 // refuses the new, it does not sacrifice the live.
253 assert.Equal(t, "echo:still mine", say(t, held, "still mine"))
254 }
255
256 func TestUDPConvergeEvictsASessionWhoseGuestMoved(t *testing.T) {
257 first, second := newFakeUDPGuest(t), newFakeUDPGuest(t)
258 where := map[string]string{"vm1": first.addr}
259 m := NewManager(func(vmID string) string { return where[vmID] })
260 t.Cleanup(m.StopAll)
261
262 // Both fakes answer on 127.0.0.1, so the guest PORT is what tells them
263 // apart; the pin under test is the address, so move the VM to an address
264 // nothing is at and prove the session goes.
265 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", first.port, 0)})
266 c := udpClient(t, boundUDPPort(t, m, "e1"))
267 require.Equal(t, "echo:before", say(t, c, "before"))
268 s := oneSession(t, m, "e1")
269
270 where["vm1"] = "127.0.0.2"
271 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", first.port, 0)})
272 assert.Equal(t, 0, sessions(t, m, "e1"), "a pin that no longer matches is not a session to keep")
273 _, err := s.guest.Write([]byte("orphan"))
274 assert.Error(t, err, "the evicted session's socket is closed, not leaked")
275
276 // A guest the host has lost track of entirely is no different: there is
277 // nowhere to send, so there is no session.
278 where["vm1"] = second.addr
279 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", second.port, 0)})
280 require.Equal(t, "echo:after", say(t, udpClient(t, boundUDPPort(t, m, "e1")), "after"))
281 where["vm1"] = ""
282 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", second.port, 0)})
283 assert.Equal(t, 0, sessions(t, m, "e1"), "no address is not an address to keep sending to")
284 }
285
286 func TestUDPSessionEndsWhenItsGuestSocketFails(t *testing.T) {
287 g := newFakeUDPGuest(t)
288 m := newTestManager(t, map[string]string{"vm1": g.addr})
289 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", g.port, 0)})
290 c := udpClient(t, boundUDPPort(t, m, "e1"))
291 require.Equal(t, "echo:up", say(t, c, "up"))
292
293 // Break the socket toward the guest the way an ICMP port-unreachable or a
294 // descriptor going bad would: the relay's read fails, and the session that
295 // cannot reach its guest stops being one.
296 s := oneSession(t, m, "e1")
297 require.NoError(t, s.guest.Close())
298 waitSessions(t, m, "e1", 0)
299
300 // The port is still published; the next datagram simply starts a fresh
301 // session, which is what makes the failure survivable.
302 assert.Equal(t, "echo:again", say(t, c, "again"))
303 }
304
305 func TestUDPDropsADatagramForAGuestWithNoAddress(t *testing.T) {
306 m := newTestManager(t, map[string]string{}) // the guest is still leasing
307 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", 8080, 0)})
308
309 c := udpClient(t, boundUDPPort(t, m, "e1"))
310 _, err := c.Write([]byte("anyone"))
311 require.NoError(t, err)
312 require.NoError(t, c.SetReadDeadline(time.Now().Add(300*time.Millisecond)))
313 buf := make([]byte, 64)
314 _, err = c.Read(buf)
315 assert.Error(t, err, "nowhere to send is a drop, not a wait")
316 assert.Equal(t, 0, sessions(t, m, "e1"), "a session pinned to nowhere is not a session")
317 }
318
319 func TestUDPConvergeReportsABindFailure(t *testing.T) {
320 g := newFakeUDPGuest(t)
321 m := newTestManager(t, map[string]string{"vm1": g.addr})
322
323 squatter, err := net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4zero})
324 require.NoError(t, err)
325 held := uint32(squatter.LocalAddr().(*net.UDPAddr).Port)
326
327 got := m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", g.port, held)})
328 require.Len(t, got, 1)
329 assert.Equal(t, "failed", got[0].GetState())
330 assert.NotEmpty(t, got[0].GetReason(), "the report carries what the OS said")
331
332 require.NoError(t, squatter.Close())
333 var state string
334 for i := 0; i < 20; i++ {
335 got = m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", g.port, held)})
336 state = got[0].GetState()
337 if state == "active" {
338 break
339 }
340 time.Sleep(50 * time.Millisecond)
341 }
342 assert.Equal(t, "active", state, "a bind-time refusal is a level to converge on, not an error path")
343 }
344
345 func TestChangingTheProtocolRebinds(t *testing.T) {
346 tcpGuest := newFakeGuest(t)
347 udpGuest := newFakeUDPGuest(t)
348 m := newTestManager(t, map[string]string{"vm1": tcpGuest.addr})
349
350 m.Converge([]*pb.ExposureDesired{desired("e1", "vm1", tcpGuest.port, 0)})
351 assert.Equal(t, "echo:tcp", speak(t, boundPort(t, m, "e1"), "tcp"))
352
353 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", udpGuest.port, 0)})
354 m.mu.Lock()
355 ex := m.live["e1"]
356 m.mu.Unlock()
357 assert.Nil(t, ex.ln, "the listener the old protocol needed is gone")
358 assert.Equal(t, "echo:udp", say(t, udpClient(t, boundUDPPort(t, m, "e1")), "udp"))
359 }
360
361 func TestStopAllClosesPacketSocketsToo(t *testing.T) {
362 g := newFakeUDPGuest(t)
363 m := NewManager(func(string) string { return g.addr })
364 m.Converge([]*pb.ExposureDesired{desiredUDP("e1", "vm1", g.port, 0)})
365 addr := boundUDPPort(t, m, "e1")
366 c := udpClient(t, addr)
367 require.Equal(t, "echo:live", say(t, c, "live"))
368 s := oneSession(t, m, "e1")
369
370 m.StopAll()
371
372 // The socket is unbindable-again proof: the port frees, so something else
373 // can take it. The session that hung off it is closed with it.
374 freed, err := net.ListenUDP("udp4", mustUDPAddr(t, addr))
375 require.NoError(t, err, "the published port survived StopAll")
376 freed.Close()
377 _, err = s.guest.Write([]byte("orphan"))
378 assert.Error(t, err, "a session outliving its socket would hold a descriptor nothing owns")
379 }
380
381 func mustUDPAddr(t *testing.T, addr string) *net.UDPAddr {
382 t.Helper()
383 ua, err := net.ResolveUDPAddr("udp4", addr)
384 require.NoError(t, err)
385 return ua
386 }
internal/mcpserver/server.go
Old New
@@ -60,9 +60,9 @@ func NewServer(t *Tools, opts Options) *mcp.Server {
60 register(s, "vm_exec", "Run a shell command in a VM over SSH; returns stdout/stderr/exit code.", t.VMExec) 60 register(s, "vm_exec", "Run a shell command in a VM over SSH; returns stdout/stderr/exit code.", t.VMExec)
61 register(s, "vm_write_file", "Write content to a file in a VM (parents created).", t.VMWriteFile) 61 register(s, "vm_write_file", "Write content to a file in a VM (parents created).", t.VMWriteFile)
62 register(s, "vm_read_file", "Read a file from a VM (capped at 1 MiB).", t.VMReadFile) 62 register(s, "vm_read_file", "Read a file from a VM (capped at 1 MiB).", t.VMReadFile)
63 register(s, "vm_expose", "Publish a VM's guest TCP port on its host and return the address to dial. Omit host_port to allocate one from 30000-32767. WARNING: a published port has NO AUTHENTICATION in front of it — whoever can reach the host on that port reaches the service. Publish only what is meant to be reachable.", t.VMExpose) 63 register(s, "vm_expose", "Publish a VM's guest port on its host, TCP or UDP, and return the address to dial. Omit host_port to allocate one from 30000-32767. WARNING: a published port has NO AUTHENTICATION in front of it — whoever can reach the host on that port reaches the service, and a UDP one answers whatever address a datagram claims to come from. Publish only what is meant to be reachable.", t.VMExpose)
64 register(s, "vm_exposures", "List a VM's published ports, with the address to dial and each listener's state. These ports are unauthenticated.", t.VMExposures) 64 register(s, "vm_exposures", "List a VM's published ports, with the protocol, the address to dial and each socket's state. These ports are unauthenticated.", t.VMExposures)
65 register(s, "vm_unexpose", "Stop publishing a VM's guest port; the host closes the listener.", t.VMUnexpose) 65 register(s, "vm_unexpose", "Stop publishing a VM's guest port; the host closes the socket.", t.VMUnexpose)
66 register(s, "vm_destroy", "Destroy a VM by id or EXACT name. Explicit-only; never called automatically.", t.VMDestroy) 66 register(s, "vm_destroy", "Destroy a VM by id or EXACT name. Explicit-only; never called automatically.", t.VMDestroy)
67 register(s, "ca_upload", 67 register(s, "ca_upload",
68 "Register your SSH user CA's PUBLIC key with your tenant, so your guests trust certificates it signs. "+ 68 "Register your SSH user CA's PUBLIC key with your tenant, so your guests trust certificates it signs. "+
internal/mcpserver/tools.go
Old New
@@ -25,7 +25,7 @@ type api interface {
25 DeleteVM(ctx context.Context, id string) error 25 DeleteVM(ctx context.Context, id string) error
26 ListHosts(ctx context.Context) ([]client.Host, error) 26 ListHosts(ctx context.Context) ([]client.Host, error)
27 FirstEligibleHost(ctx context.Context) (client.Host, error) 27 FirstEligibleHost(ctx context.Context) (client.Host, error)
28 CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64) (client.Exposure, error) 28 CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64, protocol string) (client.Exposure, error)
29 ListExposures(ctx context.Context, vmID string) ([]client.Exposure, error) 29 ListExposures(ctx context.Context, vmID string) ([]client.Exposure, error)
30 DeleteExposure(ctx context.Context, id string) error 30 DeleteExposure(ctx context.Context, id string) error
31 RegisterUserCA(ctx context.Context, caLine, label string) error 31 RegisterUserCA(ctx context.Context, caLine, label string) error
@@ -608,16 +608,19 @@ type ExposureView struct {
608 ID string `json:"id"` 608 ID string `json:"id"`
609 GuestPort int64 `json:"guest_port"` 609 GuestPort int64 `json:"guest_port"`
610 HostPort int64 `json:"host_port"` 610 HostPort int64 `json:"host_port"`
611 Protocol string `json:"protocol"`
611 Address string `json:"address,omitempty"` 612 Address string `json:"address,omitempty"`
612 State string `json:"state"` 613 State string `json:"state"`
613 Reason string `json:"reason,omitempty"` 614 Reason string `json:"reason,omitempty"`
614 } 615 }
615 616
616 // exposureView folds an API exposure into the MCP view, joining the host 617 // exposureView folds an API exposure into the MCP view, joining the host
617 // address and host port into one dialable string. 618 // address and host port into one dialable string. The protocol travels beside
619 // it rather than inside it: an address is a host and a port, and which of the
620 // two protocols to send is the caller's next decision, not part of the name.
618 func exposureView(e client.Exposure) ExposureView { 621 func exposureView(e client.Exposure) ExposureView {
619 v := ExposureView{ 622 v := ExposureView{
620 ID: e.ID, GuestPort: e.GuestPort, HostPort: e.HostPort, 623 ID: e.ID, GuestPort: e.GuestPort, HostPort: e.HostPort, Protocol: e.Protocol,
621 State: e.State, Reason: e.Reason, 624 State: e.State, Reason: e.Reason,
622 } 625 }
623 if e.HostAddr != "" { 626 if e.HostAddr != "" {
@@ -628,8 +631,9 @@ func exposureView(e client.Exposure) ExposureView {
628 631
629 type VMExposeIn struct { 632 type VMExposeIn struct {
630 VM string `json:"vm" jsonschema:"VM id or exact name"` 633 VM string `json:"vm" jsonschema:"VM id or exact name"`
631 GuestPort int64 `json:"guest_port" jsonschema:"TCP port the service listens on inside the guest"` 634 GuestPort int64 `json:"guest_port" jsonschema:"port the service listens on inside the guest"`
632 HostPort int64 `json:"host_port,omitempty" jsonschema:"port to bind on the host; omit to allocate one from 30000-32767, a named port must be >= 1024"` 635 HostPort int64 `json:"host_port,omitempty" jsonschema:"port to bind on the host; omit to allocate one from 30000-32767, a named port must be >= 1024"`
636 Protocol string `json:"protocol,omitempty" jsonschema:"tcp (the default) or udp; the same host port can carry one of each"`
633 } 637 }
634 638
635 type VMExposeOut struct { 639 type VMExposeOut struct {
@@ -644,7 +648,7 @@ func (t *Tools) VMExpose(ctx context.Context, in VMExposeIn) (VMExposeOut, error
644 if err != nil { 648 if err != nil {
645 return VMExposeOut{}, err 649 return VMExposeOut{}, err
646 } 650 }
647 e, err := t.API.CreateExposure(ctx, vm.ID, in.GuestPort, in.HostPort) 651 e, err := t.API.CreateExposure(ctx, vm.ID, in.GuestPort, in.HostPort, in.Protocol)
648 if err != nil { 652 if err != nil {
649 return VMExposeOut{}, fmt.Errorf("expose vm %s port %d: %w", vm.Name, in.GuestPort, err) 653 return VMExposeOut{}, fmt.Errorf("expose vm %s port %d: %w", vm.Name, in.GuestPort, err)
650 } 654 }
@@ -679,12 +683,14 @@ type VMUnexposeIn struct {
679 VM string `json:"vm" jsonschema:"VM id or exact name"` 683 VM string `json:"vm" jsonschema:"VM id or exact name"`
680 GuestPort int64 `json:"guest_port" jsonschema:"the guest port to stop publishing"` 684 GuestPort int64 `json:"guest_port" jsonschema:"the guest port to stop publishing"`
681 HostPort int64 `json:"host_port,omitempty" jsonschema:"host port, only needed when the same guest port is published more than once"` 685 HostPort int64 `json:"host_port,omitempty" jsonschema:"host port, only needed when the same guest port is published more than once"`
686 Protocol string `json:"protocol,omitempty" jsonschema:"tcp or udp, only needed when the same guest port is published in both"`
682 } 687 }
683 688
684 type VMUnexposeOut struct { 689 type VMUnexposeOut struct {
685 ID string `json:"id"` 690 ID string `json:"id"`
686 GuestPort int64 `json:"guest_port"` 691 GuestPort int64 `json:"guest_port"`
687 HostPort int64 `json:"host_port"` 692 HostPort int64 `json:"host_port"`
693 Protocol string `json:"protocol"`
688 } 694 }
689 695
690 // VMUnexpose revokes one of a VM's published ports. The caller names the guest 696 // VMUnexpose revokes one of a VM's published ports. The caller names the guest
@@ -699,25 +705,26 @@ func (t *Tools) VMUnexpose(ctx context.Context, in VMUnexposeIn) (VMUnexposeOut,
699 if err != nil { 705 if err != nil {
700 return VMUnexposeOut{}, fmt.Errorf("list exposures for vm %s: %w", vm.Name, err) 706 return VMUnexposeOut{}, fmt.Errorf("list exposures for vm %s: %w", vm.Name, err)
701 } 707 }
702 e, err := matchExposure(exps, vm.Name, in.GuestPort, in.HostPort) 708 e, err := matchExposure(exps, vm.Name, in.GuestPort, in.HostPort, in.Protocol)
703 if err != nil { 709 if err != nil {
704 return VMUnexposeOut{}, err 710 return VMUnexposeOut{}, err
705 } 711 }
706 if err := t.API.DeleteExposure(ctx, e.ID); err != nil { 712 if err := t.API.DeleteExposure(ctx, e.ID); err != nil {
707 return VMUnexposeOut{}, fmt.Errorf("unexpose vm %s port %d: %w", vm.Name, in.GuestPort, err) 713 return VMUnexposeOut{}, fmt.Errorf("unexpose vm %s port %d: %w", vm.Name, in.GuestPort, err)
708 } 714 }
709 return VMUnexposeOut{ID: e.ID, GuestPort: e.GuestPort, HostPort: e.HostPort}, nil 715 return VMUnexposeOut{ID: e.ID, GuestPort: e.GuestPort, HostPort: e.HostPort, Protocol: e.Protocol}, nil
710 } 716 }
711 717
712 // matchExposure picks the one exposure of vmName publishing guestPort. Nothing 718 // matchExposure picks the one exposure of vmName publishing guestPort. Nothing
713 // stops a guest port being published on two host ports, so hostPort (0 = any) 719 // stops a guest port being published on two host ports, or in both protocols,
714 // disambiguates; an ambiguous match refuses rather than guessing which 720 // so hostPort (0 = any) and protocol ("" = any) disambiguate; an ambiguous
715 // listener to close. Errors name what IS published, so the model can correct 721 // match refuses rather than guessing which socket to close. Errors name what IS
716 // itself without a second listing call. 722 // published, so the model can correct itself without a second listing call.
717 func matchExposure(exps []client.Exposure, vmName string, guestPort, hostPort int64) (client.Exposure, error) { 723 func matchExposure(exps []client.Exposure, vmName string, guestPort, hostPort int64, protocol string) (client.Exposure, error) {
718 var matches []client.Exposure 724 var matches []client.Exposure
719 for _, e := range exps { 725 for _, e := range exps {
720 if e.GuestPort == guestPort && (hostPort == 0 || e.HostPort == hostPort) { 726 if e.GuestPort == guestPort && (hostPort == 0 || e.HostPort == hostPort) &&
727 (protocol == "" || e.Protocol == protocol) {
721 matches = append(matches, e) 728 matches = append(matches, e)
722 } 729 }
723 } 730 }
@@ -727,18 +734,18 @@ func matchExposure(exps []client.Exposure, vmName string, guestPort, hostPort in
727 case 0: 734 case 0:
728 return client.Exposure{}, fmt.Errorf("vm %s publishes no guest port %d (published: %s)", vmName, guestPort, describeExposures(exps)) 735 return client.Exposure{}, fmt.Errorf("vm %s publishes no guest port %d (published: %s)", vmName, guestPort, describeExposures(exps))
729 default: 736 default:
730 return client.Exposure{}, fmt.Errorf("vm %s publishes guest port %d more than once (%s); name host_port to pick one", vmName, guestPort, describeExposures(matches)) 737 return client.Exposure{}, fmt.Errorf("vm %s publishes guest port %d more than once (%s); name host_port or protocol to pick one", vmName, guestPort, describeExposures(matches))
731 } 738 }
732 } 739 }
733 740
734 // describeExposures renders exposures as "guest:host" pairs for an error. 741 // describeExposures renders exposures as "guest:host/protocol" for an error.
735 func describeExposures(exps []client.Exposure) string { 742 func describeExposures(exps []client.Exposure) string {
736 if len(exps) == 0 { 743 if len(exps) == 0 {
737 return "none" 744 return "none"
738 } 745 }
739 parts := make([]string, 0, len(exps)) 746 parts := make([]string, 0, len(exps))
740 for _, e := range exps { 747 for _, e := range exps {
741 parts = append(parts, fmt.Sprintf("%d:%d", e.GuestPort, e.HostPort)) 748 parts = append(parts, fmt.Sprintf("%d:%d/%s", e.GuestPort, e.HostPort, e.Protocol))
742 } 749 }
743 return strings.Join(parts, ", ") 750 return strings.Join(parts, ", ")
744 } 751 }
internal/mcpserver/tools_test.go
Old New
@@ -94,15 +94,19 @@ func (f *fakeToolsAPI) FirstEligibleHost(ctx context.Context) (client.Host, erro
94 } 94 }
95 95
96 // CreateExposure mirrors the control plane: host port 0 is allocated from the 96 // CreateExposure mirrors the control plane: host port 0 is allocated from the
97 // reserved range, and the host address comes back with the grant. 97 // reserved range, an unnamed protocol is tcp, and the host address comes back
98 func (f *fakeToolsAPI) CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64) (client.Exposure, error) { 98 // with the grant.
99 func (f *fakeToolsAPI) CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64, protocol string) (client.Exposure, error) {
99 if hostPort == 0 { 100 if hostPort == 0 {
100 hostPort = 30000 + int64(len(f.exposures[vmID])) 101 hostPort = 30000 + int64(len(f.exposures[vmID]))
101 } 102 }
103 if protocol == "" {
104 protocol = "tcp"
105 }
102 e := client.Exposure{ 106 e := client.Exposure{
103 ID: fmt.Sprintf("x-%d", len(f.exposures[vmID])+1), VMID: vmID, HostID: "h1", 107 ID: fmt.Sprintf("x-%d", len(f.exposures[vmID])+1), VMID: vmID, HostID: "h1",
104 GuestPort: guestPort, HostPort: hostPort, HostAddr: "10.0.0.4", 108 GuestPort: guestPort, HostPort: hostPort, HostAddr: "10.0.0.4",
105 Protocol: "tcp", Scope: "host", State: "pending", 109 Protocol: protocol, Scope: "host", State: "pending",
106 } 110 }
107 if f.exposures == nil { 111 if f.exposures == nil {
108 f.exposures = map[string][]client.Exposure{} 112 f.exposures = map[string][]client.Exposure{}
@@ -513,6 +517,32 @@ func TestExposePublishesGuestPortWithDialAddress(t *testing.T) {
513 assert.Equal(t, "10.0.0.4:31500", out.Exposure.Address) 517 assert.Equal(t, "10.0.0.4:31500", out.Exposure.Address)
514 } 518 }
515 519
520 func TestExposeCarriesTheProtocolAndUnexposePicksByIt(t *testing.T) {
521 api := &fakeToolsAPI{vms: []client.VM{{ID: "abc123", Name: "web-1", Lifecycle: "ready"}}}
522 tl := newTestTools(api, &fakeRunner{})
523
524 tcp, err := tl.VMExpose(t.Context(), VMExposeIn{VM: "web-1", GuestPort: 53, HostPort: 30053})
525 require.NoError(t, err)
526 assert.Equal(t, "tcp", tcp.Exposure.Protocol, "an unnamed protocol is tcp, and the view says which it got")
527
528 udp, err := tl.VMExpose(t.Context(), VMExposeIn{VM: "web-1", GuestPort: 53, HostPort: 30053, Protocol: "udp"})
529 require.NoError(t, err)
530 assert.Equal(t, "udp", udp.Exposure.Protocol)
531
532 // One guest port, both protocols: the same ambiguity two host ports create,
533 // and protocol is the other way to resolve it.
534 _, err = tl.VMUnexpose(t.Context(), VMUnexposeIn{VM: "web-1", GuestPort: 53})
535 require.Error(t, err)
536 assert.ErrorContains(t, err, "53:30053/udp", "the error names the protocol of what IS published")
537 assert.Empty(t, api.revoked)
538
539 out, err := tl.VMUnexpose(t.Context(), VMUnexposeIn{VM: "web-1", GuestPort: 53, Protocol: "udp"})
540 require.NoError(t, err)
541 assert.Equal(t, udp.Exposure.ID, out.ID)
542 assert.Equal(t, "udp", out.Protocol)
543 assert.Equal(t, []string{udp.Exposure.ID}, api.revoked)
544 }
545
516 func TestExposeUnknownVM(t *testing.T) { 546 func TestExposeUnknownVM(t *testing.T) {
517 tl := newTestTools(&fakeToolsAPI{}, &fakeRunner{}) 547 tl := newTestTools(&fakeToolsAPI{}, &fakeRunner{})
518 _, err := tl.VMExpose(t.Context(), VMExposeIn{VM: "nope", GuestPort: 80}) 548 _, err := tl.VMExpose(t.Context(), VMExposeIn{VM: "nope", GuestPort: 80})
internal/pb/sync.pb.go
Old New
@@ -1396,18 +1396,18 @@ func (x *TCPOpened) GetError() string {
1396 } 1396 }
1397 1397
1398 // ExposureDesired is one published guest port a host should be serving: bind 1398 // ExposureDesired is one published guest port a host should be serving: bind
1399 // host_port on the host, pipe every accepted connection to guest_port inside 1399 // host_port on the host, pipe every accepted connection (or every datagram)
1400 // the guest. It rides the snapshot at TOP LEVEL rather than nested in 1400 // to guest_port inside the guest. It rides the snapshot at TOP LEVEL rather
1401 // VMDesired, because exposures are their own objects converging on their own 1401 // than nested in VMDesired, because exposures are their own objects converging
1402 // cadence — an exposure can be created while its VM is still imaging, and it 1402 // on their own cadence — an exposure can be created while its VM is still
1403 // binds immediately. 1403 // imaging, and it binds immediately.
1404 type ExposureDesired struct { 1404 type ExposureDesired struct {
1405 state protoimpl.MessageState `protogen:"open.v1"` 1405 state protoimpl.MessageState `protogen:"open.v1"`
1406 Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"` 1406 Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
1407 VmId string `protobuf:"bytes,2,opt,name=vm_id,json=vmId,proto3" json:"vm_id,omitempty"` 1407 VmId string `protobuf:"bytes,2,opt,name=vm_id,json=vmId,proto3" json:"vm_id,omitempty"`
1408 GuestPort uint32 `protobuf:"varint,3,opt,name=guest_port,json=guestPort,proto3" json:"guest_port,omitempty"` 1408 GuestPort uint32 `protobuf:"varint,3,opt,name=guest_port,json=guestPort,proto3" json:"guest_port,omitempty"`
1409 HostPort uint32 `protobuf:"varint,4,opt,name=host_port,json=hostPort,proto3" json:"host_port,omitempty"` 1409 HostPort uint32 `protobuf:"varint,4,opt,name=host_port,json=hostPort,proto3" json:"host_port,omitempty"`
1410 Protocol string `protobuf:"bytes,5,opt,name=protocol,proto3" json:"protocol,omitempty"` // "tcp" 1410 Protocol string `protobuf:"bytes,5,opt,name=protocol,proto3" json:"protocol,omitempty"` // "tcp"|"udp"
1411 unknownFields protoimpl.UnknownFields 1411 unknownFields protoimpl.UnknownFields
1412 sizeCache protoimpl.SizeCache 1412 sizeCache protoimpl.SizeCache
1413 } 1413 }
@@ -1478,14 +1478,14 @@ func (x *ExposureDesired) GetProtocol() string {
1478 } 1478 }
1479 1479
1480 // ExposureActual is one exposure's state as its host observes it: "active" 1480 // ExposureActual is one exposure's state as its host observes it: "active"
1481 // once the host listener is bound, "failed" with the OS error otherwise. 1481 // once the host socket is bound, "failed" with the OS error otherwise.
1482 // "active" means the HOST half of the pipe exists — whether anything answers 1482 // "active" means the HOST half of the pipe exists — whether anything answers
1483 // inside the guest is the guest's half, and this does not pretend otherwise. 1483 // inside the guest is the guest's half, and this does not pretend otherwise.
1484 type ExposureActual struct { 1484 type ExposureActual struct {
1485 state protoimpl.MessageState `protogen:"open.v1"` 1485 state protoimpl.MessageState `protogen:"open.v1"`
1486 Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"` 1486 Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
1487 State string `protobuf:"bytes,2,opt,name=state,proto3" json:"state,omitempty"` // "active"|"failed" 1487 State string `protobuf:"bytes,2,opt,name=state,proto3" json:"state,omitempty"` // "active"|"failed"
1488 Reason string `protobuf:"bytes,3,opt,name=reason,proto3" json:"reason,omitempty"` // the OS error, when failed 1488 Reason string `protobuf:"bytes,3,opt,name=reason,proto3" json:"reason,omitempty"` // the OS error, when failed; a bound port's ongoing trouble otherwise
1489 unknownFields protoimpl.UnknownFields 1489 unknownFields protoimpl.UnknownFields
1490 sizeCache protoimpl.SizeCache 1490 sizeCache protoimpl.SizeCache
1491 } 1491 }
internal/server/api/client/client.go
Old New
@@ -150,11 +150,12 @@ func (c *Client) PatchVM(ctx context.Context, id, powerState string) error {
150 return c.do(ctx, http.MethodPatch, "/api/v1/vms/"+url.PathEscape(id), PatchVMRequest{PowerState: powerState}, nil) 150 return c.do(ctx, http.MethodPatch, "/api/v1/vms/"+url.PathEscape(id), PatchVMRequest{PowerState: powerState}, nil)
151 } 151 }
152 152
153 // CreateExposure publishes guestPort of a VM on its host. hostPort 0 asks the 153 // CreateExposure publishes guestPort of a VM on its host, for protocol "tcp"
154 // control plane to allocate one from the reserved range; the returned exposure 154 // or "udp" (empty means tcp). hostPort 0 asks the control plane to allocate one
155 // carries whichever port it ended up with. 155 // from the reserved range; the returned exposure carries whichever port it
156 func (c *Client) CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64) (Exposure, error) { 156 // ended up with.
157 req := CreateExposureRequest{GuestPort: guestPort, HostPort: hostPort} 157 func (c *Client) CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64, protocol string) (Exposure, error) {
158 req := CreateExposureRequest{GuestPort: guestPort, HostPort: hostPort, Protocol: protocol}
158 var out Exposure 159 var out Exposure
159 return out, c.do(ctx, http.MethodPost, "/api/v1/vms/"+url.PathEscape(vmID)+"/exposures", req, &out) 160 return out, c.do(ctx, http.MethodPost, "/api/v1/vms/"+url.PathEscape(vmID)+"/exposures", req, &out)
160 } 161 }
internal/server/api/client/client_test.go
Old New
@@ -479,7 +479,7 @@ func TestCreateExposure(t *testing.T) {
479 `"created_at":"2026-08-05T12:00:00Z"}`) 479 `"created_at":"2026-08-05T12:00:00Z"}`)
480 c := &client.Client{BaseURL: srv.URL, Token: "eitri_pat_x"} 480 c := &client.Client{BaseURL: srv.URL, Token: "eitri_pat_x"}
481 481
482 got, err := c.CreateExposure(context.Background(), "v-1", 8080, 0) 482 got, err := c.CreateExposure(context.Background(), "v-1", 8080, 0, "udp")
483 if err != nil { 483 if err != nil {
484 t.Fatalf("CreateExposure: %v", err) 484 t.Fatalf("CreateExposure: %v", err)
485 } 485 }
@@ -490,8 +490,8 @@ func TestCreateExposure(t *testing.T) {
490 if err := json.Unmarshal(cap.body, &sent); err != nil { 490 if err := json.Unmarshal(cap.body, &sent); err != nil {
491 t.Fatalf("decode sent body: %v", err) 491 t.Fatalf("decode sent body: %v", err)
492 } 492 }
493 if sent.GuestPort != 8080 || sent.HostPort != 0 { 493 if sent.GuestPort != 8080 || sent.HostPort != 0 || sent.Protocol != "udp" {
494 t.Errorf("sent = %+v, want guest 8080 / host 0", sent) 494 t.Errorf("sent = %+v, want guest 8080 / host 0 / udp", sent)
495 } 495 }
496 if got.ID != "x-1" || got.HostPort != 30080 { 496 if got.ID != "x-1" || got.HostPort != 30080 {
497 t.Errorf("got = %+v, want the allocated exposure back", got) 497 t.Errorf("got = %+v, want the allocated exposure back", got)
internal/server/api/exposures.go
Old New
@@ -32,15 +32,19 @@ func (a *API) exposureVM(w http.ResponseWriter, r *http.Request) (store.VM, bool
32 return vm, true 32 return vm, true
33 } 33 }
34 34
35 // validateExposurePorts checks a create request, returning (msg, status) on 35 // validateExposure checks a create request, returning (msg, status) on failure
36 // failure or ("", 0) when valid. 36 // or ("", 0) when valid.
37 // 37 //
38 // guest_port is unrestricted within the port space: the guest owns the guest, 38 // guest_port is unrestricted within the port space: the guest owns the guest,
39 // including which of its ports are worth publishing. host_port is where 39 // including which of its ports are worth publishing. host_port is where
40 // privilege lives — the 1024 floor is uniformity as much as safety, because a 40 // privilege lives — the 1024 floor is uniformity as much as safety, because a
41 // macOS agent is unprivileged and cannot bind lower, and low ports are the 41 // macOS agent is unprivileged and cannot bind lower, and low ports are the
42 // future gateway's territory. 42 // future gateway's territory.
43 func validateExposurePorts(req types.CreateExposureRequest) (string, int) { 43 //
44 // protocol is closed at two values because the agent has exactly two proxies
45 // to offer, and a third string accepted here would become a grant no host ever
46 // binds — a row that reads published and serves nothing.
47 func validateExposure(req types.CreateExposureRequest) (string, int) {
44 if req.GuestPort < 1 || req.GuestPort > 65535 { 48 if req.GuestPort < 1 || req.GuestPort > 65535 {
45 return "guest_port must be between 1 and 65535", http.StatusBadRequest 49 return "guest_port must be between 1 and 65535", http.StatusBadRequest
46 } 50 }
@@ -49,9 +53,22 @@ func validateExposurePorts(req types.CreateExposureRequest) (string, int) {
49 strconv.Itoa(store.MinAllocatedHostPort) + "-" + strconv.Itoa(store.MaxAllocatedHostPort), 53 strconv.Itoa(store.MinAllocatedHostPort) + "-" + strconv.Itoa(store.MaxAllocatedHostPort),
50 http.StatusBadRequest 54 http.StatusBadRequest
51 } 55 }
56 if req.Protocol != "" && req.Protocol != "tcp" && req.Protocol != "udp" {
57 return `protocol must be "tcp" or "udp"`, http.StatusBadRequest
58 }
52 return "", 0 59 return "", 0
53 } 60 }
54 61
62 // exposureProtocol is the protocol a create request asks for. An omitted one is
63 // tcp: the field arrived after the endpoint did, and every caller that predates
64 // it means the same thing by silence.
65 func exposureProtocol(req types.CreateExposureRequest) string {
66 if req.Protocol == "" {
67 return "tcp"
68 }
69 return req.Protocol
70 }
71
55 // exposureState folds a host's live report into one exposure's state. An 72 // exposureState folds a host's live report into one exposure's state. An
56 // exposure the host has not reported on is "pending": the grant exists, and 73 // exposure the host has not reported on is "pending": the grant exists, and
57 // nothing has said what the host made of it yet. 74 // nothing has said what the host made of it yet.
@@ -101,17 +118,18 @@ func (a *API) handleCreateExposure(w http.ResponseWriter, r *http.Request) {
101 if !decodeJSON(w, r, &req) { 118 if !decodeJSON(w, r, &req) {
102 return 119 return
103 } 120 }
104 if msg, code := validateExposurePorts(req); msg != "" { 121 if msg, code := validateExposure(req); msg != "" {
105 http.Error(w, msg, code) 122 http.Error(w, msg, code)
106 return 123 return
107 } 124 }
108 e, err := a.st.CreateExposure(vm.ID, req.GuestPort, req.HostPort) 125 protocol := exposureProtocol(req)
126 e, err := a.st.CreateExposure(vm.ID, req.GuestPort, req.HostPort, protocol)
109 if err != nil { 127 if err != nil {
110 switch { 128 switch {
111 case errors.Is(err, store.ErrExposureVMNotFound): 129 case errors.Is(err, store.ErrExposureVMNotFound):
112 http.Error(w, "not found", http.StatusNotFound) 130 http.Error(w, "not found", http.StatusNotFound)
113 case errors.Is(err, store.ErrHostPortTaken): 131 case errors.Is(err, store.ErrHostPortTaken):
114 http.Error(w, "host port already in use on that host", http.StatusConflict) 132 http.Error(w, "host port already in use on that host for "+protocol, http.StatusConflict)
115 case errors.Is(err, store.ErrNoFreeHostPort): 133 case errors.Is(err, store.ErrNoFreeHostPort):
116 http.Error(w, "no free host port in the reserved range on that host", http.StatusConflict) 134 http.Error(w, "no free host port in the reserved range on that host", http.StatusConflict)
117 default: 135 default:
@@ -123,6 +141,7 @@ func (a *API) handleCreateExposure(w http.ResponseWriter, r *http.Request) {
123 "vm_id": vm.ID, "name": vm.Name, "exposure_id": e.ID, 141 "vm_id": vm.ID, "name": vm.Name, "exposure_id": e.ID,
124 "guest_port": strconv.FormatInt(e.GuestPort, 10), 142 "guest_port": strconv.FormatInt(e.GuestPort, 10),
125 "host_port": strconv.FormatInt(e.HostPort, 10), 143 "host_port": strconv.FormatInt(e.HostPort, 10),
144 "protocol": e.Protocol,
126 }) 145 })
127 a.hub.Poke(vm.HostID) 146 a.hub.Poke(vm.HostID)
128 a.notif.notify() 147 a.notif.notify()
@@ -178,6 +197,7 @@ func (a *API) handleDeleteExposure(w http.ResponseWriter, r *http.Request) {
178 "vm_id": e.VMID, "exposure_id": e.ID, 197 "vm_id": e.VMID, "exposure_id": e.ID,
179 "guest_port": strconv.FormatInt(e.GuestPort, 10), 198 "guest_port": strconv.FormatInt(e.GuestPort, 10),
180 "host_port": strconv.FormatInt(e.HostPort, 10), 199 "host_port": strconv.FormatInt(e.HostPort, 10),
200 "protocol": e.Protocol,
181 }) 201 })
182 a.hub.Poke(e.HostID) 202 a.hub.Poke(e.HostID)
183 a.notif.notify() 203 a.notif.notify()
internal/server/api/exposures_test.go
Old New
@@ -2,6 +2,7 @@ package api
2 2
3 import ( 3 import (
4 "encoding/json" 4 "encoding/json"
5 "io"
5 "net/http" 6 "net/http"
6 "net/http/httptest" 7 "net/http/httptest"
7 "testing" 8 "testing"
@@ -76,6 +77,48 @@ func TestCreateExposureHonoursAndRefusesHostPorts(t *testing.T) {
76 assert.Equal(t, 409, resp.StatusCode) 77 assert.Equal(t, 409, resp.StatusCode)
77 } 78 }
78 79
80 func TestCreateExposureDefaultsToTCPAndTakesUDP(t *testing.T) {
81 ts, _, _ := testServer(t)
82 host := enroll(t, ts)
83 vmID := createTestVM(t, ts, host["host_id"], "web-1")
84
85 resp := do(t, "POST", ts.URL+"/api/v1/vms/"+vmID+"/exposures", testPAT,
86 map[string]any{"guest_port": 53, "host_port": 8443, "protocol": "udp"})
87 require.Equal(t, 201, resp.StatusCode)
88 var udp map[string]any
89 require.NoError(t, json.NewDecoder(resp.Body).Decode(&udp))
90 assert.Equal(t, "udp", udp["protocol"])
91
92 // The same host port carries one of each, and no more than one of either.
93 resp = do(t, "POST", ts.URL+"/api/v1/vms/"+vmID+"/exposures", testPAT,
94 map[string]any{"guest_port": 53, "host_port": 8443})
95 require.Equal(t, 201, resp.StatusCode)
96 var tcp map[string]any
97 require.NoError(t, json.NewDecoder(resp.Body).Decode(&tcp))
98 assert.Equal(t, "tcp", tcp["protocol"], "a request that names no protocol means tcp")
99
100 resp = do(t, "POST", ts.URL+"/api/v1/vms/"+vmID+"/exposures", testPAT,
101 map[string]any{"guest_port": 54, "host_port": 8443, "protocol": "udp"})
102 assert.Equal(t, 409, resp.StatusCode)
103 }
104
105 func TestCreateExposureRefusesAnotherProtocol(t *testing.T) {
106 ts, _, _ := testServer(t)
107 host := enroll(t, ts)
108 vmID := createTestVM(t, ts, host["host_id"], "web-1")
109
110 for _, proto := range []string{"sctp", "TCP", "udp4", "http"} {
111 t.Run(proto, func(t *testing.T) {
112 resp := do(t, "POST", ts.URL+"/api/v1/vms/"+vmID+"/exposures", testPAT,
113 map[string]any{"guest_port": 8080, "protocol": proto})
114 require.Equal(t, 400, resp.StatusCode)
115 body, err := io.ReadAll(resp.Body)
116 require.NoError(t, err)
117 assert.Contains(t, string(body), `"tcp" or "udp"`, "a refusal names what it would take")
118 })
119 }
120 }
121
79 func TestCreateExposureValidatesPorts(t *testing.T) { 122 func TestCreateExposureValidatesPorts(t *testing.T) {
80 ts, _, _ := testServer(t) 123 ts, _, _ := testServer(t)
81 host := enroll(t, ts) 124 host := enroll(t, ts)
internal/server/api/routes.go
Old New
@@ -233,9 +233,9 @@ var routeTable = []Route{
233 Doc: "One VM's lifecycle timeline (audit rows carrying its vm_id), newest first; survives the VM row being reaped.", 233 Doc: "One VM's lifecycle timeline (audit rows carrying its vm_id), newest first; survives the VM row being reaped.",
234 handler: (*API).handleListVMEvents, 234 handler: (*API).handleListVMEvents,
235 }, 235 },
236 // Service exposure: a tenant publishes one TCP port of one VM, and the 236 // Service exposure: a tenant publishes one port of one VM, TCP or UDP, and
237 // fleet binds it on that VM's host. There is no update verb — an exposure 237 // the fleet binds it on that VM's host. There is no update verb — an
238 // is a grant, not a document. 238 // exposure is a grant, not a document.
239 { 239 {
240 Method: "POST", 240 Method: "POST",
241 Path: "/api/v1/vms/{id}/exposures", 241 Path: "/api/v1/vms/{id}/exposures",
@@ -244,7 +244,7 @@ var routeTable = []Route{
244 Request: (*types.CreateExposureRequest)(nil), 244 Request: (*types.CreateExposureRequest)(nil),
245 Response: (*types.Exposure)(nil), 245 Response: (*types.Exposure)(nil),
246 Success: http.StatusCreated, 246 Success: http.StatusCreated,
247 Doc: "Publish a guest TCP port on the VM's host. Omit host_port to allocate one from the reserved range 30000-32767; a named port must be >= 1024 and is honored or refused.", 247 Doc: "Publish a guest port on the VM's host, protocol \"tcp\" (the default) or \"udp\". Omit host_port to allocate one from the reserved range 30000-32767; a named port must be >= 1024 and is honored or refused, and is taken only by another exposure of the same protocol.",
248 handler: (*API).handleCreateExposure, 248 handler: (*API).handleCreateExposure,
249 }, 249 },
250 { 250 {
internal/server/api/testdata/create-exposure-request.golden.json
Old New
@@ -1,4 +1,5 @@
1 { 1 {
2 "guest_port": 8080, 2 "guest_port": 8080,
3 "host_port": 30080 3 "host_port": 30080,
4 "protocol": "udp"
4 } 5 }
internal/server/api/types/types.go
Old New
@@ -229,17 +229,20 @@ type RevokeSSHCertRequest struct {
229 // CreateExposureRequest is the POST /api/v1/vms/{id}/exposures body: publish 229 // CreateExposureRequest is the POST /api/v1/vms/{id}/exposures body: publish
230 // guest_port of that VM. host_port is optional — omitted (or 0) allocates one 230 // guest_port of that VM. host_port is optional — omitted (or 0) allocates one
231 // from the reserved range 30000-32767; a named port must be >= 1024 and is 231 // from the reserved range 30000-32767; a named port must be >= 1024 and is
232 // honored or refused. Request-side only. 232 // honored or refused. protocol is "tcp" (the default when omitted) or "udp",
233 // and a host port is claimed per protocol, so the same number can carry one of
234 // each. Request-side only.
233 type CreateExposureRequest struct { 235 type CreateExposureRequest struct {
234 GuestPort int64 `json:"guest_port"` 236 GuestPort int64 `json:"guest_port"`
235 HostPort int64 `json:"host_port"` // 0 = allocate 237 HostPort int64 `json:"host_port"` // 0 = allocate
238 Protocol string `json:"protocol"` // "" = tcp
236 } 239 }
237 240
238 // Exposure is one published guest port, served by 241 // Exposure is one published guest port, served by
239 // POST and GET /api/v1/vms/{id}/exposures. HostAddr and State are 242 // POST and GET /api/v1/vms/{id}/exposures. HostAddr and State are
240 // server-derived: the address comes from the host's own report, and the state 243 // server-derived: the address comes from the host's own report, and the state
241 // from what its agent last said its listener is doing — "pending" until an 244 // from what its agent last said its socket is doing — "pending" until an
242 // agent has reported at all, then "active" (the listener is bound) or "failed" 245 // agent has reported at all, then "active" (the socket is bound) or "failed"
243 // with the OS error in Reason. "active" describes the host half of the pipe; 246 // with the OS error in Reason. "active" describes the host half of the pipe;
244 // whether anything answers inside the guest is the guest's business. 247 // whether anything answers inside the guest is the guest's business.
245 type Exposure struct { 248 type Exposure struct {
internal/server/api/wire_golden_test.go
Old New
@@ -260,6 +260,7 @@ func TestWireGolden(t *testing.T) {
260 goldenCheck(t, "create-exposure-request", types.CreateExposureRequest{ 260 goldenCheck(t, "create-exposure-request", types.CreateExposureRequest{
261 GuestPort: 8080, 261 GuestPort: 8080,
262 HostPort: 30080, 262 HostPort: 30080,
263 Protocol: "udp",
263 }) 264 })
264 265
265 goldenCheck(t, "exposure", []types.Exposure{{ 266 goldenCheck(t, "exposure", []types.Exposure{{
internal/server/store/evolve.go
Old New
@@ -44,6 +44,21 @@ func dropTable(db *sql.DB, table string) error {
44 return nil 44 return nil
45 } 45 }
46 46
47 // dropIndex removes an index the schema no longer declares. An index whose
48 // columns change is a new index under a new name plus this: SQLite's CREATE
49 // UNIQUE INDEX IF NOT EXISTS leaves an existing index of that name exactly as
50 // it was, so a redefinition under the old name would be silently ignored on
51 // every database that already had one.
52 //
53 // index is interpolated verbatim (SQLite cannot bind identifiers): pass trusted
54 // compile-time constants only, the same rule ensureColumn states.
55 func dropIndex(db *sql.DB, index string) error {
56 if _, err := db.Exec(fmt.Sprintf(`DROP INDEX IF EXISTS %s`, index)); err != nil {
57 return fmt.Errorf("drop index %s: %w", index, err)
58 }
59 return nil
60 }
61
47 // dropColumn removes a column that is no longer part of the schema, so an 62 // dropColumn removes a column that is no longer part of the schema, so an
48 // existing database stops carrying it — and stops carrying whatever was in it. 63 // existing database stops carrying it — and stops carrying whatever was in it.
49 // Idempotent: a database that never had the column, or has already dropped it, 64 // Idempotent: a database that never had the column, or has already dropped it,
internal/server/store/exposures.go
Old New
@@ -11,10 +11,10 @@ import (
11 ) 11 )
12 12
13 // Exposure is one published guest port: the fleet binds HostPort on the VM's 13 // Exposure is one published guest port: the fleet binds HostPort on the VM's
14 // host and pipes every accepted connection to GuestPort inside the guest. 14 // host and pipes every accepted connection — or every datagram, when Protocol
15 // Tenant and HostID are both derived from the VM at create — never accepted 15 // is "udp" — to GuestPort inside the guest. Tenant and HostID are both derived
16 // from a caller — so an exposure can only ever name the partition and the 16 // from the VM at create — never accepted from a caller — so an exposure can
17 // machine its VM already lives in. 17 // only ever name the partition and the machine its VM already lives in.
18 type Exposure struct { 18 type Exposure struct {
19 ID, Tenant, VMID, HostID string 19 ID, Tenant, VMID, HostID string
20 GuestPort, HostPort int64 20 GuestPort, HostPort int64
@@ -36,7 +36,7 @@ const (
36 var ErrExposureVMNotFound = errors.New("vm not found") 36 var ErrExposureVMNotFound = errors.New("vm not found")
37 37
38 // ErrHostPortTaken reports that another exposure already holds the requested 38 // ErrHostPortTaken reports that another exposure already holds the requested
39 // host port on that host. 39 // host port on that host, for the same protocol.
40 var ErrHostPortTaken = errors.New("host port already exposed on this host") 40 var ErrHostPortTaken = errors.New("host port already exposed on this host")
41 41
42 // ErrNoFreeHostPort reports that the reserved range is fully allocated on the 42 // ErrNoFreeHostPort reports that the reserved range is fully allocated on the
@@ -82,11 +82,15 @@ func queryExposures(q querier, from string, args ...any) ([]Exposure, error) {
82 return out, rows.Err() 82 return out, rows.Err()
83 } 83 }
84 84
85 // CreateExposure publishes guestPort of vmID on that VM's host. hostPort 0 asks 85 // CreateExposure publishes guestPort of vmID on that VM's host, for protocol
86 // for one from the reserved range; a named port is honored or refused. The 86 // ("tcp" or "udp" — the API is what holds callers to those two). hostPort 0
87 // whole decision — which host, which tenant, which port — happens in one 87 // asks for one from the reserved range; a named port is honored or refused,
88 // transaction, so two concurrent creates cannot agree on the same port. 88 // and it is only taken when the SAME protocol already holds it: TCP 30000 and
89 func (s *Store) CreateExposure(vmID string, guestPort, hostPort int64) (Exposure, error) { 89 // UDP 30000 are two ports on the host, and refusing the second would be the
90 // proxy inventing a scarcity the machine does not have. The whole decision —
91 // which host, which tenant, which port — happens in one transaction, so two
92 // concurrent creates cannot agree on the same port.
93 func (s *Store) CreateExposure(vmID string, guestPort, hostPort int64, protocol string) (Exposure, error) {
90 tx, err := s.db.Begin() 94 tx, err := s.db.Begin()
91 if err != nil { 95 if err != nil {
92 return Exposure{}, err 96 return Exposure{}, err
@@ -116,7 +120,7 @@ func (s *Store) CreateExposure(vmID string, guestPort, hostPort int64) (Exposure
116 e := Exposure{ 120 e := Exposure{
117 ID: random.Hex(16), Tenant: tenant, VMID: vmID, HostID: hostID, 121 ID: random.Hex(16), Tenant: tenant, VMID: vmID, HostID: hostID,
118 GuestPort: guestPort, HostPort: hostPort, 122 GuestPort: guestPort, HostPort: hostPort,
119 Protocol: "tcp", Scope: "lan", CreatedAt: time.Now().UTC(), 123 Protocol: protocol, Scope: "lan", CreatedAt: time.Now().UTC(),
120 } 124 }
121 if _, err := tx.Exec( 125 if _, err := tx.Exec(
122 `INSERT INTO exposures(id, tenant, vm_id, host_id, guest_port, host_port, protocol, scope, created_at) 126 `INSERT INTO exposures(id, tenant, vm_id, host_id, guest_port, host_port, protocol, scope, created_at)
@@ -125,7 +129,7 @@ func (s *Store) CreateExposure(vmID string, guestPort, hostPort int64) (Exposure
125 e.CreatedAt.Format(time.RFC3339), 129 e.CreatedAt.Format(time.RFC3339),
126 ); err != nil { 130 ); err != nil {
127 // SQLITE_CONSTRAINT_UNIQUE (2067): the only UNIQUE constraint this 131 // SQLITE_CONSTRAINT_UNIQUE (2067): the only UNIQUE constraint this
128 // insert can trip besides the random-hex PK is exposures_host_port. 132 // insert can trip besides the random-hex PK is exposures_host_port_proto.
129 // Matched by errno, not message text, which embeds the index's column 133 // Matched by errno, not message text, which embeds the index's column
130 // list and breaks on the next index change. 134 // list and breaks on the next index change.
131 if serr, ok := errors.AsType[*sqlite.Error](err); ok && serr.Code() == 2067 { 135 if serr, ok := errors.AsType[*sqlite.Error](err); ok && serr.Code() == 2067 {
@@ -143,10 +147,17 @@ func (s *Store) CreateExposure(vmID string, guestPort, hostPort int64) (Exposure
143 return e, nil 147 return e, nil
144 } 148 }
145 149
146 // allocateHostPort returns the lowest free port of the reserved range on 150 // allocateHostPort returns the lowest port of the reserved range that no
147 // hostID. It reads the range's held ports into memory before choosing, because 151 // exposure on hostID holds — in EITHER protocol. A caller that named nothing
148 // the store runs on a single connection: an open result set would block the 152 // gets a number it can hand to anyone without a protocol beside it, and the
149 // insert that follows in the same transaction. 153 // range is 2768 ports deep on a host that runs dozens of guests, so the ports
154 // this leaves on the table cost nothing that reasoning about half-taken
155 // numbers would not cost more. Naming a port explicitly still gets the full
156 // per-protocol answer.
157 //
158 // It reads the range's held ports into memory before choosing, because the
159 // store runs on a single connection: an open result set would block the insert
160 // that follows in the same transaction.
150 func allocateHostPort(tx *sql.Tx, hostID string) (int64, error) { 161 func allocateHostPort(tx *sql.Tx, hostID string) (int64, error) {
151 rows, err := tx.Query( 162 rows, err := tx.Query(
152 `SELECT host_port FROM exposures WHERE host_id=? AND host_port BETWEEN ? AND ?`, 163 `SELECT host_port FROM exposures WHERE host_id=? AND host_port BETWEEN ? AND ?`,
internal/server/store/exposures_test.go
Old New
@@ -21,7 +21,7 @@ func TestCreateExposureAllocatesFromTheReservedRange(t *testing.T) {
21 h := enrollHost(t, s) 21 h := enrollHost(t, s)
22 vm := makeExposureVM(t, s, h, "web-1") 22 vm := makeExposureVM(t, s, h, "web-1")
23 23
24 e, err := s.CreateExposure(vm.ID, 8080, 0) 24 e, err := s.CreateExposure(vm.ID, 8080, 0, "tcp")
25 require.NoError(t, err) 25 require.NoError(t, err)
26 assert.Equal(t, vm.ID, e.VMID) 26 assert.Equal(t, vm.ID, e.VMID)
27 assert.Equal(t, h.ID, e.HostID) 27 assert.Equal(t, h.ID, e.HostID)
@@ -38,16 +38,16 @@ func TestCreateExposureAllocatesTheLowestFreePort(t *testing.T) {
38 h := enrollHost(t, s) 38 h := enrollHost(t, s)
39 vm := makeExposureVM(t, s, h, "web-1") 39 vm := makeExposureVM(t, s, h, "web-1")
40 40
41 first, err := s.CreateExposure(vm.ID, 8080, 0) 41 first, err := s.CreateExposure(vm.ID, 8080, 0, "tcp")
42 require.NoError(t, err) 42 require.NoError(t, err)
43 second, err := s.CreateExposure(vm.ID, 8081, 0) 43 second, err := s.CreateExposure(vm.ID, 8081, 0, "tcp")
44 require.NoError(t, err) 44 require.NoError(t, err)
45 assert.Equal(t, int64(MinAllocatedHostPort), first.HostPort) 45 assert.Equal(t, int64(MinAllocatedHostPort), first.HostPort)
46 assert.Equal(t, int64(MinAllocatedHostPort+1), second.HostPort) 46 assert.Equal(t, int64(MinAllocatedHostPort+1), second.HostPort)
47 47
48 // A freed port is the lowest gap, so it is handed out again. 48 // A freed port is the lowest gap, so it is handed out again.
49 require.NoError(t, s.DeleteExposure(first.ID)) 49 require.NoError(t, s.DeleteExposure(first.ID))
50 third, err := s.CreateExposure(vm.ID, 8082, 0) 50 third, err := s.CreateExposure(vm.ID, 8082, 0, "tcp")
51 require.NoError(t, err) 51 require.NoError(t, err)
52 assert.Equal(t, int64(MinAllocatedHostPort), third.HostPort) 52 assert.Equal(t, int64(MinAllocatedHostPort), third.HostPort)
53 } 53 }
@@ -57,7 +57,7 @@ func TestCreateExposureHonoursARequestedPort(t *testing.T) {
57 h := enrollHost(t, s) 57 h := enrollHost(t, s)
58 vm := makeExposureVM(t, s, h, "web-1") 58 vm := makeExposureVM(t, s, h, "web-1")
59 59
60 e, err := s.CreateExposure(vm.ID, 8080, 8443) 60 e, err := s.CreateExposure(vm.ID, 8080, 8443, "tcp")
61 require.NoError(t, err) 61 require.NoError(t, err)
62 assert.Equal(t, int64(8443), e.HostPort) 62 assert.Equal(t, int64(8443), e.HostPort)
63 } 63 }
@@ -68,13 +68,29 @@ func TestCreateExposureRefusesAPortAnotherExposureHolds(t *testing.T) {
68 vm := makeExposureVM(t, s, h, "web-1") 68 vm := makeExposureVM(t, s, h, "web-1")
69 other := makeExposureVM(t, s, h, "web-2") 69 other := makeExposureVM(t, s, h, "web-2")
70 70
71 _, err := s.CreateExposure(vm.ID, 8080, 8443) 71 _, err := s.CreateExposure(vm.ID, 8080, 8443, "tcp")
72 require.NoError(t, err) 72 require.NoError(t, err)
73 73
74 _, err = s.CreateExposure(other.ID, 9090, 8443) 74 _, err = s.CreateExposure(other.ID, 9090, 8443, "tcp")
75 assert.ErrorIs(t, err, ErrHostPortTaken, "the unique index IS the collision check") 75 assert.ErrorIs(t, err, ErrHostPortTaken, "the unique index IS the collision check")
76 } 76 }
77 77
78 func TestCreateExposureAllowsTheSamePortInTheOtherProtocol(t *testing.T) {
79 s := newStore(t)
80 h := enrollHost(t, s)
81 vm := makeExposureVM(t, s, h, "web-1")
82
83 _, err := s.CreateExposure(vm.ID, 8080, 8443, "tcp")
84 require.NoError(t, err)
85
86 udp, err := s.CreateExposure(vm.ID, 8080, 8443, "udp")
87 require.NoError(t, err, "a host port is claimed per protocol")
88 assert.Equal(t, "udp", udp.Protocol)
89
90 _, err = s.CreateExposure(vm.ID, 9090, 8443, "udp")
91 assert.ErrorIs(t, err, ErrHostPortTaken, "the second UDP claim on that port is a collision")
92 }
93
78 func TestCreateExposureAllowsTheSamePortOnAnotherHost(t *testing.T) { 94 func TestCreateExposureAllowsTheSamePortOnAnotherHost(t *testing.T) {
79 s := newStore(t) 95 s := newStore(t)
80 h1 := enrollHost(t, s) 96 h1 := enrollHost(t, s)
@@ -86,9 +102,9 @@ func TestCreateExposureAllowsTheSamePortOnAnotherHost(t *testing.T) {
86 vm1 := makeExposureVM(t, s, h1, "web-1") 102 vm1 := makeExposureVM(t, s, h1, "web-1")
87 vm2 := makeExposureVM(t, s, h2, "web-2") 103 vm2 := makeExposureVM(t, s, h2, "web-2")
88 104
89 _, err = s.CreateExposure(vm1.ID, 8080, 8443) 105 _, err = s.CreateExposure(vm1.ID, 8080, 8443, "tcp")
90 require.NoError(t, err) 106 require.NoError(t, err)
91 _, err = s.CreateExposure(vm2.ID, 8080, 8443) 107 _, err = s.CreateExposure(vm2.ID, 8080, 8443, "tcp")
92 assert.NoError(t, err, "the port is unique per host, not per fleet") 108 assert.NoError(t, err, "the port is unique per host, not per fleet")
93 } 109 }
94 110
@@ -97,11 +113,11 @@ func TestCreateExposureRefusesAnUnknownOrTombstonedVM(t *testing.T) {
97 h := enrollHost(t, s) 113 h := enrollHost(t, s)
98 vm := makeExposureVM(t, s, h, "web-1") 114 vm := makeExposureVM(t, s, h, "web-1")
99 115
100 _, err := s.CreateExposure("no-such-vm", 8080, 0) 116 _, err := s.CreateExposure("no-such-vm", 8080, 0, "tcp")
101 assert.ErrorIs(t, err, ErrExposureVMNotFound) 117 assert.ErrorIs(t, err, ErrExposureVMNotFound)
102 118
103 require.NoError(t, s.TombstoneVM(vm.ID)) 119 require.NoError(t, s.TombstoneVM(vm.ID))
104 _, err = s.CreateExposure(vm.ID, 8080, 0) 120 _, err = s.CreateExposure(vm.ID, 8080, 0, "tcp")
105 assert.ErrorIs(t, err, ErrExposureVMNotFound, "a VM being torn down takes no new exposures") 121 assert.ErrorIs(t, err, ErrExposureVMNotFound, "a VM being torn down takes no new exposures")
106 } 122 }
107 123
@@ -120,7 +136,7 @@ func TestCreateExposureReportsRangeExhaustion(t *testing.T) {
120 require.NoError(t, err) 136 require.NoError(t, err)
121 } 137 }
122 138
123 _, err := s.CreateExposure(vm.ID, 8080, 0) 139 _, err := s.CreateExposure(vm.ID, 8080, 0, "tcp")
124 assert.ErrorIs(t, err, ErrNoFreeHostPort) 140 assert.ErrorIs(t, err, ErrNoFreeHostPort)
125 } 141 }
126 142
@@ -131,7 +147,7 @@ func TestCreateExposureBumpsTheEpoch(t *testing.T) {
131 147
132 before, err := s.Epoch() 148 before, err := s.Epoch()
133 require.NoError(t, err) 149 require.NoError(t, err)
134 _, err = s.CreateExposure(vm.ID, 8080, 0) 150 _, err = s.CreateExposure(vm.ID, 8080, 0, "tcp")
135 require.NoError(t, err) 151 require.NoError(t, err)
136 after, err := s.Epoch() 152 after, err := s.Epoch()
137 require.NoError(t, err) 153 require.NoError(t, err)
@@ -143,26 +159,41 @@ func TestCreateExposureCountsANamedPortInTheReservedRange(t *testing.T) {
143 h := enrollHost(t, s) 159 h := enrollHost(t, s)
144 vm := makeExposureVM(t, s, h, "web-1") 160 vm := makeExposureVM(t, s, h, "web-1")
145 161
146 named, err := s.CreateExposure(vm.ID, 8080, MinAllocatedHostPort) 162 named, err := s.CreateExposure(vm.ID, 8080, MinAllocatedHostPort, "tcp")
147 require.NoError(t, err) 163 require.NoError(t, err)
148 assert.Equal(t, int64(MinAllocatedHostPort), named.HostPort) 164 assert.Equal(t, int64(MinAllocatedHostPort), named.HostPort)
149 165
150 auto, err := s.CreateExposure(vm.ID, 8081, 0) 166 auto, err := s.CreateExposure(vm.ID, 8081, 0, "tcp")
151 require.NoError(t, err) 167 require.NoError(t, err)
152 assert.Equal(t, int64(MinAllocatedHostPort+1), auto.HostPort, "a named port inside the reserved range is held against auto-allocation") 168 assert.Equal(t, int64(MinAllocatedHostPort+1), auto.HostPort, "a named port inside the reserved range is held against auto-allocation")
153 } 169 }
154 170
171 func TestAllocationSkipsAPortHeldByEitherProtocol(t *testing.T) {
172 s := newStore(t)
173 h := enrollHost(t, s)
174 vm := makeExposureVM(t, s, h, "web-1")
175
176 _, err := s.CreateExposure(vm.ID, 8080, MinAllocatedHostPort, "udp")
177 require.NoError(t, err)
178
179 // The allocator hands out a number a caller can name without a protocol
180 // beside it, so the UDP claim takes the port out of the running for TCP too.
181 auto, err := s.CreateExposure(vm.ID, 8081, 0, "tcp")
182 require.NoError(t, err)
183 assert.Equal(t, int64(MinAllocatedHostPort+1), auto.HostPort)
184 }
185
155 func TestListExposuresForVMIsPortOrdered(t *testing.T) { 186 func TestListExposuresForVMIsPortOrdered(t *testing.T) {
156 s := newStore(t) 187 s := newStore(t)
157 h := enrollHost(t, s) 188 h := enrollHost(t, s)
158 vm := makeExposureVM(t, s, h, "web-1") 189 vm := makeExposureVM(t, s, h, "web-1")
159 other := makeExposureVM(t, s, h, "web-2") 190 other := makeExposureVM(t, s, h, "web-2")
160 191
161 _, err := s.CreateExposure(vm.ID, 8081, 31000) 192 _, err := s.CreateExposure(vm.ID, 8081, 31000, "tcp")
162 require.NoError(t, err) 193 require.NoError(t, err)
163 _, err = s.CreateExposure(vm.ID, 8080, 30500) 194 _, err = s.CreateExposure(vm.ID, 8080, 30500, "tcp")
164 require.NoError(t, err) 195 require.NoError(t, err)
165 _, err = s.CreateExposure(other.ID, 9090, 30001) 196 _, err = s.CreateExposure(other.ID, 9090, 30001, "tcp")
166 require.NoError(t, err) 197 require.NoError(t, err)
167 198
168 got, err := s.ListExposuresForVM(vm.ID) 199 got, err := s.ListExposuresForVM(vm.ID)
@@ -178,9 +209,9 @@ func TestListExposuresForHostSkipsTombstonedVMs(t *testing.T) {
178 live := makeExposureVM(t, s, h, "web-1") 209 live := makeExposureVM(t, s, h, "web-1")
179 dying := makeExposureVM(t, s, h, "web-2") 210 dying := makeExposureVM(t, s, h, "web-2")
180 211
181 _, err := s.CreateExposure(live.ID, 8080, 30001) 212 _, err := s.CreateExposure(live.ID, 8080, 30001, "tcp")
182 require.NoError(t, err) 213 require.NoError(t, err)
183 _, err = s.CreateExposure(dying.ID, 8080, 30002) 214 _, err = s.CreateExposure(dying.ID, 8080, 30002, "tcp")
184 require.NoError(t, err) 215 require.NoError(t, err)
185 216
186 require.NoError(t, s.TombstoneVM(dying.ID)) 217 require.NoError(t, s.TombstoneVM(dying.ID))
@@ -196,7 +227,7 @@ func TestGetExposureAndDelete(t *testing.T) {
196 h := enrollHost(t, s) 227 h := enrollHost(t, s)
197 vm := makeExposureVM(t, s, h, "web-1") 228 vm := makeExposureVM(t, s, h, "web-1")
198 229
199 e, err := s.CreateExposure(vm.ID, 8080, 0) 230 e, err := s.CreateExposure(vm.ID, 8080, 0, "tcp")
200 require.NoError(t, err) 231 require.NoError(t, err)
201 232
202 got, err := s.GetExposure(e.ID) 233 got, err := s.GetExposure(e.ID)
@@ -216,7 +247,7 @@ func TestDeleteExposureBumpsTheEpoch(t *testing.T) {
216 s := newStore(t) 247 s := newStore(t)
217 h := enrollHost(t, s) 248 h := enrollHost(t, s)
218 vm := makeExposureVM(t, s, h, "web-1") 249 vm := makeExposureVM(t, s, h, "web-1")
219 e, err := s.CreateExposure(vm.ID, 8080, 0) 250 e, err := s.CreateExposure(vm.ID, 8080, 0, "tcp")
220 require.NoError(t, err) 251 require.NoError(t, err)
221 252
222 before, err := s.Epoch() 253 before, err := s.Epoch()
@@ -231,7 +262,7 @@ func TestReapingAVMDestroysItsExposures(t *testing.T) {
231 s := newStore(t) 262 s := newStore(t)
232 h := enrollHost(t, s) 263 h := enrollHost(t, s)
233 vm := makeExposureVM(t, s, h, "web-1") 264 vm := makeExposureVM(t, s, h, "web-1")
234 e, err := s.CreateExposure(vm.ID, 8080, 0) 265 e, err := s.CreateExposure(vm.ID, 8080, 0, "tcp")
235 require.NoError(t, err) 266 require.NoError(t, err)
236 267
237 require.NoError(t, s.TombstoneVM(vm.ID)) 268 require.NoError(t, s.TombstoneVM(vm.ID))
@@ -245,7 +276,7 @@ func TestForceRemovingAHostDestroysItsExposures(t *testing.T) {
245 s := newStore(t) 276 s := newStore(t)
246 h := enrollHost(t, s) 277 h := enrollHost(t, s)
247 vm := makeExposureVM(t, s, h, "web-1") 278 vm := makeExposureVM(t, s, h, "web-1")
248 e, err := s.CreateExposure(vm.ID, 8080, 0) 279 e, err := s.CreateExposure(vm.ID, 8080, 0, "tcp")
249 require.NoError(t, err) 280 require.NoError(t, err)
250 281
251 purged, err := s.ForceRemoveHost(h.ID) 282 purged, err := s.ForceRemoveHost(h.ID)
internal/server/store/store.go
Old New
@@ -214,8 +214,9 @@ CREATE UNIQUE INDEX IF NOT EXISTS idx_tenant_user_cas_pubkey ON tenant_user_cas(
214 -- whoever inserts first, and the second insert fails at the storage layer 214 -- whoever inserts first, and the second insert fails at the storage layer
215 -- rather than after a read-then-write race. 215 -- rather than after a read-then-write race.
216 -- 216 --
217 -- protocol is 'tcp' and scope is 'lan', both stored so 'udp' and 'public' are 217 -- protocol is 'tcp' or 'udp' — two different ports on the same number, which
218 -- additive values later rather than a migration. 218 -- is why it is part of the unique index. scope is 'lan', stored so 'public' is
219 -- an additive value later rather than a migration.
219 -- 220 --
220 -- ON DELETE CASCADE against vms is what keeps the promise that no listener 221 -- ON DELETE CASCADE against vms is what keeps the promise that no listener
221 -- outlives the thing it pointed at: hard-deleting a reaped VM takes its 222 -- outlives the thing it pointed at: hard-deleting a reaped VM takes its
@@ -231,7 +232,7 @@ CREATE TABLE IF NOT EXISTS exposures (
231 scope TEXT NOT NULL DEFAULT 'lan', 232 scope TEXT NOT NULL DEFAULT 'lan',
232 created_at DATETIME NOT NULL 233 created_at DATETIME NOT NULL
233 ); 234 );
234 CREATE UNIQUE INDEX IF NOT EXISTS exposures_host_port ON exposures(host_id, host_port); 235 CREATE UNIQUE INDEX IF NOT EXISTS exposures_host_port_proto ON exposures(host_id, host_port, protocol);
235 236
236 -- Console sessions. Server-side so revocation works and restarts keep 237 -- Console sessions. Server-side so revocation works and restarts keep
237 -- users signed in. id is 256-bit random hex; expiry enforced on read. 238 -- users signed in. id is 256-bit random hex; expiry enforced on read.
@@ -344,6 +345,15 @@ func Open(path, cidrPool string) (*Store, error) {
344 return nil, err 345 return nil, err
345 } 346 }
346 347
348 // A host port is claimed per protocol, so TCP 30000 and UDP 30000 are two
349 // grants rather than a collision. The index that spanned only (host, port)
350 // would refuse the second one, so it goes — exposures_host_port_proto above
351 // has already replaced it by the time this runs.
352 if err := dropIndex(db, "exposures_host_port"); err != nil {
353 db.Close()
354 return nil, err
355 }
356
347 // One identity binds at most one tenant (per issuer). Partial index so 357 // One identity binds at most one tenant (per issuer). Partial index so
348 // unbound rows (empty issuer+subject) don't collide. 358 // unbound rows (empty issuer+subject) don't collide.
349 if _, err := db.Exec(`CREATE UNIQUE INDEX IF NOT EXISTS tenants_identity 359 if _, err := db.Exec(`CREATE UNIQUE INDEX IF NOT EXISTS tenants_identity
internal/server/store/store_test.go
Old New
@@ -165,7 +165,7 @@ func TestOpenDropsTheEscrowedHostKeyColumn(t *testing.T) {
165 require.NoError(t, err) 165 require.NoError(t, err)
166 // An exposure, because vms is referenced ON DELETE CASCADE: a migration that 166 // An exposure, because vms is referenced ON DELETE CASCADE: a migration that
167 // rebuilt the table by dropping it would take this row with it. 167 // rebuilt the table by dropping it would take this row with it.
168 _, err = s.CreateExposure("vm1", 22, 30001) 168 _, err = s.CreateExposure("vm1", 22, 30001, "tcp")
169 require.NoError(t, err) 169 require.NoError(t, err)
170 require.NoError(t, s.Close()) 170 require.NoError(t, s.Close())
171 171
@@ -198,6 +198,43 @@ func TestOpenDropsTheEscrowedHostKeyColumn(t *testing.T) {
198 assert.Equal(t, "cert-line", vm.SSHHostCert) 198 assert.Equal(t, "cert-line", vm.SSHHostCert)
199 } 199 }
200 200
201 func TestOpenReplacesTheProtocolBlindHostPortIndex(t *testing.T) {
202 path := t.TempDir() + "/eitri.db"
203 s, err := Open(path, "10.77.0.0/16")
204 require.NoError(t, err)
205 _, err = s.CreateTenantForIdentity("https://test-issuer", "test-subject", testTenant+"@test.local")
206 require.NoError(t, err)
207 h := enrollHost(t, s)
208 require.NoError(t, s.CreateVM(VM{
209 ID: "vm1", HostID: h.ID, Name: "legacy", ImageURL: "u", ImageSHA256: "abc",
210 VCPUs: 1, MemMB: 512, DiskGB: 5, PowerState: "running",
211 }))
212
213 // Put the database back into the shape an earlier release left it: one host
214 // port, one exposure, whatever the protocol.
215 _, err = s.db.Exec(`DROP INDEX exposures_host_port_proto`)
216 require.NoError(t, err)
217 _, err = s.db.Exec(`CREATE UNIQUE INDEX exposures_host_port ON exposures(host_id, host_port)`)
218 require.NoError(t, err)
219 _, err = s.CreateExposure("vm1", 22, 30001, "tcp")
220 require.NoError(t, err)
221 require.NoError(t, s.Close())
222
223 up, err := Open(path, "10.77.0.0/16")
224 require.NoError(t, err)
225 t.Cleanup(func() { up.Close() })
226
227 var n int
228 require.NoError(t, up.db.QueryRow(
229 `SELECT count(*) FROM sqlite_master WHERE type='index' AND name='exposures_host_port'`).Scan(&n))
230 assert.Equal(t, 0, n, "the index that spanned only (host, port) must be gone")
231
232 _, err = up.CreateExposure("vm1", 53, 30001, "udp")
233 assert.NoError(t, err, "the same host port in the other protocol is a second grant")
234 _, err = up.CreateExposure("vm1", 54, 30001, "udp")
235 assert.ErrorIs(t, err, ErrHostPortTaken, "the new index still holds the pair unique")
236 }
237
201 func TestSSHCertRevocation(t *testing.T) { 238 func TestSSHCertRevocation(t *testing.T) {
202 s := newStore(t) 239 s := newStore(t)
203 240
internal/server/syncsvc/exposures_test.go
Old New
@@ -24,7 +24,7 @@ func TestSnapshotCarriesExposures(t *testing.T) {
24 f := setup(t) 24 f := setup(t)
25 exposureVM(t, f, "vm1", "web-1") 25 exposureVM(t, f, "vm1", "web-1")
26 26
27 e, err := f.st.CreateExposure("vm1", 8080, 30080) 27 e, err := f.st.CreateExposure("vm1", 8080, 30080, "tcp")
28 require.NoError(t, err) 28 require.NoError(t, err)
29 29
30 snap, err := f.svc.buildSnapshot(f.host.ID) 30 snap, err := f.svc.buildSnapshot(f.host.ID)
@@ -44,7 +44,7 @@ func TestSnapshotCarriesExposures(t *testing.T) {
44 func TestSnapshotDropsExposuresOfATombstonedVM(t *testing.T) { 44 func TestSnapshotDropsExposuresOfATombstonedVM(t *testing.T) {
45 f := setup(t) 45 f := setup(t)
46 exposureVM(t, f, "vm1", "web-1") 46 exposureVM(t, f, "vm1", "web-1")
47 _, err := f.st.CreateExposure("vm1", 8080, 30080) 47 _, err := f.st.CreateExposure("vm1", 8080, 30080, "tcp")
48 require.NoError(t, err) 48 require.NoError(t, err)
49 require.NoError(t, f.st.TombstoneVM("vm1")) 49 require.NoError(t, f.st.TombstoneVM("vm1"))
50 50
internal/server/syncsvc/syncsvc.go
Old New
@@ -324,8 +324,9 @@ func (s *Service) buildSnapshot(hostID string) (*pb.DesiredStateSnapshot, error)
324 return nil, fmt.Errorf("list host exposures: %w", err) 324 return nil, fmt.Errorf("list host exposures: %w", err)
325 } 325 }
326 for _, e := range exps { 326 for _, e := range exps {
327 // In-range by construction: the API validates ports at create 327 // In-range by construction: the API validates ports and closes the
328 // (validateExposurePorts), and the store accepts rows only from the API. 328 // protocol at the two the agent can bind (validateExposure), and the
329 // store accepts rows only from the API.
329 snap.Exposures = append(snap.Exposures, &pb.ExposureDesired{ 330 snap.Exposures = append(snap.Exposures, &pb.ExposureDesired{
330 Id: e.ID, VmId: e.VMID, 331 Id: e.ID, VmId: e.VMID,
331 GuestPort: uint32(e.GuestPort), HostPort: uint32(e.HostPort), 332 GuestPort: uint32(e.GuestPort), HostPort: uint32(e.HostPort),
internal/smoke/exposure.go
Old New
@@ -21,11 +21,15 @@ const sshBannerPrefix = "SSH-2.0"
21 // bannerFunc reads the first bytes a TCP peer sends after accepting. 21 // bannerFunc reads the first bytes a TCP peer sends after accepting.
22 type bannerFunc func(ctx context.Context, addr string) (string, error) 22 type bannerFunc func(ctx context.Context, addr string) (string, error)
23 23
24 // echoFunc sends one datagram to a published UDP port and returns what came
25 // back on the same socket.
26 type echoFunc func(ctx context.Context, addr, payload string) (string, error)
27
24 // proveExposure publishes the smoke VM's ssh port on its host, dials the 28 // proveExposure publishes the smoke VM's ssh port on its host, dials the
25 // address the grant names, and expects an SSH banner. The exposure is revoked 29 // address the grant names, and expects an SSH banner. The exposure is revoked
26 // before the leg returns, whatever the outcome. 30 // before the leg returns, whatever the outcome.
27 func proveExposure(ctx context.Context, c vmAPI, vmID string, now func() time.Time, sleep func(time.Duration), dial bannerFunc) error { 31 func proveExposure(ctx context.Context, c vmAPI, vmID string, now func() time.Time, sleep func(time.Duration), dial bannerFunc) error {
28 exp, err := c.CreateExposure(ctx, vmID, 22, 0) 32 exp, err := c.CreateExposure(ctx, vmID, 22, 0, "tcp")
29 if err != nil { 33 if err != nil {
30 return fmt.Errorf("create exposure: %w", err) 34 return fmt.Errorf("create exposure: %w", err)
31 } 35 }
@@ -73,6 +77,32 @@ func truncateBanner(s string) string {
73 return s 77 return s
74 } 78 }
75 79
80 // readEcho is the real echoFunc: send payload to a published UDP port and read
81 // the answer off a connected socket, so a reply from anywhere but the port the
82 // grant names is not the fleet answering and does not count. The deadline
83 // bounds a datagram that goes nowhere — every hop here may drop one silently,
84 // which is what the caller's retry loop is for.
85 func readEcho(ctx context.Context, addr, payload string) (string, error) {
86 var d net.Dialer
87 conn, err := d.DialContext(ctx, "udp", addr)
88 if err != nil {
89 return "", err
90 }
91 defer conn.Close()
92 if err := conn.SetDeadline(time.Now().Add(5 * time.Second)); err != nil {
93 return "", err
94 }
95 if _, err := conn.Write([]byte(payload)); err != nil {
96 return "", err
97 }
98 buf := make([]byte, 512)
99 n, err := conn.Read(buf)
100 if err != nil {
101 return "", err
102 }
103 return string(buf[:n]), nil
104 }
105
76 // readBanner is the real bannerFunc: dial addr and read the peer's first line, 106 // readBanner is the real bannerFunc: dial addr and read the peer's first line,
77 // bounded by a deadline so a listener that binds but never answers fails the 107 // bounded by a deadline so a listener that binds but never answers fails the
78 // leg instead of hanging it. It reads until a newline (or 256 bytes) rather 108 // leg instead of hanging it. It reads until a newline (or 256 bytes) rather
internal/smoke/exposure_test.go
Old New
@@ -19,7 +19,7 @@ func TestProveExposureReadsTheBannerBack(t *testing.T) {
19 clock := &fakeClock{} 19 clock := &fakeClock{}
20 deleted := "" 20 deleted := ""
21 api := &testAPI{ 21 api := &testAPI{
22 createExposureFunc: func(ctx context.Context, vmID string, guest, host int64) (client.Exposure, error) { 22 createExposureFunc: func(ctx context.Context, vmID string, guest, host int64, proto string) (client.Exposure, error) {
23 if vmID != "vm-1" { 23 if vmID != "vm-1" {
24 t.Errorf("CreateExposure vmID = %q, want vm-1", vmID) 24 t.Errorf("CreateExposure vmID = %q, want vm-1", vmID)
25 } 25 }
@@ -56,7 +56,7 @@ func TestProveExposureReadsTheBannerBack(t *testing.T) {
56 func TestProveExposureDialsAnIPv6Uplink(t *testing.T) { 56 func TestProveExposureDialsAnIPv6Uplink(t *testing.T) {
57 clock := &fakeClock{} 57 clock := &fakeClock{}
58 api := &testAPI{ 58 api := &testAPI{
59 createExposureFunc: func(context.Context, string, int64, int64) (client.Exposure, error) { 59 createExposureFunc: func(context.Context, string, int64, int64, string) (client.Exposure, error) {
60 return client.Exposure{ID: "x-1", HostPort: 30080, HostAddr: "2001:db8::1"}, nil 60 return client.Exposure{ID: "x-1", HostPort: 30080, HostAddr: "2001:db8::1"}, nil
61 }, 61 },
62 deleteExposureFunc: func(context.Context, string) error { return nil }, 62 deleteExposureFunc: func(context.Context, string) error { return nil },
@@ -78,7 +78,7 @@ func TestProveExposureDialsAnIPv6Uplink(t *testing.T) {
78 func TestProveExposureRetriesUntilTheListenerConverges(t *testing.T) { 78 func TestProveExposureRetriesUntilTheListenerConverges(t *testing.T) {
79 clock := &fakeClock{} 79 clock := &fakeClock{}
80 api := &testAPI{ 80 api := &testAPI{
81 createExposureFunc: func(context.Context, string, int64, int64) (client.Exposure, error) { 81 createExposureFunc: func(context.Context, string, int64, int64, string) (client.Exposure, error) {
82 return client.Exposure{ID: "x-1", HostPort: 30080, HostAddr: "192.168.0.190"}, nil 82 return client.Exposure{ID: "x-1", HostPort: 30080, HostAddr: "192.168.0.190"}, nil
83 }, 83 },
84 deleteExposureFunc: func(context.Context, string) error { return nil }, 84 deleteExposureFunc: func(context.Context, string) error { return nil },
@@ -103,7 +103,7 @@ func TestProveExposureRetriesUntilTheListenerConverges(t *testing.T) {
103 func TestProveExposureFailsOnTheWrongBanner(t *testing.T) { 103 func TestProveExposureFailsOnTheWrongBanner(t *testing.T) {
104 clock := &fakeClock{} 104 clock := &fakeClock{}
105 api := &testAPI{ 105 api := &testAPI{
106 createExposureFunc: func(context.Context, string, int64, int64) (client.Exposure, error) { 106 createExposureFunc: func(context.Context, string, int64, int64, string) (client.Exposure, error) {
107 return client.Exposure{ID: "x-1", HostPort: 30080, HostAddr: "192.168.0.190"}, nil 107 return client.Exposure{ID: "x-1", HostPort: 30080, HostAddr: "192.168.0.190"}, nil
108 }, 108 },
109 deleteExposureFunc: func(context.Context, string) error { return nil }, 109 deleteExposureFunc: func(context.Context, string) error { return nil },
@@ -126,7 +126,7 @@ func TestProveExposureFailsWhenTheGrantNamesNoAddress(t *testing.T) {
126 clock := &fakeClock{} 126 clock := &fakeClock{}
127 revoked := "" 127 revoked := ""
128 api := &testAPI{ 128 api := &testAPI{
129 createExposureFunc: func(context.Context, string, int64, int64) (client.Exposure, error) { 129 createExposureFunc: func(context.Context, string, int64, int64, string) (client.Exposure, error) {
130 return client.Exposure{ID: "x-1", HostPort: 30080}, nil 130 return client.Exposure{ID: "x-1", HostPort: 30080}, nil
131 }, 131 },
132 deleteExposureFunc: func(_ context.Context, id string) error { revoked = id; return nil }, 132 deleteExposureFunc: func(_ context.Context, id string) error { revoked = id; return nil },
internal/smoke/mcp.go
Old New
@@ -165,14 +165,15 @@ func proveRemoteMCPToolset(ctx context.Context, serverURL, pat string) error {
165 165
166 // proveMCP drives one full cycle through the remote MCP endpoint and nothing 166 // proveMCP drives one full cycle through the remote MCP endpoint and nothing
167 // else: delegate access to eitri, create a VM, run a command in it, publish a 167 // else: delegate access to eitri, create a VM, run a command in it, publish a
168 // port, read the guest's banner back through that port, and destroy it. 168 // TCP port and read the guest's banner back through it, publish a UDP port and
169 // read a datagram back through that, and destroy the VM.
169 // 170 //
170 // The CA is registered BEFORE the VM is created, because a guest bakes its CA 171 // The CA is registered BEFORE the VM is created, because a guest bakes its CA
171 // set at create and a CA registered afterwards is one it will never trust. The 172 // set at create and a CA registered afterwards is one it will never trust. The
172 // DELEGATION itself may happen on either side of the create — that is the whole 173 // DELEGATION itself may happen on either side of the create — that is the whole
173 // improvement over holding a signing key, and it is worth stating plainly here 174 // improvement over holding a signing key, and it is worth stating plainly here
174 // so nobody reintroduces an ordering constraint that no longer exists. 175 // so nobody reintroduces an ordering constraint that no longer exists.
175 func proveMCP(ctx context.Context, c mcpTools, vmName string, d delegator, now func() time.Time, sleep func(time.Duration), dial bannerFunc) error { 176 func proveMCP(ctx context.Context, c mcpTools, vmName string, d delegator, now func() time.Time, sleep func(time.Duration), dial bannerFunc, echo echoFunc) error {
176 if err := proveToolList(ctx, c); err != nil { 177 if err := proveToolList(ctx, c); err != nil {
177 return err 178 return err
178 } 179 }
@@ -282,6 +283,88 @@ func proveMCP(ctx context.Context, c mcpTools, vmName string, d delegator, now f
282 if _, err := c.Call(ctx, "vm_unexpose", map[string]any{"vm": vmName, "guest_port": 22}); err != nil { 283 if _, err := c.Call(ctx, "vm_unexpose", map[string]any{"vm": vmName, "guest_port": 22}); err != nil {
283 return fmt.Errorf("vm_unexpose over MCP: %w", err) 284 return fmt.Errorf("vm_unexpose over MCP: %w", err)
284 } 285 }
286
287 return proveUDPExposure(ctx, c, vmName, now, sleep, echo)
288 }
289
290 // udpEchoPort is where the smoke's own echo listens inside the guest. A TCP
291 // exposure can be proven against the sshd that is already there; UDP has no
292 // such standing service, so the leg brings its own — which is the honest test
293 // anyway, since it proves a port nothing else on the host or the guest is
294 // touching.
295 const udpEchoPort = 17007
296
297 // udpEchoCommand starts that echo and returns. python3 is what a guest that
298 // booted through cloud-init necessarily has, so this needs no package install
299 // and no image of its own; nohup and the redirects are what let the exec's
300 // session close while the echo keeps running.
301 func udpEchoCommand() string {
302 return fmt.Sprintf(`nohup python3 -c '
303 import socket
304 s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
305 s.bind(("0.0.0.0", %d))
306 while True:
307 datagram, sender = s.recvfrom(2048)
308 s.sendto(datagram, sender)
309 ' >/dev/null 2>&1 &`, udpEchoPort)
310 }
311
312 // proveUDPExposure publishes a UDP port of the MCP leg's own VM and proves a
313 // datagram makes the whole round trip: through the host's packet proxy, into
314 // the guest, and back out to the caller through the same published port. The
315 // grant is revoked before the leg returns; the echo dies with the VM.
316 func proveUDPExposure(ctx context.Context, c mcpTools, vmName string, now func() time.Time, sleep func(time.Duration), echo echoFunc) error {
317 if _, err := c.Call(ctx, "vm_exec", map[string]any{"vm": vmName, "command": udpEchoCommand()}); err != nil {
318 return fmt.Errorf("start the guest UDP echo over MCP: %w", err)
319 }
320
321 exposed, err := c.Call(ctx, "vm_expose", map[string]any{
322 "vm": vmName, "guest_port": udpEchoPort, "protocol": "udp",
323 })
324 if err != nil {
325 return fmt.Errorf("vm_expose udp over MCP: %w", err)
326 }
327 if got, _ := exposed["exposure"].(map[string]any); got != nil {
328 if proto, _ := got["protocol"].(string); proto != "udp" {
329 return fmt.Errorf("FAIL: vm_expose published %q, want a udp exposure", proto)
330 }
331 }
332 address, err := exposureAddress(exposed)
333 if err != nil {
334 return err
335 }
336
337 nonce, err := randNonce()
338 if err != nil {
339 return err
340 }
341 var lastErr error
342 err = pollLoop(ctx, now, sleep, 60*time.Second, 3*time.Second, func() (bool, error) {
343 // Everything transient lives in this retry: the host binds the socket on
344 // its next converge, the guest's echo takes a moment to come up, and a
345 // datagram is allowed to go missing on any hop between them.
346 back, derr := echo(ctx, address, nonce)
347 if derr != nil {
348 lastErr = derr
349 return false, nil
350 }
351 if back != nonce {
352 return false, fmt.Errorf("FAIL: published UDP port %s echoed %q, want %q", address, truncateBanner(back), nonce)
353 }
354 return true, nil
355 })
356 if errors.Is(err, errPollTimeout) {
357 return fmt.Errorf("FAIL: no echo from the MCP-published UDP port %s within 60s: %v", address, lastErr)
358 }
359 if err != nil {
360 return err
361 }
362
363 if _, err := c.Call(ctx, "vm_unexpose", map[string]any{
364 "vm": vmName, "guest_port": udpEchoPort, "protocol": "udp",
365 }); err != nil {
366 return fmt.Errorf("vm_unexpose udp over MCP: %w", err)
367 }
285 return nil 368 return nil
286 } 369 }
287 370
internal/smoke/mcp_test.go
Old New
@@ -7,6 +7,7 @@ import (
7 "errors" 7 "errors"
8 "net/http" 8 "net/http"
9 "net/http/httptest" 9 "net/http/httptest"
10 "strconv"
10 "strings" 11 "strings"
11 "testing" 12 "testing"
12 "time" 13 "time"
@@ -28,8 +29,10 @@ type fakeMCP struct {
28 // hold it: a certificate signed by anything else is refused. 29 // hold it: a certificate signed by anything else is refused.
29 trusted []ssh.PublicKey 30 trusted []ssh.PublicKey
30 pubLine string 31 pubLine string
31 // uploaded records the CA lines ca_upload received. 32 // uploaded records the CA lines ca_upload received, and exposedAs the
32 uploaded []string 33 // protocol each vm_expose asked for.
34 uploaded []string
35 exposedAs []string
33 // delegated is what tenant_info reports; set once a certificate lands. 36 // delegated is what tenant_info reports; set once a certificate lands.
34 delegated bool 37 delegated bool
35 38
@@ -53,7 +56,6 @@ func newFakeMCP(t *testing.T, trusted ...ssh.Signer) *fakeMCP {
53 results: map[string]map[string]any{ 56 results: map[string]map[string]any{
54 "vm_create": {"id": "v-1", "name": "smoke-mcp"}, 57 "vm_create": {"id": "v-1", "name": "smoke-mcp"},
55 "vm_exec": {"stdout": "", "exit_code": 0.0}, 58 "vm_exec": {"stdout": "", "exit_code": 0.0},
56 "vm_expose": {"exposure": map[string]any{"id": "x-1", "address": "10.0.0.1:30001"}},
57 "vm_unexpose": {"id": "x-1"}, 59 "vm_unexpose": {"id": "x-1"},
58 "vm_destroy": {"id": "v-1"}, 60 "vm_destroy": {"id": "v-1"},
59 }, 61 },
@@ -93,6 +95,22 @@ func (f *fakeMCP) Call(_ context.Context, name string, args map[string]any) (map
93 if name == "delegate_complete" { 95 if name == "delegate_complete" {
94 return f.complete(args) 96 return f.complete(args)
95 } 97 }
98 if name == "vm_expose" {
99 // The control plane answers with the exposure it made, protocol
100 // included, and an unnamed protocol is tcp. A scripted result stands in
101 // for a plane answering something else.
102 proto, _ := args["protocol"].(string)
103 if proto == "" {
104 proto = "tcp"
105 }
106 f.exposedAs = append(f.exposedAs, proto)
107 if scripted, ok := f.results[name]; ok {
108 return scripted, nil
109 }
110 return map[string]any{"exposure": map[string]any{
111 "id": "x-1", "address": "10.0.0.1:30001", "protocol": proto,
112 }}, nil
113 }
96 out := map[string]any{} 114 out := map[string]any{}
97 for k, v := range f.results[name] { 115 for k, v := range f.results[name] {
98 out[k] = v 116 out[k] = v
@@ -144,6 +162,10 @@ func (e echoingMCP) Call(ctx context.Context, name string, args map[string]any)
144 162
145 func okBannerDial(context.Context, string) (string, error) { return "SSH-2.0-OpenSSH_9.6\r\n", nil } 163 func okBannerDial(context.Context, string) (string, error) { return "SSH-2.0-OpenSSH_9.6\r\n", nil }
146 164
165 // okEcho is a published UDP port with the guest's echo behind it: whatever went
166 // in comes back.
167 func okEcho(_ context.Context, _, payload string) (string, error) { return payload, nil }
168
147 // newCA returns a signer standing in for somebody's own SSH user CA. 169 // newCA returns a signer standing in for somebody's own SSH user CA.
148 func newCA(t *testing.T) ssh.Signer { 170 func newCA(t *testing.T) ssh.Signer {
149 t.Helper() 171 t.Helper()
@@ -175,14 +197,50 @@ func TestProveMCPDrivesTheWholeCycle(t *testing.T) {
175 f := echoingMCP{newFakeMCP(t, ca)} 197 f := echoingMCP{newFakeMCP(t, ca)}
176 clock := &fakeClock{} 198 clock := &fakeClock{}
177 199
178 require.NoError(t, proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial)) 200 require.NoError(t, proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial, okEcho))
179 201
180 assert.Equal(t, []string{authorizedLine(ca.PublicKey())}, f.uploaded, 202 assert.Equal(t, []string{authorizedLine(ca.PublicKey())}, f.uploaded,
181 "the CA is registered through the tool, before anything is created") 203 "the CA is registered through the tool, before anything is created")
182 assert.Equal(t, 204 assert.Equal(t,
183 []string{"ca_upload", "delegate_begin", "delegate_complete", "delegate_complete", 205 []string{"ca_upload", "delegate_begin", "delegate_complete", "delegate_complete",
184 "tenant_info", "vm_create", "vm_exec", "vm_expose", "vm_unexpose", "vm_destroy"}, 206 "tenant_info", "vm_create", "vm_exec", "vm_expose", "vm_unexpose",
185 f.calls) 207 "vm_exec", "vm_expose", "vm_unexpose", "vm_destroy"},
208 f.calls, "the UDP leg starts its own listener in the guest before publishing a port for it")
209 assert.Equal(t, []string{"tcp", "udp"}, f.exposedAs)
210 }
211
212 // TestProveUDPExposureRequiresTheDatagramBack pins what the UDP leg is for: a
213 // published port that binds and answers nothing passes no gate.
214 func TestProveUDPExposureRequiresTheDatagramBack(t *testing.T) {
215 f := newFakeMCP(t)
216 clock := &fakeClock{}
217
218 // A port that answers something else entirely is a failure on the spot —
219 // something is listening, and it is not the guest's echo.
220 wrong := func(context.Context, string, string) (string, error) { return "not-the-nonce", nil }
221 err := proveUDPExposure(t.Context(), f, "smoke-mcp", clock.now, clock.sleep, wrong)
222 require.Error(t, err)
223 assert.Contains(t, err.Error(), "echoed")
224
225 // A port that answers nothing at all is retried to the deadline and then
226 // fails with what the last attempt said.
227 silent := func(context.Context, string, string) (string, error) {
228 return "", errors.New("i/o timeout")
229 }
230 err = proveUDPExposure(t.Context(), f, "smoke-mcp", clock.now, clock.sleep, silent)
231 require.Error(t, err)
232 assert.Contains(t, err.Error(), "no echo from the MCP-published UDP port")
233 assert.Contains(t, err.Error(), "i/o timeout")
234 }
235
236 // TestUDPEchoCommandDetachesFromTheExecSession pins the two things about the
237 // guest-side echo that a passing leg depends on: it binds the port the leg
238 // publishes, and it outlives the exec that started it.
239 func TestUDPEchoCommandDetachesFromTheExecSession(t *testing.T) {
240 cmd := udpEchoCommand()
241 assert.Contains(t, cmd, strconv.Itoa(udpEchoPort), "the echo binds the port the leg publishes")
242 assert.Contains(t, cmd, "nohup", "the echo must survive the session that started it")
243 assert.Contains(t, cmd, ">/dev/null 2>&1 &", "a background process holding stdout would hang the exec")
186 } 244 }
187 245
188 // TestProveMCPRequiresTheUnregisteredCAToBeRefused is the negative leg itself: 246 // TestProveMCPRequiresTheUnregisteredCAToBeRefused is the negative leg itself:
@@ -193,7 +251,7 @@ func TestProveMCPRequiresTheUnregisteredCAToBeRefused(t *testing.T) {
193 f := echoingMCP{newFakeMCP(t, ca, stranger)} // a plane that trusts everyone 251 f := echoingMCP{newFakeMCP(t, ca, stranger)} // a plane that trusts everyone
194 clock := &fakeClock{} 252 clock := &fakeClock{}
195 253
196 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial) 254 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial, okEcho)
197 require.Error(t, err) 255 require.Error(t, err)
198 assert.Contains(t, err.Error(), "accepted a certificate from an unregistered CA") 256 assert.Contains(t, err.Error(), "accepted a certificate from an unregistered CA")
199 } 257 }
@@ -207,7 +265,7 @@ func TestProveMCPRejectsAWrongPrincipal(t *testing.T) {
207 f.results["delegate_begin"] = map[string]any{"public_key": f.pubLine, "principal": "root"} 265 f.results["delegate_begin"] = map[string]any{"public_key": f.pubLine, "principal": "root"}
208 clock := &fakeClock{} 266 clock := &fakeClock{}
209 267
210 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial) 268 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial, okEcho)
211 require.Error(t, err) 269 require.Error(t, err)
212 assert.Contains(t, err.Error(), `asked for principal "root"`) 270 assert.Contains(t, err.Error(), `asked for principal "root"`)
213 } 271 }
@@ -219,7 +277,7 @@ func TestProveMCPRejectsADelegationWithNoExpiry(t *testing.T) {
219 f := noExpiryMCP{newFakeMCP(t, ca)} 277 f := noExpiryMCP{newFakeMCP(t, ca)}
220 clock := &fakeClock{} 278 clock := &fakeClock{}
221 279
222 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial) 280 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial, okEcho)
223 require.Error(t, err) 281 require.Error(t, err)
224 assert.Contains(t, err.Error(), "reports no expiry") 282 assert.Contains(t, err.Error(), "reports no expiry")
225 } 283 }
@@ -242,7 +300,7 @@ func TestProveMCPDestroysItsVMOnFailure(t *testing.T) {
242 f.failOn = "vm_exec" 300 f.failOn = "vm_exec"
243 clock := &fakeClock{} 301 clock := &fakeClock{}
244 302
245 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial) 303 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial, okEcho)
246 require.Error(t, err) 304 require.Error(t, err)
247 assert.Contains(t, err.Error(), "vm_exec over MCP") 305 assert.Contains(t, err.Error(), "vm_exec over MCP")
248 assert.Contains(t, f.calls, "vm_destroy", "the leg must reap its own VM even when it fails") 306 assert.Contains(t, f.calls, "vm_destroy", "the leg must reap its own VM even when it fails")
@@ -294,7 +352,7 @@ func TestProveMCPRejectsAShortToolset(t *testing.T) {
294 f.tools = f.tools[:5] 352 f.tools = f.tools[:5]
295 clock := &fakeClock{} 353 clock := &fakeClock{}
296 354
297 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial) 355 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial, okEcho)
298 require.Error(t, err) 356 require.Error(t, err)
299 assert.Contains(t, err.Error(), "advertises 5 tools") 357 assert.Contains(t, err.Error(), "advertises 5 tools")
300 assert.Empty(t, f.calls, "a wrong toolset must fail before any tool is called") 358 assert.Empty(t, f.calls, "a wrong toolset must fail before any tool is called")
@@ -312,7 +370,7 @@ func TestProveMCPRejectsAMissingDelegateTool(t *testing.T) {
312 } 370 }
313 clock := &fakeClock{} 371 clock := &fakeClock{}
314 372
315 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial) 373 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial, okEcho)
316 require.Error(t, err) 374 require.Error(t, err)
317 assert.Contains(t, err.Error(), "does not advertise delegate_begin") 375 assert.Contains(t, err.Error(), "does not advertise delegate_begin")
318 } 376 }
@@ -325,7 +383,7 @@ func TestProveMCPRejectsAnExposureWithNoAddress(t *testing.T) {
325 f.results["vm_expose"] = map[string]any{"exposure": map[string]any{"id": "x-1"}} 383 f.results["vm_expose"] = map[string]any{"exposure": map[string]any{"id": "x-1"}}
326 clock := &fakeClock{} 384 clock := &fakeClock{}
327 385
328 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial) 386 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial, okEcho)
329 require.Error(t, err) 387 require.Error(t, err)
330 assert.Contains(t, err.Error(), "no address to dial") 388 assert.Contains(t, err.Error(), "no address to dial")
331 } 389 }
@@ -338,7 +396,7 @@ func TestProveMCPRejectsAWrongBanner(t *testing.T) {
338 clock := &fakeClock{} 396 clock := &fakeClock{}
339 397
340 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, 398 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep,
341 func(context.Context, string) (string, error) { return "HTTP/1.1 200 OK", nil }) 399 func(context.Context, string) (string, error) { return "HTTP/1.1 200 OK", nil }, okEcho)
342 require.Error(t, err) 400 require.Error(t, err)
343 assert.Contains(t, err.Error(), "want an SSH-2.0 banner") 401 assert.Contains(t, err.Error(), "want an SSH-2.0 banner")
344 } 402 }
@@ -350,7 +408,7 @@ func TestProveMCPRejectsAnEchoThatDoesNotComeBack(t *testing.T) {
350 f := newFakeMCP(t, ca) // plain fake: vm_exec returns an empty stdout 408 f := newFakeMCP(t, ca) // plain fake: vm_exec returns an empty stdout
351 clock := &fakeClock{} 409 clock := &fakeClock{}
352 410
353 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial) 411 err := proveMCP(t.Context(), f, "smoke-mcp", testDelegator(ca, stranger), clock.now, clock.sleep, okBannerDial, okEcho)
354 require.Error(t, err) 412 require.Error(t, err)
355 assert.Contains(t, err.Error(), "want it to contain") 413 assert.Contains(t, err.Error(), "want it to contain")
356 } 414 }
internal/smoke/run.go
Old New
@@ -168,7 +168,7 @@ func Run() error {
168 stranger: stranger, 168 stranger: stranger,
169 principal: cfg.SmokeVMUser, 169 principal: cfg.SmokeVMUser,
170 now: time.Now, 170 now: time.Now,
171 }, time.Now, time.Sleep, readBanner) 171 }, time.Now, time.Sleep, readBanner, readEcho)
172 } 172 }
173 173
174 msg, err := runScenario(ctx, vmName, api, api.DialConsole, gate, mcp, time.Now, time.Sleep, realReadPubKey, readBanner) 174 msg, err := runScenario(ctx, vmName, api, api.DialConsole, gate, mcp, time.Now, time.Sleep, realReadPubKey, readBanner)
internal/smoke/scenario.go
Old New
@@ -53,7 +53,7 @@ type vmAPI interface {
53 ListVMs(ctx context.Context) ([]client.VM, error) 53 ListVMs(ctx context.Context) ([]client.VM, error)
54 DeleteVM(ctx context.Context, id string) error 54 DeleteVM(ctx context.Context, id string) error
55 PatchVM(ctx context.Context, id, powerState string) error 55 PatchVM(ctx context.Context, id, powerState string) error
56 CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64) (client.Exposure, error) 56 CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64, protocol string) (client.Exposure, error)
57 DeleteExposure(ctx context.Context, id string) error 57 DeleteExposure(ctx context.Context, id string) error
58 } 58 }
59 59
@@ -193,7 +193,9 @@ func runScenario(ctx context.Context, vmName string, c vmAPI, dialConsole consol
193 exposureOK = true 193 exposureOK = true
194 194
195 // Remote MCP: the same fleet, driven by a bearer PAT over HTTP with no 195 // Remote MCP: the same fleet, driven by a bearer PAT over HTTP with no
196 // local install and no uploaded CA. It creates and destroys its own VM. 196 // local install and no uploaded CA. It creates and destroys its own VM,
197 // and it is where the UDP published port is proven — the proof needs a
198 // listener inside the guest, and this is the leg that can start one.
197 mcpOK := false 199 mcpOK := false
198 if mcp != nil { 200 if mcp != nil {
199 if err := mcp(ctx, vmName+"-mcp"); err != nil { 201 if err := mcp(ctx, vmName+"-mcp"); err != nil {
@@ -288,7 +290,7 @@ func runScenario(ctx context.Context, vmName string, c vmAPI, dialConsole consol
288 msg += ", exposed port: ok" 290 msg += ", exposed port: ok"
289 } 291 }
290 if mcpOK { 292 if mcpOK {
291 msg += ", remote MCP: ok" 293 msg += ", remote MCP: ok, published UDP port: ok"
292 } 294 }
293 return msg, nil 295 return msg, nil
294 } 296 }
internal/smoke/scenario_test.go
Old New
@@ -70,7 +70,7 @@ type testAPI struct {
70 listVMsFunc func(ctx context.Context) ([]client.VM, error) 70 listVMsFunc func(ctx context.Context) ([]client.VM, error)
71 deleteVMFunc func(ctx context.Context, id string) error 71 deleteVMFunc func(ctx context.Context, id string) error
72 patchVMFunc func(ctx context.Context, id, powerState string) error 72 patchVMFunc func(ctx context.Context, id, powerState string) error
73 createExposureFunc func(ctx context.Context, vmID string, guestPort, hostPort int64) (client.Exposure, error) 73 createExposureFunc func(ctx context.Context, vmID string, guestPort, hostPort int64, protocol string) (client.Exposure, error)
74 deleteExposureFunc func(ctx context.Context, id string) error 74 deleteExposureFunc func(ctx context.Context, id string) error
75 } 75 }
76 76
@@ -90,11 +90,11 @@ func (a *testAPI) PatchVM(ctx context.Context, id, powerState string) error {
90 } 90 }
91 return a.patchVMFunc(ctx, id, powerState) 91 return a.patchVMFunc(ctx, id, powerState)
92 } 92 }
93 func (a *testAPI) CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64) (client.Exposure, error) { 93 func (a *testAPI) CreateExposure(ctx context.Context, vmID string, guestPort, hostPort int64, protocol string) (client.Exposure, error) {
94 if a.createExposureFunc == nil { 94 if a.createExposureFunc == nil {
95 return client.Exposure{ID: "x-fake", HostPort: 30080, HostAddr: "192.168.0.190"}, nil 95 return client.Exposure{ID: "x-fake", HostPort: 30080, HostAddr: "192.168.0.190", Protocol: protocol}, nil
96 } 96 }
97 return a.createExposureFunc(ctx, vmID, guestPort, hostPort) 97 return a.createExposureFunc(ctx, vmID, guestPort, hostPort, protocol)
98 } 98 }
99 99
100 func (a *testAPI) DeleteExposure(ctx context.Context, id string) error { 100 func (a *testAPI) DeleteExposure(ctx context.Context, id string) error {
proto/eitri/v1/sync.proto
Old New
@@ -198,25 +198,25 @@ message TCPOpened {
198 } 198 }
199 199
200 // ExposureDesired is one published guest port a host should be serving: bind 200 // ExposureDesired is one published guest port a host should be serving: bind
201 // host_port on the host, pipe every accepted connection to guest_port inside 201 // host_port on the host, pipe every accepted connection (or every datagram)
202 // the guest. It rides the snapshot at TOP LEVEL rather than nested in 202 // to guest_port inside the guest. It rides the snapshot at TOP LEVEL rather
203 // VMDesired, because exposures are their own objects converging on their own 203 // than nested in VMDesired, because exposures are their own objects converging
204 // cadence — an exposure can be created while its VM is still imaging, and it 204 // on their own cadence — an exposure can be created while its VM is still
205 // binds immediately. 205 // imaging, and it binds immediately.
206 message ExposureDesired { 206 message ExposureDesired {
207 string id = 1; 207 string id = 1;
208 string vm_id = 2; 208 string vm_id = 2;
209 uint32 guest_port = 3; 209 uint32 guest_port = 3;
210 uint32 host_port = 4; 210 uint32 host_port = 4;
211 string protocol = 5; // "tcp" 211 string protocol = 5; // "tcp"|"udp"
212 } 212 }
213 213
214 // ExposureActual is one exposure's state as its host observes it: "active" 214 // ExposureActual is one exposure's state as its host observes it: "active"
215 // once the host listener is bound, "failed" with the OS error otherwise. 215 // once the host socket is bound, "failed" with the OS error otherwise.
216 // "active" means the HOST half of the pipe exists — whether anything answers 216 // "active" means the HOST half of the pipe exists — whether anything answers
217 // inside the guest is the guest's half, and this does not pretend otherwise. 217 // inside the guest is the guest's half, and this does not pretend otherwise.
218 message ExposureActual { 218 message ExposureActual {
219 string id = 1; 219 string id = 1;
220 string state = 2; // "active"|"failed" 220 string state = 2; // "active"|"failed"
221 string reason = 3; // the OS error, when failed 221 string reason = 3; // the OS error, when failed; a bound port's ongoing trouble otherwise
222 } 222 }
web/src/lib/api-types.ts
Old New
@@ -1393,7 +1393,7 @@ export interface paths {
1393 }; 1393 };
1394 }; 1394 };
1395 put?: never; 1395 put?: never;
1396 /** Publish a guest TCP port on the VM's host. Omit host_port to allocate one from the reserved range 30000-32767; a named port must be >= 1024 and is honored or refused. */ 1396 /** Publish a guest port on the VM's host, protocol "tcp" (the default) or "udp". Omit host_port to allocate one from the reserved range 30000-32767; a named port must be >= 1024 and is honored or refused, and is taken only by another exposure of the same protocol. */
1397 post: { 1397 post: {
1398 parameters: { 1398 parameters: {
1399 query?: never; 1399 query?: never;
@@ -1516,6 +1516,7 @@ export interface components {
1516 CreateExposureRequest: { 1516 CreateExposureRequest: {
1517 guest_port?: number; 1517 guest_port?: number;
1518 host_port?: number; 1518 host_port?: number;
1519 protocol?: string;
1519 }; 1520 };
1520 CreateVMRequest: { 1521 CreateVMRequest: {
1521 cloud_init?: string; 1522 cloud_init?: string;
web/src/lib/fleet.svelte.ts
Old New
@@ -21,8 +21,9 @@ export type VMEvent = components['schemas']['AuditEvent'];
21 export type UserCA = components['schemas']['UserCA']; 21 export type UserCA = components['schemas']['UserCA'];
22 22
23 /** Exposure is one published guest port: the fleet binds host_port on the VM's 23 /** Exposure is one published guest port: the fleet binds host_port on the VM's
24 * host and pipes it to guest_port inside the guest. host_addr is the address 24 * host and pipes it to guest_port inside the guest, in protocol ('tcp' or
25 * to dial; state is what the host says its listener is doing. */ 25 * 'udp'). host_addr is the address to dial; state is what the host says its
26 * socket is doing. */
26 export type Exposure = components['schemas']['Exposure']; 27 export type Exposure = components['schemas']['Exposure'];
27 28
28 export type CreateVMRequest = components['schemas']['CreateVMRequest']; 29 export type CreateVMRequest = components['schemas']['CreateVMRequest'];
@@ -265,17 +266,19 @@ export async function listExposures(id: string): Promise<Exposure[]> {
265 return (await req('GET', `/api/v1/vms/${id}/exposures`)).json(); 266 return (await req('GET', `/api/v1/vms/${id}/exposures`)).json();
266 } 267 }
267 268
268 /** createExposure publishes a guest port. hostPort 0 asks the control plane to 269 /** createExposure publishes a guest port over protocol ('tcp' or 'udp').
269 * allocate one from the reserved range. */ 270 * hostPort 0 asks the control plane to allocate one from the reserved range. */
270 export async function createExposure( 271 export async function createExposure(
271 id: string, 272 id: string,
272 guestPort: number, 273 guestPort: number,
273 hostPort: number 274 hostPort: number,
275 protocol: string
274 ): Promise<Exposure> { 276 ): Promise<Exposure> {
275 return ( 277 return (
276 await req('POST', `/api/v1/vms/${id}/exposures`, { 278 await req('POST', `/api/v1/vms/${id}/exposures`, {
277 guest_port: guestPort, 279 guest_port: guestPort,
278 host_port: hostPort 280 host_port: hostPort,
281 protocol
279 }) 282 })
280 ).json(); 283 ).json();
281 } 284 }
@@ -427,9 +430,9 @@ export function eventLabel(ev: VMEvent): string {
427 case 'vm.reap': 430 case 'vm.reap':
428 return `Destroyed (${detail.reason ?? '?'})`; 431 return `Destroyed (${detail.reason ?? '?'})`;
429 case 'exposure.create': 432 case 'exposure.create':
430 return `Exposed guest :${detail.guest_port ?? '?'} on host :${detail.host_port ?? '?'}`; 433 return `Exposed guest :${detail.guest_port ?? '?'}/${detail.protocol ?? 'tcp'} on host :${detail.host_port ?? '?'}`;
431 case 'exposure.delete': 434 case 'exposure.delete':
432 return `Exposure removed (guest :${detail.guest_port ?? '?'})`; 435 return `Exposure removed (guest :${detail.guest_port ?? '?'}/${detail.protocol ?? 'tcp'})`;
433 default: 436 default:
434 return ev.action; 437 return ev.action;
435 } 438 }
web/src/routes/vms/[id]/+page.svelte
Old New
@@ -71,6 +71,7 @@
71 let exposuresFailed = $state(false); 71 let exposuresFailed = $state(false);
72 let guestPortInput = $state(''); 72 let guestPortInput = $state('');
73 let hostPortInput = $state(''); 73 let hostPortInput = $state('');
74 let protocolInput = $state('tcp');
74 75
75 async function loadExposures(forId: string | undefined) { 76 async function loadExposures(forId: string | undefined) {
76 if (!forId) return; 77 if (!forId) return;
@@ -116,10 +117,12 @@
116 if (!vm) return; 117 if (!vm) return;
117 const guest = Number(guestPortInput); 118 const guest = Number(guestPortInput);
118 const host = hostPortInput === '' ? 0 : Number(hostPortInput); 119 const host = hostPortInput === '' ? 0 : Number(hostPortInput);
120 const proto = protocolInput;
119 const vmId = vm.id; 121 const vmId = vm.id;
120 if (await action(() => createExposure(vmId, guest, host))) { 122 if (await action(() => createExposure(vmId, guest, host, proto))) {
121 guestPortInput = ''; 123 guestPortInput = '';
122 hostPortInput = ''; 124 hostPortInput = '';
125 protocolInput = 'tcp';
123 await loadExposures(vmId); 126 await loadExposures(vmId);
124 } 127 }
125 } 128 }
@@ -251,7 +254,7 @@
251 <tbody> 254 <tbody>
252 {#each exposures as e (e.id)} 255 {#each exposures as e (e.id)}
253 <tr> 256 <tr>
254 <th>guest :{e.guest_port}</th> 257 <th>guest :{e.guest_port}/{e.protocol}</th>
255 <td> 258 <td>
256 {#if exposureTarget(e)} 259 {#if exposureTarget(e)}
257 <code>{exposureTarget(e)}</code> 260 <code>{exposureTarget(e)}</code>
@@ -288,6 +291,10 @@
288 bind:value={hostPortInput} 291 bind:value={hostPortInput}
289 placeholder="host port (optional)" 292 placeholder="host port (optional)"
290 /> 293 />
294 <select bind:value={protocolInput} aria-label="protocol">
295 <option value="tcp">tcp</option>
296 <option value="udp">udp</option>
297 </select>
291 <button type="submit">Expose</button> 298 <button type="submit">Expose</button>
292 </form> 299 </form>
293 {/if} 300 {/if}