9672785e

feat(release): a tag publishes once, and nothing needs purging

a73x 2026-08-09 15:08

Commit message

feat(release): a tag publishes once, and nothing needs purging

A versioned download URL is immutable, so eitri makes it true rather than
declaring it. `scripts/release.sh` builds byte-reproducibly: the same tagged
tree with the same manifest base yields the same tarballs, the same SHA256SUMS
and the same manifest.json, however many times it runs. Go builds carry no host
paths, archive entries are sorted with ownership zeroed and every timestamp
pinned to the tag's own commit, gzip records neither a filename nor a moment,
and file modes come from a fixed umask rather than the invoking shell's.

The pipeline holds a tag to that. Before stage 3 pushes the site image that
bakes `dist/<tag>` in — the last moment before those bytes become
`https://<site>/dl/<tag>/` — it asks the plane what it already serves for this
tag. Absent is a first publish. Identical is an honest re-run, which is how the
pipeline keeps its promise that re-running from the top is always safe; the
site image is left where it is, since rebuilding it would bake the same webroot
under a new digest and move a released tag's image for nothing. Different is
refused, naming the tag and saying what the fix is: cut a new one. Those bytes
are already in the wild and an agent verifies its upgrade against the sha the
manifest named, so a new payload behind the old URL is not an option. Each
plane is asked about itself, because artifacts embed their own plane's `/dl`
base and stg and prod therefore hold genuinely different bytes for one tag,
neither of them wrong. The server image is rebuilt and pushed on every branch:
nothing outside the cluster holds a contract on its bytes.

The manifest is written before the sums are taken, so SHA256SUMS covers
everything the plane serves — manifest.json included. The manifest is
precisely the artifact whose bytes legitimately differ per plane, since it
carries the URLs artifacts are served from, and that makes it exactly the file
the byte-compare must see: a manifest pointing somewhere else must read as
different, never ride unexamined beneath identical sums. BuildManifest hashes
the artifacts it names by reading them and never reads the sums, so the
ordering costs reproducibility nothing. SHA256SUMS itself is the one row the
downloads page shows without a checksum, because a sums file cannot list
itself.

With immutable URLs immutable in fact, there is nothing to invalidate.
`site/nginx.conf` already states the whole cache contract — `/dl/v*` immutable,
everything mutable `no-cache`, including the `/dl/latest/` alias the fleet
re-polls daily — so the pipeline ends at the roll. No CDN credentials, no
zone-wide eviction to resist, no step that runs at prod and nowhere else.

/dl carries one set of bytes per platform per release. A host bundle is the
artifact: the tarball an operator unpacks by hand is the one manifest.json names
for an agent's upgrade, and the self-updater takes it from there — it verifies
the sha, sniffs gzip, and extracts the bundle's eitri-agent member.
cloud-hypervisor and the guest firmware stay bare, since agents bootstrap those
directly and they duplicate nothing.

`make ship` forwards the pipeline's own knobs — FROM and SKIP_SMOKE — so the
Makefile entry and the script speak one interface.

Makefile

Old		New
@@ -74,7 +74,7 @@ release: web
74	# failure; the script's other flags are for direct use.	74	# failure; the script's other flags are for direct use.
75	ship:	75	ship:
76	@$(if $(and $(TARGET),$(TAG)),,$(error ship: TARGET and TAG are required, e.g. make ship TARGET=stg TAG=v0.0.4-pre.1))	76	@$(if $(and $(TARGET),$(TAG)),,$(error ship: TARGET and TAG are required, e.g. make ship TARGET=stg TAG=v0.0.4-pre.1))
77	./scripts/ship.sh --target $(TARGET) --tag $(TAG)$(if $(FROM), --from $(FROM))	77	./scripts/ship.sh --target $(TARGET) --tag $(TAG)$(if $(FROM), --from $(FROM))$(if $(SKIP_SMOKE), --skip-smoke)
78		78
79	# --- quality gates -----------------------------------------------------------	79	# --- quality gates -----------------------------------------------------------
80		80

deploy/server/README.md

Old		New
@@ -25,7 +25,6 @@ deployment serving the static site and its `/dl` artifacts.
25	\| guest CIDR pool \| `10.78.0.0/16` \| `10.79.0.0/16` \|	25	\| guest CIDR pool \| `10.78.0.0/16` \| `10.79.0.0/16` \|
26	\| sign-in \| Google \| bundled `eitri-oidc` at `oidc.stg.eitri.sh` \|	26	\| sign-in \| Google \| bundled `eitri-oidc` at `oidc.stg.eitri.sh` \|
27	\| backups \| nightly \| none — the database is disposable \|	27	\| backups \| nightly \| none — the database is disposable \|
28	\| CDN purge \| yes \| no — served cache-bypassed \|
29		28
30	The pod runs `hostNetwork`, so those ports are the node's: the two planes	29	The pod runs `hostNetwork`, so those ports are the node's: the two planes
31	share a node and cannot share a port triple. Deployment, Service, PVC and	30	share a node and cannot share a port triple. Deployment, Service, PVC and
@@ -118,19 +117,50 @@ failure.
118	2. Build the artifacts into `dist/<tag>`, with the manifest base pointed at	117	2. Build the artifacts into `dist/<tag>`, with the manifest base pointed at
119	the target's own `/dl`. This is the one build input the planes legitimately	118	the target's own `/dl`. This is the one build input the planes legitimately
120	differ on, and it is why the stg run proves the download and upgrade paths	119	differ on, and it is why the stg run proves the download and upgrade paths
121	rather than rehearsing them.	120	rather than rehearsing them. The build is byte-reproducible, which stage 3
122	3. Build and push the images, server and site, tagged with the version.	121	relies on.
		122	3. Build and push the images, server and site, tagged with the version —
		123	after checking that this plane does not already serve this tag as something
		124	else. The site image bakes `dist/<tag>` in, so the check happens here, at
		125	the last moment before those bytes become an immutable URL. Absent means
		126	first publish; identical means an honest re-run and the site image is left
		127	alone; different is a hard failure, and the fix is a new tag. The server
		128	image is rebuilt and pushed either way.
123	4. Check the config (see below).	129	4. Check the config (see below).
124	5. Apply the plane's shape — namespace, middleware, storage, service,	130	5. Apply the plane's shape — namespace, middleware, storage, service,
125	certificates, routes. Nothing here restarts anything.	131	certificates, routes. Nothing here restarts anything.
126	6. Roll the site, then purge. Order is load-bearing: the server fetches the	132	6. Roll the site. Order is load-bearing: the server fetches the release
127	release manifest at boot and pins the answer for 24 hours, so a server rolled	133	manifest at boot and pins the answer for 24 hours, so a server rolled ahead
128	ahead of its site serves the previous release to the whole fleet for a day.	134	of its site serves the previous release to the whole fleet for a day.
129	7. Roll the control plane. Recreate strategy — a brief gap; agents redial.	135	7. Roll the control plane. Recreate strategy — a brief gap; agents redial.
130	8. Hosted smoke: the same `eitri-smoke` the branch gate runs, against this	136	8. Hosted smoke: the same `eitri-smoke` the branch gate runs, against this
131	plane's public names.	137	plane's public names.
132	9. Report what is actually running.	138	9. Report what is actually running.
133		139
		140	### A tag publishes once
		141
		142	`site/nginx.conf` states the cache contract the site is served under:
		143	`/dl/v*` is `immutable`, everything else — the `/dl/latest/` alias the fleet
		144	re-polls daily, the pages, the console bundle — is `no-cache`. That is the
		145	right contract only if a versioned URL really never changes, so the pipeline
		146	makes it true rather than assuming it.
		147
		148	`scripts/release.sh` builds byte-reproducibly: the same tagged tree and the
		149	same manifest base yield the same tarballs, the same `SHA256SUMS` and the same
		150	`manifest.json`, run after run. Stage 3 then compares what it is about to
		151	publish against what the plane already serves for that tag. Absent is a first
		152	publish. Identical is a re-run, which is the pipeline's whole idempotence
		153	promise honoured at the one stage that touches the outside world; the site
		154	image stays as it is, because rebuilding it would produce the same webroot
		155	under a new digest and move a released tag's image for no reason. Different is
		156	refused: those bytes are already in the wild and an agent verifies its upgrade
		157	against the sha the manifest named, so the answer is a new tag, not a new
		158	payload behind the old URL.
		159
		160	Each plane is asked about itself. Artifacts embed their own plane's `/dl`
		161	base, so stg and prod hold genuinely different bytes for one tag and neither
		162	is wrong.
		163
134	### Stage 4, the one that justifies the script	164	### Stage 4, the one that justifies the script
135		165
136	Every incident on v0.0.3 release day was a hosted-shape failure the local gate	166	Every incident on v0.0.3 release day was a hosted-shape failure the local gate
@@ -168,17 +198,13 @@ Ordered by when they bite.
168	`ship.env` names and never creates one.	198	`ship.env` names and never creates one.
169	5. Firewall openings for the plane's sync and gate ports, in the cloud	199	5. Firewall openings for the plane's sync and gate ports, in the cloud
170	security list and the node's host firewall.	200	security list and the node's host firewall.
171	6. The Cloudflare purge token, once, into	201	6. DNS. Verify rather than create: the `*.eitri.sh` wildcard covers the HTTP
172	`~/eitri-deploy/pipeline/cf-purge.env` (`CF_ZONE_ID`, `CF_API_TOKEN`). Until
173	it exists a prod run refuses to finish without `ALLOW_MANUAL_PURGE=1`, which
174	is how you say out loud that you will purge by hand.
175	7. DNS. Verify rather than create: the `*.eitri.sh` wildcard covers the HTTP
176	names. `gate` and `sync` are the exception and need explicit A records — see	202	names. `gate` and `sync` are the exception and need explicit A records — see
177	"Known gaps".	203	"Known gaps".
178	8. Minting the operator PAT — prod only. Where the plane's issuer is one of	204	7. Minting the operator PAT — prod only. Where the plane's issuer is one of
179	ours, the smoke signs in and mints its own, so there is no token to paste and	205	ours, the smoke signs in and mints its own, so there is no token to paste and
180	none to rotate.	206	none to rotate.
181	9. Tagging. The pipeline verifies tags; it never creates or pushes them.	207	8. Tagging. The pipeline verifies tags; it never creates or pushes them.
182		208
183	## One-time bring-up of a plane	209	## One-time bring-up of a plane
184		210
@@ -481,14 +507,6 @@ guests exist at the time.
481		507
482	## Known gaps	508	## Known gaps
483		509
484	The CDN purge's first real execution is at promote time. stg is served
485	cache-bypassed, so a purge there would invalidate nothing and make the prod step
486	look practiced when it is not. The mitigation is to keep the step small enough
487	that an unrehearsed run is safe: one `curl` with an explicit file list built
488	from `dist/<tag>`, no cache tags, no zone-wide purge. A zone-wide purge is the
489	tempting simplification and the wrong one — it would evict the whole site's
490	cache on every release.
491
492	`gate` and `sync` do not inherit the wildcard usefully. Every `*.eitri.sh`	510	`gate` and `sync` do not inherit the wildcard usefully. Every `*.eitri.sh`
493	name resolves to the home origin, where Traefik runs, but the control-plane pod	511	name resolves to the home origin, where Traefik runs, but the control-plane pod
494	is `hostNetwork` on the cloud node — so its SSH gate and QUIC sync bind that	512	is `hostNetwork` on the cloud node — so its SSH gate and QUIC sync bind that

deploy/server/plane.prod.env

Old		New
@@ -49,6 +49,3 @@ SITE_TLS_SECRET=web-tls
49		49
50	# Nightly sqlite backup CronJob: prod's database is the one nobody can rebuild.	50	# Nightly sqlite backup CronJob: prod's database is the one nobody can rebuild.
51	BACKUPS=1	51	BACKUPS=1
52	# The CDN purge runs at prod and only at prod — stg is served cache-bypassed,
53	# so there is nothing there to invalidate.
54	CDN_PURGE=1

deploy/server/plane.stg.env

Old		New
@@ -50,6 +50,3 @@ SITE_TLS_SECRET=stg-web-tls
50	# No backups: stg's database is disposable by design, and a second nightly	50	# No backups: stg's database is disposable by design, and a second nightly
51	# sqlite job against local-path storage on the same node buys nothing.	51	# sqlite job against local-path storage on the same node buys nothing.
52	BACKUPS=0	52	BACKUPS=0
53	# No purge: stg is served cache-bypassed, so a purge here would be theatre that
54	# made the prod step look rehearsed when it is not. See README, "Known gaps".
55	CDN_PURGE=0

docs/upgrade.md

Old		New
@@ -10,8 +10,9 @@ newer release (it polls `https://eitri.sh/dl/latest/manifest.json` daily), an
10	`↑` button appears next to hosts that are behind. Clicking it tells that one	10	`↑` button appears next to hosts that are behind. Clicking it tells that one
11	agent to upgrade itself:	11	agent to upgrade itself:
12		12
13	1. The agent downloads the new binary from eitri.sh and verifies its sha256	13	1. The agent downloads the host bundle from eitri.sh—the same tarball you
14	against the release manifest.	14	would install by hand—verifies its sha256 against the release manifest, and
		15	takes the `eitri-agent` out of it.
15	2. It swaps the binary in place—the old one is kept next to it as	16	2. It swaps the binary in place—the old one is kept next to it as
16	`eitri-agent.prev`—and re-execs. The process keeps its PID; running VMs	17	`eitri-agent.prev`—and re-execs. The process keeps its PID; running VMs
17	are untouched and stay under the agent's care throughout.	18	are untouched and stay under the agent's care throughout.
@@ -42,14 +43,18 @@ On Linux, set `V` to the release you want (shown at
42	throughout:	43	throughout:
43		44
44	```sh	45	```sh
45	V=v0.0.3	46	V=v0.0.4
46	curl -fsSLO "https://eitri.sh/dl/$V/eitri-agent_linux_amd64"	47	curl -fsSLO "https://eitri.sh/dl/$V/eitri-server_${V}_linux_amd64.tar.gz"
47	curl -fsSL "https://eitri.sh/dl/$V/SHA256SUMS" \| grep ' eitri-agent_linux_amd64$' \| sha256sum -c -	48	curl -fsSL "https://eitri.sh/dl/$V/SHA256SUMS" \| grep " eitri-server_${V}_linux_amd64.tar.gz$" \| sha256sum -c -
		49	tar xzf "eitri-server_${V}_linux_amd64.tar.gz"
48	sudo cp /usr/local/bin/eitri-agent /usr/local/bin/eitri-agent.prev	50	sudo cp /usr/local/bin/eitri-agent /usr/local/bin/eitri-agent.prev
49	sudo install -m 0755 eitri-agent_linux_amd64 /usr/local/bin/eitri-agent	51	sudo install -m 0755 "eitri-server_${V}_linux_amd64/eitri-agent" /usr/local/bin/eitri-agent
50	sudo systemctl restart eitri-agent	52	sudo systemctl restart eitri-agent
51	```	53	```
52		54
		55	The host bundle is the artifact—there is one set of bytes per platform per
		56	release, and it is the same one the console's button fetches.
		57
53	Running VMs survive the restart—the unit's `KillMode=process` (see Operations	58	Running VMs survive the restart—the unit's `KillMode=process` (see Operations
54	below) leaves the guests alone. The `.prev` copy mirrors what the button path	59	below) leaves the guests alone. The `.prev` copy mirrors what the button path
55	keeps, so rollback is the same either way.	60	keeps, so rollback is the same either way.

internal/site/cli.go

Old		New
@@ -40,7 +40,7 @@ func runBuild(args []string) error {
40	func runManifest(args []string) error {	40	func runManifest(args []string) error {
41	fs := flag.NewFlagSet("eitri-site manifest", flag.ExitOnError)	41	fs := flag.NewFlagSet("eitri-site manifest", flag.ExitOnError)
42	ver := fs.String("version", "", "release version (vX.Y.Z)")	42	ver := fs.String("version", "", "release version (vX.Y.Z)")
43	dist := fs.String("dist", "", "dist/<version> dir holding bare agent binaries")	43	dist := fs.String("dist", "", "dist/<version> dir holding the release artifacts")
44	base := fs.String("base", "", "base URL artifacts are served from")	44	base := fs.String("base", "", "base URL artifacts are served from")
45	out := fs.String("out", "", "output path (default <dist>/manifest.json)")	45	out := fs.String("out", "", "output path (default <dist>/manifest.json)")
46	if err := fs.Parse(args); err != nil {	46	if err := fs.Parse(args); err != nil {

internal/site/cli_test.go

Old		New
@@ -36,7 +36,7 @@ func TestRunCLIBuildError(t *testing.T) {
36	// subcommand, to both an explicit -out and the default <dist>/manifest.json.	36	// subcommand, to both an explicit -out and the default <dist>/manifest.json.
37	func TestRunCLIManifest(t *testing.T) {	37	func TestRunCLIManifest(t *testing.T) {
38	dist := t.TempDir()	38	dist := t.TempDir()
39	if err := os.WriteFile(filepath.Join(dist, "eitri-agent_linux_amd64"), []byte("x"), 0o755); err != nil {	39	if err := os.WriteFile(filepath.Join(dist, "eitri-server_v0.0.1_linux_amd64.tar.gz"), []byte("x"), 0o644); err != nil {
40	t.Fatal(err)	40	t.Fatal(err)
41	}	41	}
42		42

internal/site/dl.go

Old		New
@@ -41,7 +41,7 @@ func downloadsMarkdown(distDir string) (string, error) {
41	if err != nil {	41	if err != nil {
42	return "", err	42	return "", err
43	}	43	}
44	// manifest.json and SHA256SUMS itself carry no checksum entry; an	44	// SHA256SUMS cannot list itself, so its own row has no checksum; an
45	// empty code span would render as literal backticks.	45	// empty code span would render as literal backticks.
46	sha := "—"	46	sha := "—"
47	if s := sums[name]; s != "" {	47	if s := sums[name]; s != "" {

internal/site/dl_test.go

Old		New
@@ -15,8 +15,8 @@ func fixtureDist(t *testing.T) string {
15	}	15	}
16	files := map[string]string{	16	files := map[string]string{
17	"eitri-server_v0.0.1_linux_amd64.tar.gz": strings.Repeat("x", 2048),	17	"eitri-server_v0.0.1_linux_amd64.tar.gz": strings.Repeat("x", 2048),
18	"eitri-agent_linux_amd64": "binary",	18	"eitri-cli_v0.0.1_linux_amd64.tar.gz": "bundle",
19	"SHA256SUMS": "abc123 eitri-server_v0.0.1_linux_amd64.tar.gz\ndef456 eitri-agent_linux_amd64\n",	19	"SHA256SUMS": "abc123 eitri-server_v0.0.1_linux_amd64.tar.gz\ndef456 eitri-cli_v0.0.1_linux_amd64.tar.gz\n",
20	}	20	}
21	for name, body := range files {	21	for name, body := range files {
22	if err := os.WriteFile(filepath.Join(dist, name), []byte(body), 0o644); err != nil {	22	if err := os.WriteFile(filepath.Join(dist, name), []byte(body), 0o644); err != nil {
@@ -67,11 +67,11 @@ func TestDownloadsPageRequiresSums(t *testing.T) {
67	}	67	}
68	}	68	}
69		69
		70	// TestDownloadsPageUnsummedFilesGetPlaceholder covers the one row that can
		71	// carry no checksum: SHA256SUMS, which cannot list itself. Everything else a
		72	// release stages — manifest.json included — has an entry.
70	func TestDownloadsPageUnsummedFilesGetPlaceholder(t *testing.T) {	73	func TestDownloadsPageUnsummedFilesGetPlaceholder(t *testing.T) {
71	dist := fixtureDist(t)	74	dist := fixtureDist(t)
72	if err := os.WriteFile(filepath.Join(dist, "manifest.json"), []byte("{}"), 0o644); err != nil {
73	t.Fatal(err)
74	}
75	md, err := downloadsMarkdown(dist)	75	md, err := downloadsMarkdown(dist)
76	if err != nil {	76	if err != nil {
77	t.Fatal(err)	77	t.Fatal(err)

internal/site/manifest.go

Old		New
@@ -13,9 +13,14 @@ import (
13	"github.com/a73x/eitri/internal/server/release"	13	"github.com/a73x/eitri/internal/server/release"
14	)	14	)
15		15
16	// barePat matches the bare agent binaries the release stage drops in dist/:	16	// bundlePat matches the host bundles the release stage drops in dist/: a Linux
17	// eitri-agent_<os>_<arch>. These are what the agent self-updater downloads.	17	// host takes eitri-server_<v>_linux_<arch>.tar.gz (server, agent and their
18	var barePat = regexp.MustCompile(`^eitri-agent_([a-z0-9]+)_([a-z0-9]+)$`)	18	// units), a Mac eitri-agent_<v>_darwin_<arch>.tar.gz (agent and its installer).
		19	// Each carries an eitri-agent member, so the tarball an operator unpacks by
		20	// hand is the same artifact a self-updating agent downloads — it verifies the
		21	// sha and extracts that member. The client and issuer bundles carry no agent
		22	// and are deliberately outside the pattern.
		23	var bundlePat = regexp.MustCompile(`^eitri-(?:server\|agent)_[^_]+_([a-z0-9]+)_([a-z0-9]+)\.tar\.gz$`)
19		24
20	// chPat matches the pinned cloud-hypervisor binaries mirrored into dist/:	25	// chPat matches the pinned cloud-hypervisor binaries mirrored into dist/:
21	// cloud-hypervisor_<os>_<arch>. What the agent bootstraps its runtime from.	26	// cloud-hypervisor_<os>_<arch>. What the agent bootstraps its runtime from.
@@ -25,11 +30,11 @@ var chPat = regexp.MustCompile(`^cloud-hypervisor_([a-z0-9]+)_([a-z0-9]+)$`)
25	// edk2 CLOUDHV is x86-64 only, so it manifests under a single platform key.	30	// edk2 CLOUDHV is x86-64 only, so it manifests under a single platform key.
26	const firmwareName = "CLOUDHV.fd"	31	const firmwareName = "CLOUDHV.fd"
27		32
28	// BuildManifest scans distDir for bare eitri-agent binaries — required — plus	33	// BuildManifest scans distDir for host bundles — required — plus optional
29	// optional runtime artifacts (a pinned cloud-hypervisor and guest firmware)	34	// runtime artifacts (a pinned cloud-hypervisor and guest firmware) and
30	// and produces the release manifest the server polls and the agent	35	// produces the release manifest the server polls and the agent bootstraps
31	// bootstraps from. Sharing release.Manifest with the consumers is	36	// from. Sharing release.Manifest with the consumers is deliberate: the wire
32	// deliberate: the wire contract lives in one type.	37	// contract lives in one type.
33	func BuildManifest(version, distDir, baseURL string) (release.Manifest, error) {	38	func BuildManifest(version, distDir, baseURL string) (release.Manifest, error) {
34	if version == "" {	39	if version == "" {
35	return release.Manifest{}, fmt.Errorf("version required")	40	return release.Manifest{}, fmt.Errorf("version required")
@@ -48,8 +53,8 @@ func BuildManifest(version, distDir, baseURL string) (release.Manifest, error) {
48	}	53	}
49	key, platform := "", ""	54	key, platform := "", ""
50	switch {	55	switch {
51	case barePat.MatchString(e.Name()):	56	case bundlePat.MatchString(e.Name()):
52	match := barePat.FindStringSubmatch(e.Name())	57	match := bundlePat.FindStringSubmatch(e.Name())
53	key, platform = "eitri-agent", match[1]+"/"+match[2]	58	key, platform = "eitri-agent", match[1]+"/"+match[2]
54	case chPat.MatchString(e.Name()):	59	case chPat.MatchString(e.Name()):
55	match := chPat.FindStringSubmatch(e.Name())	60	match := chPat.FindStringSubmatch(e.Name())
@@ -76,7 +81,7 @@ func BuildManifest(version, distDir, baseURL string) (release.Manifest, error) {
76	}	81	}
77	}	82	}
78	if len(m.Artifacts["eitri-agent"]) == 0 {	83	if len(m.Artifacts["eitri-agent"]) == 0 {
79	return release.Manifest{}, fmt.Errorf("no bare eitri-agent binaries in %s", distDir)	84	return release.Manifest{}, fmt.Errorf("no host bundles in %s", distDir)
80	}	85	}
81	return m, nil	86	return m, nil
82	}	87	}

internal/site/manifest_test.go

Old		New
@@ -11,17 +11,26 @@ import (
11	"github.com/a73x/eitri/internal/server/release"	11	"github.com/a73x/eitri/internal/server/release"
12	)	12	)
13		13
14	func TestBuildManifestFromBareBinaries(t *testing.T) {	14	func TestBuildManifestFromHostBundles(t *testing.T) {
15	dist := t.TempDir()	15	dist := t.TempDir()
16	body := []byte("fake agent binary")	16	body := []byte("fake host bundle")
17	for _, name := range []string{"eitri-agent_linux_amd64", "eitri-agent_linux_arm64"} {	17	for _, name := range []string{
18	if err := os.WriteFile(filepath.Join(dist, name), body, 0o755); err != nil {	18	"eitri-server_v0.0.1_linux_amd64.tar.gz",
		19	"eitri-server_v0.0.1_linux_arm64.tar.gz",
		20	} {
		21	if err := os.WriteFile(filepath.Join(dist, name), body, 0o644); err != nil {
19	t.Fatal(err)	22	t.Fatal(err)
20	}	23	}
21	}	24	}
22	// Non-bare files must be ignored.	25	// The bundles that carry no agent stay out of the manifest: an agent
23	if err := os.WriteFile(filepath.Join(dist, "eitri-server_v0.0.1_linux_amd64.tar.gz"), body, 0o644); err != nil {	26	// pointed at one would swap itself for a binary that is not an agent.
24	t.Fatal(err)	27	for _, name := range []string{
		28	"eitri-cli_v0.0.1_linux_amd64.tar.gz",
		29	"eitri-oidc_v0.0.1_linux_amd64.tar.gz",
		30	} {
		31	if err := os.WriteFile(filepath.Join(dist, name), body, 0o644); err != nil {
		32	t.Fatal(err)
		33	}
25	}	34	}
26		35
27	m, err := BuildManifest("v0.0.1", dist, "https://eitri.sh/dl/v0.0.1")	36	m, err := BuildManifest("v0.0.1", dist, "https://eitri.sh/dl/v0.0.1")
@@ -37,7 +46,7 @@ func TestBuildManifestFromBareBinaries(t *testing.T) {
37	}	46	}
38	sum := sha256.Sum256(body)	47	sum := sha256.Sum256(body)
39	want := release.Artifact{	48	want := release.Artifact{
40	URL: "https://eitri.sh/dl/v0.0.1/eitri-agent_linux_amd64",	49	URL: "https://eitri.sh/dl/v0.0.1/eitri-server_v0.0.1_linux_amd64.tar.gz",
41	SHA256: hex.EncodeToString(sum[:]),	50	SHA256: hex.EncodeToString(sum[:]),
42	}	51	}
43	if agents["linux/amd64"] != want {	52	if agents["linux/amd64"] != want {
@@ -58,9 +67,9 @@ func TestBuildManifestFromBareBinaries(t *testing.T) {
58	}	67	}
59	}	68	}
60		69
61	func TestBuildManifestRequiresBinaries(t *testing.T) {	70	func TestBuildManifestRequiresHostBundles(t *testing.T) {
62	if _, err := BuildManifest("v0.0.1", t.TempDir(), "https://eitri.sh/dl/v0.0.1"); err == nil {	71	if _, err := BuildManifest("v0.0.1", t.TempDir(), "https://eitri.sh/dl/v0.0.1"); err == nil {
63	t.Fatal("want error when no bare eitri-agent binaries exist")	72	t.Fatal("want error when no host bundles exist")
64	}	73	}
65	}	74	}
66		75
@@ -72,7 +81,7 @@ func TestBuildManifestRequiresVersion(t *testing.T) {
72		81
73	func TestBuildManifestNormalizesBaseURL(t *testing.T) {	82	func TestBuildManifestNormalizesBaseURL(t *testing.T) {
74	dist := t.TempDir()	83	dist := t.TempDir()
75	if err := os.WriteFile(filepath.Join(dist, "eitri-agent_linux_amd64"), []byte("x"), 0o755); err != nil {	84	if err := os.WriteFile(filepath.Join(dist, "eitri-server_v0.0.1_linux_amd64.tar.gz"), []byte("x"), 0o644); err != nil {
76	t.Fatal(err)	85	t.Fatal(err)
77	}	86	}
78	m, err := BuildManifest("v0.0.1", dist, "https://eitri.sh/dl/v0.0.1/")	87	m, err := BuildManifest("v0.0.1", dist, "https://eitri.sh/dl/v0.0.1/")
@@ -80,22 +89,22 @@ func TestBuildManifestNormalizesBaseURL(t *testing.T) {
80	t.Fatal(err)	89	t.Fatal(err)
81	}	90	}
82	got := m.Artifacts["eitri-agent"]["linux/amd64"].URL	91	got := m.Artifacts["eitri-agent"]["linux/amd64"].URL
83	if got != "https://eitri.sh/dl/v0.0.1/eitri-agent_linux_amd64" {	92	if got != "https://eitri.sh/dl/v0.0.1/eitri-server_v0.0.1_linux_amd64.tar.gz" {
84	t.Errorf("trailing-slash base not normalized: %q", got)	93	t.Errorf("trailing-slash base not normalized: %q", got)
85	}	94	}
86	}	95	}
87		96
88	func TestBuildManifestIncludesRuntimeArtifacts(t *testing.T) {	97	func TestBuildManifestIncludesRuntimeArtifacts(t *testing.T) {
89	dist := t.TempDir()	98	dist := t.TempDir()
90	agent := []byte("fake agent binary")	99	bundle := []byte("fake host bundle")
91	ch := []byte("fake cloud-hypervisor binary")	100	ch := []byte("fake cloud-hypervisor binary")
92	fw := []byte("fake CLOUDHV.fd")	101	fw := []byte("fake CLOUDHV.fd")
93	files := map[string][]byte{	102	files := map[string][]byte{
94	"eitri-agent_linux_amd64": agent,	103	"eitri-server_v0.0.1_linux_amd64.tar.gz": bundle,
95	"eitri-agent_linux_arm64": agent,	104	"eitri-server_v0.0.1_linux_arm64.tar.gz": bundle,
96	"cloud-hypervisor_linux_amd64": ch,	105	"cloud-hypervisor_linux_amd64": ch,
97	"cloud-hypervisor_linux_arm64": ch,	106	"cloud-hypervisor_linux_arm64": ch,
98	"CLOUDHV.fd": fw,	107	"CLOUDHV.fd": fw,
99	}	108	}
100	for name, body := range files {	109	for name, body := range files {
101	if err := os.WriteFile(filepath.Join(dist, name), body, 0o755); err != nil {	110	if err := os.WriteFile(filepath.Join(dist, name), body, 0o755); err != nil {
@@ -137,7 +146,7 @@ func TestBuildManifestIncludesRuntimeArtifacts(t *testing.T) {
137		146
138	func TestBuildManifestRuntimeArtifactsOptional(t *testing.T) {	147	func TestBuildManifestRuntimeArtifactsOptional(t *testing.T) {
139	dist := t.TempDir()	148	dist := t.TempDir()
140	if err := os.WriteFile(filepath.Join(dist, "eitri-agent_linux_amd64"), []byte("x"), 0o755); err != nil {	149	if err := os.WriteFile(filepath.Join(dist, "eitri-server_v0.0.1_linux_amd64.tar.gz"), []byte("x"), 0o644); err != nil {
141	t.Fatal(err)	150	t.Fatal(err)
142	}	151	}
143		152
@@ -165,10 +174,11 @@ func TestBuildManifestRuntimeArtifactsOptional(t *testing.T) {
165	func TestBuildManifestCarriesADarwinAgent(t *testing.T) {	174	func TestBuildManifestCarriesADarwinAgent(t *testing.T) {
166	dist := t.TempDir()	175	dist := t.TempDir()
167	for _, name := range []string{	176	for _, name := range []string{
168	"eitri-agent_linux_amd64", "eitri-agent_darwin_arm64",	177	"eitri-server_v0.0.3_linux_amd64.tar.gz",
		178	"eitri-agent_v0.0.3_darwin_arm64.tar.gz",
169	"cloud-hypervisor_linux_amd64", "CLOUDHV.fd",	179	"cloud-hypervisor_linux_amd64", "CLOUDHV.fd",
170	} {	180	} {
171	if err := os.WriteFile(filepath.Join(dist, name), []byte(name), 0o755); err != nil {	181	if err := os.WriteFile(filepath.Join(dist, name), []byte(name), 0o644); err != nil {
172	t.Fatal(err)	182	t.Fatal(err)
173	}	183	}
174	}	184	}
@@ -180,7 +190,7 @@ func TestBuildManifestCarriesADarwinAgent(t *testing.T) {
180	if !ok {	190	if !ok {
181	t.Fatal("no eitri-agent artifact for darwin/arm64")	191	t.Fatal("no eitri-agent artifact for darwin/arm64")
182	}	192	}
183	if want := "https://eitri.sh/dl/v0.0.3/eitri-agent_darwin_arm64"; art.URL != want {	193	if want := "https://eitri.sh/dl/v0.0.3/eitri-agent_v0.0.3_darwin_arm64.tar.gz"; art.URL != want {
184	t.Errorf("url = %q, want %q", art.URL, want)	194	t.Errorf("url = %q, want %q", art.URL, want)
185	}	195	}
186	if art.SHA256 == "" {	196	if art.SHA256 == "" {

scripts/release.sh

Old		New
@@ -6,16 +6,17 @@
6	# eitri-cli_<v>_<os>_<arch>.tar.gz client CLI (eitri) for linux+darwin	6	# eitri-cli_<v>_<os>_<arch>.tar.gz client CLI (eitri) for linux+darwin
7	# eitri-oidc_<v>_linux_{amd64,arm64}.tar.gz bundled OIDC issuer + its unit	7	# eitri-oidc_<v>_linux_{amd64,arm64}.tar.gz bundled OIDC issuer + its unit
8	# (optional sidecar; not in manifest.json)	8	# (optional sidecar; not in manifest.json)
9	# eitri-agent_{linux_amd64,linux_arm64,darwin_arm64}
10	# bare binaries — what the agent
11	# self-updater downloads and
12	# sha-verifies, for one more release
13	# (see "the bare-binary bridge" below)
14	# cloud-hypervisor_linux_{amd64,arm64} pinned runtime, mirrored from upstream	9	# cloud-hypervisor_linux_{amd64,arm64} pinned runtime, mirrored from upstream
15	# CLOUDHV.fd guest UEFI firmware (if FIRMWARE_SRC set)	10	# CLOUDHV.fd guest UEFI firmware (if FIRMWARE_SRC set)
16	# SHA256SUMS over everything above
17	# manifest.json agent-upgrade + agent-bootstrap manifest	11	# manifest.json agent-upgrade + agent-bootstrap manifest
18	# (shared type with internal/relmanifest)	12	# (shared type with internal/relmanifest)
		13	# SHA256SUMS over everything above, manifest included
		14	#
		15	# Everything a host installs is a bundle. manifest.json points the agent
		16	# upgrade at the very tarball you would unpack by hand, and the self-updater
		17	# takes it from there: it sniffs gzip and extracts the bundle's eitri-agent
		18	# member (internal/agent/selfupdate). cloud-hypervisor and the firmware stay
		19	# bare — agents bootstrap those directly and they duplicate nothing.
19	#	20	#
20	# MANIFEST_BASE (env, optional) overrides the URL base written into	21	# MANIFEST_BASE (env, optional) overrides the URL base written into
21	# manifest.json — default https://eitri.sh/dl/<version>; set it for staging.	22	# manifest.json — default https://eitri.sh/dl/<version>; set it for staging.
@@ -24,6 +25,17 @@
24	# (the agent-upgrade path compares versions numerically, so an unparsable	25	# (the agent-upgrade path compares versions numerically, so an unparsable
25	# version silently disables the upgrade button fleet-wide); an untagged HEAD	26	# version silently disables the upgrade button fleet-wide); an untagged HEAD
26	# warns but proceeds, for staging runs.	27	# warns but proceeds, for staging runs.
		28	#
		29	# The build is byte-reproducible. The same tagged tree with the same
		30	# MANIFEST_BASE yields the same tarballs, the same SHA256SUMS and the same
		31	# manifest.json, however many times it is run: Go builds carry no host paths,
		32	# archive metadata is pinned to the tag's own commit, and gzip records neither
		33	# a name nor a moment. That is what makes a versioned /dl URL immutable in
		34	# fact rather than only in a cache header — scripts/ship.sh compares these
		35	# bytes against what the target already serves and refuses to republish a tag
		36	# as something else.
		37	#
		38	# GNU userland, like the sha256sum below: releases are built on Linux.
27	set -euo pipefail	39	set -euo pipefail
28	cd "$(dirname "$0")/.."	40	cd "$(dirname "$0")/.."
29		41
@@ -51,27 +63,28 @@ mkdir -p "$OUT"
51	STAGE_ROOT="$(mktemp -d)"	63	STAGE_ROOT="$(mktemp -d)"
52	trap 'rm -rf "$STAGE_ROOT"' EXIT	64	trap 'rm -rf "$STAGE_ROOT"' EXIT
53		65
54	# ---- the bare-binary bridge: this release emits it, the next one does not ----	66	# Two inputs the artifacts would otherwise take from whoever is running this:
55	# Each host bundle below is followed by a `cp` that drops the same eitri-agent	67	# the invoking shell's umask, which lands in every archived file's mode, and
56	# bytes into dist/ a second time, bare. That copy is a compatibility hop, not an	68	# the wall clock. Fix both. The tag's own commit time is the natural reference
57	# artifact anyone is meant to fetch by hand: manifest.json points the	69	# — it is a property of what is being released, so a rebuild years later still
58	# eitri-agent entries at it, and a fielded agent swaps in WHATEVER bytes the	70	# reproduces it.
59	# manifest names. Hand an agent from an older release a tarball and it verifies	71	umask 022
60	# the sha, writes the archive over its own binary, and crash-loops.	72	SOURCE_DATE_EPOCH="$(git log -1 --pretty=%ct)"
61	#	73
62	# As of this release the self-updater reads a tarball — it sniffs gzip and	74	# bundle_tar archives a staged bundle directory reproducibly: entries in name
63	# extracts the bundle's eitri-agent member (internal/agent/selfupdate) — so	75	# order, ownership zeroed, every timestamp the tag's, and a gzip stream that
64	# every agent running this version or later can be pointed at the bundle. These	76	# records neither the input filename nor the moment it was compressed.
65	# bare copies exist only to carry the ones already in the field across.	77	bundle_tar() {
66	#	78	local stage="$1" bundle="$2" out="$3"
67	# NEXT RELEASE, once no agent older than this one is still out there: point	79	tar -C "$stage" --format=gnu --sort=name \
68	# manifest.json at eitri-server_<v>_linux_<arch>.tar.gz and	80	--owner=0 --group=0 --numeric-owner \
69	# eitri-agent_<v>_darwin_arm64.tar.gz (widen the matcher in	81	--mtime="@$SOURCE_DATE_EPOCH" \
70	# internal/site.BuildManifest) and delete the two `cp` lines below. /dl then	82	-cf - "$bundle" \| gzip -9 -n >"$out"
71	# serves tarballs and nothing is shipped twice. cloud-hypervisor and CLOUDHV.fd	83	}
72	# stay bare in either case — agents bootstrap those directly and they duplicate	84
73	# nothing.	85	# The Linux host bundle, and the artifact manifest.json names for a Linux
74	# ------------------------------------------------------------------------------	86	# agent's upgrade: one set of bytes on /dl, whether a person unpacks it or an
		87	# agent swaps itself for the eitri-agent inside it.
75	for arch in amd64 arm64; do	88	for arch in amd64 arm64; do
76	bundle="eitri-server_${VERSION}_linux_${arch}"	89	bundle="eitri-server_${VERSION}_linux_${arch}"
77	stage="$STAGE_ROOT/$arch"	90	stage="$STAGE_ROOT/$arch"
@@ -81,9 +94,8 @@ for arch in amd64 arm64; do
81	-o "$stage/$bundle/eitri-server" ./cmd/eitri-server	94	-o "$stage/$bundle/eitri-server" ./cmd/eitri-server
82	CGO_ENABLED=0 GOOS=linux GOARCH="$arch" go build -trimpath -ldflags "$LDFLAGS" \	95	CGO_ENABLED=0 GOOS=linux GOARCH="$arch" go build -trimpath -ldflags "$LDFLAGS" \
83	-o "$stage/$bundle/eitri-agent" ./cmd/eitri-agent	96	-o "$stage/$bundle/eitri-agent" ./cmd/eitri-agent
84	cp scripts/eitri-agent.service scripts/eitri-server.service "$stage/$bundle/"	97	install -m 0644 scripts/eitri-agent.service scripts/eitri-server.service "$stage/$bundle/"
85	tar -C "$stage" -czf "$OUT/$bundle.tar.gz" "$bundle"	98	bundle_tar "$stage" "$bundle" "$OUT/$bundle.tar.gz"
86	cp "$stage/$bundle/eitri-agent" "$OUT/eitri-agent_linux_${arch}" # bridge; drop next release
87	done	99	done
88		100
89	# macOS host bundle: the agent and the script that installs it as a LaunchAgent.	101	# macOS host bundle: the agent and the script that installs it as a LaunchAgent.
@@ -106,15 +118,14 @@ mkdir -p "$mac_stage/$mac_bundle"
106	echo "==> building darwin/arm64"	118	echo "==> building darwin/arm64"
107	CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -trimpath -ldflags "$LDFLAGS" \	119	CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -trimpath -ldflags "$LDFLAGS" \
108	-o "$mac_stage/$mac_bundle/eitri-agent" ./cmd/eitri-agent	120	-o "$mac_stage/$mac_bundle/eitri-agent" ./cmd/eitri-agent
109	cp scripts/eitri-agent-launchagent.sh "$mac_stage/$mac_bundle/"	121	install -m 0755 scripts/eitri-agent-launchagent.sh "$mac_stage/$mac_bundle/"
110	tar -C "$mac_stage" -czf "$OUT/$mac_bundle.tar.gz" "$mac_bundle"	122	bundle_tar "$mac_stage" "$mac_bundle" "$OUT/$mac_bundle.tar.gz"
111	cp "$mac_stage/$mac_bundle/eitri-agent" "$OUT/eitri-agent_darwin_arm64" # bridge; drop next release
112		123
113	# Bundled OIDC issuer — its own tarball (binary + systemd unit) so running with	124	# Bundled OIDC issuer — its own tarball (binary + systemd unit) so running with
114	# or without local OIDC is a pure deployment choice: a fleet fronted by an	125	# or without local OIDC is a pure deployment choice: a fleet fronted by an
115	# external IdP never downloads it. Like eitri-cli, it is not an agent/runtime	126	# external IdP never downloads it. Like eitri-cli, it carries no agent, so it
116	# artifact, so it stays out of manifest.json (BuildManifest matches only the	127	# stays out of manifest.json (BuildManifest matches the host bundles, the
117	# bare eitri-agent/cloud-hypervisor binaries and CLOUDHV.fd).	128	# cloud-hypervisor binaries and CLOUDHV.fd).
118	for arch in amd64 arm64; do	129	for arch in amd64 arm64; do
119	bundle="eitri-oidc_${VERSION}_linux_${arch}"	130	bundle="eitri-oidc_${VERSION}_linux_${arch}"
120	stage="$STAGE_ROOT/oidc-$arch"	131	stage="$STAGE_ROOT/oidc-$arch"
@@ -122,8 +133,8 @@ for arch in amd64 arm64; do
122	echo "==> building eitri-oidc linux/$arch"	133	echo "==> building eitri-oidc linux/$arch"
123	CGO_ENABLED=0 GOOS=linux GOARCH="$arch" go build -trimpath -ldflags "$LDFLAGS" \	134	CGO_ENABLED=0 GOOS=linux GOARCH="$arch" go build -trimpath -ldflags "$LDFLAGS" \
124	-o "$stage/$bundle/eitri-oidc" ./cmd/eitri-oidc	135	-o "$stage/$bundle/eitri-oidc" ./cmd/eitri-oidc
125	cp scripts/eitri-oidc.service "$stage/$bundle/"	136	install -m 0644 scripts/eitri-oidc.service "$stage/$bundle/"
126	tar -C "$stage" -czf "$OUT/$bundle.tar.gz" "$bundle"	137	bundle_tar "$stage" "$bundle" "$OUT/$bundle.tar.gz"
127	done	138	done
128		139
129	# Client CLI, cross-compiled for laptops (pure Go, CGO-free).	140	# Client CLI, cross-compiled for laptops (pure Go, CGO-free).
@@ -135,7 +146,7 @@ for platform in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64; do
135	echo "==> building client $platform"	146	echo "==> building client $platform"
136	CGO_ENABLED=0 GOOS="$os" GOARCH="$arch" go build -trimpath -ldflags "$LDFLAGS" \	147	CGO_ENABLED=0 GOOS="$os" GOARCH="$arch" go build -trimpath -ldflags "$LDFLAGS" \
137	-o "$stage/$bundle/eitri" ./cmd/eitri	148	-o "$stage/$bundle/eitri" ./cmd/eitri
138	tar -C "$stage" -czf "$OUT/$bundle.tar.gz" "$bundle"	149	bundle_tar "$stage" "$bundle" "$OUT/$bundle.tar.gz"
139	done	150	done
140		151
141	# Mirror the pinned cloud-hypervisor (agents bootstrap it from the manifest).	152	# Mirror the pinned cloud-hypervisor (agents bootstrap it from the manifest).
@@ -171,12 +182,24 @@ fi
171		182
172	# The API contract rides in the release too (drift-gated in ci, so the	183	# The API contract rides in the release too (drift-gated in ci, so the
173	# committed copy is authoritative).	184	# committed copy is authoritative).
174	cp docs/openapi.json "$OUT/openapi.json"	185	install -m 0644 docs/openapi.json "$OUT/openapi.json"
175
176	(cd "$OUT" && LC_ALL=C sha256sum -- * > SHA256SUMS)
177		186
		187	# The manifest is written before the sums, so the sums cover it. It is the one
		188	# file in here the fleet polls on its own — every agent's upgrade starts by
		189	# reading it — and it is also the one file whose bytes differ per plane, since
		190	# it carries the URLs artifacts are served from. Left out of SHA256SUMS, it is
		191	# invisible to the publish-once check in scripts/ship.sh: two runs of a tag
		192	# against different SITE_HOSTs compare identical while the manifest under that
		193	# immutable URL says something else. Ordering is safe in one direction only,
		194	# and this is it — BuildManifest hashes the artifacts it names by reading them
		195	# and never reads SHA256SUMS, so the manifest depends on the tarballs and the
		196	# sums depend on both.
178	go run ./cmd/eitri-site manifest -version "$VERSION" -dist "$OUT" \	197	go run ./cmd/eitri-site manifest -version "$VERSION" -dist "$OUT" \
179	-base "${MANIFEST_BASE:-https://eitri.sh/dl/$VERSION}"	198	-base "${MANIFEST_BASE:-https://eitri.sh/dl/$VERSION}"
180		199
		200	# The glob is expanded before the redirection creates the file, so SHA256SUMS
		201	# is the only artifact it cannot list: itself.
		202	(cd "$OUT" && LC_ALL=C sha256sum -- * > SHA256SUMS)
		203
181	echo "==> release staged in $OUT"	204	echo "==> release staged in $OUT"
182	ls -lh "$OUT"	205	ls -lh "$OUT"

scripts/ship.env.example

Old		New
@@ -3,7 +3,7 @@
3	# Copy this to ~/eitri-deploy/<target>/ship.env (override the whole path with	3	# Copy this to ~/eitri-deploy/<target>/ship.env (override the whole path with
4	# $EITRI_SHIP_ENV) and fill it in. This file is site-specific and points at	4	# $EITRI_SHIP_ENV) and fill it in. This file is site-specific and points at
5	# secrets — keep it OUT of the repo. What the PLANE is (hostnames, host ports,	5	# secrets — keep it OUT of the repo. What the PLANE is (hostnames, host ports,
6	# namespace, which plane gets backups and a purge) is committed instead, in	6	# namespace, which plane gets backups) is committed instead, in
7	# deploy/server/plane.<target>.env.	7	# deploy/server/plane.<target>.env.
8	#	8	#
9	# The same layout as scripts/deploy.env.example on purpose: one convention.	9	# The same layout as scripts/deploy.env.example on purpose: one convention.

scripts/ship.sh

Old		New
@@ -14,13 +14,14 @@
14	# Every stage is idempotent: re-running from the top is always safe and is the	14	# Every stage is idempotent: re-running from the top is always safe and is the
15	# documented default. Every remote or destructive action is gated on --target,	15	# documented default. Every remote or destructive action is gated on --target,
16	# and the plane's values come from deploy/server/plane.<target>.env — the	16	# and the plane's values come from deploy/server/plane.<target>.env — the
17	# hostnames, the host-port triple, and which plane gets backups and a purge.	17	# hostnames, the host-port triple, and which plane gets backups.
18	#	18	#
19	# ONE STEP IS UNREHEARSED, DELIBERATELY: the CDN purge runs at prod and only at	19	# A tag publishes once. scripts/release.sh builds byte-reproducibly, so a
20	# prod. stg is served cache-bypassed, so a purge there would invalidate nothing	20	# re-run rebuilds the artifacts the target already serves; stage 3 recognises
21	# and make the prod step look practiced when it is not. Its first real execution	21	# them, leaves the published site image where it is, and carries on. What it
22	# is at promote time. That is why it is kept to a single curl with an explicit	22	# will not do is put different bytes behind a URL the world has already
23	# file list — small enough that an unrehearsed run is safe.	23	# fetched: /dl/<tag>/ is served immutable, so those bytes are the tag, and
		24	# changing them means cutting a new one.
24	#	25	#
25	# The pipeline reads the plane's config Secret and validates it. It never	26	# The pipeline reads the plane's config Secret and validates it. It never
26	# writes one: a script that can write config secrets is a script that can	27	# writes one: a script that can write config secrets is a script that can
@@ -38,7 +39,7 @@ SKIP_SMOKE=0
38	RENDER_ONLY=0	39	RENDER_ONLY=0
39		40
40	usage() {	41	usage() {
41	sed -n '2,30p' "$0" \| sed 's/^#\{1,2\} \{0,1\}//'	42	sed -n '2,32p' "$0" \| sed 's/^#\{1,2\} \{0,1\}//'
42	exit "${1:-1}"	43	exit "${1:-1}"
43	}	44	}
44		45
@@ -233,10 +234,79 @@ fi
233	if [[ "$FROM" -le 3 ]]; then	234	if [[ "$FROM" -le 3 ]]; then
234	bold "3. Build and push images at $TAG"	235	bold "3. Build and push images at $TAG"
235	[[ -d "dist/$TAG" ]] \|\| fail "dist/$TAG missing — run from stage 2"	236	[[ -d "dist/$TAG" ]] \|\| fail "dist/$TAG missing — run from stage 2"
		237
		238	# Set by the publish check below when the plane already serves exactly
		239	# these artifacts, which is the one case where the site image is left
		240	# alone.
		241	site_published=0
		242
		243	# Publish once, per plane. The site image about to be pushed bakes
		244	# dist/$TAG in, so this is the last moment before those bytes become
		245	# https://$SITE_HOST/dl/$TAG/ — a URL served immutable and cached
		246	# accordingly, downloaded by hand and by every agent taking the upgrade.
		247	# Ask the plane what it already serves for this tag and compare.
		248	#
		249	# The two planes legitimately hold different bytes for one tag: the
		250	# artifacts name their own plane's /dl (stage 2), which is what makes the
		251	# stg run a proof rather than a rehearsal. So the comparison is against
		252	# THIS target and no other.
		253	sums_url="https://$SITE_HOST/dl/$TAG/SHA256SUMS"
		254	published="$(mktemp)"
		255	# curl writes 000 for a request that never got a status line; the explicit
		256	# assignment on failure keeps a non-zero exit from appending a second one.
		257	code="$(curl -sS --max-time 30 -o "$published" -w '%{http_code}' "$sums_url")" \|\| code=000
		258	case "$code" in
		259	200)
		260	if cmp -s "dist/$TAG/SHA256SUMS" "$published"; then
		261	echo "publish ok: $TAG is already published at $SITE_HOST with these exact bytes"
		262	site_published=1
		263	else
		264	rm -f "$published"
		265	fail "$TAG is ALREADY PUBLISHED at $SITE_HOST, with different bytes.
		266	$sums_url does not match dist/$TAG/SHA256SUMS. A versioned download URL
		267	is immutable — someone may already hold the published artifacts, and an
		268	agent verifies its upgrade against the sha the manifest named. The fix
		269	is a new tag, never a new payload under the old one.
		270	(The build is reproducible, so an honest re-run of the same tree
		271	matches. A difference means the tag moved or the tree did.)"
		272	fi
		273	;;
		274	404)
		275	echo "publish ok: $TAG is not yet published at $SITE_HOST — this is its first publish"
		276	;;
		277	000)
		278	warn "$SITE_HOST did not answer for $sums_url, so what it serves for $TAG is
		279	unknown and this run publishes over it. Expected while a plane is being
		280	brought up for the first time; anywhere else, check the site is serving
		281	before letting the push proceed."
		282	;;
		283	*)
		284	rm -f "$published"
		285	fail "$sums_url answered HTTP $code, so whether $TAG is already published
		286	cannot be established. Resolve it before publishing over the answer."
		287	;;
		288	esac
		289	rm -f "$published"
		290
236	make -C "$REPO_ROOT" web	291	make -C "$REPO_ROOT" web
237	"$REPO_ROOT/scripts/server-image.sh"	292	"$REPO_ROOT/scripts/server-image.sh"
238	make -C "$REPO_ROOT" site SITE_DIST="dist/$TAG"	293
239	"$REPO_ROOT/scripts/site-image.sh"	294	# A site image already published under this tag is left where it is. Its
		295	# content would be identical — the same dist/$TAG baked into the same
		296	# webroot — but a rebuild lands under a new digest, and the registry tag
		297	# for an already-published release quietly pointing somewhere new is the
		298	# confusion this stage exists to end. Stage 6 rolls the digest already
		299	# behind the tag.
		300	#
		301	# The server image is rebuilt and pushed on every branch. Nothing outside
		302	# the cluster holds a contract on its bytes, and making it conditional
		303	# would only complicate what --from resumes into.
		304	if [[ "$site_published" == "1" ]]; then
		305	echo "site image: skipped — $SITE_HOST already serves /dl/$TAG from the image at this tag"
		306	else
		307	make -C "$REPO_ROOT" site SITE_DIST="dist/$TAG"
		308	"$REPO_ROOT/scripts/site-image.sh"
		309	fi
240	fi	310	fi
241		311
242	# ── 4. Config schema and value check ──────────────────────────────────────────	312	# ── 4. Config schema and value check ──────────────────────────────────────────
@@ -423,63 +493,21 @@ if [[ "$FROM" -le 5 ]]; then
423	done	493	done
424	fi	494	fi
425		495
426	# ── 6. Roll the site, then purge ──────────────────────────────────────────────	496	# ── 6. Roll the site ──────────────────────────────────────────────────────────
427	# ORDER IS LOAD-BEARING. The server fetches the release manifest at boot and	497	# ORDER IS LOAD-BEARING. The server fetches the release manifest at boot and
428	# pins the answer for 24 hours, so a server rolled ahead of its site serves the	498	# pins the answer for 24 hours, so a server rolled ahead of its site serves the
429	# previous release to the whole fleet for a day.	499	# previous release to the whole fleet for a day.
		500	#
		501	# Nothing follows this but the roll. site/nginx.conf states the cache contract
		502	# the site is served under — /dl/v* immutable, everything else no-cache — and
		503	# stage 3 holds the tag to it, so the new URLs were never cached under an older
		504	# answer and the mutable ones are re-fetched on their own.
430	if [[ "$FROM" -le 6 ]]; then	505	if [[ "$FROM" -le 6 ]]; then
431	bold "6. Roll the site to $TAG (before the server)"	506	bold "6. Roll the site to $TAG (before the server)"
432	require_render_vars	507	require_render_vars
433	render site-deployment.yaml \| kubectl apply -f -	508	render site-deployment.yaml \| kubectl apply -f -
434	kubectl -n "$NAMESPACE" rollout status deployment/web --timeout=5m	509	kubectl -n "$NAMESPACE" rollout status deployment/web --timeout=5m
435	echo "site serving $TAG at https://$SITE_HOST/dl/$TAG/"	510	echo "site serving $TAG at https://$SITE_HOST/dl/$TAG/"
436
437	PURGED="not attempted"
438	if [[ "$CDN_PURGE" != "1" ]]; then
439	echo "purge: skipped — $TARGET is served cache-bypassed, so there is nothing to invalidate"
440	PURGED="not applicable at $TARGET"
441	else
442	CF_ENV="${EITRI_CF_ENV:-$HOME/eitri-deploy/pipeline/cf-purge.env}"
443	if [[ ! -f "$CF_ENV" ]]; then
444	if [[ "${ALLOW_MANUAL_PURGE:-0}" == "1" ]]; then
445	warn "no $CF_ENV — purge NOT run. Do it by hand before announcing:
446	every URL under https://$SITE_HOST/dl/$TAG/, plus /dl/latest/manifest.json,
447	/dl/ and /docs/releases/. Serving a stale artifact is how a release breaks."
448	PURGED="DEFERRED — run it by hand"
449	else
450	fail "no $CF_ENV, so the CDN purge cannot run.
451	/dl is served immutable: without a purge the CDN keeps serving the
452	previous bytes. Create the file with CF_ZONE_ID and CF_API_TOKEN, or
453	re-run with ALLOW_MANUAL_PURGE=1 to say out loud that you will do it
454	by hand."
455	fi
456	else
457	# shellcheck disable=SC1090
458	source "$CF_ENV"
459	: "${CF_ZONE_ID:?set in $CF_ENV}" "${CF_API_TOKEN:?set in $CF_ENV}"
460	# An explicit file list, never a zone-wide purge: the tempting
461	# simplification would evict the whole site's cache on every release.
462	purge_urls="$(
463	{
464	find "dist/$TAG" -maxdepth 1 -type f -printf "https://$SITE_HOST/dl/$TAG/%f\n"
465	printf 'https://%s/dl/latest/manifest.json\n' "$SITE_HOST"
466	printf 'https://%s/dl/\n' "$SITE_HOST"
467	printf 'https://%s/docs/releases/\n' "$SITE_HOST"
468	} \| jq -R . \| jq -s .
469	)"
470	# The API takes at most 30 files per call.
471	while read -r batch; do
472	curl -fsS -X POST \
473	"https://api.cloudflare.com/client/v4/zones/$CF_ZONE_ID/purge_cache" \
474	-H "Authorization: Bearer $CF_API_TOKEN" \
475	-H "Content-Type: application/json" \
476	--data "$batch" >/dev/null
477	done < <(jq -c --argjson u "$purge_urls" -n \
478	'[range(0; ($u \| length); 30) as $i \| {files: $u[$i:$i+30]}][]')
479	echo "purge: $(jq 'length' <<<"$purge_urls") URLs invalidated"
480	PURGED="ran"
481	fi
482	fi
483	fi	511	fi
484		512
485	# ── 7. Roll the server ────────────────────────────────────────────────────────	513	# ── 7. Roll the server ────────────────────────────────────────────────────────
@@ -564,7 +592,6 @@ if [[ "$LOCAL_OIDC" == "1" ]]; then
564	fi	592	fi
565	echo "mcp https://$CONSOLE_HOST/mcp (proxied) and https://$API_HOST/mcp"	593	echo "mcp https://$CONSOLE_HOST/mcp (proxied) and https://$API_HOST/mcp"
566	echo "downloads https://$SITE_HOST/dl/$TAG/"	594	echo "downloads https://$SITE_HOST/dl/$TAG/"
567	echo "cdn purge: ${PURGED:-not reached this run}"
568	# An if, not a trailing &&: as the script's last command, a false conditional	595	# An if, not a trailing &&: as the script's last command, a false conditional
569	# would be the whole run's exit status.	596	# would be the whole run's exit status.
570	if [[ "$SKIP_SMOKE" == "1" ]]; then	597	if [[ "$SKIP_SMOKE" == "1" ]]; then