docs/superpowers/specs/2026-09-05-server-authoritative-collab-design.md

Ref: main   Size: 15.3 KiB   History

# Server-Authoritative Collaboration — Design & Roadmap

Date: 2026-09-05
Status: Draft

## Purpose

Retire the distributed collab-over-refs machinery and make the git-collab
server the single authority for live collaboration state: issues, comments,
patches, reviews, and a new primitive, work leases. Git remains the store for
code and for patch revision commits; SQLite on the server becomes the store
for everything that is *state* rather than *content*.

The driving use case: agents in ephemeral VMs (provisioned by eitri, reachable
via mux) that claim work, do it, and participate in review — plus humans doing
the same over the web UI, CLI, and IRC. The current substrate cannot serve it:

- **Claiming work needs arbitration.** Git's only compare-and-swap is the ref
  update on push. A claim today is fetch → append event → push → maybe
  rejected → refetch → replay → discover you lost. Seconds of latency and a
  lottery, not a lock.
- **Review back-and-forth needs subscription.** Git has no watch primitive;
  the latency floor for a conversation is the polling interval plus a full
  fetch/reconcile/push cycle per utterance.
- **Ephemeral workers can't afford the bootstrap.** Every fresh VM pays
  clone + `init-key` + trust-allowlist distribution + refspec setup before it
  can say one word.
- **The consistency model is design debt by our own admission.** Concurrent
  status events resolve by lexicographic timestamp comparison
  (`docs/design-debt.md`): forgeable, clock-skewed, non-deterministic on ties.
- **Signed events freeze the schema forever.** Every historical `event.json`
  shape is load-bearing permanently (see the `PatchCreate.commit` history in
  `src/event.rs`). A server-owned store migrates; a signed DAG cannot.
- **Per-repo ref islands cannot cross-reference.** An issue about mux filed
  from inside eitri's repo has no home. One forge, one database makes a
  cross-repo link a row with two foreign keys.

All five problems share one cause: the refs substrate makes git act as a
replicated multi-writer database. Roughly 7.3k lines (`sync.rs`, `state.rs`'s
DAG replay, `trust.rs`, `dag.rs`, `sync_lock.rs`, `merge_scan.rs`'s sync-time
half, the signed-event plumbing) exist to prop up that role. With a server we
already run, every one of those problems has a boring, standard answer.

## The design test, inverted

The 2026-08-09 patch-revision-refs design stated:

> If a feature only works when the server is `git-collab-server`, it is not a
> git-collab feature.

That test is hereby retired. It was the honest formulation of the original
bet — collaboration between any two clones with no server — and the bet's
value did not survive contact with the actual use case. The replacement test:

> **Contributing code must require nothing but plain git. Everything else may
> require the server.**

Gerrit is the precedent for the whole shape: it stores review data in git
(NoteDb, `refs/changes/*`) but behind a single server that is the only writer
and the arbiter of all state. Git as *storage* was never the problem; git as
the *multi-writer sync protocol* was. This design keeps the Gerrit-style data
model already built (numbered revisions, revision-anchored comments,
interdiffs, timelines, trailer-recorded merges) and relocates authority.

## Decision ledger

| Concern | Old home | New home |
|---|---|---|
| Code, branches, tags | git | git (unchanged) |
| Patch revision commits | git, refs written by clients + sync | git, refs written **by the server** on push |
| Issues, comments, reviews, status | signed event DAG in `refs/collab/*` | SQLite on the server |
| Work claims | (impossible) | SQLite lease rows, atomic server operation over SSH exec |
| Merge recording | sync-time scan on every clone | push-time scan on the server |
| Auth / identity | trust allowlist file, distributed by hand | server-side `authorized_keys`; agents are keypairs authorized once; SSH principal = identity |
| Conflict resolution | timestamp-wins DAG replay | none needed — the DB serializes writes |
| Event schema | frozen forever (signed) | migratable (DB), versioned (webhooks) |
| Portability of the record | live sync of `refs/collab/*` | optional one-way `export` to `refs/collab/*` as an archival artifact |
| Notifications | none (poll `fetch`) | webhooks (machines) + IRC bridge (humans) |

## Architecture

### One event bus, three projections

Every mutation on the forge emits one internal event — `issue.filed`,
`issue.claimed`, `patch.created`, `revision.pushed`, `review.posted`,
`lease.expired`, `merge.recorded` — with three consumers:

1. **SQLite** — the authoritative record.
2. **Webhooks** — the machine feed. `POST` to subscriber URLs (triage bot,
   foreman, CI). Payloads reuse the `event.rs` vocabulary, versioned and
   evolvable now that nothing signs them into permanence. Payloads are signed
   with the forge's Ed25519 key (`signing.rs`, relocated) so subscribers can
   verify origin — GitHub's HMAC pattern with the keys we already believe in.
3. **IRC bridge** — the human feed. Announcements mirrored into channels;
   scrollback becomes an ambient audit trail.

Rule that keeps the seams clean: **machines talk to the forge directly
(SSH exec in, webhooks out), humans talk IRC, and
the bridge translates only at the edges.** IRC carries intent only when a
human is speaking (DM intake, explicit commands like `claim`). No
machine-originated action ever rides the channel; automation consumes
webhooks. IRC is intake and awareness, never correctness, transport, or
record.

### Leases

A claim is one conditional update:

```sql
UPDATE tasks
SET lease_holder = :agent, lease_token = lease_token + 1,
    lease_expires = :now + :ttl
WHERE id = :task
  AND (lease_holder IS NULL OR lease_expires < :now);
```

One row affected = claimed; zero = lost the race. The database's write
serialization is the arbiter; no lock service.

**Transport and auth: SSH exec verbs, not HTTP.** The server already has an
authenticated command channel — the `collab-release` pattern: the client
shells out to `ssh`, `auth_publickey` maps the key to a principal, and
per-repo policies authorize the verb. Leases follow it (`collab-lease
acquire '<repo>' '<issue>' [--ttl <secs>]`), and `lease_holder` is the SSH
principal. This sets the rule for every later phase: **mutations ride SSH
(authenticated by key), HTTP stays read-only (web UI + git data), webhooks
are outbound and signed.** One auth system for humans and agents alike — an
agent's keypair is an `authorized_keys` line, and it already needs that key
to clone. If a non-SSH client ever matters, mint bearer tokens over SSH
(`collab-token new`); not before.

- **TTL + heartbeat.** Agents die mid-task (VM reaped, OOM, wedged). Leases
  expire on their own; live holders renew (`collab-lease renew`). Humans
  get an open-ended lease that renders as "assigned" — one mechanism, two
  tenure policies.
- **Fencing tokens.** `lease_token` increments on every acquire. Every write
  a holder makes (revision push, status change, completion) carries its
  token; the forge rejects stale tokens. Expiry alone protects liveness;
  fencing protects correctness against zombie workers waking up post-expiry.
- **Idempotent acquire.** Re-claiming a lease you already hold succeeds, so a
  client retrying after a dropped connection does not deadlock against
  itself.

Scale is a non-issue by construction: a lease op costs ~1ms of forge time
against tasks costing minutes of VM work, and contention is per-task (N racers
= one winner, N−1 cheap 409s). The pattern is what GitHub Actions runners,
Kubernetes `Lease` objects, and Buildkite agents all do. The thing that
actually fails to scale is polling, and the architecture is push-based
(webhooks/IRC announce; agents claim on signal, with jitter).

### Patch flow (Gerrit-style, magic ref)

- `git push origin HEAD:refs/for/<base>` — the server's receive hook creates
  a patch (or matches an existing one by the `Patch:` trailer), pins the
  commit as the next revision under `refs/collab/patches/<id>/rev/<oid>`,
  links `--fixes` issues, and prints the patch URL in the push response.
  Plain git is the only client requirement — this is the new design test
  holding.
- The existing `commit-msg` hook (`hooks.rs`) is our Change-Id equivalent:
  it stamps the `Patch:` trailer at commit time, and amends carry it. It
  survives verbatim.
- `git-collab patch create/revise` remain as CLI conveniences over the same
  server operations (set title/body without the web UI).
- Local branches are private and never leave the machine. The patch is the
  unit; revisions are immutable commits; comments anchor to the revision they
  were written on; interdiff works exactly as today.
- Merge: land the commit with its `Patch:` trailer however you merge
  (including squash). The server scans at push time (`merge_scan.rs`
  relocated from sync time) and records the merge. `patch merge` stays as
  the manual fallback.

### Identity

One keypair per principal, human or agent. Humans: the SSH keys the server
already authorizes. Agents: a keypair minted (or injected) at VM boot,
authorized once on the forge; every clone, push, comment, and claim it makes
is attributed to that key. Provenance chains are recorded where actions are
brokered: an issue filed by the triage bot from a DM reads *filed by triage,
on behalf of alex, via DM*.

### Cross-repo

Issues belong to a repo; references are qualified (`mux#4f2a91`). `relates_to`
generalizes to cross-repo foreign keys in the one database. A worker that
finds a side-issue in another repo files it (one exec verb / API call),
links it, and stays on task — the default policy is **file, link, continue**;
the lease system keeps workers honest about what they claimed.

## Companion pieces (out of this repo's scope)

Per the standalone-tools philosophy, each seam lands as a feature of the tool
it belongs to, and the "platform" remains a personal composition. The glue
daemons are designed in the **`workshop` repo** (`~/code/rad/workshop`):
`heimdall` (IRC bridge) and `durin` (foreman), plus how `brokkr` (the LAN
LLM gateway) provisions workers with model access:

- **eitri: job mode.** Boot a VM, inject key + task context, run, report,
  tear down. Useful to anyone doing CI/sandboxed builds on their own
  hardware, independent of this forge.
- **heimdall + durin: personal glue, unshipped** (see `workshop`). heimdall
  mirrors forge events into IRC and translates human commands/DMs into API
  calls. durin is a dumb elasticity loop: unclaimed `agent`-labeled issues >
  idle workers → ask eitri for a VM (up to a cap); idle worker > N minutes →
  reap. Lease TTLs already handle worker death, so recruitment can afford to
  be naive.
- **Triage: a webhook subscriber**, not a forge feature. Dedup, labels,
  severity, clarifying questions — all ordinary API calls.

Nothing in this repo may depend on any of these existing.

## Codebase impact

**Survives, largely intact** — the parts that were never the problem:

- `src/server/` — SSH, HTTP, repos, releases, governance, web. Promoted from
  "optional" to the center of the product.
- The Gerrit reconstruction: `patch.rs` diff/interdiff, `timeline.rs`,
  revision refs (now server-written), TUI and web rendering.
- `merge_scan.rs` (relocated to push time), `hooks.rs` (unchanged role).
- The CLI: same verbs, thin client over SSH exec (the `release.rs` pattern)
  instead of a local ref-writer.
- `event.rs` vocabulary: becomes the webhook payload schema.
- `signing.rs`: signs webhook payloads; agent keys remain Ed25519.

**Dies** — the replicated-database tax (~6k lines):

- `sync.rs`, `sync_lock.rs`, `dag.rs`, `trust.rs`.
- The DAG-replay materializers in `state.rs` (become SQL queries).
- Timestamp-wins conflict resolution and its design-debt entry.
- Signed event-trees as the live storage format, and with them the
  frozen-schema archaeology.

**New, all small:**

- SQLite schema: issues, comments, patches, revisions, reviews, leases,
  webhook subscriptions.
- The lease endpoints (~50 lines + fencing checks on write paths).
- The SSH exec verbs for issues/comments/reviews/leases (the mutation API).
- `refs/for/<base>` receive-hook handling.
- The event bus + webhook dispatcher.
- `git-collab export`: one-way materialization of the record into
  `refs/collab/*` so "clone carries the conversation" survives as an
  archival feature rather than as the transport.

## Roadmap

Each phase is independently shippable and gets its own implementation plan
(in `docs/superpowers/plans/`) when picked up. Order matters: every phase is
useful the day it lands, and none blocks on the companions.

1. **Leases.** ✅ **Done** (`docs/superpowers/plans/2026-09-05-issue-leases.md`).
   SQLite lease store, `collab-lease` SSH exec verb (acquire/renew/release/
   list), `issue claim|unclaim|renew|claims` on the CLI, claims shown in the
   web UI. Transport is the SSH verb; no new auth machinery. Fencing tokens
   are stored and reported but not yet *enforced* — nothing server-mediated
   exists to fence until phase 3. (The seam the whole agent story hangs on,
   and the one thing git structurally cannot express.)
2. **Issues and comments to SQLite.** Exec verbs + web UI read/write the
   DB; a comment becomes one round trip. Kills the review burden. Includes
   a one-time importer that replays existing `refs/collab/*` DAGs into the
   DB (the current `state.rs` materializer, run once, then retired).
3. **Server-maintained revision refs + `refs/for/<base>`.** The receive hook
   creates/updates patches; push-time merge scanning. Kills the last
   client-side collab-ref writes. CLI verbs become API calls.
4. **Event bus + webhooks + IRC bridge hook points.** Signed payloads,
   subscription management. Unblocks triage/foreman/CI as external
   subscribers.
5. **Export and burial.** `git-collab export`; delete `sync.rs`,
   `sync_lock.rs`, `dag.rs`, `trust.rs`, the `state.rs` replay, and the
   design-debt entry. Update README: the pitch becomes *the lightest forge
   with real code review*.

## Non-goals

- **Offline multi-writer collaboration.** The property this design
  deliberately gives up. The record stays portable (export); the *transport*
  stops being git.
- **Nostr / NIP-34 / federation.** The relay-authoritative model here is
  compatible in spirit (Buzz reaches the same conclusions), and the event
  vocabulary could become kinds later. Not now; the forge API is smaller and
  we own both ends.
- **Multi-node forge.** SQLite and one process outlive any realistic personal
  fleet by orders of magnitude. Revisit at ~10⁴ concurrent claimants, i.e.
  never.
- **IRC as a dependency.** The forge must be fully usable with no bridge and
  no bot.

## Open questions

- **Human lease semantics.** Open-ended lease vs. long TTL with soft nag?
  Leaning open-ended; an unassign is a manual act either way.
- **Agent key authorization flow.** Foreman-signed enrollment vs. manual
  authorize-once per agent identity. Start manual; automate when it hurts.
- **Export format.** Reuse today's signed event-tree layout (readable by
  existing tooling) vs. a simpler unsigned JSON log. Leaning: keep the tree
  layout, sign with the forge key — provenance without frozen schemas.
- **What happens to `refs/collab/*` data in repos that never import?** The
  importer is per-repo and opt-in; old refs stay readable by old binaries.
- **TUI scope.** Point the dashboard at the API. Cheap if the API mirrors
  today's read model; decide during phase 2.