Canon

One file. Many machines. One truth.

Opaque bytes. Durable commits. Consistent reads. Recoverable across machines.

Canon is an experimental, Raft-backed single durable file. It gives applications one logical /data byte sequence across a cluster without requiring them to implement consensus, understand file contents, or depend on a database. Linux FUSE is an optional frontend; the portable API and CLI are the core interface.

Status: pre-alpha engineering prototype. Not yet independently reviewed or qualified for irreplaceable data. See the philosophy and alpha readiness.

---

Implementation and quickstart

Exactly one file: /data. Three fixed Raft voters replicate its bytes. Every node accepts access, the leader orders mutations, and successful mutations wait for durable majority commitment. Linux native FUSE, a CLI, and HTTP are three interfaces to that same file—not three different storage models.

This is an alpha-preparation build, not a qualified alpha release. It retains the small custom Raft implementation. Use disposable or independently backed-up data. Independent review, this build's real-kernel tests, multi-host soak, and OS-crash qualification remain open. See all 30 checklist items and their actual status.

There is no database, object-storage dependency, multi-file API, directory creation, dynamic membership, or new transport. The server uses only Go's standard library. Internal WAL/checkpoint files are private recovery machinery, not extra files in the mounted namespace.

What changed from v2

Full-file snapshots now preserve the applied index/term, bytes/version, and retry results. Automatic log compaction publishes a checked generation before deleting the old one. Lagging replicas can install a snapshot. Retry identifiers have a bounded, explicitly rejectable expiry model. Backup/restore, TLS, separate client/peer credentials, preflight checks, diagnostics, and a repeatable process-failure campaign are included.

The fast path stays native Go, pooled TCP/HTTP, opportunistic group commit, cheap metadata, and log-free quorum-checked reads. Full checkpoints currently run under the node mutex: expect occasional pauses during checkpointing rather than a claim of unchanged tail latency.

Start a fresh local demo

Use a currently supported Go toolchain; the source language level is Go 1.22. No module downloads or CGO are required.

go build -o canon .
./canon version
./canon demo

Leave the demo running. It starts three separate processes on 127.0.0.1:7401..7403, with persistent state under demo-state/. Ctrl-C stops the children; the same command reopens their state. Three processes on one host do not protect against losing that host or its disk.

In another terminal:

./canon replace --text 'hello'
./canon append --text ' world'
./canon read
./canon doctor
./canon status

Commands accept --addr URL[,URL...]. Mutations print their request ID to stderr before sending. Retrying a failed/ambiguous mutation requires the same ID and payload, not a fresh append. Read the retry contract.

Native mount on Arch

With the servers running and fuse3 installed:

mkdir -p mnt1
./canon doctor --mount ./mnt1
./canon mount ./mnt1

The mount stays in the foreground. Another terminal can use:

printf 'hello\n' > mnt1/data
printf 'appended\n' >> mnt1/data
cat mnt1/data

The directory contains only data (plus the usual . and ..). Creating a second file, renaming, or unlinking fails. This is not an atomic-save editor workspace or a general-purpose POSIX filesystem. Run as your normal user, not root. Unmount with Ctrl-C or fusermount3 -u mnt1.

No Python, uv, fusepy, or native C library is needed for this frontend. The older Python adapter remains in adapters/ as a compatibility/testing aid; it now understands retry epochs and optional HTTPS. It is not the primary alpha frontend.

Embedded mount: no local HTTP hop on the leader

Stop the demo before using the default ports. Initialize a new three-voter state root once:

./canon init --dir native-state --cluster my-onefile
mkdir -p mnt1 mnt2 mnt3

Run one command in each of three terminals:

./canon server --id n1 --dir native-state/n1 --cluster my-onefile --mount ./mnt1
./canon server --id n2 --dir native-state/n2 --cluster my-onefile --mount ./mnt2
./canon server --id n3 --dir native-state/n3 --cluster my-onefile --mount ./mnt3

The leader's mount calls its backend directly. Follower mounts route to the leader over persistent TCP. Killing an embedded node kills that node's own mount; it does not imply that surviving mounts should stop working.

server refuses missing identity state unless explicitly bootstrapped. Never use --bootstrap to replace a lost voter in an existing cluster. Recovery rules.

Upgrade from your v2 build

Full stop required. No rolling upgrade or downgrade is supported. Peer probes and state format changed. Keep a stopped copy of the entire old three-node state and the old executable before upgrading.

1. Unmount, stop every old node, and verify none remains running.
2. Copy the old state root to a separate backup destination while stopped.
3. Build this executable and start all three nodes against their original state directories, identical cluster name, IDs, peer URLs, and ports.
4. Check contents and run backup before destructive experiments.

For a default v2 demo, from this extracted source directory:

go build -o canon .
# Replace the path with your actual STOPPED v2 state root.
./canon demo --dir /absolute/path/to/durable-onefile-v2/demo-state

A stopped v2 upgrade, retained retry result, all-node compaction, restart, and new-cluster restore were executed in the CLI smoke test. That is evidence for that scenario, not every possible legacy state. Very large legacy dedup tables have an explicit migration limit; see recovery.

Backup and recovery

While quorum is healthy:

./canon backup --out onefile.backup
./canon inspect-backup --file onefile.backup
./canon diagnostics --out diagnostics.json

Output files must not already exist. Backup is a consistent logical snapshot, includes retry results, and is verified by reading back the saved bytes. It contains the entire file: treat it as sensitive. Diagnostics omit file contents, credentials, raw logs, and endpoint URLs.

Restore into a new directory and cluster identity, never over a live voter:

# Stop/fence the old cluster before reusing its ports.
./canon restore --file onefile.backup --dir recovered-state --cluster recovered-onefile
./canon server --id n1 --dir recovered-state/n1 --cluster recovered-onefile
# Start n2 and n3 in their own terminals with matching IDs/directories.

For different endpoints add the identical --peers map to restore and all server commands. Do not use demo for restored/custom-cluster state; its cluster name is deliberately fixed to onefile-demo.

Offline maintenance, only after stopping the selected node:

./canon compact --dir native-state/n1

Automatic compaction normally makes this unnecessary. There is no command that force-bootstraps a damaged minority as the old cluster.

Multiple hosts and security

Loopback demo HTTP is intentionally unauthenticated. Non-loopback servers require TLS and distinct client and peer credentials. Use the tested TLS setup pattern, protect the state directories, and keep this build off the public Internet. Peer tokens authorize an entire trusted peer set; this is not Byzantine protection or multi-tenant isolation.

server --config PATH reads a strict JSON object of flag names to string values. Explicit command-line flags override it. Use token-file paths, not secrets embedded in JSON. Example configuration.

Tests you can run

go test -race -count=3 -timeout 180s ./...
go vet ./...
./canon exercise
./canon campaign --events 100 --seed 42 --dir campaign-new
python3 tools/release_smoke.py --binary ./onefile

The campaign creates a new disposable cluster; it never points at your current file. It saves events and node logs. The seed controls payloads; fault kinds rotate predictably. OS scheduling is not deterministic.

On Linux with /dev/fuse, run both actual kernel tests:

ONEFILE_FUSE_TEST=1 go test \
  -run '^TestNativeFUSEKernel' -v -count=1 -timeout 120s

The second test exercises surviving mounts during leader isolation and compaction. These tests are opt-in; without the environment variable they skip. The proposed Linux CI mount job treats an absent device as failure, not a successful skip. This build has not been mounted in the current execution environment.

Executed report · Contract · Recovery · Security · Checklist status · Review guide

Limits and known risks

Item	Current bound or behavior
User-visible namespace	One /data file, fixed forever
Logical file	16 MiB, held in memory on each replica
Mutation / FUSE callback data	256 KiB
Voters	Exactly three; fixed identity and peer map
Compaction trigger	2,048 applied entries or 8 MiB of WAL tail
WAL tail guard	128 MiB; never silently overwrite
Snapshot envelope	64 MiB maximum encoded header; full snapshots only
Retry epochs	Rotate after 4,096 new applied protocol-v2 operations; current + previous epoch accepted
Incoming connections	256 per server
Concurrent client handlers / strict-read checks	128 / 64 per node
Uncommitted proposals / snapshot installs	64 / 1 per node
Native mount handles / workers	4,096 / 64 per mount
Disk headroom	doctor checks 256 MiB; no automatic disk reserve allocator
Fault model	Crash/omission/partition, honest peers, sync-respecting storage; not all-copy loss

These are admission limits, not a measured RSS or disk-space guarantee. Snapshot encoding/copying, concurrent large reads, migration records, and temporary generations need extra memory and disk. Automatic compaction cannot make broken storage healthy. A storage write/sync failure poisons that node until controlled restart and diagnosis.

Linux x86-64 is the only runtime executed here. Linux ARM64 and macOS/Windows builds cross-compile; their runtime/recovery behavior is not qualified. Windows lacks a portable directory-sync implementation here, so power-loss durability of checkpoint publication is explicitly not claimed.