Expanse turns ordinary hardware — the design target is a decade-old laptop — into a node of a self-healing cluster: replicated storage, scheduled services, VIP failover and a live web console. Built on NixOS from proven parts, not hand-rolled distributed systems.
:8443. Right: the read-only host console every installed node shows on tty1.
Illustrative mock-ups rendered in HTML, not screenshots.
One bootable ISO installs a node. Nodes find each other, join with single-use tokens, and share one strongly consistent control plane. Everything below ships in 1.1.9 and is exercised by NixOS VM tests that kill nodes, partition networks and pull disks.
A hybrid UEFI/BIOS ISO with a full-screen installer TUI or a one-file unattended mode. Declarative btrfs + LVM layout, an impermanent root that is wiped on every boot, and a two-disk mirror layout that boots from either disk alone.
Installer detailsRaft-replicated state over mTLS, linearizable reads by default, single-writer leases as the anti-split-brain primitive, witness nodes for 2+1 topologies, and a read-only degraded mode when a node cannot reach a leader.
ClusteringDRBD 9 on LVM thin pools: create, resize, snapshot and restore, online repair and verified split-brain recovery. A volume starts on one node and grows to its replication target as nodes join, one fully synced replica at a time.
StorageDeclare a service, its replicas, resources and placement; the scheduler runs it as hardened systemd units. Rolling updates, anti-affinity, singleton and daemonset placement, cross-node logs, and explain when something will not place.
A WireGuard overlay between every node, lease-fenced VIPs that move with their service, L4 and L7 balancing, cluster DNS for <block>.<ns>.expanse.local, and a default-deny nftables firewall per node.
Every fleet-wide configuration change is a generation: diff any two, roll back behind a confirm. Upgrade one node at a time with no quorum loss; a switch watchdog rolls a node back automatically if a switch goes wrong.
UpgradesEvery node serves the UI on :8443: dashboard, cluster, nodes, health, blocks, volumes, generations and events, updating over Server-Sent Events. Password login plus optional OIDC single sign-on with a fail-closed allow-list.
A Prometheus /metrics endpoint on every node, a shipped alert-rule file and Grafana dashboard, and an in-UI health page that derives the same critical alerts with no external tooling at all.
A per-cluster CA with zero-downtime rotation, HMAC single-use join tokens, mTLS on every internal port, age-encrypted secrets, and a browser-facing UI CA that is separate from the cluster's own.
SecurityOne installed node is already a usable cluster. Each step below is the real command, not a sketch.
Write the ISO to a USB stick and boot the target. The installer picks disks, network and an SSH key, shows what it will destroy, then installs. Headless? SSH in and run it there.
$ nix build github:team-expanse/expanse/v1.1.9#iso $ sudo dd if=result/iso/*.iso of=/dev/sdX bs=4M # boot the target from the stick, then: $ expanse install --tui
Bootstrap the first node as the sole voter. It prints a single-use join command; run it on each further node. Volumes and blocks gain replicas automatically as nodes join.
$ systemctl stop expansed $ expanse cluster init --expect 1 cluster initialized: 5c0b… (name "expanse") Join command (single-use, 15m): expanse cluster join --address 10.0.0.11:7446 \ --token expanse-join-… $ systemctl start expansed
Open the web console on any node, or use the CLI: apply a block, create a volume, watch health. Everything the UI does, the CLI does too — it is one control plane with two ways in.
$ expanse ctl block apply -f web.yaml created default/web $ expanse ctl volume create pg --size 10Gi $ expanse cluster status quorum: 3/2 # 3 voters, 2 needed
Writes are forwarded to the Raft leader, reads are linearizable by default, and every action the web console offers is a CLI verb underneath. When a block will not place, explain tells you why each node was rejected.
block apply is a no-op when nothing changed and a rolling update when something did.block logs default/web --replica 1 streams from whichever node runs it.$ expanse cluster status cluster: expanse (5c0b3e2a-…) version: 1.1.9 generation: 12 leader: n1 quorum: 3/2 nodes: 3 ID ROLE STATE RAFT API n1 voter leader 10.0.0.11:7444 10.0.0.11:7443 n2 voter voter 10.0.0.12:7444 10.0.0.12:7443 n3 voter voter 10.0.0.13:7444 10.0.0.13:7443 $ expanse ctl block apply -f web.yaml --dry-run # validated, nothing written $ expanse ctl block apply -f web.yaml created default/web $ expanse ctl block explain web web: 3/3 placed (phase RUNNING) $ expanse ctl volume list ID NAME SIZE STATE REPLICAS NODES (PRIMARY) vol-… pgdata 10Gi healthy 3/3 n1,n2,n3 (n1) $ expanse doctor storage --vg expanse CHECK STAT DETAIL drbd-module PASS loaded, version 9.2.14 volume-group PASS expanse: 41.3% free of 943718400000 bytes thin-pools PASS 1 pool(s) healthy: pool system-mirror PASS md md127 RAID1 across 2 devices drbd-resources PASS 1 resource(s) healthy
The design rule is simple: adopt a proven component for the data plane, build only the control plane that ties them together. That keeps the whole system small enough to reason about — and to test with real failures.
| Concern | Adopted component | What Expanse adds |
|---|---|---|
| Host OS and upgrades | NixOS, systemd-boot | Generations, rollback, a switch watchdog, an impermanent root |
| Cluster state | hashicorp/raft, memberlist | Leases, generations, join protocol, mTLS CA with rotation |
| Volumes | DRBD 9, LVM thin, btrfs, md RAID1 | Placement, repair, split-brain resolution, grow-as-you-join |
| Services | systemd units with sandboxing | A scheduler with rolling updates, anti-affinity, singleton leases |
| Network | WireGuard, nftables, miekg/dns | Mesh reconciliation, lease-fenced VIPs, L4/L7 balancing, cluster DNS |
| Backup and metrics | restic, Prometheus, Grafana | Consistent snapshot paths, one-command restore, shipped rules and dashboard |
Expanse's code, tests and documentation, and this website, were built by a human maintainer working with AI coding assistants. The AI wrote much of the code and prose. The maintainer directs the work and decides what ships.
Co-Authored-By trailer. Read the commit log.The quickstart takes a blank machine to a one-node cluster with a web console, and shows how each further node joins with a single command.