A block is a service described in YAML that the cluster schedules onto nodes and runs as a sandboxed systemd unit. Apply one with expanse ctl block apply -f <file>; add --dry-run to validate it first. Getting started walks through a first block, and Features covers scheduling, rolling updates and self-healing. On a node, expanse ctl catalog list lists the types and expanse ctl catalog show <type> prints a type's schema.
Every type can also be deployed and edited without YAML from the web UI: Workloads → Blocks → Deploy block lists the catalog by category, graying out types no node can run, and builds a form for each from its schema and starting template. Errors show on the field they concern, a pre-flight shows the node each replica would run on (or why no node takes it), and "Show as YAML" gives the same block as a manifest. The keys on each type's page are the settings the form shows.
Fields every block shares
Every block has the same outer shape; only spec.config differs by type.
Field under spec | Meaning |
|---|---|
type | The catalog type, <category>/<name>: one of the types below. |
replicas | How many replicas to run. Omit it for a daemonset; a singleton without it runs one. |
strategy.kind | Unset: active-active replicas. SINGLETON: exactly one instance, fenced by a Raft lease. DAEMONSET: one replica on every node. |
strategy.update | mode (UPDATE_MODE_ROLLING or UPDATE_MODE_RECREATE), maxUnavailable, maxSurge and minReadySeconds. |
resources.requests | cpu and memory the scheduler reserves on the node. A VM uses them as its vCPUs and RAM. |
storage[] | Volumes: name, size, replication, mountPath, and filesystem: none for a raw device. A singleton with storage always runs where its volume's primary is. |
placement | antiAffinity: ANTI_AFFINITY_NODE keeps replicas on different nodes; requiredCapabilities such as [kvm] limits the nodes. |
network.ports[] | name, port, target_port, protocol, and expose: EXPOSE_VIP to put the port on the block's VIP, which forwards to target_port. Several ports share one VIP. The VIP holder binds the exposed port on its node, so a workload on that node must listen on a different target_port. |
network.health_check | readiness and liveness probes, PROBE_TCP or PROBE_HTTP. A port on a VIP needs a readiness probe; traffic skips a replica whose probe fails. A failing liveness probe restarts the replica, then moves it. |
config | The type's own settings, checked against its schema. Each type's page lists them. |
Block secrets
Passwords, tokens and secret keys in config, the fields a type's schema marks writeOnly, are taken out of the block before it is stored, whether they come from the web UI's form or from expanse ctl block apply. Each is sealed with age, keyed from the cluster secret, and the block keeps a spec.secrets reference in its place. The values are not in block get, the web UI, generation diffs or the replica's spec file; the node running a replica hands them to it as systemd credentials.
- Leaving a field out of an update keeps its value; an empty string removes it. In the web UI, an empty password field keeps it too.
- A changed value restarts the replicas with it. Applying the same value again restarts nothing.
- On Deploy, the web UI generates any required password left empty and shows it once.
- Old values stay sealed until the block is deleted, so a generation rollback can restore them. Deleting a block deletes its secrets.
- Blocks created before Expanse sealed secrets keep their passwords in their config until their next update, which seals them.
util/restic-backup'senvmap is not sealed yet.
Programs on the node
Most types run an upstream program, such as postgres or caddy. Nodes do not carry every type's programs, which together are about 5 GiB. vm/instance's qemu is built into every node; any other type's programs are fetched from the NixOS binary cache the first time a block of that type is placed on a node, so that first placement needs network access and waits for the download. util/echo, web/whoami and web/static-site are built into Expanse and need nothing.
| Type | Programs fetched on first placement |
|---|---|
web/caddy | pkgs.caddy |
net/haproxy | pkgs.haproxy |
web/nginx | pkgs.nginx |
web/static-site | None: built in |
web/whoami | None: built in |
db/postgres | pkgs.postgresql_18 |
db/mariadb | pkgs.mariadb |
db/redis | pkgs.redis |
share/nfs | pkgs.nfs-ganesha |
storage/s3 | pkgs.garage |
iscsi/target | pkgs.targetcli-fb |
share/smb | pkgs.samba |
vm/instance | Built in (headless qemu) |
monitor/prometheus | pkgs.prometheus |
monitor/grafana | pkgs.grafana and Expanse's packages.grafana-plugins |
monitor/node-exporter | pkgs.prometheus-node-exporter |
monitor/uptime-kuma | pkgs.uptime-kuma |
dev/forgejo | pkgs.forgejo |
security/vaultwarden | pkgs.vaultwarden and pkgs.vaultwarden.webvault |
media/jellyfin | pkgs.jellyfin |
media/immich | pkgs.immich, pkgs.immich.machine-learning, pkgs.redis and Postgres 18 with VectorChord |
util/restic-backup | pkgs.restic |
util/echo | None: built in |
ai/ollama | pkgs.ollama |
Web and proxies
web/caddy
Caddy with automatic HTTPS; its certificates, ACME account and CA fail over on a replicated volume.
Configuration and usagenet/haproxy
HAProxy balancing TCP and HTTP over health-checked servers, as stateless replicas behind a VIP.
Configuration and usageweb/nginx
nginx serving one virtual host from a directory, as stateless replicas.
Configuration and usageweb/static-site
A built-in Go file server that serves one configured HTML page.
Configuration and usageweb/whoami
Answers each request with its replica number, to watch load balancing.
Configuration and usageDatabases
db/postgres
PostgreSQL replicas on their own volumes, streaming replication, and live promotion behind one VIP.
Configuration and usagedb/mariadb
One MariaDB server on a replicated volume behind a VIP; every acknowledged commit survives a failover.
Configuration and usagedb/redis
Independent Redis servers used as a cache; nothing is kept across restarts.
Configuration and usageStorage and file sharing
share/nfs
One NFSv4.1/4.2 export served by NFS-Ganesha; clients reclaim their opens and locks after a failover.
Configuration and usagestorage/s3
An S3-compatible object store on Garage, with metadata, objects and keys on a replicated volume.
Configuration and usageiscsi/target
A raw replicated volume exported as one iSCSI LUN through LIO; initiators reconnect on their own after a failover.
Configuration and usageshare/smb
A Samba SMB3 share on a replicated volume. Deploys, but its failover has an open bug.
Configuration and usageVirtual machines
vm/instance
A QEMU/KVM guest on a raw replicated disk; node loss is a cold boot elsewhere at the same address.
Configuration and usageMonitoring
monitor/prometheus
Prometheus scraping every node and evaluating Expanse's shipped alert rules, its TSDB on a replicated volume.
Configuration and usagemonitor/grafana
Grafana with the Expanse Prometheus datasource and cluster-health dashboard set up.
Configuration and usagemonitor/node-exporter
The Prometheus node exporter on every node, as a daemonset.
Configuration and usagemonitor/uptime-kuma
Uptime Kuma with its monitors, history and SQLite database on a replicated volume behind a VIP.
Configuration and usageDeveloper tools
dev/forgejo
Forgejo with its repositories, database, SSH host key and secrets on a replicated volume behind a VIP.
Configuration and usageSecurity
security/vaultwarden
Vaultwarden, a Bitwarden-compatible password manager, with its vaults and database on a replicated volume behind a VIP.
Configuration and usageMedia
media/jellyfin
Jellyfin with its library database and settings on a replicated volume behind a VIP, streaming media every node mounts.
Configuration and usagemedia/immich
Immich with its uploads, Postgres database and job queue on a replicated volume behind a VIP.
Configuration and usageUtilities
util/restic-backup
Backs up block volumes on a schedule from crash-consistent snapshots, following each volume's primary.
Configuration and usageutil/echo
Answers every HTTP request with a configured body. Runs on any node.
Configuration and usageAI
ai/ollama
The Ollama LLM runtime. Part of the paused on-prem LLM subsystem.
Configuration and usage