Block reference

Every block type Expanse ships.

Expanse 1.19.1 ships twenty-four block types. Each page covers what the block runs, an example to start from, every configuration key with its default, how to use it, and what happens when a node fails.

A block is a service described in YAML that the cluster schedules onto nodes and runs as a sandboxed systemd unit. Apply one with expanse ctl block apply -f <file>; add --dry-run to validate it first. Getting started walks through a first block, and Features covers scheduling, rolling updates and self-healing. On a node, expanse ctl catalog list lists the types and expanse ctl catalog show <type> prints a type's schema.

Every type can also be deployed and edited without YAML from the web UI: Workloads → Blocks → Deploy block lists the catalog by category, graying out types no node can run, and builds a form for each from its schema and starting template. Errors show on the field they concern, a pre-flight shows the node each replica would run on (or why no node takes it), and "Show as YAML" gives the same block as a manifest. The keys on each type's page are the settings the form shows.

Fields every block shares

Every block has the same outer shape; only spec.config differs by type.

Field under specMeaning
typeThe catalog type, <category>/<name>: one of the types below.
replicasHow many replicas to run. Omit it for a daemonset; a singleton without it runs one.
strategy.kindUnset: active-active replicas. SINGLETON: exactly one instance, fenced by a Raft lease. DAEMONSET: one replica on every node.
strategy.updatemode (UPDATE_MODE_ROLLING or UPDATE_MODE_RECREATE), maxUnavailable, maxSurge and minReadySeconds.
resources.requestscpu and memory the scheduler reserves on the node. A VM uses them as its vCPUs and RAM.
storage[]Volumes: name, size, replication, mountPath, and filesystem: none for a raw device. A singleton with storage always runs where its volume's primary is.
placementantiAffinity: ANTI_AFFINITY_NODE keeps replicas on different nodes; requiredCapabilities such as [kvm] limits the nodes.
network.ports[]name, port, target_port, protocol, and expose: EXPOSE_VIP to put the port on the block's VIP, which forwards to target_port. Several ports share one VIP. The VIP holder binds the exposed port on its node, so a workload on that node must listen on a different target_port.
network.health_checkreadiness and liveness probes, PROBE_TCP or PROBE_HTTP. A port on a VIP needs a readiness probe; traffic skips a replica whose probe fails. A failing liveness probe restarts the replica, then moves it.
configThe type's own settings, checked against its schema. Each type's page lists them.

Block secrets

Passwords, tokens and secret keys in config, the fields a type's schema marks writeOnly, are taken out of the block before it is stored, whether they come from the web UI's form or from expanse ctl block apply. Each is sealed with age, keyed from the cluster secret, and the block keeps a spec.secrets reference in its place. The values are not in block get, the web UI, generation diffs or the replica's spec file; the node running a replica hands them to it as systemd credentials.

  • Leaving a field out of an update keeps its value; an empty string removes it. In the web UI, an empty password field keeps it too.
  • A changed value restarts the replicas with it. Applying the same value again restarts nothing.
  • On Deploy, the web UI generates any required password left empty and shows it once.
  • Old values stay sealed until the block is deleted, so a generation rollback can restore them. Deleting a block deletes its secrets.
  • Blocks created before Expanse sealed secrets keep their passwords in their config until their next update, which seals them.
  • util/restic-backup's env map is not sealed yet.

Programs on the node

Most types run an upstream program, such as postgres or caddy. Nodes do not carry every type's programs, which together are about 5 GiB. vm/instance's qemu is built into every node; any other type's programs are fetched from the NixOS binary cache the first time a block of that type is placed on a node, so that first placement needs network access and waits for the download. util/echo, web/whoami and web/static-site are built into Expanse and need nothing.

TypePrograms fetched on first placement
web/caddypkgs.caddy
net/haproxypkgs.haproxy
web/nginxpkgs.nginx
web/static-siteNone: built in
web/whoamiNone: built in
db/postgrespkgs.postgresql_18
db/mariadbpkgs.mariadb
db/redispkgs.redis
share/nfspkgs.nfs-ganesha
storage/s3pkgs.garage
iscsi/targetpkgs.targetcli-fb
share/smbpkgs.samba
vm/instanceBuilt in (headless qemu)
monitor/prometheuspkgs.prometheus
monitor/grafanapkgs.grafana and Expanse's packages.grafana-plugins
monitor/node-exporterpkgs.prometheus-node-exporter
monitor/uptime-kumapkgs.uptime-kuma
dev/forgejopkgs.forgejo
security/vaultwardenpkgs.vaultwarden and pkgs.vaultwarden.webvault
media/jellyfinpkgs.jellyfin
media/immichpkgs.immich, pkgs.immich.machine-learning, pkgs.redis and Postgres 18 with VectorChord
util/restic-backuppkgs.restic
util/echoNone: built in
ai/ollamapkgs.ollama

Web and proxies

web/caddy

Caddy with automatic HTTPS; its certificates, ACME account and CA fail over on a replicated volume.

Singleton on a replicated volume

Configuration and usage

Databases

db/mariadb

One MariaDB server on a replicated volume behind a VIP; every acknowledged commit survives a failover.

Singleton on a replicated volume

Configuration and usage

Storage and file sharing

share/nfs

One NFSv4.1/4.2 export served by NFS-Ganesha; clients reclaim their opens and locks after a failover.

Singleton on a replicated volume

Configuration and usage

storage/s3

An S3-compatible object store on Garage, with metadata, objects and keys on a replicated volume.

Singleton on a replicated volume

Configuration and usage

iscsi/target

A raw replicated volume exported as one iSCSI LUN through LIO; initiators reconnect on their own after a failover.

Singleton on a raw replicated volume

Configuration and usage

Virtual machines

vm/instance

A QEMU/KVM guest on a raw replicated disk; node loss is a cold boot elsewhere at the same address.

Singleton on a raw replicated volume

Configuration and usage

Monitoring

Developer tools

dev/forgejo

Forgejo with its repositories, database, SSH host key and secrets on a replicated volume behind a VIP.

Singleton on a replicated volume

Configuration and usage

Security

Media

media/jellyfin

Jellyfin with its library database and settings on a replicated volume behind a VIP, streaming media every node mounts.

Singleton on a replicated volume

Configuration and usage

Utilities

AI