Overview
Runs Caddy as a web server and reverse proxy with automatic HTTPS. The block is a singleton that runs where its volume's primary is, behind one VIP that serves both 80 and 443. Caddy's data directory is on the volume: certificates and their keys, the ACME account and Caddy's internal CA. After a failover the new node serves the same certificates instead of requesting new ones, which matters under Let's Encrypt's rate limits.
The block writes the Caddyfile from sites on every start. To write your own, give a whole caddyfile instead.
- Shape
- Singleton on a replicated volume
- Reached at
- VIP on 80 and 443
- Node package
pkgs.caddy
Fetched on demand: the first time a block of this type is placed on a node, its agent fetches pkgs.caddy, built from the node's own nixpkgs so the NixOS binary cache serves it. That first placement needs network access (or a binary cache on the LAN) and waits for the download; later placements on that node start at once.
Example
apiVersion: expanse.io/v1 kind: Block metadata: name: edge namespace: default spec: type: web/caddy replicas: 1 strategy: kind: SINGLETON resources: requests: cpu: 100m memory: 128Mi storage: - name: caddy-data size: 1Gi replication: 3 mountPath: /var/lib/caddy config: email: ops@example.com sites: - host: app.example.com reverseProxy: ["10.0.0.21:8080", "10.0.0.22:8080"] - host: status.example.com tls: "off" respond: ok network: ports: - name: http port: 80 target_port: 18080 protocol: tcp expose: EXPOSE_VIP - name: https port: 443 target_port: 18443 protocol: tcp expose: EXPOSE_VIP health_check: readiness: type: PROBE_TCP port: 18080 period_seconds: 2
$ expanse ctl block apply -f caddy.yaml --dry-run # validate only $ expanse ctl block apply -f caddy.yaml
Configuration
Give either sites or caddyfile.
Keys go under spec.config and are checked against the type's JSON Schema when the block is applied. A default is what the block uses when the key is unset.
| Key | Default | Meaning |
|---|---|---|
sites | required, or caddyfile | Sites served, one host name each. The deploy form lists existing blocks' addresses for the reverse proxy targets. |
sites[].host | required | Host name the site answers for. A wildcard such as *.example.com needs a DNS challenge, so use caddyfile for it. |
sites[].tls | acme | acme: a public certificate; internal: Caddy's own CA; off: plain HTTP only. |
sites[].reverseProxy | one of these two | Upstreams as host:port, round robin. A failing upstream is skipped for 30 s. |
sites[].respond | one of these two | A fixed one-line body, for a placeholder or a health endpoint. |
email | none | ACME account email. |
acmeCA | Let's Encrypt, then ZeroSSL | ACME directory URL, such as Let's Encrypt's staging server. |
port | 18080 | Caddy's HTTP port, the target_port of 80. |
httpsPort | 18443 | Caddy's HTTPS port, the target_port of 443. |
caddyfile | none | A whole Caddyfile, used instead of everything above. Set the ports with the http_port and https_port global options. The data directory is still the volume. |
Using it
Point each site's DNS name at the block's VIP (expanse ctl block get edge). port and httpsPort must differ from the exposed 80 and 443, because the VIP holder binds those on the node Caddy runs on.
A site with tls: acme or tls: internal answers HTTPS, and its plain-HTTP requests get a 308 redirect to https://<host>. ACME's HTTP challenge is answered on port 80, so for acme sites the VIP must be reachable from the internet on 80 or 443. On a private network use tls: internal; clients then need Caddy's root certificate, which is at caddy/pki/authorities/local/root.crt on the volume.
When a node fails
When the serving node dies, the block, its volume's primary and its VIP move together to a survivor, and Caddy starts there with the same data directory. Requests in flight fail and clients reconnect to the same address, which serves the same certificate. A renewal that was under way is done again on the survivor.
Caddy does not sync the files it writes, so the block flushes the volume every 2 s. A certificate issued in the last 2 s before a crash can be lost; the survivor then issues it again. In Expanse's 3-node VM test the VIP was on a survivor 55 s after the serving node was killed, and HTTPS answered again a fraction of a second later with the same certificate.
Limits
- One Caddy serves the sites; there is no shared certificate storage for several instances.
- Caddy's admin API is off, so a configuration change is a redeploy, which restarts Caddy.
- HTTP/3 over UDP: VIPs forward TCP only.
- To serve files, use
web/nginx, or acaddyfilewithfile_server.
Source
- docs/CADDY.md: the Expanse guide this page follows.
- nix/blocks/web/caddy: the type's identity and JSON Schema.