Blocks · AI

ai/ollama

Ollama (paused). The Ollama LLM runtime. Part of the paused on-prem LLM subsystem.

Overview

Runs ollama serve on port. The on-prem LLM subsystem this type belongs to is paused, because no accelerator hardware has been available to validate it, so the type does less than its schema suggests.

Paused. Models are not pulled and are not kept: the runtime stores them in a directory private to the unit, which is discarded when it stops.

Shape
Replicas
Reached at
Node port
Node package
pkgs.ollama

Fetched on demand: the first time a block of this type is placed on a node, its agent fetches pkgs.ollama, built from the node's own nixpkgs so the NixOS binary cache serves it. That first placement needs network access (or a binary cache on the LAN) and waits for the download; later placements on that node start at once.

Example

ollama.yaml
apiVersion: expanse.io/v1
kind: Block
metadata:
  name: llm
  namespace: default
spec:
  type: ai/ollama
  replicas: 1
  resources:
    requests:
      cpu: 2000m
      memory: 8Gi
  config:
    port: 11434
$ expanse ctl block apply -f ollama.yaml --dry-run   # validate only
$ expanse ctl block apply -f ollama.yaml

Configuration

Keys go under spec.config and are checked against the type's JSON Schema when the block is applied. A default is what the block uses when the key is unset.

KeyDefaultMeaning
portnone: always set itPort of the Ollama API. Unset, Ollama takes a random free port. The deploy form sets 21434 and puts 11434 on the VIP.
modelsnoneModel names to pull on start, per the schema; not applied. Pull models through the API instead.
keepAlivenoneModel keep-alive; not applied.

Using it

$ curl http://<node address>:11434/api/tags
$ curl http://<node address>:11434/api/pull -d '{"model": "<model>"}'

When a node fails

A failed node's replica is placed again elsewhere and starts with no models.

Limits

  • Paused: no GPU validation.
  • Models are lost when the replica stops.

Source

All block types · Fields every block shares