Tutorial 4 Rig

← 3. Your first application · Tutorial index · 5. Build a Distribution + publish to S3 →

Chapter 4 — Write and shape `rig.py`

Your .art says what the system is. A rig says where it runs — which machines exist and which processes land on each. This is the deployment step: assigning your FCs into the supervised tree, per machine.

A rig is a small Python file (manifest/<target>/rig.py) that builds a DeploymentLayer using an algebra of layers. You start from a shared base and apply your target's transform.

4.1 The model: layers + the Append/Remove algebra

A deployment is composed of orthogonal layers that you combine:

MachineSetLayer — the boards (MachineLayers: name, arch, os, cores, whether it hosts etcd, …).
ExecutionLayer — process placement (ProcessLayers: which machine each process runs on).
ApplicationSetLayer — the applications (your cluster), each bound to a host machine.

You combine layers with .combine(...), and within a layer you edit the set with Append(...) / Remove(...) and pin a field with Explicit(...). This monoid algebra lets one base shape serve several targets (a single rig vs a split rig) with small, readable overrides.

Reference: references/manifest-py-syntax.md in the Theia skill has the full algebra. The in-repo demo/manifest/{single,split}/rig.py are the worked examples this chapter is modelled on.

4.2 A single-machine rig

Create manifest/single/__init__.py (empty) and manifest/single/rig.py — one machine (central) runs everything:

from __future__ import annotations
from artheia.manifest.algebra import Append, Explicit
from artheia.manifest.deployment import (
    ApplicationLayer, ApplicationSetLayer, DeploymentLayer,
    ExecutionLayer, MachineLayer, MachineSetLayer, ProcessLayer,
    _members as _set_members,
)

# The generated apps manifest (a base DeploymentLayer with machines open).
# Not present until `artheia gen-manifest` is run — guard the import so the
# rig still serializes on a fresh workspace (produces an empty-but-valid deploy).
try:
    from manifest.apps.manifest import DEPLOYMENT as _APPS
except Exception:
    _APPS = DeploymentLayer()

_PROCESS_NAMES = sorted(p.name for p in _set_members(_APPS.execution.processes))

try:
    from manifest.apps.manifest import PROCESS_NODES
except Exception:
    PROCESS_NODES = {}

try:
    from manifest.apps.manifest import PROCESS_PARAMS
except Exception:
    PROCESS_PARAMS = {}

try:
    from manifest.apps.manifest import PROCESS_CONFIG_DEFAULTS
except Exception:
    PROCESS_CONFIG_DEFAULTS = {}

try:
    from manifest.apps.executor import SUPERVISORS
except Exception:
    SUPERVISORS = []

RIG = _APPS.combine(DeploymentLayer(
    machines=MachineSetLayer(machines={
        # One machine, the MASTER role. It hosts etcd (there is exactly ONE etcd
        # per cluster — the coordinator board).
        MachineLayer(name="central", role=Explicit("master"),
                     arch=Explicit("x86_64"), etcd=Explicit(True),
                     cores={0, 1, 2, 3}, machine_states={"Startup", "Running"}),
    }),
    execution=ExecutionLayer(processes={
        # bind every process to central
        Append(ProcessLayer(name=n, machine=Explicit("central")))
        for n in _PROCESS_NAMES
    }),
    applications=ApplicationSetLayer(applications={
        # your cluster runs on central
        Append(ApplicationLayer(name="apps", host_machine=Explicit("central"))),
    }),
))

The pieces to notice:

role=Explicit("master") — the machine's deployment role. master is the coordinator (etcd + the deployment-wide singletons + the mender gateway); a worker board takes role="zonal". colony provisions the manifest/<role>/ slice, so a role names what a board runs, independent of its name. Omit it and the role defaults to the machine name (a lone central is its own master). See §4.3 for a master + zonal split.
etcd=Explicit(True) — declares that this machine hosts the cluster's etcd. Provisioning reads this (it installs etcd on the etcd: true machine only). On a single-machine rig, the sole board hosts it.
machine_states — the Function-Group states the supervisor brings the board through (Startup → Running).
The apps application is your cluster, bound to a host machine.
The four re-exported sidecars (PROCESS_NODES, PROCESS_PARAMS, PROCESS_CONFIG_DEFAULTS, SUPERVISORS) are what serialize-manifest reads from the rig module. Each is generated into manifest/apps/ by artheia gen-manifest; the rig re-exports them so serialize-manifest can see them through the rig module boundary. Without SUPERVISORS the supervisor tree is empty (no children launched). Without PROCESS_PARAMS/PROCESS_CONFIG_DEFAULTS the per-FC config/<fc>.json files are not emitted and theia install falls back to a legacy skeleton. Always include all four guards.

4.3 A split (multi-machine) rig — the interesting case

Now a two-machine rig: central (the coordinator) + compute (runs your app's heavy work). Create manifest/split/__init__.py (empty) and manifest/split/rig.py. It re-uses the same guarded _APPS base from the single rig — put the common imports at the top, then define which processes go where:

from __future__ import annotations
from artheia.manifest.algebra import Append, Explicit
from artheia.manifest.deployment import (
    ApplicationLayer, ApplicationSetLayer, DeploymentLayer,
    ExecutionLayer, MachineLayer, MachineSetLayer, ProcessLayer,
    _members as _set_members,
)

try:
    from manifest.apps.manifest import DEPLOYMENT as _APPS
except Exception:
    _APPS = DeploymentLayer()

try:
    from manifest.apps.manifest import PROCESS_NODES
except Exception:
    PROCESS_NODES = {}

try:
    from manifest.apps.manifest import PROCESS_PARAMS
except Exception:
    PROCESS_PARAMS = {}

try:
    from manifest.apps.manifest import PROCESS_CONFIG_DEFAULTS
except Exception:
    PROCESS_CONFIG_DEFAULTS = {}

try:
    from manifest.apps.executor import SUPERVISORS
except Exception:
    SUPERVISORS = []

_ALL_PROCS = sorted(p.name for p in _set_members(_APPS.execution.processes))

# Which processes run on which machine. Tune to your app's topology.
# Here everything is on compute; central hosts only the ARA services (add them
# if you init'd with --with-services).
ON_CENTRAL: list[str] = []                    # e.g. ["sm", "com", "per"]
ON_COMPUTE: list[str] = _ALL_PROCS

_SPLIT = _APPS.combine(DeploymentLayer(
    machines=MachineSetLayer(machines={
        # central = the MASTER (etcd lives here ONLY; compute connects to it).
        MachineLayer(name="central", role=Explicit("master"), etcd=Explicit(True),
                     cores={0, 1, 2, 3}, machine_states={"Startup", "Running"}),
        # compute = a ZONAL worker (no etcd — reaches the master's over TIPC).
        MachineLayer(name="compute", role=Explicit("zonal"),
                     cores={0, 1, 2, 3, 4, 5, 6, 7}, machine_states={"Startup", "Running"}),
    }),
    execution=ExecutionLayer(processes={
        *(Append(ProcessLayer(name=n, machine=Explicit("central"))) for n in ON_CENTRAL),
        *(Append(ProcessLayer(name=n, machine=Explicit("compute"))) for n in ON_COMPUTE),
    }),
    applications=ApplicationSetLayer(applications={
        Append(ApplicationLayer(name="apps", host_machine=Explicit("compute"))),
    }),
))

# One base shape, two arch overrides at the bottom:
DOCKER = _SPLIT.combine(DeploymentLayer(machines=MachineSetLayer(machines={
    Append(MachineLayer(name="central", arch=Explicit("x86_64"))),   # all-x86 (dev/CI)
    Append(MachineLayer(name="compute", arch=Explicit("x86_64"))),
})))
HW = _SPLIT.combine(DeploymentLayer(machines=MachineSetLayer(machines={
    Append(MachineLayer(name="central", arch=Explicit("aarch64"))),  # rpi4 + jetson
    Append(MachineLayer(name="compute", arch=Explicit("aarch64"))),
})))

This shows the algebra's payoff: one SPLIT base, two arch flavours (DOCKER for an all-x86 dev box, HW for real arm64 boards) — no duplicated rig. The attribute you select at serialize time (--attr DOCKER) decides which one is emitted.

One master, N zonal is the rule for any multi-machine rig: the master role is the coordinator (it hosts the single per-cluster etcd + the deployment-wide singletons); every worker is zonal and connects to it. The role — not the machine name — is what colony provisions against (it pulls the manifest/<role>/ slice), so a fleet of workers all share the one zonal slice. You declare the roles + etcd in the rig, not a deploy script — the manifest is the source of truth.

4.4 Serialize the rig → JSON manifests

theia manifest <target> runs the address-collision gate, then serializes the rig to the per-machine JSON manifest set:

theia manifest single                 # the single rig
theia manifest split --attr DOCKER     # the split rig, DOCKER flavour

It writes, under dist/manifest/:

dist/manifest/
  machines.json                  the deployment: { machines, role_map, app, roles, arity, on }
  central/
    machine.json                 arch, os, cores, machine_states, etcd, role
    application.json             the apps this machine runs
    executor.json                the supervised process tree for this machine
    execution.json, service.json
  compute/   (split only)
    …

Things that land in machines.json that matter downstream:

app / roles / arity — the user Software Package name, the machine list it spans, and the arity (1 = single rig, 2 = central+compute). This is what the .deb / SWP is named from, and what the Distribution model reads.
on — which machine(s) actually run the SWP's processes (the overlay target).
role_map — name → deployment role ({"central": "central"}). A role is the master/zone distinction a Distribution binds to a board: central is the master (etcd + the deployment-wide singletons + the mender gateway); a zonal board is a zone-of-responsibility worker (the minimal FC set + a mender agent). Provisioning keys off it. A machine's role defaults to its name, so a lone central is its own master with no extra authoring.

And machine.json carries etcd: true/false and role per the rig — provisioning uses both.

Runtime vs your app. The single/split rigs above are for your app. The framework runtime (supervisor + ARA services) no longer uses single/split shapes at all — it ships one role-based rig (manifest/services/rig.py, exports RIG = central-only and MULTI = central+zonal), and a Distribution's roles pick which boards materialize. You never author that rig; you consume the runtime it produces.

The manifest is the single source of truth. Don't hand-write ad-hoc JSON downstream; if you need more deploy info, put it in the rig and let theia manifest serialize it. (That's how etcd and the SWP arity/roles got there.)

4.5 Verify

cat dist/manifest/machines.json
# → { "machines": ["central"], "app": "single", "role_map": {"central": "master"},
#     "roles": ["central"], "arity": 1, "on": ["central"] }
#   (role_map is "master" because the §4.2 rig set role=Explicit("master");
#    omit that and the role defaults to the machine name, "central".)

python3 -c 'import json; d=json.load(open("dist/manifest/central/machine.json")); print(d["etcd"], d["role"])'
# → True master   (central hosts etcd and takes the explicit master role)

You now have: a rig (single and/or split) that places your cluster onto machines, serialized to JSON manifests — with etcd placement and SWP arity baked in.

Next: Chapter 5 — Make a Distribution & publish to S3, where you turn the build + manifest into a deployable, versioned bundle.

← 3. Your first application · Tutorial index · 5. Build a Distribution + publish to S3 →

Theia Wiki

Home

Tutorial

Tutorial index
1. Install
2. Workspace
3. First app
4. Rig
5. Distribution
6. Provision + Deploy
7. RF test
8. Operator GUI
9. Advanced

Reference

Uh oh!

Tutorial 4 Rig

Chapter 4 — Write and shape rig.py

4.1 The model: layers + the Append/Remove algebra

4.2 A single-machine rig

4.3 A split (multi-machine) rig — the interesting case

4.4 Serialize the rig → JSON manifests

4.5 Verify

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Theia Wiki

Related wikis

Clone this wiki locally

Chapter 4 — Write and shape `rig.py`