Configuration and Migration

Configuration & Migration

Theia has two distinct node-configuration mechanisms — split by lifecycle, not format. Do not conflate them: params are static read-only deploy knobs; config is the dynamic, etcd-backed, versioned state. Migration applies only to config.

params vs config — the split

	params (static)	config (dynamic)
What	per-FC deploy knobs (read-only at runtime)	per-node structured state that evolves
Declared in .art	a node's params { … } block	a node's config <Msg> binding
Format on disk	per-FC JSON — install/<machine>/config/<fc>.json (one section per node)	binary proto bytes in etcd at /theia/config/<node>
Read	once at startup into the theia::runtime::get_config() process singleton (a tiny hand-rolled JSON reader — no JSON dep)	on demand from services/per, hot-reloadable via a ConfigUpdated cast
Mutability	read-only — fixed for the process lifetime	mutable; digest-versioned (content-hash of the shape)
Owner	the runtime (each FC reads its own file)	services/per — the sole etcd client + config gatekeeper
Generated by	artheia gen-params (+ gen-config-defaults for .art field defaults)	artheia gen-schema (the digest+shape), seeded via artheia gen-etcd

In short: params is a per-FC JSON file of RO parameters, staged next to the binary and read once. config is a per-node protobuf message stored in etcd, versioned by a digest, and reloadable while the FC runs. gen-etcd seeds the etcd store; it does not make params dynamic.

params — read once, never changes

A node declares params { … } in its .art; theia install runs artheia gen-params to emit config/<fc>.json, staged at install/<machine>/config/<fc>.json. The generated main.cc calls init_config() early, and a node reads its own section via the process singleton:

auto cfg = ::theia::runtime::get_config();   // read once, at startup

Use params for deployment-time knobs that never change at runtime (buffer sizes, feature flags, machine-specific tunables).

config — etcd-backed, versioned, hot-reloadable

A node binds one config <Msg> (a protobuf message). services/per stores it as binary proto bytes at /theia/config/<node>, tagged with a digest — a content-hash of the message shape. per is the sole etcd client; other FCs never touch etcd. A config change is pushed to a live FC as a ConfigUpdated cast (per-field STATIC / HOT_RELOAD / RESTART_REQUIRED).

Migration — evolving a config <Msg> across versions

When you change a config <Msg> shape, its digest changes. A stored value whose digest ≠ the current schema digest needs a migration. services/per is the migration gatekeeper:

• Migrations are strictly 1:1 payload evolution (one config per node). Storage-topology changes (key moves / splits / merges) are out of scope.
• Steps chain v1 → v2 → v3 from adjacent from_digest → to_digest entries (BFS in the MigrationRegistry).
• They run lazily on read (GetConfig) and in bulk (PerManager.MigrateBulk).

Two engines, one rule-set (keep them lockstep)

The same migration rules run in two places that must agree:

engine	where	works on
tools/migrate/migrate.py	dev box	decoded JSON (design / preview bench)
the dlopen'd plugin	services/per	the nanopb struct (the runtime — fast, no JSON/libprotobuf)

To extend the rule vocabulary you edit migrate.apply_rule and transform_codegen._emit_rules together; a regression test asserts they produce identical output (the lockstep invariant).

The tool chain

.art (config <Msg>)                              ← evolve the shape
  │  artheia gen-schema <component.art> --out schema_vN.json   (config_type → digest + fields)
  ▼   (keep the PREVIOUS schema_v{N-1}.json — the diff baseline)
artheia gen-migration --from schema_v{N-1}.json --to schema_vN.json --out migration/
  │   → one <node>_v1_to_v2.json per CHANGED config (auto from/to digest +
  │     add/remove + same-tag rename heuristic + custom stub), each guess flagged
  ▼   ← REVIEW the scaffold: confirm renames, set add-defaults, fill custom hooks
tdb get-snapshot <label> --schema schema_vN.json     → snapshot.json (decoded JSON)
  │  migrate.py --snapshot snap.json --transform <node>_v1_to_v2.json --out next.json
  ▼   ← DESIGN: eyeball the v_{n+1} snapshot, iterate on the transform .json
artheia gen-transform <node>_v1_to_v2.json --schema schema_vN.json --out <node>_v1_to_v2.cc
  │   → the migration plugin .cc (+ a _custom.cc for any {custom} rule)
  ▼  bazel build //migration:libper_migrate_<node>.so
deploy the plugin → per migrates lazily on read + in bulk

The migration can be driven and verified from rf-theia — see the rf-theia wiki's testing pages.

Commands

command	emits
artheia gen-params <art>	the per-FC static params JSON (config/<fc>.json)
artheia gen-config-defaults <art>	the .art-declared config field defaults
artheia gen-schema <art>	the combined config schema (digest → type → shape)
artheia gen-etcd <art>	the etcd seed for the config store
artheia gen-migration --from … --to …	the per-node migration scaffolds
artheia gen-transform <transform.json>	the dlopen'd migration plugin .cc

See also: Deployment + Serialization, Architecture.