Skip to content

atelet state persistence: recover per-actor state across restarts #1372

Description

Problem

atelet keeps per-actor state only in memory. The first concrete case is the
system-info volume refresher (#1231), which tracks each running actor's
volume specs so ClusterTrustBundle rotations rewrite projected files live.
Sandboxes outlive an atelet restart, so actors keep running afterwards, but
live refresh breaks until the next Run/Restore re-registers them. An actor
that never suspends keeps a stale egress MITM trust bundle and loses TLS
through the egress gateway on the next CA rotation (#1226).

More data sources will need the same treatment (JWTs, certificates), so this
should be solved once rather than per subsystem.

For comparison, kubelet persists nothing here: on restart it re-fetches its
pods' specs from kube-apiserver and reconstructs the rest from disk, so
recovery and enrollment are the same operation.

Options

  1. Reacquire from ateapi on startup (kubelet model). Needs a new RPC to list
    an atelet's placed actors with rendered WorkloadSpecs; has scaling
    implications as ateapi moves off a singleton.
  2. Query the node's ateoms on startup. Keeps recovery node-local, but shifts
    spec ownership toward ateom and interacts with [EPIC] Multi-Actor Worker Support #1266.
  3. Persist a minimal spec slice on the node. Cheapest, but locally persisted
    derived state has been painful in prior systems.

Re-rendering from the ActorTemplate is out: the template may have changed
since the actor started.

Scope

Pick the on-node source of truth and recovery flow, then converge existing
subsystems on it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions