Incubate a transactional content-addressed Agent Spec catalog

[schickling-assistant]

Problem

The folder catalog currently makes one mutable agent.kdl carry several
different identities at once:

authored bytes
desired selection
publication generation
runtime reconciliation input

That works for hand-edited static catalogs, but it does not give generated or
dynamic Agent Specs a transaction boundary.

Three concrete failures make the distinction load-bearing:

1. Publishing new bytes under the same task ID is immediately visible to the
   resident watcher, while reconciliation may adopt the old process. Desired
   bytes can say g2 while the process still executes g1
   (#40,
   #44).
2. Per-seat atomic files cannot preserve catalog-wide invariants. Two changes
   can each validate against the same base yet combine into duplicate lowered
   task IDs or dangling supervisor references. A multi-seat rename/swap may
   have a valid final catalog but no valid first per-seat write.
3. Agent Spec requirements and durable resource bindings have no shared
   admission identity. Publishing the spec first can expose missing/old
   bindings; publishing bindings first can expose extraneous or over-broad
   access.

The existing exact-byte publication proposal
(#41) is the right starting
point, but an atomic agent.kdl replacement alone cannot close the
catalog-wide or spec/resource admission gaps.

The original runnable reproduction remains:

https://github.com/schickling-repros/2026-07-st2-dynamic-agent-spec-lifecycle

The corresponding portable eval direction is tracked in
compoundingtech/evals#37.

Proposed incubation model

Use a Git-like immutable object graph with one catalog-wide visibility point:

mutable catalog head --CAS--> immutable catalog-root commit
                                `-- seat -> immutable Seat Admission
                                             |-- ref commit
                                             |    `-- revision/tree/blobs
                                             `-- resource-binding commit

The identities are deliberately separate:

• Revision: exact accepted Agent Spec bytes plus catalog-owned immutable
  materialization inputs.
• Ref commit: per-seat manager custody, lineage, operation identity, and
  target revision/tombstone. Expected-parent CAS closes A -> B -> A.
• Seat Admission: exact join of one ref commit and its compatible typed
  resource-binding commit.
• Catalog-root commit: complete seat map and the only catalog-wide atomic
  visibility boundary.
• Realization receipt: later runtime evidence for the exact
  root/admission/revision/launch actually running; not part of this first store
  slice.

Publication is nondestructive:

prepare/import immutable closure
  -> construct manager-scoped patch
  -> validate complete candidate catalog
  -> write immutable objects
  -> CAS one catalog head

It does not imply materialization, replacement, running, or readiness.

Why a catalog root is required

Catalog-wide validation plus a daemon mutex is insufficient if readers can see
seat files one at a time. For example:

old seat owns task T
dependent seat points to old seat

desired atomic patch:
  remove old seat
  add new seat owning task T
  repoint dependent seat to new seat

Every first per-seat write is invalid:

• add new first: duplicate task T;
• remove old first: dangling supervisor/reference;
• repoint dependent first: dangling destination.

An immutable complete root followed by one head CAS exposes either the complete
old catalog or complete new catalog across crashes and concurrent readers.

Proposed first PR slice

Incubate, rather than prematurely standardize, these primitives:

• exact-byte immutable spec/revision objects;
• immutable ref commits with manager, parent, operation ID, and replay-safe
  terminal outcome;
• immutable resource-binding commits and Seat Admissions;
• immutable complete catalog-root commits plus expected-root CAS;
• whole-catalog validation before root visibility;
• safe rebase across unrelated changes only when touched-seat preconditions
  still match and every foreign seat is preserved byte-for-byte;
• a small explicitly experimental JSON CLI used by executable evals; and
• proposed VRS plus crash/concurrency evidence.

The first slice should remain policy-free. A manager identity is stored and
fenced, but caller authentication and authorization policy are separate work.

Required eval outcomes

The model-free agent-spec-lifecycle-catalog-* family should prove:

1. prepare/import persists exact immutable closure without moving the head;
2. source mutation cannot poison an already prepared object;
3. concurrent stale publishers have one winner;
4. request replay after a lost acknowledgement returns the same commit/root;
5. ref-commit CAS rejects stale publication after A -> B -> A;
6. separately valid seat patches cannot create an invalid union;
7. a multi-seat patch becomes visible through one root commit;
8. a desired revision never becomes visible without its compatible resource
   binding;
9. crash barriers expose complete old or complete new roots, never dangling
   objects or mixed admissions; and
10. corrupt objects fail closed when consumed.

Stable state boundary

Writable resources, inbox, context, status, progress, secrets, runtime handles,
and observed-task pins remain outside immutable desired objects. A stable
host-qualified agent-state directory survives revision and catalog-root
changes.

The immutable Seat Admission references the binding generation; it does not
content-address mutable payload bytes. Detaching desired access also does not
prove an already-open runtime handle was revoked.

Follow-up work, not hidden in this slice

• daemon-local control API and sole online linearizer;
• backend-issued generation/launch receipts;
• explicit realize/replace with durable crash recovery;
• authoritative retirement/tombstone after all PTY and exec tasks are absent
  (#42);
• authenticated managers and resource authorities;
• host moves/ownership transfer;
• physical garbage collection, replication, signing, and distributed
  consensus.

Non-goals

• no SeatIntent, persona, harness-profile, or Nix grammar in st2;
• no automatic process restart on publication;
• no mutable resource payloads in the content-addressed store;
• no /nix/store runtime dependency, derivations, or substitution protocol;
• no compatibility dual writes or fallback reads as permanent architecture;
• no claim that hashes provide authorization or runtime readiness.

Acceptance direction

The VRS/implementation PR should remain draft while the API is incubating. The
eval PR should contain real product-conformance cells that invoke st2 directly,
not a green shim. Individual cells land in maintained evals only with the st2
commit that makes their target outcomes green.

Posted on behalf of @schickling

field	value
agent_name	co2-pine
agent_session_id	cdd15622-c01a-4731-bbfe-ab8cce0cad54
agent_tool	Codex CLI
agent_tool_version	0.145.0
agent_runtime	Codex CLI 0.145.0
agent_model	unknown
runtime_profile	/nix/store/ph8rlhdj25mg71v81jsfzy6dq4xpcs9m-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest	/nix/store/lsykz8x5481xrpbgk280xh3pypk1c5jy-agent-skills-corpus/share/agent-skills/manifest.json
worktree	tmp
machine	dev3
tooling_profile	dotfiles@3649b53

[schickling-assistant]

The first executable incubation slice is now open:

• st2 VRS + prototype: prototype: incubate content-addressed catalog admission #55
• direct-product eval family: eval: prove content-addressed catalog admission evals#39
• exact st2 prototype commit: 16ee7eb953508801a4603edd056ab32f28986601

The experiments narrowed the first slice from the full target architecture:

• green now: exact KDL-byte import, staged ref CAS/replay/ABA fencing, immutable
  seat admissions, one multi-seat root CAS, mixed-manager seat preservation,
  stable mutable agent_dir, selected-graph corruption/path/state-root
  rejection, and single-snapshot inspection;
• deliberately later: content-addressed render-input closure, typed resource
  compatibility, authenticated managers, resident root consumption,
  realization/replacement receipts, public crash failpoints, power-loss
  durability proof, and GC.

In particular, the current crash evidence establishes process-level atomic
visibility around the root head, not filesystem power-loss durability. The
eval PR contains only the three cells backed by real product surfaces; it does
not merge known-RED typed-resource or crash-failpoint placeholders.

Posted on behalf of @schickling

field	value
agent_name	co2-pine
agent_session_id	cdd15622-c01a-4731-bbfe-ab8cce0cad54
agent_tool	Codex CLI
agent_tool_version	0.145.0
agent_runtime	Codex CLI 0.145.0
agent_model	unknown
runtime_profile	/nix/store/ph8rlhdj25mg71v81jsfzy6dq4xpcs9m-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest	/nix/store/lsykz8x5481xrpbgk280xh3pypk1c5jy-agent-skills-corpus/share/agent-skills/manifest.json
worktree	st2/schickling-assistant/2026-07-29-content-addressed-catalog
machine	dev3
tooling_profile	dotfiles@3649b53