Skip to content

docs(architecture): refine CBAR-SUBSTRATE — pin trait to shipped types, add engram-analyzer worked example, codex derive-macro gate#1324

Merged
joelteply merged 2 commits into
canaryfrom
joel/docs-cbar-substrate-refine
May 16, 2026
Merged

docs(architecture): refine CBAR-SUBSTRATE — pin trait to shipped types, add engram-analyzer worked example, codex derive-macro gate#1324
joelteply merged 2 commits into
canaryfrom
joel/docs-cbar-substrate-refine

Conversation

@joelteply
Copy link
Copy Markdown
Contributor

What

Four structural refinements to `docs/architecture/CBAR-SUBSTRATE-ARCHITECTURE.md`. Doc-only PR, no code touched.

Previously parked on `joel/docs-cbar-substrate-refine` waiting on the precommit-gate fix (now landed via #1319) and on incorporating codex's derive-macro review from #cambriantech.

Why

This document is the substrate contract — what every Rust concern in continuum-core inherits. The earlier draft was already directionally right (RTOS / CBAR pattern) but had three load-bearing weaknesses that this refresh fixes:

  1. The `RuntimeModule` trait sketch named types (`ArtifactSelector`, `CadencePolicy`, `RuntimeFrame`, `ModuleResult`) as if they shipped — they don't. A reader couldn't tell what was real and what was proposed.
  2. The "for free" framing was stated without showing how a new module actually inherits concurrency / memory pressure / device pressure / telemetry. The engram-analyzer worked example, asked for explicitly by Joel ("need a new engram analyzer? works in its own thread with zero effort..."), now demonstrates this concretely.
  3. No acceptance gate on the derive-macro layer. Codex correctly flagged on #cambriantech that the derive macro is the load-bearing piece — if it ships sloppy, every module that uses it inherits the sloppiness invisibly.

Changes

1. Continuum Translation — anchor the trait sketch in shipped types

The section now first shows the actually-shipped `ServiceModule` + `ModuleConfig` from `src/workers/continuum-core/src/runtime/service_module.rs` with the real field set, then shows the proposed `RuntimeModule: ServiceModule` extension trait that adds typed `ArtifactSelector` / `CadencePolicy` / `RuntimeFrame` / `ModuleResult`. Each new type is labelled as a Lane D deliverable. A side-by-side table makes the today-vs-after-Lane-D delta explicit.

2. New section "The 'For Free' Triplet" + engram-analyzer worked example

Names the three things that have to ship together: base trait, `#[derive(RuntimeModule)]` macro, `just scaffold-module` generator. The engram-analyzer worked example shows the literal Rust source the developer writes (four config attributes + a handler body) and a per-concern inheritance table tracing what they get for free: registration, tokio/dedicated-thread choice, memory + CPU + device pressure response, concurrency cap, coalescing, spans / timing / VDD emission, typed failure path, replay test fixture, ts-rs exposure, lifecycle.

3. Substrate Gap Analysis cross-linked to ALPHA-GAP lanes

The previous "six numbered missing pieces" list now sits in a table that assigns each piece to its lane (A–G) and adds two pieces the earlier list omitted: piece 3 (the for-free triplet companion to the typed contract), and piece 8 (deletion of pre-broker concurrency hacks with the concrete `inference-grpc/main.rs::get_num_workers()` example calling out the `INFERENCE_WORKERS=` config.env read + system-memory heuristic as a Lane E deletion target).

4. Derive-macro acceptance gate (codex review)

Five gates the `#[derive(RuntimeModule)]` macro must clear before landing:

  1. Thin — generated code is what a careful human would write; a reviewer can read a small module's expansion in one screen.
  2. Contract-preserving — emits exactly the trait the hand-written version would; no smuggled behavior.
  3. Inspectable — `cargo expand` output is auditable in 30 seconds, not identifier soup.
  4. Tested — golden-file or trybuild tests over every supported attribute permutation, including failure modes with useful errors.
  5. No hidden behavior — resource leases, scheduling decisions, and fallback/degradation paths remain visible in the macro output. The macro saves typing, not auditability.

Also

  • Replaces the two-section "Extension Bar" + "Test Contract" pair with a single "Acceptance Criteria For Substrate-Done" section organized by what the substrate proves.
  • "See Also" footer names ALPHA-GAP as the planning document and reasserts the precedence rule (CBAR-SUBSTRATE wins on substrate contract).

Validation

Companion PRs (canary doc track)

This PR closes the parked block on CBAR-SUBSTRATE. The four-doc set (this one + #1316 + #1317 + #1320) now cross-reference each other consistently, with this doc named as the precedence-winner on substrate-shape questions.

…ram example, codex derive-macro gate, cross-link to ALPHA-GAP

Four structural refinements to CBAR-SUBSTRATE-ARCHITECTURE.md, in
service of the "philosophy of docs: extreme elegance" bar:

1. Continuum Translation: anchor the trait sketch in shipped types.
   The previous sketch named lane(), subscriptions(), cadence(),
   handle(), and an unnamed ModuleResult as if the supporting types
   already existed. They do not. The section now first shows the
   actually shipped ServiceModule + ModuleConfig (from
   src/workers/continuum-core/src/runtime/service_module.rs) with the
   real field set, then shows the proposed
   RuntimeModule: ServiceModule extension trait that adds typed
   ArtifactSelector, CadencePolicy, RuntimeFrame, and ModuleResult —
   each labelled as a Lane D deliverable. A side-by-side table makes
   the delta explicit.

2. New section "The 'For Free' Triplet" + worked engram-analyzer
   example. The earlier doc said modules should get concurrency /
   memory pressure / telemetry "for free" but didn't show how. The
   refined section names the three things that have to ship together:
   (a) base trait, (b) #[derive(RuntimeModule)] macro, (c)
   just scaffold-module generator. The engram-analyzer worked example
   shows the literal Rust source the developer writes (four config
   attributes + a handler body) and a per-concern inheritance table
   tracing what they get for free.

3. Substrate Gap Analysis cross-linked to ALPHA-GAP lanes.
   The previous "six numbered missing pieces" list now sits in a table
   that assigns each piece to its lane (A–G) and adds two pieces the
   earlier list omitted: the for-free triplet companion to the typed
   contract, and deletion of pre-broker concurrency hacks (with the
   concrete inference-grpc/main.rs::get_num_workers() example).

4. Derive-macro acceptance gate (incorporated from codex review on
   #cambriantech). The derive macro is the load-bearing piece of the
   for-free triplet; if it ships sloppy, every module that uses it
   inherits the sloppiness invisibly. Five gates must be cleared before
   landing the macro: (1) thin — a reviewer can read the expansion of
   a small module in one screen; (2) contract-preserving — exactly the
   same trait the hand-written version would emit, no smuggled
   behavior; (3) inspectable — cargo expand output is auditable in 30s,
   not identifier soup; (4) tested — golden-file or trybuild tests
   over every supported attribute permutation including failure modes
   with useful errors; (5) no hidden behavior — resource leases,
   scheduling decisions, and fallback/degradation paths must remain
   visible in the macro output. The macro saves typing, not auditability.

Also:

- Replaces the two-section "Extension Bar" + "Test Contract" pair with
  a single "Acceptance Criteria For Substrate-Done" section organized
  by what the substrate proves rather than by what kind of test is run.
- Adds a "See Also" footer that names ALPHA-GAP as the planning
  document and reasserts the precedence rule (this doc wins on
  substrate contract).

Doc-only change. No code touched.
joelteply pushed a commit that referenced this pull request May 16, 2026
Three updates to ALPHA-GAP-ANALYSIS.md following continuum#1327:

1. Lane H added to the lane status table: Substrate governor + tiered
   genome cache. Sibling to Lane E (broker owns admission; governor
   owns sizing). 7-PR implementation sequence detailed in
   GENOME-FOUNDRY-SENTINEL.md Part 13. Currently Proposed, needs owner
   claim.

2. Lane claim update at end of the lane discussion: Lane H proposed
   via continuum#1327 with full design pinned to that doc; sibling to
   Lane E with the boundary stated explicitly.

3. Document Map gets GENOME-FOUNDRY-SENTINEL entry under "Runtime
   substrate (load-bearing)" — the artifact-sharing economy on top of
   the CBAR substrate. Tiered genome cache, page faults, foundry as JIT,
   sentinel-AI as profile-guided optimizer, demand-aligned recall,
   composer + speculator, SubstrateGovernor (DVFS).

4. Immediate Next Actions step 9 added: claim Lane H. Step 10 (formerly
   step 9) updated to reflect what's landed in this doc batch
   (CBAR-SUBSTRATE refinement via #1324, CONTINUUM-ARCHITECTURE refresh
   via #1317, CONTINUUM-VISION refresh via #1320, GENOME-FOUNDRY-SENTINEL
   via #1327) and what's next (CLAUDE.md substrate pointer; stale-section
   deprecations in UNIVERSAL-SENSORY / LEARNING / QUEUE-DRIVEN-COGNITION).
Bidirectional cross-link: CBAR-SUBSTRATE is the floor (what every cell
inherits); GENOME-FOUNDRY-SENTINEL is what every cell recalls /
composes / evolves through. The two docs are paired and should
reference each other. GENOME-FOUNDRY-SENTINEL already references
back; this commit closes the loop.

Also updates the ALPHA-GAP lane reference from A–G to A–H now that
Lane H (Substrate Governor + Tiered Genome Cache) is proposed via
continuum#1327.
@joelteply joelteply merged commit 203a7ab into canary May 16, 2026
2 checks passed
@joelteply joelteply deleted the joel/docs-cbar-substrate-refine branch May 16, 2026 21:09
joelteply added a commit that referenced this pull request May 16, 2026
… document map; lane status truth-up (#1316)

* docs(alpha): refresh status against 2026-05-16 canary

Three changes to ALPHA-GAP-ANALYSIS.md:

1. Header date 2026-05-13 -> 2026-05-16. Add explicit cross-link to
   CBAR-SUBSTRATE-ARCHITECTURE.md as the runtime substrate spec.

2. Restructure the Document Map (was a flat list) into categorized
   references (Runtime substrate / Cognition migration / Memory paging /
   Model registry / Grid), and add the precedence rule: if any supporting
   doc disagrees with ALPHA-GAP on the substrate contract (concurrency,
   scheduling, memory, pressure, telemetry, artifact handles), defer to
   CBAR-SUBSTRATE-ARCHITECTURE.md.

3. Refresh the Current Snapshot table against canary @ 2026-05-16:
   - Rust core row reflects the PressureBroker bootstrap stack
     (#1307 / #1308 / #1310), runtime lease broker (#1313), cognition
     oxidization (#1284 / #1290 / #1291 / #1293 / #1298 / #1301 / #1303
     / #1292), dead-Candle deletes (#1277 / #1279 / #1281 / #1288), and
     the inference-grpc fail-closed (#1314). GRID-INFERENCE-ROUTING
     PR-1 announcer in flight on feat/grid-inference-routing-pr2-announcer.
   - Node/TS row notes net-negative trend (~2500 LOC TS deleted via the
     8-PR cognition stacks).
   - Docker row records Docker tier Phase 1 (#1297).
   - Config row records SQLite-first default (#1271).
   - Tests row records the no-CPU-fallback contract gap: the existing
     regression test in workers/continuum-core covers llama.cpp / ORT
     only, not the Candle-side paths where the orpheus + inference-grpc
     fallbacks lived before #1314.

* docs(alpha): refresh lane status table and immediate-next-actions

Two updates to ALPHA-GAP-ANALYSIS.md:

1. Lane status table now reflects actual state @ 2026-05-16, not aspiration:
   - Lane A: in progress, model_registry/ exists with admission resolver.
   - Lane B: Phase 1 landed (#1297 docker-tier-stats); GPU profile +
     tier-pool eviction (#1238 / #1239) still open.
   - Lane C: structured RuntimeMetric emits from inference paths;
     vdd-report-command not yet bound.
   - Lane D: UNSTARTED — flagged as the highest-leverage open lane because
     Lane E (PressureBroker) and the inbox coalescing pattern both
     presuppose RuntimeFrame / CognitionTurnFrame.
   - Lane E: bootstrap landed (#1307 / #1308 / #1310 / #1313); paging and
     pre-broker concurrency-hack deletion remain. Concrete deletion target
     called out: get_num_workers() in inference-grpc/main.rs, which reads
     INFERENCE_WORKERS from config.env and otherwise picks worker count
     from system memory at startup — both branches violate the
     "we do not hard code" / "dynamic, broker-owned concurrency" rule.
   - Lane F: ~2500 LOC TS deleted manually this session; mechanical CI
     ratchet still not landed (deletion is reversible until it is).
   - Lane G: refresh in flight on joel/docs-alpha-refresh.

   Adds an "adjacent active workstream" note for GRID-INFERENCE-ROUTING
   (PR-1 announcer + probe + registry in flight on
   feat/grid-inference-routing-pr2-announcer) as the grid-side counterpart
   to Lane A.

2. Immediate Next Actions reordered by alpha leverage, not by who is
   online. Top three items are Lane D claim, the universal-trait "for free"
   triplet (RuntimeModule base trait + derive macro + scaffold generator
   from CBAR-SUBSTRATE-ARCHITECTURE.md), and the
   get_num_workers() deletion. Adds the Lane C VDD report command and the
   widening of no_cpu_fallback_contract.rs to cover Candle paths. Adds
   doc-refresh follow-ups so each supporting doc gets cross-linked back
   into the Document Map.

* docs(alpha): add Lane H + GENOME-FOUNDRY-SENTINEL cross-links

Three updates to ALPHA-GAP-ANALYSIS.md following continuum#1327:

1. Lane H added to the lane status table: Substrate governor + tiered
   genome cache. Sibling to Lane E (broker owns admission; governor
   owns sizing). 7-PR implementation sequence detailed in
   GENOME-FOUNDRY-SENTINEL.md Part 13. Currently Proposed, needs owner
   claim.

2. Lane claim update at end of the lane discussion: Lane H proposed
   via continuum#1327 with full design pinned to that doc; sibling to
   Lane E with the boundary stated explicitly.

3. Document Map gets GENOME-FOUNDRY-SENTINEL entry under "Runtime
   substrate (load-bearing)" — the artifact-sharing economy on top of
   the CBAR substrate. Tiered genome cache, page faults, foundry as JIT,
   sentinel-AI as profile-guided optimizer, demand-aligned recall,
   composer + speculator, SubstrateGovernor (DVFS).

4. Immediate Next Actions step 9 added: claim Lane H. Step 10 (formerly
   step 9) updated to reflect what's landed in this doc batch
   (CBAR-SUBSTRATE refinement via #1324, CONTINUUM-ARCHITECTURE refresh
   via #1317, CONTINUUM-VISION refresh via #1320, GENOME-FOUNDRY-SENTINEL
   via #1327) and what's next (CLAUDE.md substrate pointer; stale-section
   deprecations in UNIVERSAL-SENSORY / LEARNING / QUEUE-DRIVEN-COGNITION).

---------

Co-authored-by: Test <test@test.com>
joelteply pushed a commit that referenced this pull request May 16, 2026
…ning; navigate to MODULE-CATALOG queue

Second refresh of ALPHA-GAP Immediate Next Actions to reflect work
landed since #1316 merged. Six items closed; navigation into
MODULE-CATALOG queue made explicit.

Closed: #6 contract widening (#1341), #8 GRID-INFERENCE-ROUTING PR-1
(#1315), CBAR-PIECE-5 end-to-end (#1331/#1333/#1335/#1338),
PIECE-8 inference-grpc hardcoded-clamps (#1340), doc family
architecture surface (#1324/#1327/#1332/#1336/#1337 open;
#1316/#1317/#1320/#1329 merged).

Item #9 reorganized to point at MODULE-CATALOG's 'Next Modules To
Build' queue (audit-recorder → threat-detector → working-set-manager
→ demand-aligned-recall → substrate-governor).

Adds closeout summary section listing what's done, what's open
(5 architecture-doc PRs ready for review + 2 airc PRs), and what's
queued (5 modules with dependency state + LoC + acceptance criteria
in MODULE-CATALOG).

Doc-driven development cycle is working: doc spec → implementing
agent picks up → ships PR → next spec referenced.
joelteply added a commit that referenced this pull request May 16, 2026
…ning; navigate to MODULE-CATALOG queue (#1342)

Second refresh of ALPHA-GAP Immediate Next Actions to reflect work
landed since #1316 merged. Six items closed; navigation into
MODULE-CATALOG queue made explicit.

Closed: #6 contract widening (#1341), #8 GRID-INFERENCE-ROUTING PR-1
(#1315), CBAR-PIECE-5 end-to-end (#1331/#1333/#1335/#1338),
PIECE-8 inference-grpc hardcoded-clamps (#1340), doc family
architecture surface (#1324/#1327/#1332/#1336/#1337 open;
#1316/#1317/#1320/#1329 merged).

Item #9 reorganized to point at MODULE-CATALOG's 'Next Modules To
Build' queue (audit-recorder → threat-detector → working-set-manager
→ demand-aligned-recall → substrate-governor).

Adds closeout summary section listing what's done, what's open
(5 architecture-doc PRs ready for review + 2 airc PRs), and what's
queued (5 modules with dependency state + LoC + acceptance criteria
in MODULE-CATALOG).

Doc-driven development cycle is working: doc spec → implementing
agent picks up → ships PR → next spec referenced.

Co-authored-by: Test <test@test.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant