v0.15.0 — Legible self-structuring streaming substrate
Legible self-structuring streaming: an append-only causal-log substrate that makes dynamic swarms streamable, with background-compacted hot/cold durability and author-owned context bounding.
Added
-
Append-only causal event log + typed void-edges (#282).
swarm_stream_eventsis promoted into the substrate's single source of truth: every event is appended in DB-sequenced causal order and never mutated or deleted in place. Course-corrections are recorded as typed void-edges — aSwarmCausalVoidEdgeevent that points at theevent_uuidof the event itsupersedes,replaces, orabandons, with a reason — so the voided event stays in the log and every later shaping is a read-time fold (the fold layer arrives in #283). A new database-onlyCausalLogStorecontract (extendingStreamEventStore) exposesappendVoidEdge()andisSealed();DatabaseCausalLogStoreimplements it. The append is guarded so it fails loud rather than corrupting history: voiding a sealed target throwsSealedCausalWindowException, voiding an unknown target throwsUnknownCausalTargetException, and the check + insert run under a row lock (lockForUpdate()) so a concurrent seal can never slip past the guard (the seal itself arrives with the #287 compactor; the guard ships first). The migration adds five nullable columns toswarm_stream_events(event_uuid,void_type,void_target_event_uuid,void_reason,sealed_at) plus two run-scoped indexes; existing rows and all non-void events read back unchanged. Void-edge rows inherit the table's existingexpires_atTTL andswarm:pruneretention — no new prune hook. There is no behavior change for existingstream()replay: the databaseStreamEventStorebinding now resolves theDatabaseCausalLogStoresubclass, which inherits the priorrecord()/events()contract. -
Read-policy fold layer over the causal log (#283). A read-time view,
CausalLogView, lets a consumer request different shapes of the same log without changing it — the log stays truth; every shaping is a pure fold (deterministic, idempotent, never writes to the store). The view folds along two independent axes: an order axis (ViewOrder::Causalkeeps events in stored append order;ViewOrder::Presentationreorders sibling node open/close into the order a parent declared viachild_node_ids, a stable sort that leaves undeclared nodes in causal order) and a supersession axis (ViewSupersession::Cleanhonors void-edges — suppressing asupersedes/replacestarget, and anabandonstarget and its node subtree;ViewSupersession::Everythingexposes voided events asVoidedEventwrappers carrying their void type and reason — including every member of an abandoned subtree, each wrapped under the abandoning edge's reason, so an audit view never shows a retracted event without its mark). Construct it from a run (CausalLogView::forRun($store, $runId)) or anyiterable<SwarmStreamEvent>, then callfold($order, $supersession). The fold reads structure from each event'stoArray()payload by string key (node_id,parent_node_id,child_node_ids), so it is decoupled from the node-event classes that carry that structure; where a payload has nonode_id— every #282-era event — it degrades gracefully (presentation order collapses to causal order, andabandonssuppresses only its single target). Dangling void-edges (target absent from the window) are a no-op in the clean view and still shown in everything; multiple edges on one target apply in causal order. Tiered paging from cold storage is out of scope (arrives with #286). -
Structural stream-event grammar — structure as payload (#284). A run's structure becomes first-class on the causal log: three new
SwarmStreamEventsubclasses —SwarmNodeOpened,SwarmNodeChildrenDecided, andSwarmNodeClosed(event typesswarm_node_opened/swarm_node_children_decided/swarm_node_closed) — bracket each run-structure node and declare its children, so a reader can reconstruct the run's shape from the log alone (the read-policy fold that consumes them arrives in #283). Every substantive event now also carries a nullablenode_idtagging the node it belongs to: anode_idproperty +withNodeId()are added to the swarmStreamEventbase beside the vendor'sinvocationId, every concrete event'stoArray()emits the key, andSwarmStreamEvent::fromArray()restores it centrally. The carry is additive and non-breaking — an absent or nullnode_idis a top-level event with no enclosing node, and a pre-grammar persisted log (which lacks the key) rehydrates to null.node_openedis self-identifying (node_id == id) and records itsparent_node_id(null at the root),node_children_decidedlistschild_node_idsin chosen order (the list order is the declared sibling/presentation order), andnode_closedcarries the node'sresult. The streamed decider path is wired inStaticHierarchicalStreamRunner: a sequential worker node opens before any event tagged with its id (causal completeness), streams its deliberation (text/reasoning/tool deltas, each tagged with the node id), terminates that deliberation in anode_children_decideddeclaring the child it routes forward to, and closes — with a looping node opening once and closing once at its iteration bound. The concurrent/parallel fan-out branches are intentionally left un-bracketed for now; the executor that walks the structure is deferred to #285. -
Streaming
Topology::Hierarchicalswarms (#285).SwarmRunner::stream()now routesHierarchicaltopology through a newHierarchicalStreamRunner(extendsStaticHierarchicalStreamRunner). Becauselaravel/aidoes not support streamingHasStructuredOutputagents, the coordinator step runs synchronously viaprompt()— wrapped inActiveRunContext::enter/exitso memory and guardrails work identically to the sync path — and only structural causal-log events are emitted for it:SwarmNodeOpened(rolecoordinator, id__coordinator__),SwarmStepStart/SwarmStepEnd(index 0),SwarmNodeChildrenDecided(the plan'sstartAtas the sole child), andSwarmNodeClosed. Worker nodes then stream as normal via the inherited plan-walk generator, starting at step index 1 with__coordinator__as the initial parent node. Budget accounting counts the coordinator as step 0, so a swarm with#[MaxAgentSteps(N)]may have at mostN - 1worker executions. The planner now rejects node ids that start with__(reserved prefix, validated inHierarchicalRoutePlanner::normalizeAndValidateWithCoordinatorClass()). A newswarm.hierarchical.stream_parallel_branchesconfig key (honoured alongside#[StreamParallelBranches]) controls concurrent vs. sequential parallel-branch streaming for this topology independently ofswarm.static_hierarchical.stream_parallel_branches.DispatchValidator::ensureStreamableTopology()now acceptsHierarchicalalongsideSequentialandStaticHierarchical. The plan-walk loop is extracted fromStaticHierarchicalStreamRunner::executeStaticPlan()into a new protecteddrivePlanNodes()generator — enabling reuse without duplicating 500 lines — which both runners call with topology-specific starting arguments (nextIndex: 0 / initialParentNodeId: nullfor static,nextIndex: 1 / initialParentNodeId: '__coordinator__'for dynamic). -
Hot/cold tiering:
ColdArchiveDrivercontract with atomic base-pointer swap, addressable-by-coordinate cold storage, and separate snapshot (resume) + raw-event (audit) retention paths (#286). -
TieredStreamEventStore: transparent decorator that yields cold events below the base pointer then hot events at/above it, with a half-open seam guarantee (no gap, no duplicate) (#286). -
DatabaseColdArchiveDriver: default database-backed cold archive driver withswarm_cold_archivestable (#286). -
Resume reads on cold snapshots use
openStrict(decrypt-or-throw), consistent with the established #212 convention —APP_KEYrotation after graduation triggers a re-dispatchableSwarmException(#286). -
Background compactor — seal-barriers, graduate-to-cold, base-pointer CAS (#287). The hot event log is now bounded without losing audit history. Streaming runners emit a
SwarmCausalSealBarrierimmediately afterSwarmStreamEnd— a lightweight internal marker (typeswarm_causal_seal_barrier) whose DB auto-incrementidIS the graduation boundary; it is filtered fromevents()replay output and never visible to stream consumers.DatabaseColdArchiveDrivergainsgraduate()(cold write →sealed_atUPDATE → CAS base-pointer advance, all in one transaction) andreclaim()(DELETE from hot WHERE id < barrier, called only after a successful CAS).CausalLogView::snapshot()serialises the fold's accumulated state for cold storage.SwarmCompactordrives the per-run compaction cycle: acquire a compaction lease (CAS on newcompaction_token/compaction_leased_untilcolumns inswarm_durable_runs), locate the latest barrier, read events from the current base pointer to the barrier, fold them, graduate, and reclaim; a run whose graduation throws is quarantined (compaction_quarantined_at) and logged, never crash-looped; fail-safe: hot is never reclaimed on doubt.CompactSwarmRunqueues the cycle per run.swarm:compactdiscovers eligible runs (have a barrier, not quarantined) and dispatches the queue job per run. Ordering spine invariants: cold-durable → base-pointer advance → reclaim, never inverted; CAS prevents concurrent advance; base pointer is monotonically non-decreasing.TieredStreamEventStore::events()correctly stitches cold + hot after reclaim so a consumer sees the complete history with no gap. Operator note:swarm:compactmust be explicitly scheduled by the consumer application — the package does not auto-schedule it. The recommended schedule is$schedule->command('swarm:compact')->hourly()inbootstrap/app.php. The command discovers eligible runs and dispatches aCompactSwarmRunqueue job for each; compaction work happens asynchronously on the queue. Configure the lease duration withSWARM_COMPACTION_LEASE_SECONDS(default300). If a run's graduation repeatedly throws, the run is quarantined (compaction_quarantined_atis set inswarm_durable_runs) and excluded from future sweeps to prevent crash-looping. To recover: inspect thewarninglog entry forrun_idandexception_class, resolve the underlying issue, clear the flag (DB::table('swarm_durable_runs')->where('run_id', $runId)->update(['compaction_quarantined_at' => null])), then re-trigger withswarm:compact --run-id={run_id}. -
Forward-compatibility sentinel for unknown persisted event types (#287).
SwarmStreamEvent::fromArray()previously threwSwarmExceptionon anytypestring not in its dispatch registry. This became load-bearing when the compactor (also in #287) introducedswarm_causal_seal_barrierrows written by new workers: a v0.14.x worker resuming a run after a barrier was written would call the old (unfiltered)events(), hitfromArray()withtype='swarm_causal_seal_barrier', and throw — crashing the durable job. A new@internal SwarmUnknownEventsentinel replaces the throw:fromArray()'sdefaultarm now returnsnew SwarmUnknownEvent($payload), and all three storage-layer iterators —DatabaseStreamEventStore::events(),DatabaseStreamEventStore::eventsFrom(), andDatabaseColdArchiveDriver::readEvents()— filter it out before yielding to consumers. Unknown future event types are silently skipped rather than surfaced or thrown. This gives v0.15.x (and later) workers forward-compatibility against infrastructure types introduced by a future package version — a v0.16.0 worker writing a new event type will not crash a co-deployed v0.15.0 worker. Rolling-deploy note (v0.14.x → v0.15.0): this sentinel was not backported to v0.14.x. During a rolling deploy from v0.14.x to v0.15.0, new workers begin writingswarm_causal_seal_barrierrows immediately after the firstSwarmStreamEnd; a v0.14.x worker that later resumes that run will throw. A coordinated full-fleet restart (all workers updated to v0.15.0 before any long-running durable run completes) is the safe upgrade path. For most deployments a standard rolling deploy that finishes in under one compaction window (default lease: 300 s) is safe in practice, since the window is short. -
Rollup nodes — author context-bounding idiom (#289). A workflow author can now place a
'type' => 'rollup'node in a hierarchical plan that digests a generation and bounds what flows downstream. A rollup runs a digester agent exactly like a worker; itswith_outputsdoes double duty — it is the digester's prompt context and names the generation being digested. After the digest output is recorded, the walk runs three rollup-only effects: (1) operational bounding — the digested nodes are pruned from the in-process node-output map (and thus from the persisted hierarchical boundary that survives resume), so a downstream node can only read the digest, never the raw generation; (2) display fold — a newCausalVoidEdgeType::RolledUpvoid-edge is appended against each digested node's current step-end, soCausalLogViewsuppresses exactly those nodes' events by node_id (a new membership pass, distinct fromabandonssubtree suppression — because the sequential walk records each node as the parent of itsnext, a subtree fold would erase the entire forward chain including the rollup itself), surfacing the digested events in theEverythingview asVoidedEvents carrying adigestNodeIdpointer to the summary; (3) sealability — a mid-runSwarmCausalSealBarrierso the #287 compactor can graduate the digested window to cold mid-run rather than only at completion. Context-unreferenceability is enforced, not assumed: plan materialization (bothfromStaticPlanand the coordinator-drivenfromCoordinatorOutput, so static and dynamic swarms alike) rejects any node ordered after a rollup that references a digested node viawith_outputsor a finish node'soutput_from— it fails loud before the walk runs, naming the rollup to reference instead, never a silent empty read on resume. The edges + barrier are written in one transaction (CausalLogStore::sealRollup()) so a crash can never leave a partial rollup the compactor would graduate, and the seal is idempotent (a target already carrying arolled_upedge is skipped) so a re-dispatched/re-executed pass never double-voids or throws. The edges + barrier are best-effort and require the database causal log (matching the existing run-end seal barrier); off the database driver only the operational prune applies and the display fold degrades to showing the raw nodes. A rollup composes with bounded loops — placed in a loop body it digests each iteration's fresh outputs by targeting the live, unsealed step-end (never the once-only node-open event, which a prior iteration's barrier may have sealed). Scope: hierarchical topology only (thedrivePlanNodeswalk — static-plan and coordinator-driven); the pureSequentialrunner has no plan nodes and is unaffected. Operator note: a rollup bounds the agent prompt context (downstream reads one digest, not the raw generation) and the hot causal log (the digested window becomes sealable → reclaimable to cold), and it makes the #288degrade_to_coldnudge functional by creating a sealable barrier mid-run; it does not lower the context-growth budget metric, which counts cumulative emitted stream events per segment and is monotonic by design — so an operator watching that number will not see it drop after a rollup. In synchronousrun()and durablequeue()execution (the non-streaming paths), a rollup node runs as an ordinary digester worker: it bounds the operational node-output map but emits no seal barrier or void-edges — those are streaming causal-log constructs, so there is no mid-run seal or display fold off the stream path. See the author guide and operator runbook (#291). -
Context-growth policy ladder + operator hard-cap (#288). A streaming run's hot working set is now governed by declared intent rather than surprise. A workflow author declares a
#[ContextGrowthPolicy(GrowthPolicy::…)]on the swarm; the operator supplies the budget number and an optional hard-cap veto via config. The ladder is a set of cumulative severity bands — a declared rung includes the behaviour of every lower one:ignore(take no action; the hard-cap still applies) →warn(emit telemetry + a throttled warning) →degrade_to_cold(warn, and nudge background compaction to reclaim what it can — the framework default, loud and least-destructive) →backpressure(warn, degrade, and insert a bounded delay) →refuse(warn, then abort the run loud with a re-dispatchableContextBudgetExceededException). The policy is evaluated at each step boundary by the streaming runners (SequentialStreamRunnerand the shareddrivePlanNodes()plan-walk), measuring the run's hot working set as its in-process stream-event count — the same rows compaction reclaims. The operator hard cap clamps author intent: a breach refuses regardless of the declared policy (evenignore), mirroring theswarm.limits.*precedent. The governor (ContextGrowthGovernor) is stateless / Octane-safe — per-run throttle memory is threaded through the runner's generator scope, never an instance field, so two runs in one worker never share warn/nudge bookkeeping — and fail-safe: the whole evaluation is wrapped so a throwing, slow, or mis-measuring policy degrades to the least-destructive action and the run proceeds; the only intentional throw is therefuse/hard-capContextBudgetExceededException, which is re-raised, never swallowed. Every over-budget evaluation emits acontext_growth.actiontelemetry event attributing the action to the declared policy. New config blockswarm.context_growth.{policy,budget_events,hard_cap_events,backpressure_delay_ms}(envSWARM_CONTEXT_GROWTH_*); the budget and hard-cap default to null (inert — the package ships the machinery and the author's intent, never an imposed number). Supplying a budget activates the framework defaultdegrade_to_cold(aCompactSwarmRunnudge per over-budget run) unless an author declares a different rung — setting a number is the operator's opt-in to the default behaviour, not just to measurement. The working set is measured per stream segment (the in-process event count, which resets to zero when a run resumes in a fresh process), so the budget and hard-cap are enforced per segment rather than across a run's full history — a deliberate trade (no persisted counter, idempotent on resume) consistent with the hard-cap being best-effort. Scope: the streaming substrate only — non-streamingprompt()runs do not accumulate a hot causal log and are out of scope. Within a single non-rollup run there may be no sealed prefix fordegrade_to_coldto reclaim until completion (the nudge materialises when a seal barrier exists — run end, or once rollup nodes #289 land); the timely live-growth responses arebackpressureandrefuse. The hard cap is best-effort governance, not a correctness invariant — consistent with "delegate preference, never correctness." See the author guide and operator runbook (#291). -
Per-node streaming from durable execution (#298). A durable run can now stream each node's events into the causal log instead of producing one blocking
prompt()response per node, so operators get a live, replay-safe signal from a durable run — gated behind a per-swarm#[DurableStreaming]opt-in (#310), off by default. Without the attribute the blockingprompt()path is byte-for-byte unchanged; with it,DurableSequentialStepAdvancercalls the agent'sstream()and appends each mappedSwarmStreamEventto theCausalLogStoreunder the run id. Every streamed event is stamped with its node id and a durable attempt epoch — two new queryable, indexed columns onswarm_stream_events(node_id,attempt_epoch; nullable and additive, so existing rows and all non-durable-streamed events read back unchanged) — promoted out of the JSON payload so the resume-time rollback is a metadata-only lookup that never decrypts. Crash-resume is retractable, not corrupting: the attempt epoch is the run'srecovery_count, bumped before any recovery/retry re-dispatch, so a node that re-executes after a crash always streams under a strictly higher epoch than its crashed attempt (the nullable vendorinvocation_idcannot key attempt-distinction). On resume the advancer retracts the crashed attempt with a single idempotentCausalVoidEdgeType::NodeReexecutedvoid-edge — located bylatestAttemptEpochBelow()and applied viavoidNodeAttempt()— before the fresh attempt emits anything;CausalLogViewthen suppresses the retracted attempt by(node_id, epoch)membership while the resumed attempt (same node id, higher epoch) survives in the clean fold and the crashed one surfaces in theEverythingview as aVoidedEvent. Seal-follows-commit is the load-bearing invariant: the per-node seal barrier is emitted in the same lease/execution-token-fenced step-commit transaction that advances the durable cursor (DurableRunRecorder::checkpointSequential), so an uncommitted (crashed) node's events are always above the last committed barrier — unsealed, hence retractable — and background compaction (#287/#288/#289) only graduates below a barrier, so it can never seal an in-flight node out from under the resume void (which would otherwise leave the crashed attempt frozen beside the fresh one). The seal is deliberately not best-effort: a barrier-write failure rolls the checkpoint back so the step re-executes safely on recovery, rather than advancing the cursor without its barrier. Octane-safe by construction (#298 F4): nothing per-attempt is held on the advancer — the per-event sink is a fresh closure peradvance()call, capturing the node id and epoch in its own scope, so two concurrent durable runs in one worker never share a step's buffer. As part of this the liveSequentialRunner::stream()loop and the new durablestreamSingleStep()now fold provider events through one sharedStreamEventMapper(#310), so the two paths cannot drift. Dispatch gate (F7): with the opt-in on, dispatch fails loud unless the database causal log is available and migrated for streaming, rather than silently dropping events or falling back toprompt(). Scope:Sequentialdurable runs (the issue's acceptance scope); theCausalLogStorecontract is@internal. The contract gainsvoidNodeAttempt()andlatestAttemptEpochBelow(); a new migration adds thenode_id+attempt_epochcolumns and a run-scoped(run_id, node_id, attempt_epoch)index. -
Per-swarm
#[DurableStreaming]opt-in + sharedStreamEventMapper(#310). Durable per-node streaming (#298) is now opted in per swarm class with the#[DurableStreaming]attribute instead of a global flag, and the decision is pinned onto the durable run row at run-start (durable_streamingcolumn, additive migration, defaultfalse) — so every resume reads the value the run started with, never live config, and a swarm streams (or does not) for its entire life regardless of a mid-run redeploy.SwarmAttributeResolver::resolveDurableStreaming()reads the attribute (no config fallback; absent ⇒ off); the dispatch gate and the runtime void/seal/sink sites all read the one pinned value, so they can never disagree (no dispatch-on/runtime-off dropped events or runtime-on/dispatch-off ungated writes). The vendor→swarm event fold is extracted fromSequentialRunner::mapStreamEvent()into a standalone, injectable, identity-agnosticStreamEventMapper(it never stamps node id / attempt epoch — that stays in the per-attempt sink), so the durable hierarchical and parallel-branch paths (#311/#312) reuse one fold rather than forking it; a golden-sequence test locks the live and durable paths to an identical event sequence. A new operator kill-switchswarm.durable.streaming_enabled(envSWARM_DURABLE_STREAMING_ENABLED, defaulttrue) lets an operator pause durable streaming fleet-wide at runtime without a redeploy; it gates only emission — when off, opted-in runs fall back toprompt()but every crashed attempt is still voided and every committed node still sealed, so the causal-log fold stays consistent and flipping it mid-run is safe. Scope:Sequentialdurable runs (hierarchical and parallel-branch streaming arrive in #311/#312). Because#[DurableStreaming]is a topology-agnostic surface, the dispatch gate carries an explicit streaming-supported-topology allow-list: opting in on a not-yet-wired topology fails loud at dispatch rather than silently pinning the opt-in and never streaming — the allow-list grows as #311/#312 land. The globalswarm.durable.stream_to_causal_logflag from #298 — never in a tagged release — is removed. -
Per-node streaming for hierarchical & static-hierarchical durable runs (#311).
#[DurableStreaming]now streams hierarchical and static-hierarchical durable runs, not just sequential ones — both topologies are added to the dispatch allow-list, so opting in on either streams instead of failing loud. Each durable worker node calls the agent'sstream()and folds every event through the sharedStreamEventMapper(#310) into the causal log via a per-attempt sink that stamps it with the plan node id and the run'srecovery_countepoch, bracketed bySwarmNodeOpened/SwarmNodeClosedstructure events (#284) — exactly the sequential resume contract (#298), lifted to the per-node hierarchical walk. The same crash-resume guarantee holds per node: on re-execution the prior attempt is retracted with oneNodeReexecutedvoid-edge before the fresh attempt emits, so the clean fold shows exactly one attempt per node and theEverythingview surfaces the voided one. The per-node seal barrier is emitted insideDurableRunRecorder::checkpointHierarchical's lease-fenced transaction (the hierarchical twin ofcheckpointSequential), preserving seal-follows-commit. The coordinator step (index 0) ships structural-only: it runs via the blockingprompt()path and writes its node-structure events (open → children-decided → close) under the reserved__coordinator__node id, sealed underrecovery_count; coordinator token-streaming is a decoupled#314fast-follow. Per-attempt sink closures hold no shared state, so concurrent runs in one Octane worker never cross streams. Scope:Hierarchical+StaticHierarchicaldurable runs; the blocking (non-opt-in) path is byte-for-byte unchanged.Parallelremains gated until #312. -
Per-node streaming for durable parallel branches, with seal-on-join (#312). Durable parallel branches now stream too — both top-level
Topology::Parallelbranches and hierarchical fan-out branches, which run as independent durable jobs throughDurableBranchAdvancerand write concurrently into one run-scoped causal log.Parallelis added to the dispatch gate's streaming-supported-topology allow-list, so a#[DurableStreaming]parallel swarm streams instead of failing loud; queue-hierarchical-parallel inherits this automatically (its branches run through the same advancer), and child swarms self-gate through the allow-list on their own topology + attribute (the pin is read from the child's class, never inherited). The branch's per-event sink reuses the sharedStreamEventMapperandDurableNodeStreamRecorder::sinkFor()— no forked fold. Three invariants make concurrent branches safe: (1) Branch epoch = the branch'sattemptscounter, not the run'srecovery_count(unmoved by a branch crash) and notretry_attempt(misses hard crashes) —acquireBranchLease()bumpsattemptson every successful acquire, so any re-execution streams under a strictly higher epoch than its crashed attempt; the advancer re-reads the branch after the lease to read the post-bump value. (2) Seal-on-join, never seal-on-branch-commit — the branch advancer never seals; the branch generation is sealed by the parent's next post-joincheckpointHierarchical(the run-scoped barrier graduates everything below it). This is safe because the join gate (DurableHierarchicalCoordinator::dispatchWaitingBoundary) only releases the parent once every branch is terminal, so a retrying branch blocks the join and nothing is in-flight at seal time — a per-branch seal would instead prematurely seal a concurrent sibling's in-flight window and throwSealedCausalWindowExceptionon its crash. (3) Branch node id =node_id ?? branch_id— top-level parallel branches persistnode_id = null, which would collapse every branch into one void/fold bucket, so the void lookup and membership fold fall back to the stable, unique, resume-stablebranch_id(e.g.parallel:2); hierarchical fan-out branches carry a realnode_idand use it. Crash-resume is per-branch: on resume a branch retracts only its own crashed attempt — keyed on(branch node id, branch epoch)— so a sibling that committed while it was mid-retry is never voided and never sealed against. Per-branch sinks are call-scoped closures (Octane-safe): two branch sinks in one worker carry distinct(node_id, epoch)stamps. Blocking (non-opt-in) parallel and branch durable runs are byte-for-byte unchanged. Operator-visible asymmetry — top-level parallel events are never sealed: seal-on-join applies to a hierarchical fan-out (the parent's post-joincheckpointHierarchicalseals the branch generation, so it graduates and compacts to cold), but a top-levelTopology::Parallelrun converges by completing the run with no post-join checkpoint — so its streamed branch events are retained-but-uncompacted inswarm_stream_eventsand expire only at the run's data TTL (swarm.context.ttl), never graduating to cold storage. This is deliberate (a top-level parallel run has no subsequent node to fence), but sizeswarm_stream_eventsretention accordingly for high-fan-out top-level parallel streaming. Scope: completes the durable per-node streaming matrix for parallel and fan-out branches; hierarchical main-walk node streaming lands in #311.
Tested
- Substrate correctness suite (#290). A correctness suite encoding the design-gate invariant for the streaming substrate (#282–#289): a run's durable state reconstructs to exactly one consistent value, every read folds the true state or fails loud, and no graduated history is lost. It covers the seams the gate attacked so the later change review confirms rather than discovers — (1) folds-equal: a compacted log folds identically to its uncompacted twin (resume is unaffected by compaction), and the sealed cold snapshot reconstructs to the same value as folding the raw window (snapshot ≡ replay); (2) compaction concurrent with live resume: a crash after
graduate()succeeds but beforereclaim()leaves the run readable with no gap or duplicate (the half-open seam excludes the un-reclaimed hot rows), the reclaim/lease-expiry race (#287 OG3) cannot wipe cold then find an empty hot log, and a full restart re-folds to the same value; (3) Octane: the compactor, cold-archive driver, andContextGrowthGovernorcarry no mutable per-run instance state (asserted via reflection), two runs compacted by one singleton in one worker do not leak, the compaction lease is released infinallyeven when graduation throws, and the governor's per-run throttle memory (threaded via the caller's&$state) keeps two runs' warn/nudge bookkeeping independent; (4) rotated key: a cold-snapshot resume read under a rotatedAPP_KEYfails loud viaopenStrict()with a re-dispatchableSwarmException, never foldingnullor ciphertext, while the correct key still decrypts; (5) supersede/causal-order: a void-edge against a sealed target throwsSealedCausalWindowException(sealed history is not retractable) while an unsealed target is still retractable, and concurrent-branch causal order follows the monotonic DB sequence assigned at append time, not arrival or wall-clock order — surviving a graduation unchanged. No production defect was found; the suite confirms the shipped behavior.
Changed
stream()now acceptsTopology::Hierarchical(#285).DispatchValidator::ensureStreamableTopology()previously rejected dynamic hierarchical swarms; it now streams them alongsideSequentialandStaticHierarchical. Existing streamed topologies are unaffected — this widens whatstream()accepts, it does not change how a sequential or static-hierarchical run streams.- The database
StreamEventStorebinding now resolves a tiered store (#282, #286). Under the database persistence driver, resolvingStreamEventStorefrom the container returns aTieredStreamEventStore(cold-below-base + hot-at/above, stitched at a half-open seam) wrapping theDatabaseCausalLogStore, instead of the bareDatabaseStreamEventStore. Therecord()/events()contract is preserved, so a consumer reading a run's history sees the same complete timeline; onlyinstanceofchecks against the concrete store class are affected (both classes are@internal). SwarmStreamEvent::fromArray()no longer throws on an unrecognizedtype(#287). It now returns an@internal SwarmUnknownEventsentinel, which every storage-layer iterator filters out before yielding. This makes a v0.15.0 worker forward-compatible against infrastructure event types introduced by a future package version, rather than crashing the durable job. (See the rolling-deploy note below for the v0.14.x → v0.15.0 transition, where the older worker lacks this sentinel.)
Fixed
- swarm:compact no-ops on a non-database persistence driver (#290 review). The command now returns early with an informational message instead of throwing a QueryException when swarm.persistence.driver is not database — matching the documented operator-runbook behavior.
Documentation
- Streaming substrate guide — operator runbook + author guide, and per-release surface updates (#291). Two new docs document the v0.15.0 substrate against verified class/command/event names.
docs/operator-runbook-streaming-substrate.mdis the operator side: the hot/cold tiering model (swarm_stream_eventshot,swarm_cold_archivescold, the base pointer,TieredStreamEventStore), scheduling theswarm:compactdiscover-and-dispatch command (--run-id,--limit), how a singleSwarmCompactorcycle works (lease → seal barrier → fold → graduate → reclaim, ordering-spine invariant), the retention horizon (graduate on the compaction clock, expire on theswarm:pruneclock), the context-growth budget knobs, the recovery/quarantine decision tree (compaction_quarantined_at), and the v0.14.x→v0.15.0 rolling-deploy note.docs/streaming-substrate-author-guide.mdis the author side: streamingTopology::Hierarchicalswarms (synchronous coordinator + streamed workers), "structure as payload" and folding the log withCausalLogView(ViewOrder,ViewSupersession,VoidedEvent), declaring'type' => 'rollup'plan nodes for context bounding, and the#[ContextGrowthPolicy(GrowthPolicy::…)]ladder. Per-release surface discipline:docs/public-surface.mdgains theswarm:compactcommand, the#[ContextGrowthPolicy]attribute, theCausalLogStore/ColdArchiveDriver/TieredStreamEventStore/GrowthPolicy/ContextBudgetExceededExceptionsurfaces, the'type' => 'rollup'plan node, the newswarm.compaction.*/swarm.context_growth.*/swarm.hierarchical.stream_parallel_branchesconfig keys, and adigestNodeIdcorrection onVoidedEvent;docs/events.mdgains a Structural Stream Events section (theswarm_node_*grammar,swarm_causal_void_edge+CausalVoidEdgeType, and theswarm_causal_seal_barrier/SwarmUnknownEventinfrastructure markers). Docs only — no production code change.