Skip to content

feat(store): phase-aware subscribe — Step on subscribe, decode composes (#250)#278

Merged
joeldsouzax merged 6 commits into
mainfrom
feat/250-subscription-consumption
Jul 5, 2026
Merged

feat(store): phase-aware subscribe — Step on subscribe, decode composes (#250)#278
joeldsouzax merged 6 commits into
mainfrom
feat/250-subscription-consumption

Conversation

@joeldsouzax

Copy link
Copy Markdown
Contributor

Resolves #250. Builds on #249 (now in main).

What & why

The catch-up→live boundary is what makes a subscription a subscription (a finite read has no such boundary). So subscribe/subscribe_all now yield Step<PersistedEnvelope> / Step<(AllPosition, PersistedEnvelope)> directly — the CaughtUp marker the live loop already detects rides all the way to the consumer, instead of being thrown away by raw subscribe and re-added by a separate subscribe_typed/cursor method.

Phase and decode are orthogonal axes, composed by combinators — never welded into a method name:

subscribe(&id, from)              -> Step<PersistedEnvelope>            // raw + phase
  .events()                       -> PersistedEnvelope                  // drop phase
  .events().decoded(codec)        -> Decoded<E>                         // typed, no phase (#249)
  .decoded(codec)                 -> Step<Decoded<E>>                   // typed + phase (projections)
  .events().for_each_decoded(..)  -> zero-copy borrowing fold (KERI/rkyv)

Consuming catch-up→live is now: match Step::CaughtUp / Step::Event(d) — no magic count-drain, no hand-rolled timeout_next, nexus-owned error.

Changes

Breaking to raw subscribe + #249's .decoded() — both pre-freeze/experimental; the seam is fixed now because it is frozen forever at 1.0.

Tests (4 mandatory categories, three layers)

  • phase_subscription_tests (end-to-end): exactly-one-CaughtUp across a 2600-event multi-chunk backlog (latch survives scan reopens); $all position-order replay; resume-from-checkpoint/head; .events() strip + composition; decode error never swallowed into a spurious CaughtUp; $all concurrency.
  • step_stream_ext_tests (combinator isolation + proptest): pathological Step sequences, error-domain routing, tag preservation.
  • nexus-fjall (real LSM adapter): CaughtUp survives close/reopen replay and >CATCHUP_CHUNK ScanCursor reopens.

🤖 Generated with Claude Code

joeldsouzax and others added 6 commits July 5, 2026 17:11
Delete docs/plans/ (99 files) and docs/prompts/ (9 files) wholesale, and
strip the 10 now-dangling `docs/plans/…` pointers from CLAUDE.md. The
design rationale those pointers referenced is already captured inline in
each CLAUDE.md module bullet, so nothing substantive is lost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Design/plan docs stay on disk for review + subagent execution but are
never committed; the design intent that matters lands in CLAUDE.md and
the PR body. Complements the earlier purge of the committed plan docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…es (#250)

The catch-up→live boundary is what makes a subscription a subscription, so
`subscribe`/`subscribe_all` now yield `Step<PersistedEnvelope>` /
`Step<(AllPosition, PersistedEnvelope)>` directly — the CaughtUp marker the
live loop already detects rides all the way to the consumer. Phase and decode
are orthogonal axes composed by combinators, never welded into a
`subscribe_typed`/`cursor` method:

  StepStreamExt::events()   — drop the phase (then DecodedStreamExt applies)
  StepStreamExt::decoded()  — decode and KEEP the phase -> Step<Decoded<E>>

Adds Step::map; deletes the internal events-only `live` filter (its logic is
now the public `.events()`); subscribe builds on `live_stepped`. Migrates the
#227 example off the magic count-drain + timeout_next onto CaughtUp. Existing
events-only consumers gain `.events()`.

Breaking change to raw subscribe + #249's `.decoded()` — both pre-freeze; the
seam is fixed now because it is frozen forever at 1.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the 4 mandatory test categories across three layers:

- phase_subscription_tests (InMemoryStore, end-to-end): the CaughtUp latch
  survives internal scan reopens (exactly one CaughtUp across a 2600-event,
  multi-chunk backlog, only after the final event); $all replays in position
  order with one CaughtUp; resume-from-checkpoint / resume-from-head phase
  semantics; .events() phase-strip + .events().decoded() composition; a decode
  error mid-backlog is never swallowed into a spurious CaughtUp; $all CaughtUp
  exactly once under concurrent multi-stream writers.
- step_stream_ext_tests (combinator isolation, synthetic Step streams): .events()
  drops every marker (incl. multiple/interleaved) and passes Err through in
  place; .decoded() preserves phase, keeps the $all tag, and routes Read vs
  Decode to distinct variants; a proptest asserts .events() yields exactly the
  Event payloads for any interleaving.
- nexus-fjall subscription_tests (real LSM adapter): CaughtUp survives a full
  close/reopen replay from disk, and exactly one CaughtUp across a >CATCHUP_CHUNK
  backlog that forces real ScanCursor reopens.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@joeldsouzax joeldsouzax enabled auto-merge (squash) July 5, 2026 15:12
@joeldsouzax joeldsouzax merged commit d3b414d into main Jul 5, 2026
4 checks passed
@joeldsouzax joeldsouzax deleted the feat/250-subscription-consumption branch July 5, 2026 15:21
@github-actions github-actions Bot mentioned this pull request Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[freeze] Subscription consumption is awkward: no catch-up↔live boundary signal, no bounded next, adapter error leaks into consumer bounds

1 participant