Skip to content

Projections / Persistence Query — read-side query layer #36

@pathosDev

Description

@pathosDev

Today Journal.read(pid, fromSeq) lets you scan an event stream once. What's missing is a continuous subscribe — read events as they're appended, materialise read-models, persist offsets so projections survive restart.

Strategy: pull-model first. The first iteration polls the journal at a configurable interval (default 1 s) for new events. Simple, robust, deterministic. Push-model (subscribe via EventStream) follows later when workload demands it.

API addition on the Journal interface:

interface PersistenceQuery {
  eventsByPersistenceId(pid, fromSeq): AsyncIterator<PersistentEvent>;
  eventsByTag(tag, fromOffset): AsyncIterator<PersistentEvent>;
}

ProjectionActor lifecycle:

  • preStart: load offset from offset-store (SQLite / DurableStateStore).
  • onReceive: pump events from the query iterator, run user handler, advance offset.
  • Idempotency: handler must be safe to call twice for the same event (event re-delivery is allowed).
  • Restart: pick up from last persisted offset.

Components:

File Task
src/persistence/query/PersistenceQuery.ts (new) interface
src/persistence/query/InMemoryQuery.ts (new) for InMemoryJournal
src/persistence/query/SqliteQuery.ts (new) WHERE seq > ? OR tags LIKE '%t%'
src/persistence/query/CassandraQuery.ts (new) token-range scan + tags secondary index
src/persistence/projection/ProjectionActor.ts (new) actor wrapper + offset-store + at-least-once delivery
tests/unit/persistence/projection/*.test.ts (new) round-trip, restart, idempotency tests
examples/persistence/projection-bank-statement.ts (new) materialises account histories from existing bank-account events

Estimate: 4-6 days. Orthogonal to #34 / #35 — can land in parallel with the cluster work.

Verification:

  • 4 tests: round-trip, tag filter, restart-from-offset, concurrent-writers-don't-deadlock.
  • Example: bank-statement projection over examples/persistence/bank-account.ts.

Out of scope (for v1): push-based subscribe (would integrate with EventStream), per-event delivery guarantees beyond at-least-once, projection restart from arbitrary point in time.

See the roadmap plan for full context (item 4 of 5).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions