Skip to content

CRDTs + Replicated Event Sourcing #37

@pathosDev

Description

@pathosDev

Depends on #34 (Multi-Node-Spec) for convergence testing. This is the largest single roadmap item — recommended last.

Two related concepts in two phases:

Phase 5a: CRDT library (standalone, no persistence integration)

Conflict-free replicated data types — distributed structures that converge under replication without coordination. Akka has ddata (GCounter, PNCounter, GSet, ORSet, LWWMap, ORMap, …); we ship a starter set:

  • GCounter (grow-only counter)
  • PNCounter (positive/negative counter)
  • GSet (grow-only set)
  • ORSet (observed-remove set)
  • LWWRegister (last-writer-wins register)

Each implements merge(a, b) → c (idempotent, commutative, associative) and value(): T. A DistributedData extension on the cluster wraps them in a gossip-replicated key-value store — update(key, fn), subscribe(key), delete(key).

Phase 5b: Replicated Event Sourcing

On the persistence layer. Multiple nodes can write events for the same persistenceId in parallel; a conflict-resolution mechanism (vector clocks + pluggable resolver: LWW or custom merge) merges divergent histories deterministically.

Components:

File Task
src/crdt/{GCounter,PNCounter,GSet,ORSet,LWWRegister}.ts (new) pure CRDT data types
src/crdt/DistributedData.ts (new) extension + gossip-replicated key-value store
src/persistence/ReplicatedEventSourcedActor.ts (new) phase 5b
src/persistence/replicated/VectorClock.ts (new) logical timestamps
src/persistence/replicated/ConflictResolver.ts (new) LWW + custom merge plugin point
tests/unit/crdt/** (new) property tests for idempotency / commutativity / associativity
tests/multi-node/replicated-es.test.ts (new) requires #34
examples/crdt/shopping-cart-orset.ts (new) classic OR-Set demo
examples/persistence/replicated-counter.ts (new) replicated event-sourced counter

Estimate: 8-12 days. This estimate is optimistic if we want provably-correct garbage collection of old vector-clock entries — which we explicitly defer to v2.

Out of scope (for first iteration):

  • Cross-DC replication
  • VC garbage collection
  • Delta-CRDTs (we do state-based for simplicity first)

Verification:

  • CRDT property tests: merge(a, a) === a (idempotent), merge(a, b) === merge(b, a) (commutative), merge(merge(a, b), c) === merge(a, merge(b, c)) (associative). Property-based via fast-check or written by hand.
  • Multi-node tests: 3-node cluster, parallel updates from each, convergence check after gossip quiesces.

See the roadmap plan for full context (item 5 of 5).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions