Skip to content

v2.0.0: Early Write-Scaling Milestone

Latest
Latest

Choose a tag to compare

View all tags
@MartinKalema MartinKalema released this 02 Apr 10:15
· 104 commits to main since this release
v2.0.0
27c8ba2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Historical note, June 2026: This release records the first partitioned write-scaling milestone. Subsequent work expanded the architecture with Raft-backed durability, 2PC recovery hardening, route authority, replica replay safeguards, and clearer production-readiness boundaries. For the current milestone and active roadmap, see v2.1.0-alpha.1 and docs/issue-journal.md.

Horizontal Write Scaling for Convex

The first horizontal write scaling implementation for the Convex open-source backend. Combines patterns from CockroachDB, TiDB, Vitess, YugabyteDB, and Google Spanner to scale a reactive database with real-time subscriptions, in-memory OCC, and snapshots.

Architecture

  • Partitioned multi-writer: Two independent writer nodes, each owning specific tables, writing in parallel
  • Global Timestamp Oracle: Batch TSO via NATS KV with atomic CAS (TiDB PD pattern) — zero network calls in hot path
  • Two-phase commit: Coordinator detects cross-partition writes, orchestrates prepare/commit/rollback (Vitess 2PC pattern)
  • Raft-pattern persistence pipeline: All writes — local commits, max_repeatable_ts bumps, and replica deltas — serialized through single FuturesOrdered queue (CockroachDB/TiKV pattern)
  • Delta replication: NATS JetStream carries CommitDeltas between nodes with durable consumers, per-partition subjects, and self-delta skip
  • Table number reassignment: Remote _tables entries get locally-unique numbers to avoid collisions (CockroachDB descriptor ID pattern)
  • Replica timestamp isolation: Delta apply uses monotonic counters only — no TSO, no system clock (CockroachDB closed timestamp, TiDB resolved-ts pattern)
  • Transaction watcher: Background NATS KV scanner for 2PC crash recovery

Test Results

77 integration tests across 37 categories — all passing.

3,823 messages | 3,069 tasks | 1,390 sustained writes/node in 30 seconds | 200-doc batch replicated atomically | NATS partition survived | Full cluster restart recovery | Zero data loss.

Test patterns from CockroachDB nightly Jepsen (bank, register, sequential, set, monotonic, G2, comments), TiDB Jepsen (bank-multitable, stale read), YugabyteDB Jepsen (counter), Vitess (VDiff, partition enforcement, 2PC), CockroachDB roachtest (KV scaling, nemesis, workload check), Chaos Mesh (NATS partition), Elle anomaly classes (read skew, write skew), and boundary testing.

Since v1.0.0

v1.0.0 scaled reads with Primary-Replica architecture. v2.0.0 scales writes with partitioned multi-writer, adds 2PC for cross-partition writes, wires a global TSO, and proves correctness with 77 Jepsen-inspired tests.

Documentation

Assets 2
Loading