Skip to content

V9.16.1

Choose a tag to compare

@jeremydmiller jeremydmiller released this 18 Jul 16:43

Async daemon data-safety release: the high water detection can no longer advance past "outstanding" event sequence numbers — sequences reserved by transactions that are still in flight — which could silently skip those events in async projections under concurrent append load (bulk imports being the classic case). Root-caused and fixed from discussion #4953.

The four closed mechanisms:

  • The GapDetector command batched three statements, each reading its own READ COMMITTED snapshot — commits landing mid-command could defeat every gap check and silently advance the mark over an in-flight append, regardless of StaleSequenceThreshold. Detection is now a single statement / single snapshot.
  • Projection rebuilds and forced catch-up looped the gap-skipping detection toward the reserved sequence last_value, mowing through in-flight gaps. CheckNowAsync (JasperFx.Events 2.29.1) now targets the highest committed sequence and simply waits for in-flight appends to land.
  • The stale fallback could teleport the mark to reserved last_value - 32 across thousands of in-flight reservations on an idle-then-suddenly-busy store, because its gate measured staleness against mt_event_progression.last_updated. The threshold is now measured from when each specific gap was first observed.
  • Wall-clock stale skipping could not tell a slow transaction from a rolled-back one. Before skipping any stale gap, Marten now checks PostgreSQL for evidence that a transaction which could still fill the gap is alive (pg_locks on the mt_events tables, open transactions in pg_stat_activity, in-progress write xids from pg_current_snapshot()), and holds while any exists — by default Marten never knowingly skips past a live appender. Only provably-dead gaps (rolled-back appends) are skipped, bounded to the sequence ceiling observed with the gap, and every skip is logged at Warning with its exact range.

New knobs on StoreOptions.Projections: UseTransactionEvidenceForGapSkipping (default true; false restores the previous wall-clock behavior) and SkipStaleGapsDespiteLiveTransactionsAfter (default null = never skip a live appender; PostgreSQL's idle_in_transaction_session_timeout is the recommended backstop against leaked sessions).

What's Changed

Full Changelog: V9.16.0...V9.16.1