Skip to content

v1.8.0: opt-in batched fsync

Choose a tag to compare

@HarperZ9 HarperZ9 released this 25 Jun 08:39
· 161 commits to main since this release

The durable ledger fsyncs every append, the strongest guarantee and the right default. For a high-throughput run where that per-append fsync is the bottleneck, this release adds an opt-in way to batch it, with the tradeoff stated plainly.

  • FileStorage(dir, fsync_each=False): appends are written and flushed to the OS (so they survive a process crash) but not fsynced per call, for throughput. Call ledger.sync() to fsync the logs at a point of your choosing (a phase boundary, the run's end). The default stays fsync_each=True.
  • Ledger.sync() / Storage.sync(): a new seam to force buffered writes down. No-op for in-memory and for the default mode; the durability control for batched mode.
  • Honest tradeoff: a crash before sync() can lose the un-fsynced tail; the log stays append-only and the torn-trailing-line tolerance still applies, so what survives still verifies and replays. Power-loss durability rests on the OS honoring fsync; the process-crash guarantee is unconditional.

Pure standard library. 218 tests, plus 2 gated real-model tests.

Install: pip install forum-engine