Skip to content

v0.6.1 — a failing replica stops replication, it does not hang commits

Choose a tag to compare

@mariuz mariuz released this 02 Aug 09:13
· 2 commits to main since this release

A correction. v0.6.0 claimed that a synchronous replica which dies hangs commits, and made rather a lot of that claim. Dmitry Sibiryakov pointed out on firebird-general that this is wrong:

Synchronous replica that dies doesn't hang commit because it has no durability guarantees from the beginning. It simply self-destructs instead.

He is right. The source confirms it, in src/jrd/replication/Publisher.cpp on master:

  • disable_on_error defaults to true (Config.cpp). On error, checkStatus() clears TRA_replicating, disposes the replicator, clears ATT_replicating and logs STOP_ERROR.
  • The commit path calls checkStatus(tdbb, status, transaction, false)canThrow = false — so it cannot throw, even when report_errors is enabled.
  • DEFAULT_REPORT_ERRORS is false in any case.

So a failing replica never blocks a commit. Replication tears itself down and the primary carries on. There is no durability guarantee at stake, because Firebird's synchronous replication is not two-phase commit and never offered one — which was Sibiryakov's actual point, and the part I had inverted.

The practical consequence is arguably worse than the hang I had invented: replication stops, commits keep succeeding, nobody is told, and the replica silently rots until somebody notices and turns it back on.

What changed

  • Synchronous + unreachable replica now self-disables once, logging a STOP_ERROR, and commits continue unaffected.
  • Synchronous replication no longer writes journal segments. It sends changes down a live connection; segments are an asynchronous-only structure. The earlier model conflated the two.
  • Asynchronous + unreachable replica is unchanged — the journal keeps growing, which was right, and remains the real disk-fill hazard.
  • A returning replica does not silently resume stopped replication. It has to be switched back on, as in reality.
  • The stats bar reads STOPPED rather than HUNG.

The test suite was enforcing the bug

The old assertion checked that commits hang, so it faithfully protected the wrong behaviour. It is replaced by seven assertions covering the corrected model — including that commits keep flowing while replication is stopped, that exactly one STOP_ERROR is logged rather than one per commit, that async is unaffected by a replica going away, and that synchronous writes no segments. 138 assertions, all passing.

KNOBS.md and the README are corrected, cite the source files, and record that the earlier claim was wrong rather than quietly rewriting history.

Thanks to Dmitry Sibiryakov for the correction. This is exactly why every announcement asks for them.