Skip to content

TransactionTest2.transactionsConcurrentWithPersistitClose residual flake: commit racing Persistit.close() throws raw IllegalStateException instead of PersistitClosedException #304

Description

@vharseko

Observed failure

master push run 30078532797, matrix cell build-maven (ubuntu-latest, 21) — 8 of 9 cells green, single failure in persistit/core:

[ERROR] com.persistit.TransactionTest2.transactionsConcurrentWithPersistitClose -- Time elapsed: 1.194 s <<< FAILURE!
java.lang.AssertionError: All threads should have exited correctly expected:<0> but was:<1>

The worker thread's stack trace printed alongside the assertion:

java.lang.IllegalStateException: JOURNAL_FLUSHER is not running
	at com.persistit.JournalManager.waitForDurability(JournalManager.java:1860)
	at com.persistit.Transaction.commit(Transaction.java:897)
	at com.persistit.TransactionTest2.transfer(TransactionTest2.java:380)
	at com.persistit.TransactionTest2.runIt(TransactionTest2.java:313)
	at com.persistit.TransactionTest2$3.run(TransactionTest2.java:272)

Analysis

This is a residual failure mode of the flake previously stabilized by #255, with a different mechanism that the #255 fix (deadline-bounded join before the assert) cannot help with:

  1. The test's main thread calls _persistit.close(); JournalManager.close() nulls _flusher (JournalManager.java:1327).
  2. A worker thread is concurrently inside Transaction.commit() and reaches JournalManager.waitForDurability(), which finds _flusher == null and throws a raw IllegalStateException("JOURNAL_FLUSHER is not running") (JournalManager.java:1860).
  3. IllegalStateException is not a PersistitException, so in TransactionTest2.runIt it bypasses the expected-shutdown catch clauses (PersistitClosedException, PersistitInterruptedException, interrupted-I/O PersistitIOException) and lands in the generic catch (Throwable) branch, which records a failure and does not decrement _strandedThreads.
  4. The assertion then fails with expected:<0> but was:<1> regardless of how long the join waits — the worker has already exited, but the counter can no longer reach zero.

Root cause

A transaction that races with Persistit.close() surfaces the shutdown as a raw IllegalStateException instead of the documented PersistitClosedException. Callers (including this test) that treat PersistitClosedException as the expected outcome of a concurrent close cannot distinguish this from a genuine internal error.

Candidate fixes

  • Product side (preferred): JournalManager.waitForDurability() should throw PersistitClosedException when the flusher has been stopped by close(), so a concurrent commit() unwinds through the same path as every other closed-state operation.
  • Test side (fallback): treat IllegalStateException("JOURNAL_FLUSHER is not running") during shutdown as an expected exit in TransactionTest2.runIt.

Metadata

Metadata

Assignees

Labels

bugtestsTest code changes

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions