fix(core): fix WAL segment housekeeping edge case leading to a failure to write to WAL table

[amunra]

Fixed a possible but unlikely scenario where the WalPurgeJob could delete a wal segment before it was applied.

The scenario consists of the following:

• The purge job scans for wal segments and tracks a locked wal segment (in this scenario (1,6).
• The WalPurgeJob thread is parked.
• In the meantime, another wal segment (1,7) is created and then closed and unlocked.
• The WalPurgeJob thread is re-scheduled.
• It discovers (1,7) as finds it to be unlocked.

Faulty logic assumed that a locked segment would always be the last segment for a WAL, when this is not the case.

The logic has now been simplified, extracted out to a separate class for testability and the bug fixed.

[amunra]

It's a little difficult to tell from the diff what actually changed in the logic.

The old logic accumulated state about what walId/segmentId is next to be applied in a LongList.

The bug was in this code:

// If the current segment is locked, scroll to next WAL id.
final int walIdLimit = segmentLocked ? walId + 1 : walId;    // <--- actual bug
while (committedWalId < walIdLimit && ++committedWalSegmentIndex < committedSegmentSize) {
    long sequencerPair = sequencerWalIDSegmentIDPairs.get(committedWalSegmentIndex);
    committedWalId = Numbers.decodeHighInt(sequencerPair);
    committedSegmentId = Numbers.decodeLowInt(sequencerPair);
}

In the scenario from the test case it ended up holding committedWalId == 2 and committedSegmentId == 0, when iterating segment (1,7).

This bad state was set up in the previous iteration of the loop and caused the wrong branch in the logic to be executed.

if (walId == committedWalId) {
    ...
} else if (!segmentLocked) {
    ...  // this code got executed when it should not have been.
}

Ultimately it was easier to replace the LongList with an IntIntHashMap and remove the complex iteration logic.

[ideoma]

[PR Coverage check]

😍 pass : 169 / 173 (97.69%)

file detail

	path	covered line	new line	coverage
🔵	io/questdb/cairo/wal/WalWriter.java	47	51	92.16%
🔵	io/questdb/cairo/wal/WalPurgeJob.java	121	121	100.00%
🔵	io/questdb/cairo/TableUtils.java	1	1	100.00%