Skip to content

Issue 4804: Fix txn compaction recovery leaving entry logs unreclaimed#4803

Open
void-ptr974 wants to merge 1 commit into
apache:masterfrom
void-ptr974:fix/default-entrylogger-compaction-recovery-hex
Open

Issue 4804: Fix txn compaction recovery leaving entry logs unreclaimed#4803
void-ptr974 wants to merge 1 commit into
apache:masterfrom
void-ptr974:fix/default-entrylogger-compaction-recovery-hex

Conversation

@void-ptr974
Copy link
Copy Markdown
Contributor

@void-ptr974 void-ptr974 commented May 27, 2026

Fixes #4804

Problem

DefaultEntryLogger.incompleteCompactionLogs() rebuilds txn compaction logs from *.compacted files after a restart. Entry log file names are hex-based, but this recovery path parsed the destination log id from the hex file name and then rebuilt the final .log file name from the numeric id directly.

For example, 1f4.log.78.compacted has destination log id 1f4, which is decimal 500. Recovery should make 1f4.log available, but the old path targeted 500.log.

Impact

For affected log ids, makeAvailable() creates the recovered log under the decimal file name. Later scan/recovery resolves log id 500 through the normal hex file name and looks for 1f4.log.

That mismatch prevents txn compaction recovery from scanning and finalizing the recovered log. The .compacted marker and old source entry log remain on disk, so disk space for that compaction is not reclaimed. Ledger entry contents are not changed.

Fix

Build the recovered final .log file name with logId2HexString(compactionLogId).

Coverage

  • Recovery creates 1f4.log for log id 500 and does not create 500.log.
  • Normal txn compaction still creates the hex-named final log.
  • Upgrade case: if an older recovery attempt already left a stale 500.log while the .compacted marker remains, fixed recovery still creates and scans 1f4.log and leaves the unrelated stale file untouched.

Testing

  • mvn -pl bookkeeper-server -Dtest=DefaultEntryLogTest#testRecoverCompactedLogUsesHexFinalLogFileName+testRecoverCompactedLogWithStaleDecimalLogFileFromOldRecovery+testNewCompactionLogKeepsHexFinalLogFileName test
  • mvn -pl bookkeeper-server -Dtest=DefaultEntryLogTest test

@void-ptr974 void-ptr974 changed the title Fix compacted entry log recovery filename Fix compaction recovery bug caused by decimal entry log filenames May 27, 2026
@void-ptr974 void-ptr974 changed the title Fix compaction recovery bug caused by decimal entry log filenames [Bug] Txn compaction recovery can fail after bookie restart with compacted entry logs May 27, 2026
@void-ptr974 void-ptr974 force-pushed the fix/default-entrylogger-compaction-recovery-hex branch 4 times, most recently from 68bf7a2 to 91c4a33 Compare May 28, 2026 00:37
@void-ptr974 void-ptr974 changed the title [Bug] Txn compaction recovery can fail after bookie restart with compacted entry logs Issue 4804: Fix txn compaction recovery leaving entry logs unreclaimed May 28, 2026
@void-ptr974 void-ptr974 force-pushed the fix/default-entrylogger-compaction-recovery-hex branch from 91c4a33 to d3dd87d Compare May 28, 2026 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Txn compaction recovery can leave entry logs unreclaimed after restart

1 participant