Compaction does not remove entry files when remaining file size < Compaction Threshold

[sourabhaggrawal]

BUG REPORT

Describe the bug
Whenever major or minor compaction runs, it does not remove the log files which has utilised more than threshold value. But only remove the files with usage < threshold.

A clear and concise description of what the bug is.
I am seeing high disk usage in ledger directory in 3 bookie cluster, all 3 nodes have couple of log files with size 1.1GB which is the default max entry log file size before it creates a new one.
As per the documentation here for Major and Minor Compaction, it is supposed to delete the log files when remaining log file size reaches below threshold.
When trying to debug this in code found that condition is checking for meta.Usage > threshold and ignore the meta file which is supposed to get deleted, what I feel it should be checking the meta.Remaining > Threshold.

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java

Line 573 in dd80e96

if (meta.getUsage() >= threshold || (maxTimeMillis > 0 && timeDiff.getValue() >= maxTimeMillis)

To Reproduce

Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

• Compaction should have removed the file if remaining file size < threshold.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

[dlg99]

When trying to debug this in code found that condition is checking for meta.Usage > threshold and ignore the meta file which is supposed to get deleted, what I feel it should be checking the meta.Remaining > Threshold.

meta.remaining is bytes. Threshold is a cutoff utilization (as percentage), meta.usage is meta.remainingSize / meta.totalSize.
remainingSize is the size used by ledgers/entries that are live (not deleted).
As the docs say, For those entry log files whose remaining size **percentage** reaches below this threshold will be compacted

Let's say total size of the entry log is 1000 Mbytes, remainingSize is 750 Mbytes. So the usage is 0.75
In case threshold set to 0.6 the compaction won't be performed on this entry log because the usage > threshold.

The logic there is the i/o is not free and rewrite of 750Mbytes to free up a bit of space will be a perf hit on the bookie. If the disk space is a concern and the perf is not, you can set the threshold higher (0.8 or so) and this log will be compacted.

[sourabhaggrawal]

Thanks for helping me understand it more @dlg99 , I will close the issue.