Skip to content

[To rel/1.0][IOTDB-5147]Optimize compaction schedule when priority is BALANCE#9172

Merged
xingtanzjr merged 2 commits intoapache:rel/1.0from
choubenson:CompactionComparator1.0
Mar 3, 2023
Merged

[To rel/1.0][IOTDB-5147]Optimize compaction schedule when priority is BALANCE#9172
xingtanzjr merged 2 commits intoapache:rel/1.0from
choubenson:CompactionComparator1.0

Conversation

@choubenson
Copy link
Contributor

The changes in this pr are as follows:

  1. Change the size of the candidateCompactionTaskQueue from 1000 to 50. The advantage is to avoid tasks blocked in the queue occupying files for a long time, resulting in no files being selected for cross-space compaction. But another impact is that once the number of files is large, because the number of tasks in the queue shrinks, the inner-space compaction will not be able to compact the latest files.
  2. Priority comparison of tasks: Inner-space compaction tasks no longer distinguish between seq and unseq, but are sorted according to unified rules; under the configuration of BALANCE, cross-space compaction tasks and inner-space compaction tasks follow the first-come-first-executed rule . One thing to note is that since the priority sorting of the queue is implemented through the minimum heap sorting, and each task is not in partial order, the actual result is not strictly that the one with the smaller seriesID will be executed first.
  3. Inner seq task and inner unseq task is submitted one by one, to avoid submitting too many inner seq tasks at one time, filling up the queue, resulting in no opportunity to submit inner unseq tasks.

@xingtanzjr xingtanzjr merged commit 482922e into apache:rel/1.0 Mar 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants