Add level merge to "merge" command#2605
Conversation
jt2594838
left a comment
There was a problem hiding this comment.
How you intend to solve the problem is a bit out of my expectation. Removing current locks may introduce other issues potentially.
I think the problem is actually one merge does not know that another merge is compacting its chosen files. Can we avoid this from the beginning somehow? For example, a merging mark for those files which are already chosen.
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
Outdated
Show resolved
Hide resolved
Why it affects query? |
Every time this ci problem occurs, there will be a file not found error and it is all windows ci. Although the wrong test was not manually merged, I guessed that it was because the last test reported 'file not found' error affected this test |
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
Outdated
Show resolved
Hide resolved
|
SonarCloud Quality Gate failed. |
Currently, the cli


mergecan just execute unseq compaction, it is not conveniently for user to execute compaction. So in this PR, I change themergecommand to execute both level compaction and unseq merge. And also, this PR may fix the 'file not found' ci problem.There is a windows ci problem now.
Every time this ci problem occurs, there will be a file not found error and it is all windows ci. Although the wrong test was not manually merged, I guessed that it was because the last test reported 'file not found' error affected this test
So I speculate that it is caused by an unseq compaction problem.
This error happens because we submit an unseq compaction task manually when a level compaction is working, the unseq compaction will get the current tsfile list and wait for the level compaction to be finished. However, the level compaction may merge the file which will be used in the unseq compaction process. The unseq compaction then cannot find the file to use. Then cause the ci problem.
So in this pr, we change the function of
mergecli from the execution of unseq compaction to the execution of level compaction+unseq compaction. It just starts a normal compaction process which do not have this file-related problem.