Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IOTDB-1741] Fix double close in CompactionUtils #4186

Merged
merged 1 commit into from Oct 19, 2021

Conversation

THUMarkLau
Copy link
Contributor

@THUMarkLau THUMarkLau commented Oct 19, 2021

Description

During level compaction, the write stream of target file sometimes is closed unexpectedly.The error log is shown below.
image

Analysis

By consulting related information, we know that when an OuputStream is closed and then written, there will be a Stream Closed exception. We troubleshoot the problem based on this idea. But in the process of level compaction execution, tasks are executed in a single thread in a static function (as shown in the figure).
1

Moreover, the Writer of each target file is declared in a function, so this Writer and its corresponding Stream are local variables which cannot be accessed outside this function.
image

In the end of compaction, the Writer's endFile function wll be called, in which Writer.close will be called once, and all the contents in the Writer buffer will be written to the file at this time. After calling endFile, the close function of Writer was manually called again. Theoretically, the second close does not produce any exceptions, because the buffer has been emptied and the Writer will not write anything to the file. But because we can't find other reasons, we can only attribute the cause of this exception to the two calls of the close function for the time being.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.01%) to 67.457% when pulling a75c011 on THUMarkLau:to_master_fix_double_close into e4b7f64 on apache:master.

@qiaojialin qiaojialin changed the title [To master] Fix double close in CompactionUtils [IOTDB-1741] Fix double close in CompactionUtils Oct 19, 2021
@qiaojialin qiaojialin merged commit b807e4a into apache:master Oct 19, 2021
cornmonster pushed a commit to cornmonster/iotdb that referenced this pull request Oct 25, 2021
@THUMarkLau THUMarkLau deleted the to_master_fix_double_close branch November 30, 2021 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants