Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bugs around compaction #1807

Merged
merged 1 commit into from
Dec 13, 2023
Merged

Fix bugs around compaction #1807

merged 1 commit into from
Dec 13, 2023

Conversation

zipper-meng
Copy link
Member

@zipper-meng zipper-meng commented Dec 4, 2023

Required checklist

  • Sample config files updated (config,meta/config and default config)
  • If there are user-facing changes, the documentation needs to be updated prior to approving the PR( Link )
  • If there are any breaking changes to public APIs, please add the api change label.
  • Signed CLA (if not already signed)

Which issue does this PR close?

Related #.

Rationale for this change

Fix some bugs:

1. Compaction job failed to start;

CompactJobInner::enable_compaction is initialized as true but the job starts only when it compare_exchange(false, true) == Ok(false).

if self
.enable_compaction
.compare_exchange(
false,
true,
atomic::Ordering::Acquire,
atomic::Ordering::Relaxed,
)
.is_err()
{
info!("Compaction: enable_compaction is false, later to start vnode compaction job");
return;
}

2. Errors in compact::chunk_merged_block();

There may be an error when chunking big blocks, this makes data lost.

3. Merge two tmp-fields in CompactIterator for better performance;

Merge tmp_tsm_blk_meta_iters: Vec<BlockMetaIterator> and tmp_tsm_blk_tsm_reader_idx: Vec<usize> into tmp_tsm_blk_meta_iters: Vec<(usize, BlockMetaIterator)>.

4. Tombstone files not removed when removing TSM file

Tombstone files may be removed when dropping a ColumnFile.

Are there any user-facing changes?

…ged_block(); 3.merge two tmp-fields in CompactIterator for better performance; 4.tombstone files not removed when removing tsm file
@zipper-meng zipper-meng changed the title fix: 1.compaction job failed to start; 2.errors in compact::chunk_mer… fix: bus around compaction Dec 4, 2023
@zipper-meng zipper-meng changed the title fix: bus around compaction Fix bugs around compaction Dec 4, 2023
@roseboy-liu roseboy-liu merged commit cc1a11b into main Dec 13, 2023
7 checks passed
@roseboy-liu roseboy-liu deleted the fix/compaction branch December 13, 2023 03:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants