Skip to content

Commit

Permalink
[BugFix] Fix pindex read-write concurrency between commit and apply (#…
Browse files Browse the repository at this point in the history
…33220)

During the execution of apply on the primary key table, the apply thread maybe clear or modify _l1_vec of persistent index.
And during the execution commit, the commit thread will update primary index memory usage and need to access _l1_vec.
So there are read/write conflicts between apply thread and commit thread which may cause BE crash. And we add _lock to prevent the read-write concurrency in previous pr but there are some scenarios where locks were inadvertently missed.

Signed-off-by: zhangqiang <qiangzh95@gmail.com>
(cherry picked from commit 6bf3e88)
  • Loading branch information
sevev authored and mergify[bot] committed Oct 20, 2023
1 parent 0349297 commit edf51e4
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions be/src/storage/persistent_index.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4167,14 +4167,15 @@ Status PersistentIndex::_merge_compaction_advance() {
RETURN_IF_ERROR(writer->finish());
std::vector<std::unique_ptr<ImmutableIndex>> new_l1_vec;
std::vector<int> new_l1_merged_num;
size_t merge_num = _l1_merged_num[merge_l1_start_idx];
for (int i = 0; i < merge_l1_start_idx; i++) {
new_l1_vec.emplace_back(std::move(_l1_vec[i]));
new_l1_merged_num.emplace_back(_l1_merged_num[i]);
}

size_t merge_num = 0;
{
std::unique_lock wrlock(_lock);
merge_num = _l1_merged_num[merge_l1_start_idx];
for (int i = 0; i < merge_l1_start_idx; i++) {
new_l1_vec.emplace_back(std::move(_l1_vec[i]));
new_l1_merged_num.emplace_back(_l1_merged_num[i]);
}

for (int i = merge_l1_start_idx; i < _l1_vec.size(); i++) {
_l1_vec[i]->destroy();
}
Expand Down

0 comments on commit edf51e4

Please sign in to comment.