Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix 00975_move_partition_merge_tree #30717

Merged
merged 3 commits into from
Oct 27, 2021
Merged

Fix 00975_move_partition_merge_tree #30717

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
66285fa
Revert "Fix race between MOVE PARTITION and merges/mutations for Merg…
tavplubix Oct 26, 2021
901f386
fix
tavplubix Oct 26, 2021
5d96466
Update StorageMergeTree.cpp
alesapin Oct 27, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 3 additions & 5 deletions src/Storages/StorageMergeTree.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1179,6 +1179,7 @@ bool StorageMergeTree::optimize(

ActionLock StorageMergeTree::stopMergesAndWait()
{
/// TODO allow to stop merges in specific partition only (like it's done in ReplicatedMergeTree)
std::unique_lock lock(currently_processing_in_background_mutex);

/// Asks to complete merges and does not allow them to start.
Expand Down Expand Up @@ -1332,6 +1333,7 @@ void StorageMergeTree::replacePartitionFrom(const StoragePtr & source_table, con
{
auto lock1 = lockForShare(local_context->getCurrentQueryId(), local_context->getSettingsRef().lock_acquire_timeout);
auto lock2 = source_table->lockForShare(local_context->getCurrentQueryId(), local_context->getSettingsRef().lock_acquire_timeout);
auto merges_blocker = stopMergesAndWait();
auto source_metadata_snapshot = source_table->getInMemoryMetadataPtr();
auto my_metadata_snapshot = getInMemoryMetadataPtr();

Expand Down Expand Up @@ -1403,13 +1405,9 @@ void StorageMergeTree::replacePartitionFrom(const StoragePtr & source_table, con

void StorageMergeTree::movePartitionToTable(const StoragePtr & dest_table, const ASTPtr & partition, ContextPtr local_context)
{
/// MOVE PARTITION cannot be run in parallel with merges/mutations,
/// since otherwise there can be some merge/mutation in progress,
/// that will be created in the source table after MOVE PARTITION.
std::unique_lock background_lock(currently_processing_in_background_mutex);

auto lock1 = lockForShare(local_context->getCurrentQueryId(), local_context->getSettingsRef().lock_acquire_timeout);
auto lock2 = dest_table->lockForShare(local_context->getCurrentQueryId(), local_context->getSettingsRef().lock_acquire_timeout);
auto merges_blocker = stopMergesAndWait();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!
P.S. I though that this will be too much... And by some reason I though that holding currently_processing_in_background_mutex was enough, which is obviously is incorrect, since new part will be commited only when it will be written/renamed.


auto dest_table_storage = std::dynamic_pointer_cast<StorageMergeTree>(dest_table);
if (!dest_table_storage)
Expand Down