-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
repair: accelerate repair load_history time #16927
repair: accelerate repair load_history time #16927
Conversation
@scylladb/scylla-maint please review and trigger CI for this pull request |
🟢 CI State: SUCCESS✅ - Build Build Details:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should be fine. @asias, was there a reason to load the history sequentially?
We can add some limited parallelism. If there are 10K tables, we do not want to load them all in parallelism. |
Did you see any improvements for your use case? |
🔴 CI State: ABORTED✅ - Build Build Details:
|
I think add a semaphore is a good idea. |
load_history hash using:
table_uuid:
They have are same in most_significant_bits‘s low bits. |
@scylladb/scylla-maint please review and trigger CI for this pull request |
The table id is supposed to be a make_random_uuid() which is random. Here is a sample run of loading different shards. You can see the selected shard is disturbed to all the shards.
|
You can add something like this: named_semaphore _load_parallelism_semaphore(16, named_semaphore_exception_factory{"Load repair history parallelism"}) to repair_service And add auto permit = co_await seastar::get_units(_load_parallelism_semaphore, 1); in side repair_service::load_history. This allows loading 16 tables per shard in parallel. |
3c4997d
to
246b3b2
Compare
@scylladb/scylla-maint please review and trigger CI for this pull request |
my load repair log don't look like it.
|
Thanks for your suggest. I pushed the new commit. |
@scylladb/scylla-maint please review and trigger CI for this pull request |
Convert the uuid to a shard_id which is 32 bits. It is useful to choose a shard from a uuid. Refs: scylladb#16927
Convert the uuid to a shard_id which is 32 bits. It is useful to choose a shard from a uuid. Refs: scylladb#16927
Convert the uuid to a shard_id which is 32 bits. It is useful to choose a shard from a uuid. Refs: scylladb#16927
Convert the uuid to a uint32_t using xor. It is useful to get a uint32_t number from the uuid. Refs: scylladb#16927
Convert the uuid to a uint32_t using xor. It is useful to get a uint32_t number from the uuid. Refs: scylladb#16927
Convert the uuid to a uint32_t using xor. It is useful to get a uint32_t number from the uuid. Refs: scylladb#16927
786d0cf
to
89e2a70
Compare
@scylladb/scylla-maint please review and trigger CI for this pull request |
Sorry I didn't notice that. |
89e2a70
to
b3f4711
Compare
@scylladb/scylla-maint please review and trigger CI for this pull request |
triggered |
🔴 CI State: FAILURE✅ - Build Failed Tests (2/24426):Build Details:
|
🔴 CI State: FAILURE✅ - Build Failed Tests (3/24426):
Build Details:
|
🔴 CI State: FAILURE✅ - Build Failed Tests (1/24426):Build Details:
|
@scylladb/scylla-maint please review and trigger CI for this pull request |
🔴 CI State: FAILURE✅ - Build Failed Tests (1/24453):Build Details:
|
@MyByte0 please do not merge master into you branch, please rebase it instead. |
CI failed on a known failure that was fixed yesterday - re-run. |
Using `parallel_for_each_table` instance of `for_each_table_gently` on `repair_service::load_history`, and parallel num 16 for each shard, to reduced bootstrap time.
Using uuid_xor_to_uint32 instance of table_uuid's most_significant_bits, optimize the hash conflict to shard.
1f4f095
to
9b675d1
Compare
@scylladb/scylla-maint please review and trigger CI for this pull request |
triggered. |
🟢 CI State: SUCCESS✅ - Build Build Details:
|
@MyByte0 there was a problem with your CLA submission, please check the reply you received. |
Fine. Updated it. |
Queued. |
Thanks @MyByte0 for your contribution - much appreciated! |
Convert the uuid to a uint32_t using xor. It is useful to get a uint32_t number from the uuid. Refs: scylladb#16927 Closes scylladb#17049
Using
parallel_for_each_table
instance offor_each_table_gently
onrepair_service::load_history
, to reduced bootstrap time.Using uuid_xor_to_uint32 on repair load_history dispatch to shard.
Ref: #16774