New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use more thread pools in BACKUP/RESTORE to avoid its hanging in tests #55216
Use more thread pools in BACKUP/RESTORE to avoid its hanging in tests #55216
Conversation
This is an automated comment for commit da5f48e with description of existing statuses. It's updated for the latest CI running ❌ Click here to open a full report in a separate page Successful checks
|
…USTER ASYNC. Create thread pools lazily.
ca01b5a
to
8966ae4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Too complex and general solution, which doesn't protect from the mentioned error. But ok, let's do it this way.
src/Backups/BackupsWorker.cpp
Outdated
private: | ||
const size_t num_backup_threads; | ||
const size_t num_restore_threads; | ||
std::unique_ptr<ThreadPool> thread_pools[static_cast<size_t>(ThreadPoolId::MAX)] TSA_GUARDED_BY(mutex); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe better to make unordered map for simplicity?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@vitlibar WDYT? |
I rewrote my code using |
Why it doesn't protect? I mean now the parallel tasks waiting each others are going to be always in different thread pools. And the reason of that deadlock was exactly that a task could wait another task from the same thread pool. |
5d1602f
to
da5f48e
Compare
Changelog category:
This PR is a follow-up to #55132, it adds two more thread pools to handle
BACKUP ON CLUSTER ASYNC
andRESTORE ON CLUSTER ASYNC
without possible hanging. The PR also introduces lazily creation of such thread pools.