New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix IO throttling during copying whole directories #53338
Conversation
This is an automated comment for commit aaa68a5 with description of existing statuses. It's updated for the latest CI running
|
This is likely internal CI issue. And this PR does not have a test yet. |
May I ask when will it be repaired? @azat |
This should not be required for merging this PR, it just needs some attention. @serxa maybe you can take a look? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. As for tests, at least a simple check for the duration of explicit ALTER TABLE MOVE PARTITION would be good enough.
v2: pass local context during ALTER TABLE MOVE PARTITION TO VOLUME/DISK as well Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Yeah, I was just lazy and thought maybe someone will contribute the test in the mean time, but this did not happen, so I've added the test. |
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
…ttling ALTER TABLE MOVE PARTITION TO DISK/VOLUME should not be replicated since replicas will not contains this part: azat:~/ch/tmp/53338$ for i in clickhouse-server*.log.zst; do echo -n "$i: " && zstd -cdq $i | grep -m1 -e Executed.*ALTER.*test_cnmf4xnb.test_move_partition_throttling -e Exception.*ALTER.*test_cnmf4xnb.test_move_partition_throttling; done clickhouse-server1.log.zst: 2023.08.29 16:46:53.960065 [ 1843 ] {19b13d67-54c0-496d-96b9-b1ec09df3618} <Error> executeQuery: Code: 232. DB::Exception: Nothing to move (check that the partition exists). (NO_SUCH_DATA_PART) (version 23.8.1.2862) (from 0.0.0.0:0) (comment: 02870_move_partition_to_volume_io_throttling.sql) (in query: /* ddl_entry=query-0000000005 */ ALTER TABLE test_cnmf4xnb.test_move_partition_throttling MOVE PARTITION tuple() TO VOLUME 'remote' SETTINGS max_remote_write_network_bandwidth = 1600000), Stack trace (when copying this message, always include the lines below): clickhouse-server2.log.zst: 2023.08.29 16:46:53.959560 [ 1842 ] {3cd2b5e8-24b9-4cfd-aa47-854e634936f2} <Error> executeQuery: Code: 232. DB::Exception: Nothing to move (check that the partition exists). (NO_SUCH_DATA_PART) (version 23.8.1.2862) (from 0.0.0.0:0) (comment: 02870_move_partition_to_volume_io_throttling.sql) (in query: /* ddl_entry=query-0000000005 */ ALTER TABLE test_cnmf4xnb.test_move_partition_throttling MOVE PARTITION tuple() TO VOLUME 'remote' SETTINGS max_remote_write_network_bandwidth = 1600000), Stack trace (when copying this message, always include the lines below): clickhouse-server.log.zst: 2023.08.29 16:46:53.950730 [ 721 ] {df6b20ee-1903-404a-9398-7fead209ccd7} <Debug> DDLWorker(test_cnmf4xnb): Executed query: /* ddl_entry=query-0000000005 */ ALTER TABLE test_cnmf4xnb.test_move_partition_throttling MOVE PARTITION tuple() TO VOLUME 'remote' SETTINGS max_remote_write_network_bandwidth = 1600000 CI: https://s3.amazonaws.com/clickhouse-test-reports/53338/ed401cba9b8254b4a29a5ed9e5ad838c26ffaac1/stateless_tests__release__databasereplicated__[4_4].html Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fix IO throttling during copying whole directories (MOVE PARTITION TO VOLUME, fetching/cloning parts, mutations, ...)
P.S. help with tests is highly appreciated (for now I've tested only locally
MOVE PARTITION TO VOLUME
)Fixes: #52753 (cc @den-crane)
Fixes: #52183 (cc @coulsonm)