-
Notifications
You must be signed in to change notification settings - Fork 7.8k
Fix AzureBlobStorage copy #87826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix AzureBlobStorage copy #87826
Conversation
|
After some manual tests I see its not possible to compare authentication methods in Azure Blob Storage especially for WorkloadIdentity (& ManagedIdenitty). So I have updated the approach, Now we first try native copy & if we get unauthorized error we go for read & write. I have tested manually with WorkloadIdentity that native copy works. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR relaxes the condition under which native copy is performed, which seems Okay.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
|
Made a small fix for applying setting "use_native_copy", (https://github.com/ClickHouse/ClickHouse/pull/87826/files#diff-efbc3c03efb87660e06a2c7d91d2de5a25ff91611d5f60167636a7e5d169bd9cR630-R634) Previously the map contained url (as given in config) & settings. When reading it we use |
|
PTAL at tests in private: https://github.com/ClickHouse/clickhouse-private/pull/38961#issuecomment-3369805601 |
src/Disks/ObjectStorages/AzureBlobStorage/AzureBlobStorageCommon.cpp
Outdated
Show resolved
Hide resolved
|
Let's publish the submodule fix as a separate PR? |
|
Moved submodule change to different PR : #88278 |
|
Unrelated fails : |
Cherry pick #87826 to 25.7: Fix AzureBlobStorage copy
Cherry pick #87826 to 25.8: Fix AzureBlobStorage copy
Cherry pick #87826 to 25.9: Fix AzureBlobStorage copy
Backport #87826 to 25.9: Fix AzureBlobStorage copy
Backport #87826 to 25.8: Fix AzureBlobStorage copy
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
In AzureBlobStorage, updated to try native copy first and go to read & write on 'Unauthroized' error (In AzureBlobStorage, if storage accounts are different for source & destination we get 'Unauthorized' error). And fix applying "use_native_copy" when endpoint is defined in configuration.
Documentation entry for user-facing changes