Send iceberg_metadata_file_path setting to swarm node #898
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Setting
iceberg_metadata_file_path
is sent to swarm node in distributed queries.Solve #897.
For working with Iceberg table clickhouse needs Iceberg metadata file. When info about metadata absent in query, clickhouse tries to find it on storage. For this clickhouse uses
listObject
request.https://github.com/Altinity/ClickHouse/blob/antalya-25.3/src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp#L350
S3 Tables don't support
listObject
request. S3 tables works only with catalog, and in single-node variant clickhouse gets it from catalog and adds to settings:https://github.com/Altinity/ClickHouse/blob/antalya-25.3/src/Databases/DataLake/DatabaseDataLake.cpp#L397
and later doesn't search anything, just read metadata by direct path. So this case works.
But in distributed requests swarm node does not have access to catalog, so tries to find metadata, and query failed with error
This PR adds
iceberg_metadata_file_path
to query when send it to swarm nodes.Actual changes only in
DataLakeConfiguration::createArgsWithAccessData
.https://github.com/Altinity/ClickHouse/pull/898/files#diff-589e6bce39764a9388b8e33f34dfa0d6b7c58d3874fd8b55a63ad02905cf4b90R219
But also I made refactoring, split file
DataLakeConfiguration.h
onh
andcpp
parts, all other methods moved as is, without changes.