26.3.53.20001 Altinity Antalya

26.3.53.20001 Altinity Antalya (243187f) as compared to v26.3.13.20001.altinityantalya (7c4787ee0a199114cd7d544d4fef980974625582)

New Features

• Renamed the export_merge_tree_part_schema_mismatch_mode setting for EXPORT PART/EXPORT PARTITION to export_merge_tree_part_schema_match_mode, with values position (default, same behavior as before) and name, which matches destination columns to source columns by name instead of position. Added the export_merge_tree_part_ignore_extra_source_columns setting, which allows a source column without a matching destination column to be dropped instead of rejected with NUMBER_OF_COLUMNS_DOESNT_MATCH, in either matching mode. (#2220 by @k-morozov)

Improvements

• Add Iceberg metadata file paths and data file paths to the system.replicated_partition_exports to improve observability and debugging. Partially tackles #1824 Also replaces clickhouse snowflakeid implementation with uuidv4 (#1832 by @arthurpassos)
• Implements a local backoff policy for export partition - helps with transient and local failures., It is controlled by export_merge_tree_partition_retry_initial_backoff_seconds and export_merge_tree_partition_retry_max_backoff_seconds. Removes export_merge_tree_partition_max_retries as it would conflict with a local backoff policy implementation. We already have a task time out anyways. The only limitation of relying on the task time out is that non retryable errors would be retried until it times out. Not ideal, so I implemented an err classification method that will classify between retryable and non retryable. Non retryable will fail the task immediately. (#1984 by @arthurpassos)
• Route a few more settings through export partition to export part: output_format_parquet_compression_method, output_format_compression_level, output_format_parquet_row_group_size and output_format_parquet_row_group_size_bytes. (#1990 by @arthurpassos)
• Multi-stage distributed queries (ClickHouse#106020 by @davenger via #1999)
• Using LOG_INFO for all logs in export partition might flood the system. Replace it with DEBUG and WARNING. (#2004 by @arthurpassos)
• Allow export partition through different partition expressions as long as the destination expression does not repartition the data. This is validated at schedule time through two mechanisms: 1. Structural match: in case the source and destination are identical, the destination expression is a subset of the source expression or the destination expression can be entirely computed using only constants and the exact values guaranteed (pinned) by the source. 2. Dynamic proof: the destination expression is monotonic over the source partition min/max range. (#2074 by @arthurpassos)
• Added the export_merge_tree_part_schema_mismatch_mode setting. With ignore_extra_source_columns_by_position (default: strict), EXPORT PART/EXPORT PARTITION allows a source table with extra trailing columns — they are simply ignored instead of failing with NUMBER_OF_COLUMNS_DOESNT_MATCH. (#2111 by @k-morozov)

Bug Fixes (user-visible misbehavior in an official stable release)

• Fixes an issue where currentDatabase() used as a hybrid table argument would cause an UNKNOWN_TABLE error on restart (#1995 by @mkmkme)
• Fix OAuth2 login CSRF-state lea, unbounded OIDC response in clickhouse-client and confidential clients auth (#2086 by @zvonand)
• Fixed authentication of externally defined users on Distributed / cluster() queries when remote_servers uses an interserver secret (#2114 by @zvonand)
• Fixed ALTER TABLE ... EXPORT PART and EXPORT PARTITION schema validation to prevent silently writing values into incorrect destination columns. The export now verifies, by name and position only, that the top-level column owning each PARTITION BY column or subcolumn occupies the same position in the source and destination schemas. If that column contains a named Tuple, its element names must also be declared in the same order - recursively through nested tuples and through container types such as Array and Map (both the key and value type of a Map are checked). This protects both by-name subcolumn access (PARTITION BY t.a) and positional expressions (PARTITION BY tupleElement(t, 1), PARTITION BY tupleElement(arr[1], 'a')) from silently changing meaning after the positional cast. This check only compares names, positions, and Tuple element layout of the owning columns - it does not verify that the corresponding types are compatible (e.g. it does not detect a DateTime/DateTime64 partition key column using different time zones in the source and destination tables, nor a Tuple element whose name matches but whose type differs). Such type mismatches, like all non-partition-key type mismatches, are still caught separately by the existing positional cast check (rejected unless export_merge_tree_part_allow_lossy_cast = 1 is set). (#2134 by @k-morozov)

Build report

Packages

Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/#altinityantalya as either .deb, .rpm, or .tgz

Docker images

Available for both AMD64 and Aarch64: altinity/clickhouse-server:26.3.53.20001.altinityantalya

Full Changelog: 7c4787e...243187f