26.6.8.20001 Altinity Antalya

26.6.8.20001 Altinity Antalya (9af7dce) as compared to v26.6.4.20001.altinityantalya (2073b1f88dec885444fac97edb8179ad1753c1ad)

New Features

• Add support for iceberg compaction of manifest files using optimize command. (#2193 by @subkanthi)
• Export partition on plain merge tree. (#2290 by @arthurpassos)
  • User facing & functional changes:
    • Changes how replicated partition task keys are encoded on zookeeper - this is not backwards compatible
    • Implements partition export for plain merge tree
    • Stop shipping empty parquet files on zero rows exports
    • New system.partition_exports table that covers both the previous replicated_partition_exports and the new one for plain merge tree
• Iceberg v3 has two new fields in manifest file - first_row_id and last_seq_num. This PR allows to read them as virtual columns from system tables(system.iceberg_files) (ClickHouse#115603 by @scanhex12 via #2308)
• Support for iceberg v3 unknown datatype which is maps to Nullable(Nothing) in Clickhouse. Read and write path(Parquet). (#2363 by @subkanthi)
• Aggregate function, which states are compatible with apache data sketches HLL implementation (#2398 by @UnamedRus)

Performance Improvements

• Added cas_gc_io_concurrency to run blob deletes in the CAS GC pending_deletes phase in parallel. (#2351 by @k-morozov)
• cas disk: loading a table no longer issues one object-store LIST per part file. Directory probes inside a part are answered from the part manifest, so a restart costs a fixed number of LISTs per table instead of one per file. (#2440 by @filimonov)
• Removing a data part from a cas disk writes one ref-log record instead of two manifests and six ref-log records. A create/insert/drop cycle issues 56% fewer PUT and 43% fewer GET requests and takes 44% less time. (#2452 by @filimonov)
• More optimistic Parquet footer-size estimate instead of fixed 64 KiB, fetching the footer in one read instead of two and reducing footer-read latency. (ClickHouse#115686 by @UnamedRus via #2503)
• CAS GC drains a dropped table's reference stream in one round: dead stream keys are deleted in batches on the GC I/O pool, under a soft time budget instead of one page per round (#2483 by @filimonov)

Improvements

• Allow empty object storage cluster (#2221 by @ianton-ru)
• Supports CREATE TABLE, CREATE TABLE … AS source, and DROP TABLE for DataLakeCatalog; DROP TABLE can request catalog-side data purge via the new data_lake_delete_data_on_drop setting (ClickHouse#98670 by @zvonand via #2305)
• MemoryWorker now corrects MemoryTracker using allocated memory from ASan/TSan/MSan instead of resident memory in sanitizer builds when cgroups are available, fixing inaccurate memory accounting in such builds. Closes #2299 (#2349 by @k-morozov)
• Add an Antalya-only protocol version advertised in the ServerHello. (#2378 by @zvonand)
• CAS on-store readers skip unknown optional fields in every object type, so later releases can add fields to cas_run, fold seal, cas_ref_ckpt, cas_ref_catalog and cas_gc_maintenance_state objects without a format version bump. (#2455 by @filimonov)
• Use Antalya protocol for Iceberg metadata instead of JSON (#2482 by @ianton-ru)
• CAS GC batch deletes respect the storage's batch-delete limit (objects_chunk_size_to_delete on S3), and S3 delete errors keep the store's error name. (#2484 by @filimonov)
• Use Antalya protocol for lock_object_storage_task_distribution_ms instead of JSON (#2489 by @ianton-ru)

Bug Fixes (user-visible misbehavior in an official stable release)

• Fixed an issue where Glue DataLakeCatalog tables could retain stale schema metadata after ALTER TABLE operations. #2085 (#2272 by @subkanthi)
• Added support for the FIRST and AFTER clauses with ALTER TABLE ADD COLUMN and ALTER TABLE MODIFY COLUMN for Iceberg tables (DataLakeCatalog). (#2289 by @subkanthi)
• Fixed races in the CAS GC scheduler lifecycle and made SYSTEM CAS GC START recover safely from worker scheduling failures. (#2326 by @k-morozov)
• Object-storage listings no longer try to parse every listed key as a JSON swarm command; this removes one thrown-and-caught exception per listed key on every object storage and the ASan fake-stack leak that made the CAS-S3 sanitizer lane up to 100x slower. (#2393 by @filimonov)
• Fixed a server termination when a disk write of transaction metadata (part CSN or mutation CSN) failed inside the commit or rollback of a MergeTree transaction; such writes are now retried for a bounded time. Also fixed a termination when KILL MUTATION raced with the commit of a transactional mutation. (#2396 by @filimonov)
• Fixed BACKUP of a table on a CAS disk to an S3(...) or Disk(...) destination on the same S3 endpoint as the pool. (#2415 by @k-morozov)
• Fixed Expected UInt64 column for __grouping_set, got BLOB and ColumnBLOB should be converted to a regular column before usage logical errors on a distributed query executed on a shard with enable_parallel_blocks_marshalling (enabled by default), for example with GROUP BY GROUPING SETS over a cluster that has a local replica, or when reading a Merge table that spans a local and a distributed table. BlocksMarshallingStep is no longer added to plans whose blocks are consumed in the same process (ClickHouse#111997 by @groeneai via #2427)
• Fix the CAS GC deleting the .meta condemn marker of a later condemnation of a re-published blob; graduation now always reads the marker (with read-ahead under cas_gc_io_concurrency) and the delete job removes a marker only for the round that owns it. (#2463 by @filimonov)
• Fix UNION of a local query and cluster() (or another cluster function) throwing NUMBER_OF_COLUMNS_DOESNT_MATCH or THERE_IS_NO_COLUMN on GROUP BY hostName. A server constant built on the local branch was reused for the cluster branch and folded to the initiator value. Each scope now builds its own value, so shards report their own hostName, serverUUID, tcpPort, and the other server constants. (#2471 by @ianton-ru)

Build/Testing/Packaging Improvements

• Add SBOM and license mapping to docker images (#2392 by @MyroTk)
• Run nightly checks on specified branches only if changes were added since previous run. Upload failed results to slack for easier monitoring. (#2445 by @MyroTk)

CI Fixes or Improvements

• CI: the test runner gives lldb 120 s instead of 30 s to collect stacks from debug and sanitizer servers, and a killed lldb no longer leaves the server wedged with SIGTRAP: its JIT-loader breakpoint is kept out of the server. (#2385 by @filimonov)
• Fix Inconsistent KeyCondition behavior for wider-typed key comparisons (ClickHouse#109023 by @groeneai via #2470)

Documentation

• Improve swarm documentation (#2346 by @ianton-ru)
• Remove the outdated set of documentation. The up-to-date Antalya documentation lives exclusively in docs/en/antalya from now on. (#2348 by @arthurpassos)

And ALL features from previous Antalya releases

Build report

Packages

Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/#altinityantalya as either .deb, .rpm, or .tgz

Docker images

Available for both AMD64 and Aarch64: altinity/clickhouse-server:26.6.8.20001.altinityantalya

Full Changelog: 2073b1f...9af7dce