Antalya 26.6: Implement TRUNCATE TABLE for Iceberg Engine (REST …

[zvonand]

Changelog category (leave one):

• New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Frontport for Antalya 26.3

• 26.1 Antalya port - Implement TRUNCATE TABLE for Iceberg Engine (REST catalog support) Feature: Support TRUNCATE TABLE for Iceberg engine #1529, It's a frontport from 26.1, contains:
  • feat(iceberg): Implement TRUNCATE TABLE for Iceberg Engine (REST catalog support) — Core implementation: metadata-only truncation generating a new overwrite snapshot with empty manifest list, committed atomically via REST catalog
  • fix(iceberg): pass new_snapshot to updateMetadata in IcebergStorageSink — Fixed silent breakage of all INSERTs on REST catalog tables (wrong JSON object passed to catalog->updateMetadata)
  • fix(iceberg): restore return false in RestCatalog::updateMetadata — Preserve retry contract; add LOG_WARNING for diagnostics
  • fix(iceberg): revert Mutations.cpp updateMetadata to pass new_snapshot — Same fix as IcebergStorageSink, applied to ALTER TABLE DELETE/UPDATE path
  • refactor(iceberg): add comment explaining Avro zigzag encoding — Reviewer-requested documentation for manual Avro OCF serialization
  • refactor(iceberg): address code review feedback on TRUNCATE implementation — Named zero arguments, helper functions, restart integration test (feature (iceberg): Implement TRUNCATE TABLE for Iceberg Engine (REST … #1655 by @il9ue).

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All with Aarch64
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

Cherry-picked from #1655.

---

Adaptation notes for antalya-26.3

The following API changes between antalya-26.1 and antalya-26.3 required adaptation:

• RestCatalog constructor signature updated to match antalya-26.3 calling convention in DataLakeConfiguration.h
• DatabaseDataLakeSetting member names updated
• Cargo.lock restored to antalya-26.3 version (our changes do not touch any Rust code)

All feature logic is identical to the merged antalya-26.1 implementation.

[github-actions]

Workflow [PR], commit [7076d0e]

[zvonand]

@blau-ai

[blau-ai]

CI triage

Verdict: 2 failing checks — both PR-caused, same root cause. Nothing flaky/infra/pre-existing.

Both red checks (Build (arm_tidy) and Fast test) fail at compile time on the same code. Every other job shows skipping because it is gated behind the build — those are not independent failures. This is a backport of #1655 (written against antalya-26.3), and the Iceberg-writes API on antalya-26.6 has diverged, so the ported code no longer compiles here.

There are two distinct compile problems:

1. Duplicate Iceberg field aliases in Constant.h — blocks the whole build (hit first, e.g. by Fast test)

Constant.h:145:1: error: redefinition of 'f_deleted_records'
Constant.h:126:1: note: previous definition is here
Constant.h:144:1: error: redefinition of 'f_deleted_data_files'
...
2 errors generated. → ninja: build stopped: subcommand failed.

antalya-26.6 already defines both aliases (origin/antalya-26.6:Constant.h:142-143, now at :144-145). The PR re-adds them at :126/:128, so they collide. Fix is to drop the two lines the PR added:

--- a/src/Storages/ObjectStorage/DataLakes/Iceberg/Constant.h
+++ b/src/Storages/ObjectStorage/DataLakes/Iceberg/Constant.h
@@ -123,9 +123,7 @@ DEFINE_ICEBERG_FIELD_ALIAS(partition_specs, partition-specs);
 DEFINE_ICEBERG_FIELD_ALIAS(spec_id, spec-id);
 DEFINE_ICEBERG_FIELD_ALIAS(added_records, added-records);
-DEFINE_ICEBERG_FIELD_ALIAS(deleted_records, deleted-records);
 DEFINE_ICEBERG_FIELD_ALIAS(added_data_files, added-data-files);
-DEFINE_ICEBERG_FIELD_ALIAS(deleted_data_files, deleted-data-files);
 DEFINE_ICEBERG_FIELD_ALIAS(added_delete_files, added-delete-files);

2. IcebergMetadata::truncate() uses the 26.3 FileNamesGenerator / MetadataGenerator API (arm_tidy, 5 errors)

IcebergMetadata.cpp:653:30: error: no matching constructor for initialization of 'FileNamesGenerator'
    note: candidate constructor not viable: requires 4 arguments, but 5 were provided
IcebergMetadata.cpp:659:30: error: no matching constructor for initialization of 'FileNamesGenerator'
IcebergMetadata.cpp:667:70: error: no member named 'generateMetadataName' in 'DB::FileNamesGenerator'
235321 warnings and 5 errors generated.

On antalya-26.6 the writer API changed:

• Constructor is 4-arg, FileNamesGenerator(table_location, bool use_uuid_in_metadata, compression, format) (FileNamesGenerator.h:32). There is no config_path / no separate transactional-path argument — truncate() passes 5 args. The data-path override is now set separately via setDataLocation().
• generateMetadataName() no longer exists — use generateMetadataPathWithInfo(), which returns a struct with a .path (IcebergPathFromMetadata) (FileNamesGenerator.h:47).
• generateNextMetadata(...) returns NextMetadataResult{ snapshot, manifest_list_path } — only 2 fields (MetadataGenerator.h:22-29), but truncate() unpacks 3 ([new_snapshot, manifest_list_name, storage_manifest_list_name]). Storage paths are now obtained by running the metadata path through IcebergPathResolver::resolve(...).

Good news: the is_truncate=true overload of generateNextMetadata already exists on this branch (MetadataGenerator.h:43), so no MetadataGenerator changes are needed — only the caller in truncate().

Concrete fix: rewrite the FileNamesGenerator setup and path handling in IcebergMetadata::truncate() to follow the pattern this branch's writer already uses:

• construction + setDataLocation() — see IcebergWrites.cpp:738-748;
• generateMetadataPathWithInfo() → generateNextMetadata() → resolver.resolve(manifest_list_path) — see IcebergWrites.cpp:952-970.

Note: clang-tidy stops reporting after 5 errors, so once the constructor/generateMetadataName calls are fixed, expect to reconcile a couple of follow-on mismatches in the same function (the 3-vs-2 structured binding above, and the generateManifestList / storage-path arguments). The rest of the diff (Context::createCopy, the new IDataLakeMetadata::truncate virtual, IcebergWrites.cpp zigzag helper) compiled cleanly.

---

I can prepare these fixes — the trivial Constant.h dedup is safe, and I can adapt truncate() to the 26.6 API as above (I can't build locally, so CI on the resulting PR would be the validation). Do you want me to open a blau/* PR with the fix, or commit directly to feature/antalya-26.6/pr-1655? I'll hold until you confirm the delivery.

_{Evidence: run 30468779903 — jobs Build (arm_tidy) and Fast test, head 7a636452.}

[zvonand]

@blau-ai commit the fix directly to this branch

[blau-ai]

Done — committed the fix directly to feature/antalya-26.6/pr-1655 as requested.

Commit: 984609c — Fix antalya-26.6 build: adapt Iceberg TRUNCATE to 26.6 writer API

What changed (both compile errors from the triage above)

1. Constant.h — macro redefinition (blocked the whole build)
Dropped the two field aliases the backport re-added (deleted_records, deleted_data_files); antalya-26.6 already defines both further down the file.

2. IcebergMetadata::truncate() — 26.3 writer API no longer exists (arm_tidy)
Rewrote the path/metadata handling to follow this branch's writer (IcebergStorageSink, IcebergWrites.cpp) so REST (transactional) and non-transactional catalogs go through the same IcebergPathResolver:

• 4-arg FileNamesGenerator(resolver.getTableLocation(), is_transactional, compression, format) — the old 5-arg config_path form is gone; the data-path override is now a separate setDataLocation() concern (a no-op for a metadata-only truncate, which writes no data files);
• generateMetadataPathWithInfo() replaces the removed generateMetadataName();
• generateNextMetadata(...) now returns {snapshot, manifest_list_path} (2 fields, was 3) — storage paths come from resolver.resolve(...), and the catalog path from resolver.resolveForCatalog(...);
• generateManifestList(resolver, …, {}, new_snapshot, {}, …) — first arg is the resolver (was the generator), and the empty manifest-entry/size args are now typed vectors.

The metadata-only truncate semantics (empty overwrite snapshot superseding all previous snapshots) are unchanged; the is_truncate=true overload of generateNextMetadata already exists on this branch, so no MetadataGenerator change was needed.

Validation

I can't build ClickHouse in this environment, so this is validated by CI on this PR, not locally. The two red checks (Build (arm_tidy), Fast test) both failed at compile time on exactly these two spots; a fresh run should now get past compilation. I'll be watching — ping me if anything downstream (e.g. the follow-on mismatches clang-tidy would have hidden behind its 5-error cap) still trips.