fix: use structured Summary for Iceberg REST snapshot commits

[manudiv16]

Problem

When k2i commits Iceberg snapshots to catalogs backed by apache/iceberg-rust (e.g. Lakekeeper, Nessie with the official client), the commit fails with:

Failed to deserialize the JSON body into the target type: 
updates[0]: duplicate field `operation` at line 1 column 784

Root Cause

The Snapshot.summary field was a flat HashMap<String, String> that serialized as:

{"operation": "append", "added-data-files": "1", ...}

However, apache/iceberg-rust defines Summary as a struct with operation as a named field plus #[serde(flatten)] additional_properties:

pub struct Summary {
    pub operation: Operation,
    #[serde(flatten)]
    pub additional_properties: HashMap<String, String>,
}

When the catalog deserializes k2i's JSON, it sees operation twice — once from the struct field and once from the flattened map — causing the HTTP 422 error.

This is also tracked upstream as apache/iceberg#9837 (OpenAPI spec issue with additionalProperties nesting).

Fix

• Add a Summary struct with operation as a named field + #[serde(flatten)] for additional properties
• Change Snapshot.summary from HashMap<String, String> to Summary
• Convert the existing flat HashMap to Summary at the REST API boundary (factory.rs and nessie.rs)
• SnapshotCommit.summary remains HashMap<String, String> to avoid cascading changes to non-REST catalog backends (Glue, Hive, SQL)

Serialization Before/After

Before (flat HashMap — causes duplicate field error):

{
  "summary": {
    "operation": "append",
    "added-data-files": "1"
  }
}

After (structured Summary — matches iceberg-rust spec):

{
  "summary": {
    "operation": "append",
    "added-data-files": "1"
  }
}

The JSON output is identical, but the serde model now matches what apache/iceberg-rust expects for deserialization, eliminating the duplicate field conflict.

Testing

• Updated test_table_update_serialization to verify the structured Summary serialization
• Existing tests pass (no changes to SnapshotCommit or non-REST backends)

Compatibility

• REST catalogs (factory.rs, nessie.rs): Fixed — uses structured Summary
• Official iceberg-rust (official.rs): Unaffected — uses the official crate directly
• Glue/Hive/SQL: Unaffected — SnapshotCommit.summary remains HashMap, converted only at REST boundary

[manudiv16]

Local verification ✅

Compiled and tested locally with rustc 1.96.1:

cargo check        → Finished (0 errors, 0 warnings)
cargo test -p k2i-core --lib → 268 passed, 0 failed, 1 ignored
cargo test -p k2i-core --lib -- iceberg:: → 110 passed, 0 failed

All existing tests pass, including the updated test_table_update_serialization which now verifies the structured Summary serialization.

[manudiv16]

Update: real root cause found and fixed

The initial struct fix was necessary but not sufficient. The actual root cause was in , which uses the official apache/iceberg-rust crate v0.7.0:

// official.rs - the actual code path used for REST catalog commits
let action = tx
    .fast_append()
    .set_snapshot_properties(summary)  // ← HashMap contains "operation": "append"
    .add_data_files(data_files);

The official crate's Summary struct has operation as a named field plus #[serde(flatten)] additional_properties. When set_snapshot_properties() receives a HashMap with "operation": "append", the crate serializes operation twice:

1. From the struct's named operation field
2. From the #[serde(flatten)] HashMap

This is a known issue: apache/iceberg#9837

Fix (commit aa55988)

Remove "operation" from the HashMap before passing it to the official crate:

summary.remove("operation");
let action = tx.fast_append().set_snapshot_properties(summary)...;

Verified

• cargo check ✅
• cargo test -p k2i-core --lib -- iceberg:: → 110 passed ✅
• Deployed to k3s-homelab with Lakekeeper — pod stable for 4+ minutes (previously crashed in ~30s) ✅

[sionsmith]

Follow-up review result:

PR #2 had the right direction for the Iceberg operation summary issue, but the production-ready version landed through #3 with additional hardening:

• reproduced the metadata reload failure on main with an official iceberg-rust memory catalog regression;
• kept the structured REST Summary fix;
• stripped operation before passing snapshot properties into the official iceberg-rust append API;
• centralized flat summary conversion through Summary::from_properties;
• added regression tests for committed metadata reload and summary JSON shape;
• bumped the workspace to 0.2.1 and updated changelog/docs/man page;
• fixed semver/test CI issues discovered during review.

How it was tested:

cargo fmt --all --check
git diff --check HEAD
cargo check --workspace --all-targets
cargo test --workspace --no-fail-fast
cargo clippy --workspace --all-targets -- -D warnings
scripts/e2e-docker-iceberg.sh
cargo semver-checks check-release --package k2i-core --baseline-rev origin/main

GitHub CI for #3 passed Check & Lint, Unit Tests, Integration Tests, Semver, Security Audit, and release planning.

Separate note: #1 is not part of this Iceberg fix. It is a GHCR package visibility issue for the published container image. #4 adds release CI/docs so future Docker releases verify anonymous pull access after publishing.