Skip to content

Core: Fail concurrent replace transaction commits#2

Merged
sshivampeta merged 2 commits into
mainfrom
core/fix-16232-replace-transaction-concurrency
May 9, 2026
Merged

Core: Fail concurrent replace transaction commits#2
sshivampeta merged 2 commits into
mainfrom
core/fix-16232-replace-transaction-concurrency

Conversation

@sshivampeta
Copy link
Copy Markdown
Owner

@sshivampeta sshivampeta commented May 9, 2026

Summary

Fixes apache#16232.

This change prevents REPLACE TABLE transactions from silently overwriting concurrent committed table changes. If table metadata changes after a replace transaction starts, the replace transaction now fails with CommitFailedException instead of committing stale metadata.

Problem

Before this change, replaceTransaction() could overwrite concurrent commits without conflict detection. This affected concurrent:

  • schema updates
  • table property updates
  • data appends
  • snapshot expiration
  • another replace transaction

The root problem was that replace transactions refreshed the underlying table state but still committed the stale replacement metadata built when the transaction started. In addition, replace-table update requirements skipped several optimistic concurrency checks.

This allowed committed changes to disappear silently.

Changes

  • Updated BaseTransaction.commitReplaceTransaction to fail if the table metadata changed since the replace transaction started.
  • Updated UpdateRequirements.forReplaceTable to use the same optimistic concurrency checks as ordinary table updates.
  • Added regression coverage in TestReplaceTableSafety.
  • Updated catalog tests to expect concurrent replace conflicts instead of last-writer-wins behavior.
  • Updated Hive replace transaction tests to preserve concurrent changes and expect conflict failures.
  • Updated Spark encryption replace test expectations for concurrent replace behavior.

Why this fixes the issue

A replace transaction now compares the original base metadata with the latest table metadata after refresh. If another writer committed in between, the replace transaction fails before committing stale metadata.

This prevents stale replace metadata from overwriting committed schema, property, snapshot, or data changes.

For REST-style commits, UpdateRequirements.forReplaceTable now emits normal optimistic concurrency requirements, so server-side validation can also reject conflicting replace commits.

Testing

Ran:

./gradlew :iceberg-core:test --tests "org.apache.iceberg.TestReplaceTableSafety" --tests "org.apache.iceberg.TestUpdateRequirements"
./gradlew :iceberg-hive-metastore:test --tests "org.apache.iceberg.hive.TestHiveCreateReplaceTable"
./gradlew :iceberg-hive-metastore:test
./gradlew :iceberg-spark:iceberg-spark-4.1_2.13:checkstyleTest
./gradlew :iceberg-azure:integrationTest :iceberg-aws:integrationTest --continue

Replace commits now validate that underlying table metadata is unchanged
since the replace started after refresh(); otherwise commit fails with
CommitFailedException rather than overwriting concurrent updates (apacheGH-16232).
Includes UpdateRequirements alignment, CatalogTests and Spark concurrent
replace expectations, new TestReplaceTableSafety coverage, and
hasMessageContaining("replace transaction") assertions for Checkstyle.
@cursor cursor Bot force-pushed the core/fix-16232-replace-transaction-concurrency branch from d13a3eb to d91dd14 Compare May 9, 2026 20:15
Fix the Test cases to align with the replace transaction concurrency.

authored-by : shivakumar shivampeta
@sshivampeta sshivampeta force-pushed the core/fix-16232-replace-transaction-concurrency branch from d91dd14 to d99b2fb Compare May 9, 2026 20:29
@sshivampeta sshivampeta marked this pull request as ready for review May 9, 2026 20:44
@sshivampeta sshivampeta merged commit f65c670 into main May 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Core: REPLACE TABLE transaction silently overwrites concurrent committed changes

2 participants