[tiering][iceberg] Validate partition spec before lake tiering#3639
Merged
fresh-borzoni merged 5 commits intoJul 16, 2026
Merged
Conversation
litiliu
force-pushed
the
codex/iceberg-validate-partition-spec
branch
2 times, most recently
from
July 13, 2026 09:46
ce6eb53 to
89238ca
Compare
litiliu
marked this pull request as ready for review
July 13, 2026 09:53
Contributor
Author
|
@fresh-borzoni Would you mind taking a look at this PR when you have time? Thanks. |
fresh-borzoni
left a comment
fresh-borzoni
left a comment
Member
There was a problem hiding this comment.
@litiliu Thank you for the PR, left some questions, PTAL
litiliu
force-pushed
the
codex/iceberg-validate-partition-spec
branch
from
July 15, 2026 07:18
89238ca to
7339626
Compare
fresh-borzoni
left a comment
fresh-borzoni
left a comment
Member
There was a problem hiding this comment.
@litiliu Thank you, looks good, let's close one hole left and good to merge
fresh-borzoni
approved these changes
Jul 16, 2026
fresh-borzoni
left a comment
fresh-borzoni
left a comment
Member
There was a problem hiding this comment.
@litiliu Thank you, LGTM 👍
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #3638
Iceberg lake tiering constructs partition keys from the current Iceberg partition spec, but fills them with partition and bucket values produced according to the Fluss table metadata. If an external client evolves the Iceberg partition spec, a newly created writer can silently attach incorrect partition metadata to data files.
This change validates the current Iceberg partition spec before the tiering writer is created and fails fast when the source columns, field order, transforms, or transform parameters are incompatible with the Fluss table.
Brief change log
InvalidTableExceptionon mismatch.Tests
mvn -pl fluss-lake/fluss-lake-iceberg verifyAPI and Format
No public API or storage-format changes. The shared partition-spec utility is marked
@Internal, and the existing Iceberg table mapping remains unchanged.Documentation
No documentation changes. This is a correctness fix with no new user-facing configuration.
Generative AI disclosure
Yes. Generated with OpenAI Codex and reviewed by the human developer.