Add ability to undelete a column#16089
Open
fbertsch wants to merge 9 commits into
Open
Conversation
added 8 commits
January 19, 2026 19:24
- Extract the latest instance of that column name from historical schemas - Required columns will be come non-required - The parent field must already be present in the schema - Add spark procedure call to run undelete
This is common logic that is a little confusing at first glance. Also adds a hard failure in the case that the parent field is, for some reason, not available in the schema (this should be impossible, and if it happens, will return a null pointer)
Non-null defaults are only available in v3, so set the version in the test.
fbertsch
force-pushed
the
add_undelete
branch
2 times, most recently
from
34b191a to
86b649b
Compare
- Add function to check whether data had been written for a col after the col was deleted - Fail to undelete a required col if data was written after it had been deleted - Add set_nullable parameter to force those cols as nullable, bypassing the failure (and allowing you to undelete a reqd col)
Author
|
@steveloughran @gtrettenero @yingjianwu98 the old PR was auto-closed so I had to open a new one with the changes. There are two changes:
This more closely matches the Iceberg spec. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
set_nullableto allow required columns missing data to be undeleted and made nullable