Spark: Add ExpireSnapshotsProcedure#1874
Conversation
aokolnychyi
changed the title
|
|
||
| @Test | ||
| public void testInvalidExpireSnapshotsCases() { | ||
| AssertHelpers.assertThrows("Should not allow mixed args", |
There was a problem hiding this comment.
Not sure we need some of the more generic checks here, like testing if named and pos args cannot be mixed
There was a problem hiding this comment.
It is a bit weird but we have such checks in other procedures so I added for consistency.
There was a problem hiding this comment.
Sounds fine, It doesn't really matter since they run so quickly.
| ExpireSnapshotsAction action = actions.expireSnapshots(); | ||
|
|
||
| if (olderThanMillis != null) { | ||
| action.expireOlderThan(olderThanMillis); |
There was a problem hiding this comment.
So we are going with the action default expire time? Just checking because I thougt some folks wanted that changed
There was a problem hiding this comment.
The action delegates to ExpireSnapshots table API that, in turn, respects the default table props we added recently.
There was a problem hiding this comment.
Ah! Excellent, I forgot about that
...tensions/src/test/java/org/apache/iceberg/spark/extensions/TestExpireSnapshotsProcedure.java
Outdated
Show resolved
Hide resolved
spark3/src/main/java/org/apache/iceberg/spark/procedures/ExpireSnapshotsProcedure.java
Outdated
Show resolved
Hide resolved
4 participants
This PR adds a stored procedure to expire snapshots.
The main author of this change is @liukun4515. I took commits from #1819, rebased, and added more changes to match the current state of procedures.
Resolves #1597.