[Kernel] [CatalogManaged] Committer, CommitPayload, CommitResponse, CommitFailedException APIs
#4814
+287
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
scottsand-db
changed the title
Committer, CommitPayload, CommitResponse, CommitFailedException APIs
scottsand-db
requested review from
huan233usc,
sumeet-db,
dhruvarya-db,
nicklan,
raveeram,
zachschuermann,
raveeram-db,
vkorukanti,
scovich and
prakharjain09
and removed request for
raveeram
scottsand-db
commented
Jun 24, 2025
raveeram-db
reviewed
Jun 24, 2025
kernel/kernel-api/src/main/java/io/delta/kernel/commit/CommitFailedException.java
Outdated
Show resolved
Hide resolved
raveeram-db
reviewed
Jun 24, 2025
kernel/kernel-api/src/main/java/io/delta/kernel/commit/CommitPayload.java
Outdated
Show resolved
Hide resolved
vkorukanti
approved these changes
Jun 25, 2025
nicklan
reviewed
Jun 25, 2025
| * The {@link Protocol} that was read at the beginning of the commit. Empty if a new table is | ||
| * being created. | ||
| */ | ||
| public final Optional<Protocol> readProtocolOpt; |
There was a problem hiding this comment.
Can you remind me why we need the old/new p and ms?
There was a problem hiding this comment.
Good question! For example, to detect that the catalogManaged table feature has been enabled on the table.
There was a problem hiding this comment.
And, certainly, you want the new Protocol and Metadata so you can send that to the catalog, and then it always knows what's the latets P & M, and it can give that back on the next read request to efficiently construct a ResolvedTable without having to go to the file system to resolve the latest P & M
scottsand-db
force-pushed
the
stack/kernel_catalog_managed_writes_1
branch
from
b603c47 to
072ede5
Compare
raveeram-db
reviewed
Jun 26, 2025
kernel/kernel-api/src/main/java/io/delta/kernel/commit/CommitFailedException.java
Outdated
Show resolved
Hide resolved
scottsand-db
commented
Jun 27, 2025
| * from Kernel to {@link Committer} implementations, which use the information to perform | ||
| * catalog-specific or filesystem-specific commit operations. | ||
| * | ||
| * <p><strong>Important:</strong> The actions iterator should only be consumed once. Committer |
There was a problem hiding this comment.
Engine implementations
scottsand-db
commented
Jun 27, 2025
| } | ||
|
|
||
| /** All Delta actions (data and metadata) that are part of this commit. */ | ||
| public CloseableIterator<Row> getFinalizedActions() { |
There was a problem hiding this comment.
mention again that this can only be consumed once
scottsand-db
force-pushed
the
stack/kernel_catalog_managed_writes_1
branch
from
76c689f to
75bae78
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR
Use this link to review incremental changes.
Which Delta project/connector is this regarding?
Description
This PR creates the Committer, CommitPayload, CommitResponse, CommitFailedException public APIs.
How was this patch tested?
Just new APIs.
Does this PR introduce any user-facing changes?
Yes.