New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BEAM-9344] Add support for bundle finalization execution to the Beam Java SDK. #10910
Conversation
52bc261
to
ff52f0f
Compare
This is needed for the unbounded source wrapper to be able to get an initial version working without watermark reporting. |
ff52f0f
to
a5bf78f
Compare
Run Java PreCommit |
import org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker; | ||
import org.apache.beam.sdk.transforms.splittabledofn.SplitResult; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At <a href="https://s.apache.org/splittable-do-fn>, second quote is missing that breaks all formatting in github and should break href. Can you fix typo please?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
This change adds support for passing in the BundleFinalizer as an input parameter to start/process/finish and wires it through within the SDK harness. The only supported combination that can execute this is Python ULR and Dataflow using UW which is why all the validates runner test configurations have added the test category to the exclusions list.
a5bf78f
to
fa5f428
Compare
…actory context parameter call.
…el for BundleFinalizer
Run Java PreCommit |
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username
).[BEAM-XXX] Fixes bug in ApproximateQuantiles
, where you replaceBEAM-XXX
with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.CHANGES.md
with noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
Post-Commit Tests Status (on master branch)
Pre-Commit Tests Status (on master branch)
See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.