New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BEAM-8575] Add a Python test to test windowing in DoFn finish_bundle() #10145
Conversation
8bd8d1b
to
6014a27
Compare
R: @y1chi PTAL Yichi! Thanks:) |
LGTM, I wonder would it be better to move the timestamp assignment to beam.Create() and combine the map function into the MyDoFn. |
Thanks for the quick review!
This can be done, but does not make much difference, since we still want both the process() and finish_bundle() to do something in this test, and see the reason below.
I think it's clearer to separate the Map from the test DoFn. The purpose of this test is to verify that after a DoFn with finish_bundle() implemented, it will produce results both from process() and finish_bundle(). More specifically, it wants to make sure that when windowing is involved, the output will be correct after the DoFn. The last Map is simply to print out all outputs from the test DoFn. Thoughts? |
@tvalentyn could you help to merge? |
Thanks Yichi for the review:) |
Thanks, @liumomo315 and @y1chi ! Somehow I missed the last comment, merged now! |
@liumomo315 Please run ValidatesRunner tests when marking new tests as "VR". The command to trigger the tests can be found in .test-infra/jenkins/README file, the link is available in the PR template. To run those tests you can just comment on the PR: Run Dataflow Python ValidatesRunner |
Thanks for the hints, Valentyn! Will do that for my new PRs. |
This test is the Python parity for the Java ParDo test testWindowingInStartAndFinishBundle.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username
).[BEAM-XXX] Fixes bug in ApproximateQuantiles
, where you replaceBEAM-XXX
with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.See the Contributor Guide for more tips on how to make review process smoother.
Post-Commit Tests Status (on master branch)
Pre-Commit Tests Status (on master branch)
See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.