BEAM-5993 Create SideInput Load test #7020
Conversation
kkucharc
force-pushed
the
BEAM-5993-side-input-load-test-python
branch
from
e2f4b4a
to
5a5362f
Compare
|
Do you need a review for this? : ) |
|
@pabloem I would really appreciate it! I think there is a lot to change, but I'm wondering what's your opinion. |
| ) | ||
| # pylint: disable=expression-not-assigned | ||
| (main_input | ||
| | "Merge" >> beam.ParDo( |
There was a problem hiding this comment.
By iterating through the side input in "Merge", you will consume it, and no longer need to iterate through it in "Consume" - so it may be fine if you just drop the Consume transform. Other than that, I think this must be fine : )
There was a problem hiding this comment.
That sounds good. Thanks! Consume was useful to check if join_fn correctly merges together main_input and side input. But you are right - now it's not necessary.
|
This looks fine to me. I'm glad to merge it. I believe you've run the pipeline right? |
kkucharc
force-pushed
the
BEAM-5993-side-input-load-test-python
branch
2 times, most recently
from
2ef84d0
to
9e66b0b
Compare
|
@pabloem Locally - yes. But probably as you can see it started failing with PreCommit now. I'm investigating this. |
kkucharc
force-pushed
the
BEAM-5993-side-input-load-test-python
branch
from
9e66b0b
to
453ed35
Compare
kkucharc
force-pushed
the
BEAM-5993-side-input-load-test-python
branch
from
453ed35
to
7344634
Compare
|
Run Python PreCommit |
|
Thanks Kasia. Sorry that it was just the flakiness. |
Creating Synthetic Source load test of Side Input.
Follow this checklist to help us incorporate your contribution quickly and easily:
[BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replaceBEAM-XXXwith the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.It will help us expedite review of your Pull Request if you tag someone (e.g.
@username) to look at it.Post-Commit Tests Status (on master branch)