[BEAM-3271] Improve Splittable ParDo translation#3282
Merged
Conversation
|
Build finished. |
4 tasks
kennknowles
force-pushed
the
Splittable-ParDo-translation
branch
from
db8cf1c to
082fd5a
Compare
kennknowles
force-pushed
the
Splittable-ParDo-translation
branch
from
082fd5a to
616a07c
Compare
Member
Author
|
Rebased on the latest master. This is probably easiest to review in two commits, though not so big either way. |
tgroh
approved these changes
Jun 6, 2017
|
|
||
| @NewTracker | ||
| public RestrictionTracker<Integer> newTracker(Integer restriction) { | ||
| return null; // not actually nullable |
Member
There was a problem hiding this comment.
throw new UnsupportedOperationException()?
kennknowles
force-pushed
the
Splittable-ParDo-translation
branch
from
616a07c to
1b00d95
Compare
asfgit
pushed a commit
that referenced
this pull request
Jun 6, 2017
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
[BEAM-<Jira issue #>] Description of pull requestmvn clean verify.<Jira issue #>in the title with the actual Jira issuenumber, if there is one.
Individual Contributor License Agreement.
R: @tgroh another one peeled off, this one is for Splittable ParDo.
It is different than the others in that this transform is not designed to be serializable, as it is only a post-surgery artifact.
It brings up again the decision to make a unified pre/post serialization API. We could simply not have one if we follow this workflow:
RawPTransform)RawPTransform"just works" and all "fresh" nodes from surgery now also containRawPTransform)Just musing. This latter world means that pseudo-primitives all need registered payloads. I don't totally love this, because they may contain a bunch of stuff specific to the language of the runner and it is just boilerplate to give them payloads.