Better error for reused pcollections.#3906
Conversation
| p = pipeline.Pipeline( | ||
| 'DirectRunner', PipelineOptions(sys.argv)) | ||
| else: | ||
| if any(getattr(p, 'is_ephemeral', False) for p in pipelines): |
There was a problem hiding this comment.
No, this should be False, which is the correct value if it's not present.
KesterTong
left a comment
KesterTong
left a comment
There was a problem hiding this comment.
Can you add a test for this, along the lines of
class MyObject(object):
def __init__(self, pcollection):
self.pcollection = pcollection
class CreateMyObject(beam.PTransform):
def expand(self, data):
return MyObject(data)
class TransformMyObject(beam.PTransform):
def _extract_input_pvalues(self, data_and_my_object):
data, my_object = data_and_my_object
return data_and_my_object, [data, my_object.pcollection]
def expand(self, data_and_my_object):
return data_and_my_object
...
def testErrorOnReuseOfPCollection(self):
input_data = [1, 2, 3, 4]
my_object = input_data | 'CreateMyObject' >> CreateMyObject()
_ = (input_data, my_object) | 'TransformMyObject' >> TransformMyObject()
|
@robertwb any updates on this PR? |
|
@robertwb ping |
|
Can we close this PR? |
robertwb
left a comment
robertwb
left a comment
There was a problem hiding this comment.
Added a test as requested.
| p = pipeline.Pipeline( | ||
| 'DirectRunner', PipelineOptions(sys.argv)) | ||
| else: | ||
| if any(getattr(p, 'is_ephemeral', False) for p in pipelines): |
There was a problem hiding this comment.
No, this should be False, which is the correct value if it's not present.
|
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
5 participants
Follow this checklist to help us incorporate your contribution quickly and easily:
[BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replaceBEAM-XXXwith the appropriate JIRA issue.mvn clean verifyto make sure basic checks pass. A more thorough check will be performed on your pull request automatically.