[BEAM-196] Pipeline options must be available Context in DoFn.startBundle #200
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| assertNotNull(deserializedOptions); | ||
| assertEquals(deserializedOptions, serializedOptions.getPipelineOptions()); | ||
| assertEquals(deserializedOptions, serializedOptions.getPipelineOptions()); | ||
| assertEquals(deserializedOptions, serializedOptions.getPipelineOptions()); |
There was a problem hiding this comment.
could use Hamcrest Matchers.sameInstance. Not tested, but:
assertThat(serializedOptions.getPipelineOptions(), Matchers.sameInstance(deserializedOptions));
|
LGTM |
|
R: @dhalperi Since you didn't pick someone specific ;) |
|
Thanks for the review @dhalperi! |
swegner
pushed a commit
to swegner/beam
that referenced
this pull request
Apr 22, 2016
Implement BigQueryIO.Write as a sink
4 tasks
iemejia
pushed a commit
to iemejia/beam
that referenced
this pull request
Jan 12, 2018
mareksimunek
pushed a commit
to mareksimunek/beam
that referenced
this pull request
May 9, 2018
[euphoria-build] remove unused protobuf plugin
Abacn
pushed a commit
to Abacn/beam
that referenced
this pull request
Jan 31, 2023
…gMessageErrorsAsCsv PiperOrigin-RevId: 355894522
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This gets rid of the custom Java serialization code by defaulting to serialization of the
PipelineOptionsto a byte array. So far, this has been proven the most hassle-free method for the Flink Runner. For code reuse and avoiding multiple deserialization of the byte array, theSerializedPipelineOptionsclass has been introduced.The changes also make the options accessible in the context of the
DoFnfunction.