-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BEAM-354]: Modify DatastoreIO to use the v1beta3 API #499
Conversation
Hi @dhalperi @davorbonaci, can you take a look? |
@@ -479,8 +477,7 @@ public static void main(String[] args) throws IOException { | |||
if (options.getOutputToDatastore()) { | |||
toWrite | |||
.apply(ParDo.named("FormatForDatastore").of(new FormatForDatastore(options.getKind()))) | |||
.apply(DatastoreIO.writeTo(MoreObjects.firstNonNull( | |||
options.getOutputDataset(), options.getProject()))); | |||
.apply(DatastoreIO.writeTo(options.getProject())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please keep the OutputDataset option here. You may rename it to OutputProject if you like.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if OutputProject is different from the current Project. Are dataflow pipelines allowed to work with multiple projects at the same time?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes this is a common and necessary use case.
On Mon, Jun 20, 2016 at 2:35 PM, Vikas Kedigehalli <notifications@github.com
wrote:
In
examples/java/src/main/java/org/apache/beam/examples/complete/AutoComplete.java
#499 (comment):@@ -479,8 +477,7 @@ public static void main(String[] args) throws IOException {
if (options.getOutputToDatastore()) {
toWrite
.apply(ParDo.named("FormatForDatastore").of(new FormatForDatastore(options.getKind())))
.apply(DatastoreIO.writeTo(MoreObjects.firstNonNull(
options.getOutputDataset(), options.getProject())));
.apply(DatastoreIO.writeTo(options.getProject()));
What if OutputProject is different from the current Project. Are dataflow
pipelines allowed to work with multiple projects at the same time?—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-beam/pull/499/files/834b87394f5834f7c6475032cafaed89be977ecb#r67771180,
or mute the thread
https://github.com/notifications/unsubscribe/AAgIT7OFwqMhdKeVuxt6f85WKhzfxWFuks5qNwevgaJpZM4I5_6o
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok.
Fixed Dan's review comments. |
R: @dhalperi |
Can you please post a verbose |
@dhalperi Attached the files. I looked at the dependencies brought it by beta3 but all of them were something we already used and with same versions. |
LGTM; merged |
…schema PiperOrigin-RevId: 485377181
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
[BEAM-<Jira issue #>] Description of pull request
mvn clean verify
. (Even better, enableTravis-CI on your fork and ensure the whole test matrix passes).
<Jira issue #>
in the title with the actual Jira issuenumber, if there is one.
Individual Contributor License Agreement.