[BEAM-354]: Modify DatastoreIO to use the v1beta3 API

[vikkyrk]

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

• Make sure the PR title is formatted like:
  [BEAM-<Jira issue #>] Description of pull request
• Make sure tests pass via mvn clean verify. (Even better, enable
  Travis-CI on your fork and ensure the whole test matrix passes).
• Replace <Jira issue #> in the title with the actual Jira issue
  number, if there is one.
• If this contribution is large, please file an Apache
  Individual Contributor License Agreement.

---

• Use v1beta3 datastore API and remove all v1beta2 dependencies
• Modify tests to pass.

[vikkyrk]

Hi @dhalperi @davorbonaci, can you take a look?

[dhalperi]

please keep the OutputDataset option here. You may rename it to OutputProject if you like.

[vikkyrk]

What if OutputProject is different from the current Project. Are dataflow pipelines allowed to work with multiple projects at the same time?

[dhalperi]

Yes this is a common and necessary use case.

On Mon, Jun 20, 2016 at 2:35 PM, Vikas Kedigehalli <notifications@github.com

wrote:

In
examples/java/src/main/java/org/apache/beam/examples/complete/AutoComplete.java
#499 (comment):

@@ -479,8 +477,7 @@ public static void main(String[] args) throws IOException {
if (options.getOutputToDatastore()) {
toWrite
.apply(ParDo.named("FormatForDatastore").of(new FormatForDatastore(options.getKind())))

•  .apply(DatastoreIO.writeTo(MoreObjects.firstNonNull(
  

•      options.getOutputDataset(), options.getProject())));
  

•  .apply(DatastoreIO.writeTo(options.getProject()));
  

What if OutputProject is different from the current Project. Are dataflow
pipelines allowed to work with multiple projects at the same time?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-beam/pull/499/files/834b87394f5834f7c6475032cafaed89be977ecb#r67771180,
or mute the thread
https://github.com/notifications/unsubscribe/AAgIT7OFwqMhdKeVuxt6f85WKhzfxWFuks5qNwevgaJpZM4I5_6o
.

[vikkyrk]

Ok.

[vikkyrk]

Fixed Dan's review comments.

[dhalperi]

R: @dhalperi

[dhalperi]

Can you please post a verbose mvn dependency:tree -Dverbose diff from before this change, as well as post the entire tree (without -Dverbose) after it?

[vikkyrk]

@dhalperi Attached the files. I looked at the dependencies brought it by beta3 but all of them were something we already used and with same versions.
dependencies.txt
mvndepDiff.txt

[dhalperi]

LGTM; merged