feat: add an enhanced layer for BigQuery Storage v1beta2 client #48
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change adds a simple shim layer to the BigQuery Read API client. It is modeled after the shim layer in the BigQuery Storage API v1beta1, where the parameters are simple pass-throughs to the underlying gRPC client stub. The shim currently does nothing but read default values from the base client layer and pass them back through.
This change modifies the configuration for the v1beta2 ReadRows API to add a resumption strategy. The resumption strategy will allow the connection to be resumed transparently in the case of transient errors.
googlebot
added
the
cla: yes
mmladenovski
changed the title
|
This whole "enhanced" infrastructure is just, at this point, to allow us to specify a resumption strategy in the default ReadRows settings? I'd really love to get away from this. @chingor13 @shollyman @stephaniewang526 -- any way we can get the codegen tool to do this for us? |
mmladenovski
changed the title
|
@mmladenovski should I wait to review until @kmjung's question gets answered? |
@emkornfield, please go ahead with a review. I am not sure how long it will take for automated resumption strategy generation. |
emkornfield
reviewed
Jan 29, 2020
...e/src/main/java/com/google/cloud/bigquery/storage/v1beta2/stub/EnhancedBigQueryReadStub.java
Outdated
Show resolved
Hide resolved
emkornfield
reviewed
Jan 29, 2020
...java/com/google/cloud/bigquery/storage/v1beta2/stub/readrows/ReadRowsResumptionStrategy.java
Show resolved
Hide resolved
emkornfield
reviewed
Jan 29, 2020
...java/com/google/cloud/bigquery/storage/v1beta2/stub/readrows/ReadRowsResumptionStrategy.java
Show resolved
Hide resolved
emkornfield
reviewed
Jan 29, 2020
...ystorage/src/test/java/com/google/cloud/bigquery/storage/v1beta2/BigQueryReadClientTest.java
Show resolved
Hide resolved
emkornfield
reviewed
Jan 29, 2020
...ystorage/src/test/java/com/google/cloud/bigquery/storage/v1beta2/BigQueryReadClientTest.java
Show resolved
Hide resolved
|
Took a look at manual diffs provided offline and perused the code as is, overall looks OK to me. |
emkornfield
approved these changes
Jan 29, 2020
chingor13
requested review from
stephaniewang526
and removed request for
chingor13
stephaniewang526
added
the
kokoro:force-run
yoshi-kokoro
removed
the
kokoro:force-run
Codecov Report
@@ Coverage Diff @@
## master #48 +/- ##
=========================================
Coverage ? 71.11%
Complexity ? 197
=========================================
Files ? 28
Lines ? 1357
Branches ? 3
=========================================
Hits ? 965
Misses ? 389
Partials ? 3
Continue to review full report at Codecov.
|
stephaniewang526
added
the
kokoro:force-run
yoshi-kokoro
removed
the
kokoro:force-run
stephaniewang526
approved these changes
Jan 30, 2020
shubhwip
pushed a commit
to shubhwip/java-bigquerystorage
that referenced
this pull request
Oct 7, 2023
…o v1-rev20191127-1.30.9 (googleapis#48)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request creates an enhanced shim layer in the BigQuery Storage API v1beta2 client in order to apply a streaming resumption strategy for the ReadRows method.
The changes are similar to the PR for v1beta1: googleapis/google-cloud-java#4022. The difference is that a lot of the timeout and retry settings are now automatically applied on the base (auto-generated) client.
Some tips on how to review the PR and which files to compare:
EnhancedBigQueryReadStub.javais based fromBigQueryReadStub.javaEnhancedBigQueryReadStubSettings.javais based fromBigQueryReadStubSettings.javaBigQueryReadSettings.javais based fromBaseBigQueryReadSettings.javaBigQueryReadClient.javais based fromBaseBigQueryReadClient.java@kmjung , @emkornfield, @aryann please review the changes.