Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: better default stream support in client library #750

Merged
merged 20 commits into from Dec 30, 2020

Conversation

yirutang
Copy link
Contributor

@yirutang yirutang commented Dec 18, 2020

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

@yirutang yirutang requested review from as code owners Dec 18, 2020
@yirutang yirutang requested a review from stephaniewang526 Dec 18, 2020
@generated-files-bot
Copy link

@generated-files-bot generated-files-bot bot commented Dec 18, 2020

Warning: This pull request is touching the following templated files:

  • .github/generated-files-bot.yml

@product-auto-label product-auto-label bot added the api: bigquerystorage label Dec 18, 2020
@yirutang yirutang changed the title feat: better default stream support in client library. feat: better default stream support in client library Dec 18, 2020
@google-cla google-cla bot added the cla: yes label Dec 18, 2020
@stephaniewang526
Copy link
Member

@stephaniewang526 stephaniewang526 commented Dec 21, 2020

PTAL below error:

Error:  /home/runner/work/java-bigquerystorage/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:1: Line does not match expected header line of '^/\*$'. [RegexpHeader]

@stephaniewang526
Copy link
Member

@stephaniewang526 stephaniewang526 commented Dec 21, 2020

One unit test failed:

Error:  Failures: 
Error:    JsonStreamWriterTest.testTwoParamNewBuilder_nullSchema:220 expected:<Stream[OrTable]Name is null.> but was:<Stream[]Name is null.>

yirutang and others added 15 commits Dec 22, 2020
Depends on googleapis/repo-automation-bots#1254

Fixes googleapis/repo-automation-bots#1096

Source-Author: Jeff Ching <chingor@google.com>
Source-Date: Tue Dec 15 16:16:07 2020 -0800
Source-Repo: googleapis/synthtool
Source-Sha: 3f67ceece7e797a5736a25488aae35405649b90b
Source-Link: googleapis/synthtool@3f67cee
* changes without context

        autosynth cannot find the source of changes triggered by earlier changes in this
        repository, or by version upgrades to tools such as linters.

* chore: migrate java-bigquerystorage to the Java microgenerator

Committer: @miraleung
PiperOrigin-RevId: 345311069

Source-Author: Google APIs <noreply@google.com>
Source-Date: Wed Dec 2 14:17:15 2020 -0800
Source-Repo: googleapis/googleapis
Source-Sha: e39e42f368d236203a774ee994fcb4d730c33a83
Source-Link: googleapis/googleapis@e39e42f

* feat!: Updates to BigQuery Write API V1Beta2 public interface. This includes breaking changes to the API, it is fine because the API is not officially launched yet.

PiperOrigin-RevId: 345469340

Source-Author: Google APIs <noreply@google.com>
Source-Date: Thu Dec 3 09:33:11 2020 -0800
Source-Repo: googleapis/googleapis
Source-Sha: b53c4d98aab1eae3dac90b37019dede686782f13
Source-Link: googleapis/googleapis@b53c4d9

* fix: Update gapic-generator-java to 0.0.7

Committer: @miraleung
PiperOrigin-RevId: 345476969

Source-Author: Google APIs <noreply@google.com>
Source-Date: Thu Dec 3 10:07:32 2020 -0800
Source-Repo: googleapis/googleapis
Source-Sha: 7be2c821dd88109038c55c89f7dd48f092eeab9d
Source-Link: googleapis/googleapis@7be2c82

* chore: rollback migrating java-bigquerystorage to the Java microgenerator

Committer: @miraleung
PiperOrigin-RevId: 345522380

Source-Author: Google APIs <noreply@google.com>
Source-Date: Thu Dec 3 13:28:07 2020 -0800
Source-Repo: googleapis/googleapis
Source-Sha: f8f975c7d43904e90d6c5f1684fdb6804400e641
Source-Link: googleapis/googleapis@f8f975c

* chore: migrate java-bigquerystorage to the Java microgenerator

Committer: @miraleung
PiperOrigin-RevId: 346405446

Source-Author: Google APIs <noreply@google.com>
Source-Date: Tue Dec 8 14:03:11 2020 -0800
Source-Repo: googleapis/googleapis
Source-Sha: abc43060f136ce77124754a48f367102e646844a
Source-Link: googleapis/googleapis@abc4306

* chore: update gapic-generator-java to 0.0.11

Committer: @miraleung
PiperOrigin-RevId: 347036369

Source-Author: Google APIs <noreply@google.com>
Source-Date: Fri Dec 11 11:13:47 2020 -0800
Source-Repo: googleapis/googleapis
Source-Sha: 6d65640b1fcbdf26ea76cb720de0ac138cae9bed
Source-Link: googleapis/googleapis@6d65640

* chore: update gapic-generator-java to 0.0.13

Committer: @miraleung
PiperOrigin-RevId: 347849179

Source-Author: Google APIs <noreply@google.com>
Source-Date: Wed Dec 16 10:28:38 2020 -0800
Source-Repo: googleapis/googleapis
Source-Sha: 360a0e177316b7e9811f2ccbbef11e5f83377f3f
Source-Link: googleapis/googleapis@360a0e1
@stephaniewang526
Copy link
Member

@stephaniewang526 stephaniewang526 commented Dec 23, 2020

Getting NPE in ITBigQueryWriteManualClientTest.testJsonStreamWriterBatchWriteWithDefaultStream

java.lang.NullPointerException
	at com.google.cloud.bigquery.storage.v1beta2.TableFieldSchema$Builder.setMode(TableFieldSchema.java:1647)
	at com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter.ConvertFieldSchema(BQV2ToBQStorageConverter.java:69)
	at com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter.ConvertTableSchema(BQV2ToBQStorageConverter.java:56)
	at com.google.cloud.bigquery.storage.v1beta2.JsonStreamWriter.newBuilder(JsonStreamWriter.java:253)
	at com.google.cloud.bigquery.storage.v1beta2.it.ITBigQueryWriteManualClientTest.testJsonStreamWriterBatchWriteWithDefaultStream(ITBigQueryWriteManualClientTest.java:320)
...

full trace

import com.google.cloud.bigquery.StandardSQLTypeName;
import com.google.cloud.bigquery.storage.v1alpha2.Table;
Copy link
Contributor

@yoshi-code-bot yoshi-code-bot Dec 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
import com.google.cloud.bigquery.StandardSQLTypeName;
import com.google.cloud.bigquery.storage.v1alpha2.Table;
import com.google.cloud.bigquery.StandardSQLTypeName;

@codecov
Copy link

@codecov codecov bot commented Dec 28, 2020

Codecov Report

Merging #750 (0466702) into master (f3c897f) will increase coverage by 0.08%.
The diff coverage is 82.50%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #750      +/-   ##
============================================
+ Coverage     80.80%   80.89%   +0.08%     
- Complexity      985      994       +9     
============================================
  Files            73       74       +1     
  Lines          5289     5339      +50     
  Branches        406      414       +8     
============================================
+ Hits           4274     4319      +45     
- Misses          845      848       +3     
- Partials        170      172       +2     
Impacted Files Coverage Δ Complexity Δ
...oud/bigquery/storage/v1beta2/JsonStreamWriter.java 78.33% <79.16%> (+1.14%) 14.00 <2.00> (+2.00)
...e/cloud/bigquery/storage/v1beta2/StreamWriter.java 85.01% <82.60%> (-0.08%) 38.00 <2.00> (+1.00) ⬇️
...uery/storage/v1beta2/BQV2ToBQStorageConverter.java 84.84% <84.84%> (ø) 6.00 <6.00> (?)
.../cloud/bigquery/storage/v1alpha2/StreamWriter.java 84.31% <0.00%> (+0.43%) 37.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f3c897f...0466702. Read the comment docs.

@stephaniewang526 stephaniewang526 merged commit 488f258 into googleapis:master Dec 30, 2020
18 checks passed
@yirutang yirutang deleted the default-stream-1 branch Feb 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage cla: yes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants