Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add schema update back to json writer #905

Merged
merged 7 commits into from Mar 3, 2021

Conversation

yirutang
Copy link
Contributor

@yirutang yirutang commented Mar 2, 2021

  1. Add schema update back to json writer by recreate the stream.
  2. Remove the batching support on json writer so that potentially it can be moved to use StreamWriterV2. This is to avoid users become dependent on the feature. Currently the move is not possible because StreamWriterV2 is lacking the schema update callback feature. It can probably be done inside json writer as well, but will do this later.
  3. Also remove RetrySetting ExecutorSetting to be more consistent with what we actual support right now.

@yirutang yirutang requested a review from as a code owner Mar 2, 2021
@yirutang yirutang requested a review from steffnay Mar 2, 2021
@google-cla google-cla bot added the cla: yes label Mar 2, 2021
@product-auto-label product-auto-label bot added the api: bigquerystorage label Mar 2, 2021
@codecov
Copy link

@codecov codecov bot commented Mar 2, 2021

Codecov Report

Merging #905 (5e1f092) into master (7c1bb12) will increase coverage by 0.32%.
The diff coverage is 72.50%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #905      +/-   ##
============================================
+ Coverage     80.38%   80.70%   +0.32%     
- Complexity     1015     1020       +5     
============================================
  Files            76       76              
  Lines          5576     5550      -26     
  Branches        436      410      -26     
============================================
- Hits           4482     4479       -3     
+ Misses          916      882      -34     
- Partials        178      189      +11     
Impacted Files Coverage Δ Complexity Δ
...oud/bigquery/storage/v1beta2/JsonStreamWriter.java 76.52% <63.15%> (+10.97%) 14.00 <0.00> (+2.00)
...e/cloud/bigquery/storage/v1beta2/StreamWriter.java 82.79% <80.95%> (+3.58%) 32.00 <0.00> (+1.00)
...bigquery/storage/v1alpha2/BigQueryWriteClient.java 74.02% <0.00%> (-6.50%) 38.00% <0.00%> (ø%)
.../bigquery/storage/v1beta2/BigQueryWriteClient.java 70.76% <0.00%> (-4.62%) 33.00% <0.00%> (ø%)
...ud/bigquery/storage/v1/BaseBigQueryReadClient.java 58.53% <0.00%> (-2.44%) 12.00% <0.00%> (ø%)
...gquery/storage/v1beta2/BaseBigQueryReadClient.java 58.53% <0.00%> (-2.44%) 12.00% <0.00%> (ø%)
...ery/storage/v1beta1/BaseBigQueryStorageClient.java 71.18% <0.00%> (-1.70%) 22.00% <0.00%> (ø%)
.../cloud/bigquery/storage/v1alpha2/StreamWriter.java 79.95% <0.00%> (-0.44%) 34.00% <0.00%> (ø%)
...cloud/bigquery/storage/v1beta2/StreamWriterV2.java 95.67% <0.00%> (-0.44%) 39.00% <0.00%> (ø%)
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c1bb12...1e0ea32. Read the comment docs.

@yirutang yirutang requested a review from as a code owner Mar 2, 2021
@stephaniewang526 stephaniewang526 merged commit a2adbf8 into googleapis:master Mar 3, 2021
16 checks passed
gcf-merge-on-green bot pushed a commit that referenced this issue Mar 4, 2021
🤖 I have created a release \*beep\* \*boop\*
---
## [1.14.0](https://www.github.com/googleapis/java-bigquerystorage/compare/v1.13.0...v1.14.0) (2021-03-04)


### Features

* update StreamWriterV2 to support trace id ([#895](https://www.github.com/googleapis/java-bigquerystorage/issues/895)) ([2e49ce8](https://www.github.com/googleapis/java-bigquerystorage/commit/2e49ce8c79cb059840c3307898ba16980f6892fa))


### Bug Fixes

* add schema update back to json writer ([#905](https://www.github.com/googleapis/java-bigquerystorage/issues/905)) ([a2adbf8](https://www.github.com/googleapis/java-bigquerystorage/commit/a2adbf80753161cbddd23d5a7db75e9250db58fa))
* Add unit test for concurrent issues we worried about, and fix some locking issues ([#854](https://www.github.com/googleapis/java-bigquerystorage/issues/854)) ([0870797](https://www.github.com/googleapis/java-bigquerystorage/commit/087079728195e20f93701e8d5e1e59ba29a7d21b))
* test failure testAppendWhileShutdownSuccess ([#904](https://www.github.com/googleapis/java-bigquerystorage/issues/904)) ([b80183e](https://www.github.com/googleapis/java-bigquerystorage/commit/b80183ea23c8b78611a42d22d8c62a4ba4904a80))
* testAppendWhileShutdownSuccess race ([#907](https://www.github.com/googleapis/java-bigquerystorage/issues/907)) ([d39443d](https://www.github.com/googleapis/java-bigquerystorage/commit/d39443d51d2625e4b3aee59d1e593229e9e449d3))


### Dependencies

* update dependency com.google.cloud:google-cloud-bigquery to v1.127.6 ([#909](https://www.github.com/googleapis/java-bigquerystorage/issues/909)) ([505938b](https://www.github.com/googleapis/java-bigquerystorage/commit/505938bcba5a4a7af9e618572bbc41f365702f47))
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage cla: yes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants