fix: bug fix for streamWriter & jsonStreamWriter #2122

GaoleMeng · 2023-05-25T00:37:43Z

When the user explicitly provides a table schema to JsonStreamWriter, whenever there is a schema update, we will overwrite the user provided table schema with the updated schema. And we will use this schema to set writer_schema. This will make the frontend not able to set default value, because default value is only set when the columns are not specified in the writer_schema.
Also we combined another fix for location string for multiplexing comparision, we shouldn't use '==' to compare

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
Ensure the tests and linter pass
Code coverage does not decrease (if any source code was changed)
Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

If you write sample code, please follow the samples format.

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

…eue. Since in write api server side we have total timeout of 2 minutes, it does not make sense to wait 15 minutes to determine whether we have met dead connection, let's reduce the timeout here

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

...querystorage/src/main/java/com/google/cloud/bigquery/storage/v1/SchemaAwareStreamWriter.java

explicitly provided 2. fix location string matching for multiplexing

🤖 I have created a release *beep* *boop* --- ## [2.37.1](https://togithub.com/googleapis/java-bigquerystorage/compare/v2.37.0...v2.37.1) (2023-05-25) ### Bug Fixes * Bug fix for streamWriter & jsonStreamWriter ([#2122](https://togithub.com/googleapis/java-bigquerystorage/issues/2122)) ([36964a3](https://togithub.com/googleapis/java-bigquerystorage/commit/36964a39e0f2a7ede8c55c43313a32922c06fe15)) --- This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).

GaoleMeng and others added 13 commits March 30, 2023 10:52

feat: add public api to stream writer to set the maximum wait time

1726c85

Merge branch 'googleapis:main' into main

9b9e420

🦉 Updates from OwlBot post-processor

3f0c5eb

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

modify back the readme change from owl post processor

a448c8b

🦉 Updates from OwlBot post-processor

b15b140

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Merge branch 'googleapis:main' into main

4f5c0ad

Merge branch 'googleapis:main' into main

aa6d9c6

Merge branch 'googleapis:main' into main

7055043

Merge branch 'googleapis:main' into main

ea2e8fb

fix: Reduce the timeout to 5 minutes for the requests wait time in qu…

affa11a

…eue. Since in write api server side we have total timeout of 2 minutes, it does not make sense to wait 15 minutes to determine whether we have met dead connection, let's reduce the timeout here

🦉 Updates from OwlBot post-processor

60869f7

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Merge branch 'googleapis:main' into main

fc565e0

Merge branch 'googleapis:main' into main

e86b484

GaoleMeng requested a review from a team as a code owner May 25, 2023 00:37

GaoleMeng requested a review from chalmerlowe May 25, 2023 00:37

product-auto-label bot added size: m Pull request size is medium. api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. labels May 25, 2023

GaoleMeng requested a review from yirutang May 25, 2023 00:37

GaoleMeng force-pushed the main branch from 6c4c11c to 5d86306 Compare May 25, 2023 00:55

yirutang approved these changes May 25, 2023

View reviewed changes

yirutang added the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

yirutang added kokoro:run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels May 25, 2023

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

yirutang reviewed May 25, 2023

View reviewed changes

...querystorage/src/main/java/com/google/cloud/bigquery/storage/v1/SchemaAwareStreamWriter.java Show resolved Hide resolved

GaoleMeng force-pushed the main branch from 5d86306 to 8708446 Compare May 25, 2023 03:08

yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label May 25, 2023

GaoleMeng changed the title ~~fix: disable refresh writer when schema is explicitly given~~ fix: bug fix for streamWriter & jsonStreamWriter May 25, 2023

GaoleMeng force-pushed the main branch from 8708446 to 02305cb Compare May 25, 2023 03:13

yirutang added the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

yirutang added kokoro:run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels May 25, 2023

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label May 25, 2023

yirutang added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 25, 2023

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 25, 2023

GaoleMeng added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels May 25, 2023

yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels May 25, 2023

yirutang added the kokoro:run Add this label to force Kokoro to re-run the tests. label May 25, 2023

GaoleMeng added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 25, 2023

yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels May 25, 2023

GaoleMeng force-pushed the main branch from e7b7af6 to b3bbf77 Compare May 25, 2023 20:23

fix: 1.disable refresh of stream writer when the table schema is

687ef3a

explicitly provided 2. fix location string matching for multiplexing

GaoleMeng force-pushed the main branch from b3bbf77 to 687ef3a Compare May 25, 2023 20:28

GaoleMeng added the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

GaoleMeng added the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 25, 2023

GaoleMeng merged commit 36964a3 into googleapis:main May 25, 2023
20 checks passed

release-please bot mentioned this pull request May 25, 2023

chore(main): release 2.37.1 #2123

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: bug fix for streamWriter & jsonStreamWriter #2122

fix: bug fix for streamWriter & jsonStreamWriter #2122

GaoleMeng commented May 25, 2023 •

edited

fix: bug fix for streamWriter & jsonStreamWriter #2122

fix: bug fix for streamWriter & jsonStreamWriter #2122

Conversation

GaoleMeng commented May 25, 2023 • edited

GaoleMeng commented May 25, 2023 •

edited