Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add schema update support in JsonStreamWriter #1447

Merged
merged 12 commits into from Dec 29, 2021

Conversation

stephaniewang526
Copy link
Member

@stephaniewang526 stephaniewang526 commented Dec 15, 2021

No description provided.

@stephaniewang526 stephaniewang526 requested a review from as a code owner Dec 15, 2021
@stephaniewang526 stephaniewang526 requested a review from loferris (assigned from googleapis/api-bigquery) Dec 15, 2021
@product-auto-label product-auto-label bot added the api: bigquerystorage label Dec 15, 2021
@stephaniewang526 stephaniewang526 requested review from yirutang and removed request for loferris Dec 15, 2021
@stephaniewang526 stephaniewang526 added the owlbot:run label Dec 16, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run label Dec 16, 2021
@stephaniewang526 stephaniewang526 requested a review from as a code owner Dec 16, 2021
try {
response2 = jsonStreamWriter.append(jsonArr2);
next.getAndIncrement();
} catch (IOException | DescriptorValidationException e) {
Copy link
Contributor

@yirutang yirutang Dec 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully with ignore unknown fields, we could keep writing ccc and ddd to the table and expect ddd eventually to show up in the table. This is good for now.

Copy link
Member Author

@stephaniewang526 stephaniewang526 Dec 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack. Will keep this in mind.

stephaniewang526 and others added 4 commits Dec 23, 2021
# Conflicts:
#	google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1/JsonStreamWriter.java
@stephaniewang526 stephaniewang526 requested a review from yirutang Dec 23, 2021
@stephaniewang526 stephaniewang526 added automerge owlbot:run labels Dec 24, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run label Dec 24, 2021
@stephaniewang526 stephaniewang526 added kokoro:force-run and removed automerge labels Dec 24, 2021
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Dec 24, 2021
@stephaniewang526 stephaniewang526 added the owlbot:run label Dec 28, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run label Dec 28, 2021
// Create a new underlying StreamWriter with the updated TableSchema and Descriptor
this.streamWriter =
streamWriterBuilder
.setWriterSchema(ProtoSchemaConverter.convert(this.descriptor))
Copy link
Contributor

@shollyman shollyman Dec 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to propagate anything else to the new writer like stream id, existing trace id, etc or does it all come from properties stored on the JsonStreamWriter?

Copy link
Member Author

@stephaniewang526 stephaniewang526 Dec 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, those things are needed and are already being populated in the streamWriterBuilder.

@stephaniewang526 stephaniewang526 merged commit 973afcc into googleapis:main Dec 29, 2021
18 checks passed
gcf-merge-on-green bot pushed a commit that referenced this issue Jan 4, 2022
🤖 I have created a release \*beep\* \*boop\*
---
## [2.8.0](https://www.github.com/googleapis/java-bigquerystorage/compare/v2.7.0...v2.8.0) (2021-12-30)


### Features

* Add ignoreUnknownField support in JsonWriter ([#1455](https://www.github.com/googleapis/java-bigquerystorage/issues/1455)) ([4616adb](https://www.github.com/googleapis/java-bigquerystorage/commit/4616adb30d07d7be522b74837adae080eca103a6))
* Add reconnect support to v1 client lib. ([#1446](https://www.github.com/googleapis/java-bigquerystorage/issues/1446)) ([a5157fa](https://www.github.com/googleapis/java-bigquerystorage/commit/a5157faf37354ee48a0af930a68edf4823b5be6f))
* add schema update support in JsonStreamWriter ([#1447](https://www.github.com/googleapis/java-bigquerystorage/issues/1447)) ([973afcc](https://www.github.com/googleapis/java-bigquerystorage/commit/973afccc22507b8109b3920d3fc183692dcc66c2))
* support `append()` without offset in `StreamWriter` ([#1452](https://www.github.com/googleapis/java-bigquerystorage/issues/1452)) ([cb8b0ad](https://www.github.com/googleapis/java-bigquerystorage/commit/cb8b0ad25fbf07c3509b3d94244a999b9cb90e54))


### Bug Fixes

* allow all fields to be null values ([#1450](https://www.github.com/googleapis/java-bigquerystorage/issues/1450)) ([e47ac79](https://www.github.com/googleapis/java-bigquerystorage/commit/e47ac791765486f02f027d3c440645ef2a379ba9))
* finish BIGNUMERIC support ([#1449](https://www.github.com/googleapis/java-bigquerystorage/issues/1449)) ([d9d51cd](https://www.github.com/googleapis/java-bigquerystorage/commit/d9d51cd203733a972c80e312e9e279b50fb260fa))
* fix a NullPtr when user closes a writer without connection being ever established ([#1454](https://www.github.com/googleapis/java-bigquerystorage/issues/1454)) ([b774f5d](https://www.github.com/googleapis/java-bigquerystorage/commit/b774f5d0d00275278708d1299b4912b455be4bed))
* update storageError support due to server side enhancement ([#1456](https://www.github.com/googleapis/java-bigquerystorage/issues/1456)) ([6243ad5](https://www.github.com/googleapis/java-bigquerystorage/commit/6243ad5cba61d4dae7f4ceb60b09c625e7589215))
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants