Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add ignoreUnknownField support in JsonWriter #1455

Merged
merged 5 commits into from Dec 29, 2021

Conversation

yirutang
Copy link
Contributor

@yirutang yirutang commented Dec 28, 2021

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

@yirutang yirutang requested a review from as a code owner Dec 28, 2021
@yirutang yirutang requested a review from tswast (assigned from googleapis/api-bigquery) Dec 28, 2021
@product-auto-label product-auto-label bot added the api: bigquerystorage label Dec 28, 2021
@stephaniewang526 stephaniewang526 added the owlbot:run label Dec 29, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run label Dec 29, 2021
@stephaniewang526
Copy link
Member

@stephaniewang526 stephaniewang526 commented Dec 29, 2021

We should revisit the test case for schema update while introducing this feat (related to this comment).

Copy link
Contributor Author

@yirutang yirutang left a comment

PTAL, rebased.

@stephaniewang526 stephaniewang526 added the owlbot:run label Dec 29, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run label Dec 29, 2021
*
* @return Builder
*/
public Builder ignoreUnknownFields() {
Copy link
Member

@stephaniewang526 stephaniewang526 Dec 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's more conventional and explicit(true vs. false) if we do setIgnoreUnknownFields(boolean ignoreUnknwonFields) {...} (an equivalent example in google-cloud-bigquery client).

Copy link
Contributor Author

@yirutang yirutang Dec 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -64,7 +64,8 @@ public static DynamicMessage convertJsonToProtoMessage(Descriptor protoSchema, J
Preconditions.checkNotNull(protoSchema, "Protobuf descriptor is null.");
Preconditions.checkState(json.length() != 0, "JSONObject is empty.");

return convertJsonToProtoMessageImpl(protoSchema, null, json, "root", /*topLevel=*/ true);
return convertJsonToProtoMessageImpl(
protoSchema, null, json, "root", /*topLevel=*/ true, false);
Copy link
Member

@stephaniewang526 stephaniewang526 Dec 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, topLevel is not used in convertJsonToProtoMessageImpl. Hence, we should remove it.

Copy link
Contributor Author

@yirutang yirutang Dec 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack

throw new IllegalArgumentException(
String.format("JSONObject has fields unknown to BigQuery: %s.", currentScope));
} else if (field == null) {
Copy link
Member

@stephaniewang526 stephaniewang526 Dec 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check seems redundant.

Copy link
Contributor Author

@yirutang yirutang Dec 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is null, it means we cannot proceed on further processing. Otherwise, null ptr will be thrown if we continue.

Copy link
Member

@stephaniewang526 stephaniewang526 Dec 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay makes sense -- this allows us to skip the unknown field in the middle and proceed onto other fields.

throw new IllegalArgumentException(
String.format("JSONObject has fields unknown to BigQuery: %s.", currentScope));
} else if (field == null) {
Copy link
Member

@stephaniewang526 stephaniewang526 Dec 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay makes sense -- this allows us to skip the unknown field in the middle and proceed onto other fields.

@stephaniewang526 stephaniewang526 added the owlbot:run label Dec 29, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run label Dec 29, 2021
@stephaniewang526 stephaniewang526 added automerge owlbot:run labels Dec 29, 2021
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run label Dec 29, 2021
@gcf-merge-on-green gcf-merge-on-green bot merged commit 4616adb into googleapis:main Dec 29, 2021
18 checks passed
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge label Dec 29, 2021
gcf-merge-on-green bot pushed a commit that referenced this issue Jan 4, 2022
🤖 I have created a release \*beep\* \*boop\*
---
## [2.8.0](https://www.github.com/googleapis/java-bigquerystorage/compare/v2.7.0...v2.8.0) (2021-12-30)


### Features

* Add ignoreUnknownField support in JsonWriter ([#1455](https://www.github.com/googleapis/java-bigquerystorage/issues/1455)) ([4616adb](https://www.github.com/googleapis/java-bigquerystorage/commit/4616adb30d07d7be522b74837adae080eca103a6))
* Add reconnect support to v1 client lib. ([#1446](https://www.github.com/googleapis/java-bigquerystorage/issues/1446)) ([a5157fa](https://www.github.com/googleapis/java-bigquerystorage/commit/a5157faf37354ee48a0af930a68edf4823b5be6f))
* add schema update support in JsonStreamWriter ([#1447](https://www.github.com/googleapis/java-bigquerystorage/issues/1447)) ([973afcc](https://www.github.com/googleapis/java-bigquerystorage/commit/973afccc22507b8109b3920d3fc183692dcc66c2))
* support `append()` without offset in `StreamWriter` ([#1452](https://www.github.com/googleapis/java-bigquerystorage/issues/1452)) ([cb8b0ad](https://www.github.com/googleapis/java-bigquerystorage/commit/cb8b0ad25fbf07c3509b3d94244a999b9cb90e54))


### Bug Fixes

* allow all fields to be null values ([#1450](https://www.github.com/googleapis/java-bigquerystorage/issues/1450)) ([e47ac79](https://www.github.com/googleapis/java-bigquerystorage/commit/e47ac791765486f02f027d3c440645ef2a379ba9))
* finish BIGNUMERIC support ([#1449](https://www.github.com/googleapis/java-bigquerystorage/issues/1449)) ([d9d51cd](https://www.github.com/googleapis/java-bigquerystorage/commit/d9d51cd203733a972c80e312e9e279b50fb260fa))
* fix a NullPtr when user closes a writer without connection being ever established ([#1454](https://www.github.com/googleapis/java-bigquerystorage/issues/1454)) ([b774f5d](https://www.github.com/googleapis/java-bigquerystorage/commit/b774f5d0d00275278708d1299b4912b455be4bed))
* update storageError support due to server side enhancement ([#1456](https://www.github.com/googleapis/java-bigquerystorage/issues/1456)) ([6243ad5](https://www.github.com/googleapis/java-bigquerystorage/commit/6243ad5cba61d4dae7f4ceb60b09c625e7589215))
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants