feat: add support for JSON data type #1368

olavloite · 2021-05-12T14:06:44Z

Adds support for JSON data type.

snippet-bot · 2021-05-12T14:06:54Z

Here is the summary of changes.

You are about to add 3 region tags.

samples/json-add-column.js:18, tag spanner_add_json_column
samples/json-query-parameter.js:18, tag spanner_query_with_json_parameter
samples/json-update-data.js:18, tag spanner_update_data_with_json_column

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

Refresh this comment

olavloite · 2021-05-12T14:08:53Z

samples/json-update-data.js

+      // array of objects that should be inserted into a JSON column. If we were
+      // to specify this value as an array instead of a string, the client
+      // library would encode this value as ARRAY<JSON> instead of JSON.
+      VenueDetails: `[


@zoercai Here's an example of the issue with top-level JSON arrays. The client library does not know whether this should be encoded as JSON or ARRAY<JSON> if it is specified as an actual array instead of a string. Specifying it as a string will cause the client library to just encode it as a string, which is the default encoding for JSON values.

olavloite · 2021-05-12T14:10:31Z

src/codec.ts

+  if (is.object(value)) {
+    return JSON.stringify(value);
+  }


This is where the most important part of this PR is: It ensures that any value that has been specified as an object will be converted to a valid JSON string. An array of objects will be converted to an array of valid JSON strings.

codecov · 2021-05-17T08:06:23Z

Codecov Report

Merging #1368 (52b8cf5) into master (ecaaf3b) will decrease coverage by 0.43%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1368      +/-   ##
==========================================
- Coverage   98.60%   98.16%   -0.44%     
==========================================
  Files          23       23              
  Lines       21950    21964      +14     
  Branches     1238     1219      -19     
==========================================
- Hits        21644    21562      -82     
- Misses        297      388      +91     
- Partials        9       14       +5

Impacted Files	Coverage Δ
src/codec.ts	`99.70% <100.00%> (-0.30%)`	⬇️
src/v1/database_admin_client.ts	`98.50% <0.00%> (-1.00%)`	⬇️
src/v1/instance_admin_client.ts	`98.15% <0.00%> (-0.98%)`	⬇️
src/v1/spanner_client.ts	`98.40% <0.00%> (-0.85%)`	⬇️
src/transaction-runner.ts	`97.77% <0.00%> (-0.84%)`	⬇️
src/backup.ts	`99.10% <0.00%> (-0.72%)`	⬇️
src/common-grpc/service-object.ts	`99.34% <0.00%> (-0.66%)`	⬇️
src/session-pool.ts	`99.34% <0.00%> (-0.50%)`	⬇️
src/database.ts	`99.75% <0.00%> (-0.22%)`	⬇️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ecaaf3b...52b8cf5. Read the comment docs.

src/transaction.ts

skuruppu · 2021-06-24T10:57:40Z

@olavloite I just merged in the codegen PR so you can rebase this PR now.

olavloite · 2021-06-24T19:28:35Z

I added the Do not merge label back as the system test is failing with

  1) Spanner
       types
         "before all" hook for "should throw an error for incorrect value types":
     Error: 12 UNIMPLEMENTED: JSON type is not enabled.
      at Object.callErrorFromStatus (node_modules/@grpc/grpc-js/build/src/call.js:31:26)
          -> /workspace/node_modules/@grpc/grpc-js/src/call.ts:81:24
      at Object.onReceiveStatus (node_modules/@grpc/grpc-js/build/src/client.js:179:52)
          -> /workspace/node_modules/@grpc/grpc-js/src/client.ts:338:36
      at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:80:35
          -> /workspace/node_modules/@grpc/grpc-js/src/call-stream.ts:170:27
      at Object.onReceiveStatus (node_modules/grpc-gcp/build/src/index.js:73:29)
          -> /workspace/node_modules/grpc-gcp/src/index.ts:129:15
      at InterceptingListenerImpl.onReceiveStatus (node_modules/@grpc/grpc-js/build/src/call-stream.js:75:23)
          -> /workspace/node_modules/@grpc/grpc-js/src/call-stream.ts:166:19
      at Object.onReceiveStatus (node_modules/@grpc/grpc-js/build/src/client-interceptors.js:336:141)
          -> /workspace/node_modules/@grpc/grpc-js/src/client-interceptors.ts:426:34
      at Object.onReceiveStatus (node_modules/@grpc/grpc-js/build/src/client-interceptors.js:299:181)
          -> /workspace/node_modules/@grpc/grpc-js/src/client-interceptors.ts:389:48
      at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:145:78
          -> /workspace/node_modules/@grpc/grpc-js/src/call-stream.ts:276:24
      at processTicksAndRejections (internal/process/task_queues.js:79:11)

generated-files-bot · 2021-07-03T09:00:29Z

Warning: This pull request is touching the following templated files:

README.md - README.md is managed by synthtool. However, a partials file can be used to update the README, e.g.: https://github.com/googleapis/nodejs-storage/blob/master/.readme-partials.yaml
samples/README.md - samples/README.md is managed by synthtool. However, a partials file can be used to update the README, e.g.: https://github.com/googleapis/nodejs-storage/blob/master/.readme-partials.yaml

test/codec.ts

zoercai · 2021-08-23T03:48:40Z

Hi @olavloite, we're getting ready to merge the JSON PRs now for Node, Java, and Python. You can merge this PR as soon as all the tests pass. Could you please rebase and rerun CI? Thanks!

skuruppu · 2021-08-25T02:03:11Z

@olavloite if you can please take a look at the test failure, that would be great. I ran it a couple of times and got the same error so I gathered it must be due to the new tests we added in this PR.

ERROR: Error: 6 ALREADY_EXISTS: Row [81] in table Venues already exists.

olavloite · 2021-08-25T06:21:55Z

@olavloite if you can please take a look at the test failure, that would be great. I ran it a couple of times and got the same error so I gathered it must be due to the new tests we added in this PR.

ERROR: Error: 6 ALREADY_EXISTS: Row [81] in table Venues already exists.

It was caused by a merge/rebase error. The previously added test runs for the tagging feature were also added once more by this PR, which meant that the tagging sample tests were executed twice. I've removed the duplicate test runs, and hopefully that should fix it.

skuruppu · 2021-08-25T07:11:45Z

Now asked for help from the Node.js team on how to unblock the OwlBot check.

* feat: support JSON data type * fix: remove unnecessary changes * fix: linting + skip tests on emulator * docs: simplify samples * feat: decorate commit error with additional information Decorate the error returned by a commit RPC in the specific case that the application tries to insert an array of objects into a JSON column. * test: add unit tests for decorateError * test: add test for deeply nested objects * test: add test for decoding complex JSON string * fix: typo * fix: run linter * fix: update expected query output * chore: run linter * chore: run linter * fix: remove duplicate tag test execution * fix: remove duplicate transaction tag test run Co-authored-by: skuruppu <skuruppu@google.com>

olavloite added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label May 12, 2021

olavloite requested review from skuruppu and zoercai May 12, 2021 14:06

olavloite requested review from a team as code owners May 12, 2021 14:06

google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label May 12, 2021

product-auto-label bot added the api: spanner Issues related to the googleapis/nodejs-spanner API. label May 12, 2021

olavloite commented May 12, 2021

View reviewed changes

skuruppu approved these changes May 13, 2021

View reviewed changes

olavloite commented May 20, 2021

View reviewed changes

src/transaction.ts Show resolved Hide resolved

skuruppu approved these changes Jun 9, 2021

View reviewed changes

olavloite force-pushed the json-data-type branch from 801c665 to 7a8331f Compare June 24, 2021 17:34

olavloite added do not merge Indicates a pull request not ready for merge, due to either quality or timing. and removed do not merge Indicates a pull request not ready for merge, due to either quality or timing. labels Jun 24, 2021

bcoe added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 2, 2021

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 2, 2021

skuruppu reviewed Jul 22, 2021

View reviewed changes

test/codec.ts Show resolved Hide resolved

skuruppu reviewed Jul 23, 2021

View reviewed changes

test/codec.ts Outdated Show resolved Hide resolved

skuruppu approved these changes Jul 23, 2021

View reviewed changes

olavloite removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Aug 23, 2021

olavloite mentioned this pull request Aug 23, 2021

feat: add support for JSON data type #1456

Closed

olavloite added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 23, 2021

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Aug 23, 2021

olavloite added 5 commits August 24, 2021 16:04

fix: typo

2045cc0

fix: run linter

4f79475

fix: update expected query output

654dc88

chore: run linter

9d8165c

chore: run linter

58b046c

olavloite force-pushed the json-data-type branch from 54f54dc to 58b046c Compare August 24, 2021 14:20

skuruppu removed the owlbot:ignore instruct owl-bot to ignore a PR label Aug 24, 2021

Merge branch 'master' into json-data-type

f320042

skuruppu added automerge Merge the pull request once unit tests and other checks pass. owlbot:run Add this label to trigger the Owlbot post processor. and removed automerge Merge the pull request once unit tests and other checks pass. labels Aug 24, 2021

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Aug 25, 2021

skuruppu added the owlbot:ignore instruct owl-bot to ignore a PR label Aug 25, 2021

skuruppu force-pushed the json-data-type branch from 20903b8 to f320042 Compare August 25, 2021 01:24

skuruppu added the automerge Merge the pull request once unit tests and other checks pass. label Aug 25, 2021

skuruppu removed the automerge Merge the pull request once unit tests and other checks pass. label Aug 25, 2021

olavloite added 2 commits August 25, 2021 08:19

fix: remove duplicate tag test execution

8c4d8de

fix: remove duplicate transaction tag test run

e5f9934

skuruppu added owlbot:ignore instruct owl-bot to ignore a PR and removed owlbot:ignore instruct owl-bot to ignore a PR labels Aug 25, 2021

Merge branch 'master' into json-data-type

c838090

skuruppu added the automerge Merge the pull request once unit tests and other checks pass. label Aug 26, 2021

olavloite mentioned this pull request Aug 26, 2021

feat: add support for JSON data type #1459

Closed

olavloite merged commit b8d6fe5 into googleapis:master Aug 26, 2021

gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Aug 26, 2021

olavloite deleted the json-data-type branch August 26, 2021 12:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for JSON data type #1368

feat: add support for JSON data type #1368

olavloite commented May 12, 2021 •

edited by skuruppu

Loading

snippet-bot bot commented May 12, 2021 •

edited

Loading

olavloite May 12, 2021

olavloite May 12, 2021

codecov bot commented May 17, 2021

skuruppu commented Jun 24, 2021

olavloite commented Jun 24, 2021

generated-files-bot bot commented Jul 3, 2021

zoercai commented Aug 23, 2021

skuruppu commented Aug 25, 2021

olavloite commented Aug 25, 2021

skuruppu commented Aug 25, 2021

feat: add support for JSON data type #1368

feat: add support for JSON data type #1368

Conversation

olavloite commented May 12, 2021 • edited by skuruppu Loading

snippet-bot bot commented May 12, 2021 • edited Loading

olavloite May 12, 2021

Choose a reason for hiding this comment

olavloite May 12, 2021

Choose a reason for hiding this comment

codecov bot commented May 17, 2021

Codecov Report

skuruppu commented Jun 24, 2021

olavloite commented Jun 24, 2021

generated-files-bot bot commented Jul 3, 2021

zoercai commented Aug 23, 2021

skuruppu commented Aug 25, 2021

olavloite commented Aug 25, 2021

skuruppu commented Aug 25, 2021

olavloite commented May 12, 2021 •

edited by skuruppu

Loading

snippet-bot bot commented May 12, 2021 •

edited

Loading