Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support PARSE_JSON / TRY_PARSE_JSON function #4534

Merged
merged 1 commit into from
Mar 23, 2022

Conversation

b41sh
Copy link
Member

@b41sh b41sh commented Mar 22, 2022

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Support PARSE_JSON function
Support TRY_PARSE_JSON function

Changelog

  • New Feature
  • Build/Testing/CI
  • Need Documentation (Need to add documentation to https://databend.rs)

Related Issues

Fixes #4476

Test Plan

Unit Tests

Stateless Tests

@vercel
Copy link

vercel bot commented Mar 22, 2022

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/databend/databend/q6ch5RiXBiEcBb1CusQaDyJMq7Hs
✅ Preview: https://databend-git-fork-b41sh-func-parse-json-databend.vercel.app

@mergify
Copy link
Contributor

mergify bot commented Mar 22, 2022

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@mergify mergify bot added pr-feature this PR introduces a new feature to the codebase pr-build this PR changes build/testing/ci steps labels Mar 22, 2022
return Ok(NullType::arc());
}

Ok(Arc::new(NullableType::create(VariantType::arc())))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If SUPPRESS_PARSE_ERROR is false, the return type should not be nullable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parse_json may also yield null, for example:

mysql> select parse_json('');
+----------------+
| parse_json('') |
+----------------+
| NULL           |
+----------------+
1 row in set (0.01 sec)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what's the difference between JsonValue::Null and DataValue::Null ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DataValue::Null is a SQL null, and JsonValue::Null is a JSON null. Detailed description can refer to this document:
https://docs.snowflake.com/en/sql-reference/functions/parse_json.html#usage-notes

mysql> select parse_json(null), parse_json(''), parse_json('null');
+------------------+----------------+--------------------+
| parse_json(NULL) | parse_json('') | parse_json('null') |
+------------------+----------------+--------------------+
|             NULL | NULL           | null               |
+------------------+----------------+--------------------+
1 row in set (0.02 sec)

Copy link
Member

@sundy-li sundy-li Mar 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So If SUPPRESS_PARSE_ERROR is false, it's JsonColumn not Nullable(JsonColumn).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if SUPPRESS_PARSE_ERROR is false, it should also return Nullable(JsonColumn). Because if the input value is an empty string, it should return a SQL null.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the right way is:

  1. parse_json -> SUPPRESS_PARSE_ERROR true -> JsonColumn
  2. try_parse_json -> SUPPRESS_PARSE_ERROR false -> Nullable(JsonColumn)

@BohuTANG BohuTANG merged commit d8f27da into databendlabs:main Mar 23, 2022
@BohuTANG BohuTANG mentioned this pull request May 12, 2022
55 tasks
@Xuanwo Xuanwo added this to the v0.8 milestone May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need-review pr-build this PR changes build/testing/ci steps pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: Semi-structured data function parse_json
5 participants