Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add native OpenX JSON format #16073

Merged
merged 1 commit into from
Mar 14, 2023
Merged

Add native OpenX JSON format #16073

merged 1 commit into from
Mar 14, 2023

Conversation

dain
Copy link
Member

@dain dain commented Feb 12, 2023

Description

Add native implementation of the OpenX JSON SerDe (org.openx.data.jsonserde.JsonSerDe) with extensions. The exact behavior of this format is covered in the README.md in Add native OpenX JSON format commit.

Documentation notes

This format implementation contains several extensions over the original PR as noted in the README file. This is a different approach than the other native implementations which strive for a bug-for-bug compatibility. I think we should summarize the major changes into the main documentations:

  • Added support for reading and writing DECIMAL(P, S), CHAR(N), VARCHAR(N), VARBINARY.
  • Value parse errors produce a null instead of a failure.
  • JSON object or array values for numeric or temporal types, throw an exception.
  • Values that are outside the allowed type range result in null.
    These extensions are not in Hive and Spark, so any table that relies on these extensions may not be readable and/or writable in Hive and Spark. In this case, the user would simply change the column types in the Hive Metastore to a type supported by Hive and Spark.

Release notes

# Section
* Add native OpenX JSON file format reader and writer. These can be disabled with the `openx_json_native_reader_enabled` and `openx_json_native_writer_enabled` session properties or the `openx-json.native-reader.enabled` and `openx-json.native-writer.enabled` configuration property. ({16073}``)

@1ambda
Copy link

1ambda commented May 30, 2024

For someone who searching the way to use OpenX JSON Serde,

Just use OPENX_JSON for ‘format’ the table property. Those session properties introduced in this pr was removed somehow.

@electrum
Copy link
Member

cc @mosabua we should document this

@mosabua
Copy link
Member

mosabua commented Jun 3, 2024

@electrum @dain @1ambda - #22252

@mosabua
Copy link
Member

mosabua commented Jun 4, 2024

Linked PR in preceding comment is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed hive Hive connector
Development

Successfully merging this pull request may close these issues.

4 participants