Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable serde_json arbitrary_precision feature flag #1174

Closed
tustvold opened this issue Jan 14, 2022 · 0 comments · Fixed by #1175
Closed

Disable serde_json arbitrary_precision feature flag #1174

tustvold opened this issue Jan 14, 2022 · 0 comments · Fixed by #1175
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog

Comments

@tustvold
Copy link
Contributor

tustvold commented Jan 14, 2022

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

The arbitrary_precision feature flag added in #779 alters how serde_json decodes numeric types, decoding them as maps with special keys, instead of their native types.

Unfortunately this has a tendency to break downstream code in strange ways - see here. The nature of feature flags makes this both unexpected, hard to diagnose, and impossible to opt out of.

Describe the solution you'd like

I have filed a ticket for native 128-bit support in serde_json here which I think is the ideal solution, but until then I would like to propose we encode 128-bit numbers as strings.

FWIW I'm not sure this will necessarily be any more or less broken than the status quo, as whilst the official JSON specification states that numbers should be arbitrary precision, a number of implementations assume values fit into 64-bit doubles (i.e. 52-bits of integer precision). It is for this reason that protobuf's JSON format actually encodes 64-bit integers, let alone 128-bit integers as strings - see here.

Additionally the arrow docs seem to suggest that data in JSON is intepreted as i64 - see here. It is unclear how it expects i128 to be encoded/decoded.

Describe alternatives you've considered

Add a feature flag to allow users to opt-in to using arbitrary_precision.

@tustvold tustvold added the enhancement Any new improvement worthy of a entry in the changelog label Jan 14, 2022
@alamb alamb added the arrow Changes to the arrow crate label Jan 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants