-
Notifications
You must be signed in to change notification settings - Fork 564
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow f32
and f64
map keys
#1027
Conversation
Now we allow objects with `f32` or `f64` string-serialized keys. For example: ```json { "1.123": "foo", "2.0": "bar" } ``` The `test_deny_float_key` test has been inverted, and the `deserialize_integer_key` macros have been renamed to `deserialize_numeric_key` in order to support `f32` and `f64`.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you be able to elaborate on the use case for this?
In general Rust does not make it easy to work with float map keys, because Eq
is typically required of map keys, and floats do not implement it.
This handles NaN and both positive and negative infinity.
This is useful for serializing data types like |
src/de.rs
Outdated
deserialize_numeric_key!(deserialize_f32 => visit_f32); | ||
deserialize_numeric_key!(deserialize_f64 => visit_f64); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.parse()
is not appropriate to use here for floats. That accepts a bunch of syntax that is not considered a valid float in json. For example .0
or 00
. Please make sure this accepts only representations that would be valid f32/f64 outside of a map key.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've updated this to delegate to the parent Deserializer
via self.de.method(...)
. This unfortunately makes the error messages a bit worse, as we now get something like:
expected value at line 1 column 3
Instead of:
"invalid type: string \"x\", expected f32 at line 1 column 4"
I noticed a similar issue here is present with integer keys, this example works on serde_json
master
:
extern crate serde_json;
extern crate serde;
fn main() {
dbg!(serde_json::from_str::<std::collections::HashMap<i32, i32>>(r#"{"00":0}"#));
}
Is it worth updating this as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, good call. I filed #1034 to follow up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In an effort to patch #1034, I've also updated integer key parsing to use Deserializer::*
. This still exhibits the above error degredation.
Would it make sense here to parse the key into an intermediate string buffer, and then deserialize the integer value using crate::from_str
? This would result in the original error messages for unexpected string keys, but may have a performance impact.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs a check for leading whitespace, otherwise it would end up accepting potentially invalid JSON syntax.
For example this is not a legal JSON document:
{"
1":null}
// [dependencies]
// ordered-float = { version = "3", features = ["serde"] }
// serde_json = "1"
use ordered_float::NotNan;
use std::collections::BTreeMap;
fn main() {
let invalid_json = r#"
{"
1":null}
"#;
let map: BTreeMap<NotNan<f64>, ()> = serde_json::from_str(invalid_json).unwrap();
println!("{:?}", map);
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That'll fix the error message problem too. JSON numbers must start with 0-9
or -
. If the upcoming character is not one of those, you can avoid calling the function that would produced the bad message in this situation, and directly create a useful "invalid value" error instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've patched this with a check for whitespace characters. I don't quite follow what you mean about fixing the error, the misleading text is still present, no? E.g. this example:
fn main() {
dbg!(serde_json::from_str::<std::collections::HashMap<i32, i32>>(r#"{"abc":0}"#));
}
Previously would print invalid type: string "abc", expected i32 at line 1 column 6
, but after this changeset it prints expected value at line 1 column 3
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is what I was thinking of doing to give a more meaningful error: #1035.
This ensures consistency with the default `Formatter::write_f32` output format used elsewhere.
`deserialize_numeric_key!` has been renamed back to `deserialize_integer_key!`, and a new float-specific `deserialize_float_key!` macro has been introduced. This new macro attempts to deserialize float keys by delegating to the equivalent parent deserializer method.
An `unwrap` has been changed to simply discard the `Result`. We assume that writing to a `Vec` will never fail because the `std::io::Write` implementation never creates an I/O error.
c4b9759
to
468a94a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The serialization looks good!
I left one more comment up in #1027 (review) about the deserialization.
61fe500
to
a4e2719
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [serde_json](https://github.com/serde-rs/json) | dependencies | patch | `1.0.100` -> `1.0.102` | --- ### Release Notes <details> <summary>serde-rs/json (serde_json)</summary> ### [`v1.0.102`](https://github.com/serde-rs/json/releases/tag/v1.0.102) [Compare Source](serde-rs/json@v1.0.101...v1.0.102) - Add a way to customize the serialization of byte arrays ([#​1039](serde-rs/json#1039)) ### [`v1.0.101`](https://github.com/serde-rs/json/releases/tag/v1.0.101) [Compare Source](serde-rs/json@v1.0.100...v1.0.101) - Allow f32 and f64 as keys in maps ([#​1027](serde-rs/json#1027), thanks [@​overdrivenpotato](https://github.com/overdrivenpotato)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi44LjIiLCJ1cGRhdGVkSW5WZXIiOiIzNi44LjIiLCJ0YXJnZXRCcmFuY2giOiJkZXZlbG9wIn0=--> Co-authored-by: cabr2-bot <cabr2.help@gmail.com> Co-authored-by: crapStone <crapstone01@gmail.com> Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1960 Reviewed-by: crapStone <crapstone01@gmail.com> Co-authored-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org> Co-committed-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>
Now we allow objects with
f32
orf64
string-serialized keys. For example:The
test_deny_float_key
test has been inverted, and thedeserialize_integer_key
macros have been renamed todeserialize_numeric_key
in order to supportf32
andf64
.