Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow f32 and f64 map keys #1027

Merged
merged 11 commits into from Jul 11, 2023
Merged

Conversation

overdrivenpotato
Copy link
Contributor

Now we allow objects with f32 or f64 string-serialized keys. For example:

{ "1.123": "foo", "2.0": "bar" }

The test_deny_float_key test has been inverted, and the deserialize_integer_key macros have been renamed to deserialize_numeric_key in order to support f32 and f64.

Now we allow objects with `f32` or `f64` string-serialized keys. For example:

```json
{ "1.123": "foo", "2.0": "bar" }
```

The `test_deny_float_key` test has been inverted, and the
`deserialize_integer_key` macros have been renamed to
`deserialize_numeric_key` in order to support `f32` and `f64`.
Copy link
Member

@dtolnay dtolnay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you be able to elaborate on the use case for this?

In general Rust does not make it easy to work with float map keys, because Eq is typically required of map keys, and floats do not implement it.

src/ser.rs Show resolved Hide resolved
This handles NaN and both positive and negative infinity.
@overdrivenpotato
Copy link
Contributor Author

Would you be able to elaborate on the use case for this?

This is useful for serializing data types like BTreeMap<NotNan<f32>, T>, using a type like NotNan or OrderedFloat from the ordered-float crate. A type like that could be useful in a few situations, e.g. a sorted map for timestamp -> event data, or to maintain an ordered list that allows for insertion between integer indices.

src/de.rs Outdated
Comment on lines 2168 to 2169
deserialize_numeric_key!(deserialize_f32 => visit_f32);
deserialize_numeric_key!(deserialize_f64 => visit_f64);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.parse() is not appropriate to use here for floats. That accepts a bunch of syntax that is not considered a valid float in json. For example .0 or 00. Please make sure this accepts only representations that would be valid f32/f64 outside of a map key.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated this to delegate to the parent Deserializer via self.de.method(...). This unfortunately makes the error messages a bit worse, as we now get something like:

expected value at line 1 column 3

Instead of:

"invalid type: string \"x\", expected f32 at line 1 column 4"

I noticed a similar issue here is present with integer keys, this example works on serde_json master:

extern crate serde_json;
extern crate serde;

fn main() {
    dbg!(serde_json::from_str::<std::collections::HashMap<i32, i32>>(r#"{"00":0}"#));
}

Is it worth updating this as well?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good call. I filed #1034 to follow up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In an effort to patch #1034, I've also updated integer key parsing to use Deserializer::*. This still exhibits the above error degredation.

Would it make sense here to parse the key into an intermediate string buffer, and then deserialize the integer value using crate::from_str? This would result in the original error messages for unexpected string keys, but may have a performance impact.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a check for leading whitespace, otherwise it would end up accepting potentially invalid JSON syntax.

For example this is not a legal JSON document:

{"
1":null}
// [dependencies]
// ordered-float = { version = "3", features = ["serde"] }
// serde_json = "1"

use ordered_float::NotNan;
use std::collections::BTreeMap;

fn main() {
    let invalid_json = r#"
        {"
        1":null}
    "#;

    let map: BTreeMap<NotNan<f64>, ()> = serde_json::from_str(invalid_json).unwrap();
    println!("{:?}", map);
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'll fix the error message problem too. JSON numbers must start with 0-9 or -. If the upcoming character is not one of those, you can avoid calling the function that would produced the bad message in this situation, and directly create a useful "invalid value" error instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've patched this with a check for whitespace characters. I don't quite follow what you mean about fixing the error, the misleading text is still present, no? E.g. this example:

fn main() {
    dbg!(serde_json::from_str::<std::collections::HashMap<i32, i32>>(r#"{"abc":0}"#));
}

Previously would print invalid type: string "abc", expected i32 at line 1 column 6, but after this changeset it prints expected value at line 1 column 3.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I was thinking of doing to give a more meaningful error: #1035.

src/value/ser.rs Outdated Show resolved Hide resolved
This ensures consistency with the default `Formatter::write_f32` output
format used elsewhere.
`deserialize_numeric_key!` has been renamed back to
`deserialize_integer_key!`, and a new float-specific
`deserialize_float_key!` macro has been introduced.

This new macro attempts to deserialize float keys by delegating to the
equivalent parent deserializer method.
src/value/de.rs Show resolved Hide resolved
An `unwrap` has been changed to simply discard the `Result`. We assume
that writing to a `Vec` will never fail because the `std::io::Write`
implementation never creates an I/O error.
Copy link
Member

@dtolnay dtolnay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The serialization looks good!

I left one more comment up in #1027 (review) about the deserialization.

Copy link
Member

@dtolnay dtolnay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@dtolnay dtolnay merged commit b8d8d10 into serde-rs:master Jul 11, 2023
13 checks passed
crapStone added a commit to Calciumdibromid/CaBr2 that referenced this pull request Jul 14, 2023
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [serde_json](https://github.com/serde-rs/json) | dependencies | patch | `1.0.100` -> `1.0.102` |

---

### Release Notes

<details>
<summary>serde-rs/json (serde_json)</summary>

### [`v1.0.102`](https://github.com/serde-rs/json/releases/tag/v1.0.102)

[Compare Source](serde-rs/json@v1.0.101...v1.0.102)

-   Add a way to customize the serialization of byte arrays ([#&#8203;1039](serde-rs/json#1039))

### [`v1.0.101`](https://github.com/serde-rs/json/releases/tag/v1.0.101)

[Compare Source](serde-rs/json@v1.0.100...v1.0.101)

-   Allow f32 and f64 as keys in maps ([#&#8203;1027](serde-rs/json#1027), thanks [@&#8203;overdrivenpotato](https://github.com/overdrivenpotato))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi44LjIiLCJ1cGRhdGVkSW5WZXIiOiIzNi44LjIiLCJ0YXJnZXRCcmFuY2giOiJkZXZlbG9wIn0=-->

Co-authored-by: cabr2-bot <cabr2.help@gmail.com>
Co-authored-by: crapStone <crapstone01@gmail.com>
Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1960
Reviewed-by: crapStone <crapstone01@gmail.com>
Co-authored-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>
Co-committed-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants