Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parse_json - differs parsing floats #755

Open
guusvw opened this issue Mar 20, 2024 · 2 comments
Open

parse_json - differs parsing floats #755

guusvw opened this issue Mar 20, 2024 · 2 comments
Labels
vrl: stdlib Changes to the standard library

Comments

@guusvw
Copy link

guusvw commented Mar 20, 2024

Hey dear VRL-folks 👋
I hope this is not a double posting, I only found issues regards parse_json for large ints, I'm having an issues as well for floats.

We're getting a JSON which contains the field ts and has in our test case the value 1626175065.5934923
after passing it to parse_json the object has the value of 1626175065.5934925, the last digit differs by 2.
I know, floats are hard & annoying, but is there some way to avoid/fix it?

$ message = "{\"level\":\"info\",\"ts\":1626175065.5934923,\"msg\":\"|200|   32.603µs /healthcheck\",\"statusCode\":200,\"request\":306,\"latency\":0.000032603,\"avgLatency\":0.004298489,\"ipPort\":\"1.1.1.1:12345\",\"method\":\"GET\",\"host\":\"1.1.1.1:8080\",\"path\":\"/healthcheck\",\"referer\":\"\"}"
"{\"level\":\"info\",\"ts\":1626175065.5934923,\"msg\":\"|200|   32.603µs /healthcheck\",\"statusCode\":200,\"request\":306,\"latency\":0.000032603,\"avgLatency\":0.004298489,\"ipPort\":\"1.1.1.1:12345\",\"method\":\"GET\",\"host\":\"1.1.1.1:8080\",\"path\":\"/healthcheck\",\"referer\":\"\"}"

$ obj, err = parse_json(message)

{ "avgLatency": 0.004298489, "host": "1.1.1.1:8080", "ipPort": "1.1.1.1:12345", "latency": 0.000032603, "level": "info", "method": "GET", "msg": "|200|   32.603µs /healthcheck", "path": "/healthcheck", "referer": "", "request": 306, "statusCode": 200, "ts": 1626175065.5934925 }
❯ bin/vector-0.36.1 --version
vector 0.36.1 (x86_64-apple-darwin 2857180 2024-03-11 14:32:52.417737479)
@guusvw guusvw changed the title parse_json parse_json - differs parsing floats Mar 20, 2024
@jszwedko
Copy link
Member

Hmm, interesting. I know floats have issues with representing certain numbers, but this doesn't appear to be the case for the specific example given: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=3d538d9f2ae192da51693bc12fdcfbfa

I believe we just defer to serde_json for deserialization, though, so I'm not sure why it is not preserving the precision.

@jszwedko jszwedko added the vrl: stdlib Changes to the standard library label Mar 20, 2024
@drmason13
Copy link
Contributor

I jumped on this after seeing that it is in fact solvable through a feature in serde_json.
It's a non-default feature, and has been for almost 4 years now. I'm sure we won't be the only ones caught by this :)

github-merge-queue bot pushed a commit that referenced this issue Apr 1, 2024
* fix(stdlib): parse_json improved accuracy float parsing #755

* move roundtrip float to optional dependency
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
vrl: stdlib Changes to the standard library
Projects
None yet
Development

No branches or pull requests

3 participants