Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support round-tripping 64-bit ints #64

Closed
codeinred opened this issue Dec 1, 2022 · 3 comments
Closed

Support round-tripping 64-bit ints #64

codeinred opened this issue Dec 1, 2022 · 3 comments
Labels
bug Something isn't working

Comments

@codeinred
Copy link

Description

Support round-tripping 64-bit ints. Consider the following struct:

struct Value {
    long long value;
};
template <>
struct glz::meta<Value> {
    using T = Value;
    constexpr static auto value = object(
        "value", &Value::value
    );
};

My general expectation is that, provided v doesn't contain any floating point numbers, the following holds:

glz::read_json<Value>(glz::write_json(v)) == v

However, it seems as though numbers are always parsed as floats, even if the corresponding field expects an int.

Consider the following example:

int main() {
    Value v1 { 32948729483739289 };

    std::string json = glz::write_json(v1);

    Value v2 = glz::read_json<Value>(json);

    printf("Json: %s\n", glz::prettify(json).c_str());

    if (v1.value != v2.value) {
        printf("Values not equal. v1 = %lli, v2 = %lli\n", v1.value, v2.value);
        return 1;
    } else {
        puts("All good!");
        return 0;
    }
}

This value is serialized properly (all digits intact), but when it's parsed, it's parsed as a float, then converted to an int.

This is the output of the above:

Json: {
   "value": 32948729483739289
}
Values not equal. v1 = 32948729483739289, v2 = 32948729483739288

Would it be possible to parse numbers as integers if the corresponding field calls for it?

@codeinred
Copy link
Author

Addendum: I just want to mention that this is particularly relevant in cases where nanosecond timestamps or integral numeric IDs are being stored in json. Nanosecond timestamps allow a bit of fuzzyness (if the timestamp is off by a microsecond, that's often not the end of the world), but 64-bit integral IDs need to match exactly

@mwalcott3
Copy link
Collaborator

mwalcott3 commented Dec 1, 2022

Yeah number parsing is going to change soonish (likely sometime next week). This will get fixed. Right now I would expect values between -2^53 to 2^53 would work properly since those should all be exactly representable. Values were originally parsed in as floating point numbers because we wanted to allow stuff like 1.7e10 which is handled by default in float parsers but typically not int parsers. We could quickfix this by parsing them as the proper type but it would break scientific notation and we are changing number parsing anyways.

I'm chilling in barbados right now but I'll fix this when I get back provided someone else doesn't get to it first. This is quite a big issue since as you pointed out 64bit hashes are broken when represent ed as ints as it's common for them to exceed the bounds of exactly representable ints in 64bit floating point numbers.

@mwalcott3 mwalcott3 added the bug Something isn't working label Dec 1, 2022
@mwalcott3
Copy link
Collaborator

Fixed in 6e7cab3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants