Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Floating Point Precision for Protobufs isn't intuitive #390

Open
holtskinner opened this issue Sep 6, 2023 · 0 comments
Open

Floating Point Precision for Protobufs isn't intuitive #390

holtskinner opened this issue Sep 6, 2023 · 0 comments
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@holtskinner
Copy link
Member

holtskinner commented Sep 6, 2023

Floating Point precision is based on the number of total digits, not digits after the decimal point, as most other libraries use.

        """
            float_precision (Optional(int)): If set, use this to specify float field valid digits.
                Default is None.
        Returns:
            str: The json string representation of the protocol buffer.
        """
        return MessageToJson(
            cls.pb(instance),
            use_integers_for_enums=use_integers_for_enums,
            including_default_value_fields=including_default_value_fields,
            preserving_proto_field_name=preserving_proto_field_name,
            sort_keys=sort_keys,
            indent=indent,
            float_precision=float_precision,
        )

Example

    class Squid(proto.Message):
        mass_kg = proto.Field(proto.FLOAT, number=1)

    s = Squid(mass_kg=3.14159265)

    s_dict = Squid.to_dict(s, float_precision=3)
    assert s_dict["mass_kg"] == 3.14

I would expect float_precision=3 to set the mass_kg to 3.141 instead of 3.14 (3 decimal places instead of 2)

@holtskinner holtskinner added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Sep 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

1 participant