Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect results for Float80 exact powers of two #175

Closed
tbkka opened this issue Jul 9, 2020 · 1 comment
Closed

Incorrect results for Float80 exact powers of two #175

tbkka opened this issue Jul 9, 2020 · 1 comment

Comments

@tbkka
Copy link

tbkka commented Jul 9, 2020

I've been comparing ryu_generic128 output against other implementations and believe it generates incorrect output for certain exact powers of two. One example:

  • Input float80: 0x1p-16367
  • Expected output: 1.10169395793497080013e-4927
  • Ryu_generic128 result: 1.1016939579349708001E-4927

The "expected output" here agrees with both gdtoa and Swift's standard library, so I'm reasonably certain it's correct. Similar discrepancies occur for about 4000 other exact powers of two.

This issue seems to be limited to float80. I've not seen any similar failures for float128, double, float, or float16.

ulfjack added a commit that referenced this issue Jul 10, 2020
The code compared the computed ieeeMantissa (which includes the explicit
leading one in the long double case) against 0, which is incorrect.

Fixes #175.
@ulfjack
Copy link
Owner

ulfjack commented Jul 10, 2020

Thanks for the report. The reported powers of two are all cases where the next lower floating-point number is closer than the next higher floating-point number. The code detects that case by comparing the mantissa against 0, which is correct only if the format has an implicit leading one, but incorrect for an explicit leading one, as is the case for the Float80 format. I have a fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants