-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incorrect Int Conversion #70
Comments
Hi @sanjit-bhat Thanks for your report, should be fixed in release 1.6.3. Best regards. |
buildroot-auto-update
pushed a commit
to buildroot/buildroot
that referenced
this issue
Sep 26, 2020
Bugfix release, fixing an issue with integer/float handling: foutaise/texttable#70 Adjust the .hash spacing and update the license hash for a copyright year change: foutaise/texttable@13ff0b5 Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
buildroot-auto-update
pushed a commit
to buildroot/buildroot
that referenced
this issue
Oct 2, 2020
Bugfix release, fixing an issue with integer/float handling: foutaise/texttable#70 Adjust the .hash spacing and update the license hash for a copyright year change: foutaise/texttable@13ff0b5 Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> (cherry picked from commit dc68be6) Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
buildroot-auto-update
pushed a commit
to buildroot/buildroot
that referenced
this issue
Oct 2, 2020
Bugfix release, fixing an issue with integer/float handling: foutaise/texttable#70 Adjust the .hash spacing and update the license hash for a copyright year change: foutaise/texttable@13ff0b5 Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> (cherry picked from commit dc68be6) Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
buildroot-auto-update
pushed a commit
to buildroot/buildroot
that referenced
this issue
Oct 2, 2020
Bugfix release, fixing an issue with integer/float handling: foutaise/texttable#70 Adjust the .hash spacing and update the license hash for a copyright year change: foutaise/texttable@13ff0b5 Signed-off-by: Peter Korsgaard <peter@korsgaard.com> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> (cherry picked from commit dc68be6) Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Overview
I think I found a bug in
texttable
's integer conversion logic. Specifically, the intermediate conversion to a floating-point produces an incorrect output for large integers due to floating-point imprecision.Minimum example:
Expected output:
Actual output:
Explanation of bug
This problem is with
texttable
's integer (seedtype(['i'])
above) conversion code intexttable.py
:The
cls._to_float(x)
converts the input to a float, which produces the wrong value for large ints. E.g.,float(18014398509481983) = 18014398509481984
.Possible solution idea:
Is the float conversion really necessary? If not,
str(int(round(18014398509481983))) = 18014398509481983
produces the right answer.Environment configuration:
Texttable
version: 1.6.2Python
version: 3.8.1The text was updated successfully, but these errors were encountered: