Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect Int Conversion #70

Closed
sanjit-bhat opened this issue Sep 6, 2020 · 1 comment
Closed

Incorrect Int Conversion #70

sanjit-bhat opened this issue Sep 6, 2020 · 1 comment

Comments

@sanjit-bhat
Copy link

sanjit-bhat commented Sep 6, 2020

Overview

I think I found a bug in texttable's integer conversion logic. Specifically, the intermediate conversion to a floating-point produces an incorrect output for large integers due to floating-point imprecision.

Minimum example:

from texttable import Texttable
tt = Texttable()
tt.set_cols_dtype(['i'])  # dtype 'i' is the problem here
tt.add_rows([['hello'], [18014398509481983]])
print(tt.draw())

Expected output:

+-------------------+
|       hello       |
+===================+
| 18014398509481983 |
+-------------------+

Actual output:

+-------------------+
|       hello       |
+===================+
| 18014398509481984 |
+-------------------+

Explanation of bug

This problem is with texttable's integer (see dtype(['i']) above) conversion code in texttable.py:

@classmethod
def _fmt_int(cls, x, **kw):
    """Integer formatting class-method.
    - x will be float-converted and then used.
    """
    return str(int(round(cls._to_float(x))))

The cls._to_float(x) converts the input to a float, which produces the wrong value for large ints. E.g., float(18014398509481983) = 18014398509481984.

Possible solution idea:

Is the float conversion really necessary? If not, str(int(round(18014398509481983))) = 18014398509481983 produces the right answer.

Environment configuration:

Texttable version: 1.6.2
Python version: 3.8.1

@foutaise
Copy link
Owner

foutaise commented Sep 6, 2020

Hi @sanjit-bhat

Thanks for your report, should be fixed in release 1.6.3.
(fix in commit 80ed5bd)

Best regards.

@foutaise foutaise closed this as completed Sep 6, 2020
buildroot-auto-update pushed a commit to buildroot/buildroot that referenced this issue Sep 26, 2020
Bugfix release, fixing an issue with integer/float handling:
foutaise/texttable#70

Adjust the .hash spacing and update the license hash for a copyright year
change:
foutaise/texttable@13ff0b5

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
buildroot-auto-update pushed a commit to buildroot/buildroot that referenced this issue Oct 2, 2020
Bugfix release, fixing an issue with integer/float handling:
foutaise/texttable#70

Adjust the .hash spacing and update the license hash for a copyright year
change:
foutaise/texttable@13ff0b5

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit dc68be6)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
buildroot-auto-update pushed a commit to buildroot/buildroot that referenced this issue Oct 2, 2020
Bugfix release, fixing an issue with integer/float handling:
foutaise/texttable#70

Adjust the .hash spacing and update the license hash for a copyright year
change:
foutaise/texttable@13ff0b5

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit dc68be6)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
buildroot-auto-update pushed a commit to buildroot/buildroot that referenced this issue Oct 2, 2020
Bugfix release, fixing an issue with integer/float handling:
foutaise/texttable#70

Adjust the .hash spacing and update the license hash for a copyright year
change:
foutaise/texttable@13ff0b5

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit dc68be6)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants