Casting float to int gives wrong number

[georgboe]

CircuitPython version

Adafruit CircuitPython 7.1.0-rc.1 on 2021-12-25; Raspberry Pi Pico with rp2040

Code/REPL

>>> 23.40 * 100
2340.0
>>> int(23.40 * 100)
2339

Behavior

The floating point version is 2340.0 and when cast to int becomes 2339. This works fine in regular python:

Python 3.9.7 (default, Sep 10 2021, 14:59:43) 
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 23.40 * 100
2340.0
>>> int(23.40 * 100)
2340

Description

Without knowing the implementation details, the problem reminds me of issues I've seen related to IEEE 754 elsewhere.

Additional information

No response

[dhalbert]

2340.0 cannot be represented exactly with the bits we have. See https://www.h-schmidt.net/FloatConverter/IEEE754.html

In general one should round, not truncate.

Note that the float representation we use drops the two lowest bits as well: it's a 30-bit float with two bits less of mantissa. There is more discussion of this kind of thing in the MicroPython issues, for instance: micropython#4212.

[georgboe]

Ok, thanks for explaining Dan 👍
I will close the issue.