Decimal to int conversion with engineering notation costly despite failing

[roharvey]

Bug report

Bug description:

Adding many digits to engineering notation causes cpython to work very hard trying to do the conversion, only for it to fail anyway.

>>> x = decimal.Decimal('1.0E+10000')
>>> int(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit

But the time it takes goes up when you add extra digits. It appears to be unbounded.

>>> timeit.timeit(setup="import decimal", stmt="x = decimal.Decimal('1.0E+10000'); i = int(x)", number=10)
0.027457250020233914
>>> timeit.timeit(setup="import decimal", stmt="x = decimal.Decimal('1.0E+100000'); i = int(x)", number=10)
1.8902547920006327

The decimal to int conversion should shortcut to a failure when there is no possibility of generating a real number, e.g. by examining the length of the substring after + or - (in conjunction with sys.set_int_max_str_digits or maybe some other setting).

CPython versions tested on:

3.11, 3.12

Operating systems tested on:

macOS

[picnixz]

cc @serhiy-storchaka @tim-one I'm not sure we have a specialization in the "parser" but do you think it could make sense?

[mdickinson]

Note that the conversion from Decimal succeeds. It's the attempt to print the resulting int at the prompt that fails.

Python 3.13.7 (main, Aug 15 2025, 23:01:14) [Clang 17.0.0 (clang-1700.0.13.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import Decimal
>>> x = Decimal('1e10000')
>>> i = int(x)  # succeeds
>>> i  # fails
Traceback (most recent call last):
  File "<python-input-3>", line 1, in <module>
    i
ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit

So this doesn't really have anything to do with Decimal. It's no different from trying to print 10**10000 directly. Or Fraction("1e10000"), for that matter.

>>> from fractions import Fraction
>>> f = Fraction('1e10000')
>>> i = int(f)  # succeeds
>>> i  # fails
Traceback (most recent call last):
  File "<python-input-7>", line 1, in <module>
    i  # fails
    ^
ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit

[picnixz]

Right. I always forget that we have an implicit repr(x) in the REPL. So, probably nothing to do here?

[tim-one]

Although note that it's still not all unicorns and rainbows 😉.

>>> x = Decimal('1e1000000') # add more zeroes
>>> i = int(x) # succeeds, but takes "a long time"

This is a consequence of that Decimal<->int conversions are still quadratic time. A report on that was open for years, but not long ago closed as "not planned". It would be best if (the external) libmpdec did the heavy lifting to use a sub-quadratic approach, but short of that nobody has been both willing and able to put "the smarts" into CPython's wrapping of such conversions.

[picnixz]

I'm going to close this one as a duplicate of #90716. As Gregory said, we first need a good plan if we want to work on this.

[serhiy-storchaka]

This returns almost instantly on my computer.

$ ./python -m timeit -n1 -r1 -s "from decimal import Decimal; x = Decimal('1e1_000_000')" "int(x)"
1 loop, best of 1: 262 msec per loop

Adding more zeroes can making it taking "a long time".

$ ./python -m timeit -n1 -r1 -s "from decimal import Decimal; x = Decimal('1e10_000_000')" "int(x)"
1 loop, best of 1: 10.4 sec per loop

String to integer conversion takes comparable time

$ ./python -X int_max_str_digits=0 -m timeit -n1 -r1 -s "from decimal import Decimal; x = '1'+'0'*10_000_000" "int(x)"
1 loop, best of 1: 13.6 sec per loop

so I do not think that we can optimize Decimal to integer conversion by utilizing Decimal to string to integer conversion.

[tim-one]

@serhiy-storchaka, believe it or not, 10 million digits is still on the small side(!) for O() behavior of str->int to become apparent. Increase it to 100 million digits, and you should find that it takes a lot closer to 10 times (sub-quadratic) longer than 100 times .(quadratic). On my box, about 12 times longer.

$ py -X int_max_str_digits=0 -m timeit -n1 -r1 -s "x = '1'+'0'*10_000_000" "int(x)"
1 loop, best of 1: 10.1 sec per loop

$ py -X int_max_str_digits=0 -m timeit -n1 -r1 -s "x = '1'+'0'*100_000_000" "int(x)"
1 loop, best of 1: 119 sec per loop

Ironically enough, CPython's str->int uses decimal internally to exploit its advanced (quasi-linear) multiplication.