Add Decimal.as_integer_ratio() #70116
Comments
|
In statistics, there is a FIXME on Line 250 above _decimal_to_ratio that says: # FIXME This is faster than Fraction.from_decimal, but still too slow. Half of the time is spent in a conversion in d.as_tuple(). Decimal internally stores the digits as a string, but in d.as_tuple(), the digits are individually cast to integers and returned as a tuple of integers. This is OK, but _decimal_to_ratio undoes the work that was done in d.as_tuple() by adding them all back into an integer. A similar, but slightly different approach is taken in Fractions.from_decimal, where the tuple is cast into a string and then parsed into an integer. We can be a lot faster if we use the _int instance variable directly. In the case of _decimal_to_ratio, the new code seems to be twice as fast with usage _decimal_to_ratio(Decimal(str(random.random()))): def _decimal_to_ratio(d):
sign, exp = d._sign, d._exp
if exp in ('F', 'n', 'N'): # INF, NAN, sNAN
assert not d.is_finite()
return (d, None)
num = int(d._int)
if exp < 0:
den = 10**-exp
else:
num *= 10**exp
den = 1
if sign:
num = -num
return (num, den)If the performance improvement is considered worthwhile, here are a few solutions I see.
There are probably more solutions. I lean towards 4, because it makes usage easier and avoids cluttering Decimal with methods. Here is what I used for benchmarks: ======== import timeit
old_setup = """
import random
from decimal import Decimal
def _decimal_to_ratio(d):
sign, digits, exp = d.as_tuple()
if exp in ('F', 'n', 'N'): # INF, NAN, sNAN
assert not d.is_finite()
return (d, None)
num = 0
for digit in digits:
num = num*10 + digit
if exp < 0:
den = 10**-exp
else:
num *= 10**exp
den = 1
if sign:
num = -num
return (num, den)
def run_it():
dec = Decimal(str(random.random()))
_decimal_to_ratio(dec)
"""
new_setup = """
import random
from decimal import Decimal
def _decimal_to_ratio(d):
sign, exp = d._sign, d._exp
if exp in ('F', 'n', 'N'): # INF, NAN, sNAN
assert not d.is_finite()
return (d, None)
num = int(d._int)
if exp < 0:
den = 10**-exp
else:
num *= 10**exp
den = 1
if sign:
num = -num
return (num, den)
def run_it():
dec = Decimal(str(random.random()))
_decimal_to_ratio(dec)
"""
if __name__ == '__main__':
print("Testing proposed implementation")
print("number = 10000")
print(timeit.Timer(stmt='run_it()', setup=new_setup).timeit(number=10000))
print("number = 100000")
print(timeit.Timer(stmt='run_it()', setup=new_setup).timeit(number=100000))
print("number = 1000000")
print(timeit.Timer(stmt='run_it()', setup=new_setup).timeit(number=1000000))
print("Testing old implementation")
print("number = 10000")
print(timeit.Timer(stmt='run_it()', setup=old_setup).timeit(number=10000))
print("number = 100000")
print(timeit.Timer(stmt='run_it()', setup=old_setup).timeit(number=100000))
print("number = 1000000")
print(timeit.Timer(stmt='run_it()', setup=old_setup).timeit(number=1000000)) |
|
Heres the output of running the benchmark on my machine: Testing proposed implementation |
|
I guess there's some version mixup here: From Python 3.3 on |
|
May be implement the as_integer_ratio() method and/or numerator and denominator attributes in the Decimal class? |
|
On Wed, Dec 23, 2015 at 03:18:28PM +0000, Serhiy Storchaka wrote:
That would also be good as it will decrease the API differences between |
Stefan, _int is a slot in Lib/_pydecimal.py. It should be defined on python 3.5 and tip, unsure about other versions. Python 3.5.1 (default, Dec 7 2015, 12:58:09)
[GCC 5.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import Decimal
>>> Decimal("100.00")
Decimal('100.00')
>>> Decimal("100.00")._int
'10000' |
On Wed, Dec 23, 2015 at 09:01:22PM +0000, John Walker wrote:
> Stefan, _int is a slot in Lib/_pydecimal.py. It should be defined on python 3.5 and tip, unsure about other versions.
>
> Python 3.5.1 (default, Dec 7 2015, 12:58:09)
> [GCC 5.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from decimal import Decimal
> >>> Decimal("100.00")
> Decimal('100.00')
> >>> Decimal("100.00")._int
> '10000'That should only happen if the C version did not build for some reason: Python 3.6.0a0 (default:323c10701e5d, Dec 14 2015, 14:28:41)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import Decimal
>>> Decimal("100.00")._int
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'decimal.Decimal' object has no attribute '_int'
>>> |
Ahh, gotcha. I assume one instance where this happens is when the machine doesn't have libmpdec.so |
|
No, the regular build uses the libmpdec that is shipped with |
Oh, OK. I see whats happening. My distro deletes the shipped version and compiles --with-system-libmpdec. We're on the same page now, thanks. https://projects.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/python |
|
Let's re-target this issue: Implementing as_integer_ratio() sounds reasonable, but let's hope Mark, was there perhaps a reason not to add the method? |
skrah
mannequin
changed the title
|
A visible new feature is an enhancement (even if performance is the reason for the new feature) and can only go in 3.6. |
terryjreedy
added
type-feature
|
Previously the method was rejected in bpo-8947. But the float.as_integer_ratio: 0.548023 |
In the past, there's been a desire to keep the decimal module minimal in scope, implementing little more than what's required for the specification. A number of proposed additions have been rejected on this basis. Ah, and now I read Stefan's reference to bpo-8947. I'm not sure anything has really changed since that discussion. Count me as -0E0 on the proposal. Off-topic: I wonder whether |
|
Keeping the API small is a good reason against it. There aren't many ways to destructure a Decimal though: As far Probably many people would be happy if we added as_integer_ratio()
If users want to duck-type, I think yes. |
|
Arguments against as_integer_ratio() look weighty. But may be there are less arguments against numerator and denominator? |
|
Now that there is more than one use case for Decimal.as_integer_ratio(), I'll add my support to this feature request. |
|
Here's a patch. The Python and doc parts are from Mark (bpo-8947). I did not optimize the normalization yet, in C the code is |
|
New changeset f3b09c269af0 by Stefan Krah in branch 'default': |
|
Hopefully I wasn't moving too fast, but I won't have time in the next Serhiy and Alexander (bpo-8947) would prefer more homogeneous error I wouldn't mind a patch in another issue (no argument clinic!), Thanks for the review! |
No, the patch is pretty clear. Thanks.
This would help to optimize creating a Fraction. Opened bpo-25971 for this. |
|
New changeset 510ff609cb4f by Stefan Krah in branch 'default': |
|
Steven, could you have a look at the failures in test_statistics? |
|
Ah yes, the test_statistics failures look like bpo-18841 again. |
|
I've opened bpo-25974 for the statistics.py issues. |
|
I don't think a new public method should have been added. Historically, we've been careful to not grow the API beyond what is in the spec or the dunder methods required to interface with standard Python. The feature creep is at odds with the intended goals for the module that have been present since the outset. As long as the spec remains unchanged, the API for this module should be treated as stable. Another issue is that the API for the module is already so large that it impairs usability. Please don't make it worse by adding new methods and inventing details that aren't in the spec. |
|
Raymond, you added your support in msg257097. I'm not very happy to spend my time implementing the feature and then rehashing everything after 3 months. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: