C implementation of Decimal.from_float() bypasses __new__ and __init__

[serhiy-storchaka]

BPO	27006
Nosy	@rhettinger, @facundobatista, @mdickinson, @skrah, @serhiy-storchaka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-06-20.12:58:32.226>
created_at = <Date 2016-05-12.07:15:44.525>
labels = ['extension-modules', 'type-bug']
title = 'C implementation of Decimal.from_float() bypasses __new__ and __init__'
updated_at = <Date 2016-06-20.12:58:32.225>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2016-06-20.12:58:32.225>
actor = 'skrah'
assignee = 'none'
closed = True
closed_date = <Date 2016-06-20.12:58:32.226>
closer = 'skrah'
components = ['Extension Modules']
creation = <Date 2016-05-12.07:15:44.525>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 27006
keywords = []
message_count = 6.0
messages = ['265368', '268889', '268890', '268892', '268896', '268898']
nosy_count = 6.0
nosy_names = ['rhettinger', 'facundobatista', 'mark.dickinson', 'skrah', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue27006'
versions = ['Python 3.5', 'Python 3.6']

[serhiy-storchaka]

Python implementation of Decimal.from_float() calls __new__ and __init__ methods of subclass.

>>> from _pydecimal import Decimal
>>> class D(Decimal):
...     def __new__(cls, *args, **kwargs):
...         print('__new__')
...         return Decimal.__new__(cls, *args, **kwargs)
...     def __init__(self, *args, **kwargs):
...         print('__init__')
... 
>>> print(type(D.from_float(42)))
__new__
__init__
<class '__main__.D'>
>>> print(type(D.from_float(42.0)))
__new__
__init__
<class '__main__.D'>

But C implementation doesn't.

>>> from decimal import Decimal
>>> class D(Decimal):
...     def __new__(cls, *args, **kwargs):
...         print('__new__')
...         return Decimal.__new__(cls, *args, **kwargs)
...     def __init__(self, *args, **kwargs):
...         print('__init__')
... 
>>> print(type(D.from_float(42)))
<class '__main__.D'>
>>> print(type(D.from_float(42.0)))
<class '__main__.D'>

This means that resulting instance of Decimal subclass can be in not valid state.

Example is Decimal enums (see also bpo-23640).

>>> from decimal import Decimal
>>> from enum import Enum
>>> class D(Decimal, Enum):
...     A = Decimal('3.25')
... 
>>> D(Decimal(3.25))
<D.A: Decimal('3.25')>
>>> D(3.25)
<D.A: Decimal('3.25')>
>>> D.from_float(3.25)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/enum.py", line 486, in __repr__
    self.__class__.__name__, self._name_, self._value_)
AttributeError: 'D' object has no attribute '_name_'

A solution is to reproduce Python implementation in C code:

    result = ... # create exact Decimal
    if cls is not Decimal:
        result = cls(result)
    return result

[python-dev]

New changeset dd3f48f1df86 by Stefan Krah in branch '3.5':
Issue bpo-27006: from_float(): call the subclass' __new__() and __init__().
https://hg.python.org/cpython/rev/dd3f48f1df86

[skrah]

Thank you for the detailed summary!

[serhiy-storchaka]

PyDec_CheckExact(type) always return 0. Should be PyDec_CheckExact(result).

And what about other calls of PyDecType_FromFloatExact()? Can they produce broken instance of Decimal subtype?

[python-dev]

New changeset 51a7a97c3ed4 by Stefan Krah in branch '3.5':
Issue bpo-27006: Do not use PyDec_CheckExact() on a type.
https://hg.python.org/cpython/rev/51a7a97c3ed4

[skrah]

PyDec_CheckExact(type) always return 0. Should be PyDec_CheckExact(result).

'result' is always an exact decimal, because your enum example won't work otherwise.

And what about other calls of PyDecType_FromFloatExact()? Can they produce broken instance of Decimal subtype?

The PyDecType* functions handle calling the correct allocator, they don't handle calling __new__(). Why should they? The regular non-class-method enum examples worked.