Update module attribute on namedtuple methods for introspection.

[a-feld]

BPO	33380
Nosy	@rhettinger, @pmp-p, @serhiy-storchaka, @a-feld
Files	0001-bpo-33380-Update-module-attribute-on-namedtuple-meth.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/rhettinger'
closed_at = <Date 2018-05-01.02:23:55.528>
created_at = <Date 2018-04-28.22:40:12.862>
labels = ['3.7', '3.8', 'type-feature', 'library']
title = 'Update module attribute on namedtuple methods for introspection.'
updated_at = <Date 2018-05-02.01:43:41.912>
user = 'https://github.com/a-feld'

bugs.python.org fields:

activity = <Date 2018-05-02.01:43:41.912>
actor = 'pmpp'
assignee = 'rhettinger'
closed = True
closed_date = <Date 2018-05-01.02:23:55.528>
closer = 'a-feld'
components = ['Library (Lib)']
creation = <Date 2018-04-28.22:40:12.862>
creator = 'a-feld'
dependencies = []
files = ['47555']
hgrepos = []
issue_num = 33380
keywords = ['patch']
message_count = 7.0
messages = ['315869', '315870', '315926', '315927', '315977', '315978', '316033']
nosy_count = 4.0
nosy_names = ['rhettinger', 'pmpp', 'serhiy.storchaka', 'a-feld']
pr_nums = []
priority = 'normal'
resolution = 'works for me'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue33380'
versions = ['Python 3.7', 'Python 3.8']

[a-feld]

Python 3.7 made several performance improvements to the namedtuple class as part of https://bugs.python.org/issue28638

Prior to the implementation of bpo-28638, the __module__ attribute for a namedtuple's methods (e.g. _asdict) would return the value 'namedtuple_%s' % typename (e.g. namedtuple_Point).

Due to the optimizations made, the __module__ attribute for a namedtuple's methods now returns 'collections'.

The proposed change as part of this issue is to report the more accurate derived module name for the namedtuple methods. Updating the __module__ attribute should help debug and introspection tools more accurately report the details of executing calls (in profilers for example).

Example from Python 3.6:
>>> from collections import namedtuple
>>> Point = namedtuple('Point', ('x', 'y'))
>>> p1 = Point(1,2)
>>> p1._asdict.__module__
'namedtuple_Point'

Example from Python 3.7.0b3:
>>> from collections import namedtuple
>>> Point = namedtuple('Point', ('x', 'y'))
>>> p1 = Point(1,2)
>>> p1._asdict.__module__
'collections'

Desired behavior:
>>> from collections import namedtuple
>>> Point = namedtuple('Point', ('x', 'y'))
>>> p1 = Point(1,2)
>>> p1._asdict.__module__
'__main__'

[a-feld]

Attached is a proposed change.

[rhettinger]

Am not sure I can see any value in pushing the __module__ attribute down into the methods and think it is reasonable for them to report their origin as being in the collections module.

[pmp-p]

I see that as a good fix, obviously Point definition belongs to __main__ in the sample, like would any other subclass defined there eg if "some" class is defined in awe.py module :

>>> import awe
>>> class my(awe.some):pass
... 
>>> my.__module__
'__main__'

[rhettinger]

ISTM collections is the correct module reference because that is where the code is actually defined. A debugging tool should look there instead of in the calling code. This is the way other tools work in Python as well:

    >>> from collections.abc import Set
    >>> class S(Set):
            def __init__(self): pass
            def __iter__(self): yield 0
            def __len__(self): return 0
            def __contains__(self, key): return False
            
    >>> S.__or__.__module__
    'collections.abc'

    >>> from dataclasses import make_dataclass
    >>> P = make_dataclass('P', ['x', 'y'])
    >>> P.__repr__.__module__
    'dataclasses'

Likewise, the various tools that use closures used to report the module where the closure was defined rather than the module of the caller's code (see functools.cmp_to_key or functools.lru_cache). I say "used to" because the pure Python code is now supplanted by C equivalents where the __module__ attribute__ is appropriately set to None:

    # Running from PyPy
    >>>> from functools import cmp_to_key
    >>>> c = cmp_to_key(lambda x, y: 0)
    >>>> c.__init__.__module__
    '_functools'

The Data Model section of the Language Reference defines __module__ as follows, "__module__: The name of the module the function was defined in, or None if unavailable." In the prior version of namedtuple(), the function was defined in a virtual module (an execed string). Now, that the methods are defined in a real module, the __module__ attribute should name that real module.

[a-feld]

That explanation makes sense to me. Thanks for taking the time to look into this!

[pmp-p]

Indeed thanks for the deep explanation. It seems that not finding im_self anymore (not even in the dunder clutter), i've mistaken '.__self__.__module__' with '.__module__'. How joyfull to learn anew to trace a caller id, and sorry for the noise (again).