Objects in NamedTuple class namespaces don't have __set_name__ called on them

[AlexWaygood]

Bug report

Bug description:

Generally speaking, any objects present in the namespace of a class Foo that define the special __set_name__ method will have that method called on them as part of the creation of the class Foo. (__set_name__ is generally only used for descriptors, but can be defined on any class.)

For example:

Python 3.12.0 (tags/v3.12.0:0fb18b0, Oct  2 2023, 13:03:39) [MSC v.1935 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> class Annoying:
...     def __set_name__(self, owner, name):
...         raise Exception('no')
...
>>> class Foo:
...     attr = Annoying()
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __set_name__
Exception: no
Error calling __set_name__ on 'Annoying' instance 'attr' in 'Foo'

Descriptors inside typing.NamedTuple namespaces generally work the same way as in other class namespaces...

>>> from typing import NamedTuple
>>> class Foo(NamedTuple):
...     bar = property(lambda self: 42)
...
>>> Foo().bar
42

...but with one notable exception: they don't have __set_name__ called on them!

>>> class Annoying:
...     def __set_name__(self, owner, name):
...         raise Exception('no')
...
>>> from typing import NamedTuple
>>> class Foo(NamedTuple):
...     bar = Annoying()  # this should cause the creation of the `Foo` class to fail...
...
>>> # ...but it didn't!

Why does this happen?

__set_name__ would normally be called on all members of a class dictionary during the class's creation. But the NamedTuple class Foo is created here:

cpython/Lib/typing.py

Lines 2721 to 2723 in 97c4c06

	nm_tpl = _make_nmtuple(typename, types.items(),
	defaults=[ns[n] for n in default_names],
	module=ns['__module__'])

And the bar attribute is only monkey-patched onto the Foo class after the class has actually been created. This happens a few lines lower down in typing.py, here:

cpython/Lib/typing.py

Lines 2732 to 2733 in 97c4c06

	elif key not in _special and key not in nm_tpl._fields:
	setattr(nm_tpl, key, ns[key])

__set_name__ isn't called on Foo.bar as part of the creation of the Foo class, because the Foo class doesn't have a bar attribute at the point in time when it's actually being created.

CPython versions tested on:

3.8, 3.11, 3.12, CPython main branch

Operating systems tested on:

Windows

Linked PRs

• gh-111874: Call __set_name__ on objects that define the method inside a typing.NamedTuple class dictionary as part of the creation of that class #111876

[rhettinger]

Your analysis is thorough and easy to follow. Nice work.

[serhiy-storchaka]

Is it a bugfix or a new feature? Should it be backported? Should __set_name__ be called for annotated fields (on values that became default values)?

[AlexWaygood]

Is it a bugfix or a new feature? Should it be backported?

In my opinion, it's a bugfix, though I'd be okay with not backporting the fix if we consider it to be a risky fix. The fact that this hasn't come up until now demonstrates that not many people are putting custom descriptors in the namespaces of typing.NamedTuple classes.

I initially encountered the bug when I tried to add a functools.cached_property to a typing.NamedTuple class (forgetting that functools.cached_property doesn't work with slotted classes). Trying to do this with main gives you this, pretty surprising, error message:

Python 3.13.0a1+ (heads/main:e5dfcc2b6e, Nov 15 2023, 17:23:51) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import typing, functools
>>> class Foo(typing.NamedTuple):
...     @functools.cached_property
...     def bar(self): return True
...
>>> f = Foo()
>>> f.bar
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    f.bar
  File "C:\Users\alexw\coding\cpython\Lib\functools.py", line 1003, in __get__
    raise TypeError(
TypeError: Cannot use cached_property instance without calling __set_name__ on it.

__set_name__ is a documented part of the Python data model; there could be lots of other descriptors where __set_name__ being called is crucial to ensure that the descriptor works as intended; we can't necessarily count on them all being as careful as functools.cached_property in accounting for the possibility that __set_name__ might not have been called. If __set_name__ is not called on objects in the class namespace, it could violate assumptions in many places.

[AlexWaygood]

Should __set_name__ be called for annotated fields (on values that became default values)?

I don't know; I'm interested in others' opinions here. For comparison's sake, it is the case that __set_name__ is called on annotated fields for dataclasses:

Python 3.13.0a1+ (heads/main:e5dfcc2b6e, Nov 15 2023, 17:23:51) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from dataclasses import dataclass
>>> class Vanilla:
...     def __set_name__(self, owner, name):
...         self.name = name
...
>>> @dataclass
... class Foo:
...     v: Vanilla = Vanilla()
...
>>> f = Foo()
>>> f.v.name
'v'

[JelleZijlstra]

Is it a bugfix or a new feature? Should it be backported?

It's arguably a bugfix, but it feels like too fundamental a change to backport it to earlier versions. I think we should change the behavior in 3.13 only.

[AlexWaygood]

Sure, I've removed the backport labels from the PR.