Specialized generic class does not return class attributes in dir

[kjamieson]

BPO	45755
Nosy	@gvanrossum, @miss-islington, @tirkarthi, @Fidget-Spinner
PRs	bpo-45755: [typing] Reveal class attributes of generic in generic aliases in dir() #29962 [3.10] bpo-45755: [typing] Reveal class attributes of generic in generic aliases in dir() (GH-29962) #30166

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-12-16.14:35:18.244>
created_at = <Date 2021-11-08.19:30:49.749>
labels = ['type-bug', '3.8', '3.9', '3.10', '3.11', '3.7', 'library']
title = 'Specialized generic class does not return class attributes in dir'
updated_at = <Date 2021-12-17.11:33:23.239>
user = 'https://bugs.python.org/kjamieson'

bugs.python.org fields:

activity = <Date 2021-12-17.11:33:23.239>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2021-12-16.14:35:18.244>
closer = 'kj'
components = ['Library (Lib)']
creation = <Date 2021-11-08.19:30:49.749>
creator = 'kjamieson'
dependencies = []
files = []
hgrepos = []
issue_num = 45755
keywords = ['patch']
message_count = 8.0
messages = ['405981', '406267', '406280', '407838', '408705', '408707', '408729', '408774']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'miss-islington', 'xtreak', 'kj', 'kjamieson']
pr_nums = ['29962', '30166']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45755'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10', 'Python 3.11']

[kjamieson]

This worked in Python 3.6, but in Python 3.7 and later creating a mock with a spec specifying a subscripted generic class does not mock any of the attributes of the class, because those attributes are not returned by dir().

For example:

# cat test.py
from typing import Generic, TypeVar
from unittest import mock

T = TypeVar('T')

class Foo(Generic[T]):
    def bar(self) -> None:
        pass

m = mock.MagicMock(spec=Foo[int])
m.bar()

# python3.11 test.py
Traceback (most recent call last):
  File "/root/test.py", line 11, in <module>
    m.bar()
    ^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 635, in __getattr__
    raise AttributeError("Mock object has no attribute %r" % name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: Mock object has no attribute 'bar'

[tirkarthi]

This seems to be an issue with typing than mock since mock just uses the output from dir() . I am not able to bisect the relevant change but below is the output of dir(Foo[int]) in Python 3.6 and master.

Python 3.6.9

['__abstractmethods__', '__args__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__extra__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__next_in_mro__', '__orig_bases__', '__origin__', '__parameters__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '__tree_hash__', '__weakref__', '_abc_cache', '_abc_generic_negative_cache', '_abc_generic_negative_cache_version', '_abc_registry', '_gorg', 'bar']

master branch :

['__args__', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__instancecheck__', '__le__', '__lt__', '__module__', '__mro_entries__', '__ne__', '__new__', '__or__', '__origin__', '__parameters__', '__reduce__', '__reduce_ex__', '__repr__', '__ror__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasscheck__', '__subclasshook__', '__weakref__', '_inst', '_name', '_paramspec_tvars', '_typevar_types', 'copy_with']

[gvanrossum]

Isn’t the solution to use the unspecialized class?

[Fidget-Spinner]

Not exactly sure if this is a bug, but the reason is that Foo[int] used to be a class, now it's a plain object. It's a change brought in 3.7 by PEP-560.

3.6:
>>> isinstance(Foo[int], type)
True
>>> Foo[int].__dir__
<method '__dir__' of 'object' objects
>>> type(Foo[int].__dir__)
<class 'method_descriptor'>

main:
>>> isinstance(Foo[int], type)
False
>>> Foo[int].__dir__
<built-in method __dir__ of _GenericAlias object at 0x000001B44DF0A850>
>>> type(Foo[int].__dir__)
<class 'method_descriptor'>

The fix is just a 2-line change in either _GenericAlias or _BaseGenericAlias:
+ def __dir__(self):
+ return dir(self.__origin__)

Should we go ahead with this?

[Fidget-Spinner]

New changeset d6e1374 by Ken Jin in branch 'main':
bpo-45755: [typing] Reveal class attributes of generic in generic aliases in dir() (GH-29962)
d6e1374

[Fidget-Spinner]

Hi, I won't be backporting this to 3.9 since I don't like having jarring behavior changes so late into the bugfix cycle of a Python version.

I'm tempted to backport to 3.10. For now, I'll be conservative and just merge it in main. Please do tell me if any of you feel that I should backport to 3.10 too.

3.6-3.8 are in security-fix only mode so they won't get any bugfixes.

Thanks Kevin for the interesting bug report, and Karthikeyan for saving me a ton of time debugging this!

[gvanrossum]

I think a 3.10 backport would be appreciated.

[miss-islington]

New changeset 87539cc by Miss Islington (bot) in branch '3.10':
bpo-45755: [typing] Reveal class attributes of generic in generic aliases in dir() (GH-29962)
87539cc