spec_set/autospec/spec seems to not be reading attributes defined in class body

[efagerberg]

BPO	40864
Nosy	@cjw296, @voidspace, @lisroach, @mariocj89, @tirkarthi, @efagerberg
PRs	bpo-40864: Fetch instance variables for spec #22572

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-06-04.14:28:47.540>
labels = ['3.8', 'type-bug', 'library', '3.9']
title = 'spec_set/autospec/spec seems to not be reading attributes defined in class body'
updated_at = <Date 2020-10-06.07:17:47.412>
user = 'https://github.com/efagerberg'

bugs.python.org fields:

activity = <Date 2020-10-06.07:17:47.412>
actor = 'xtreak'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-06-04.14:28:47.540>
creator = 'efagerberg'
dependencies = []
files = []
hgrepos = []
issue_num = 40864
keywords = ['patch']
message_count = 5.0
messages = ['370712', '370713', '370752', '370865', '378076']
nosy_count = 6.0
nosy_names = ['cjw296', 'michael.foord', 'lisroach', 'mariocj89', 'xtreak', 'efagerberg']
pr_nums = ['22572']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40864'
versions = ['Python 3.6', 'Python 3.8', 'Python 3.9']

[efagerberg]

Hello, I really like that this library allows for really strict mocking however one thing I have noticed is that it seems like using spec on a mock does not properly read the class body for attributes like some of the documentation claims. For example this is a snippet of the Logger class in python 3.6's logging module

class Logger(Filterer):
    name: str
    level: int
    parent: Union[Logger, PlaceHolder]
    propagate: bool
    handlers: List[Handler]
    disabled: int

Now I want to mock that class ensuring that propagate gets set to False for example

from unittest import mock
from logging import Logger

logger = mock.Mock(spec_set=Logger)
logger.propagate = False
assert logger.propagate is False
*** AttributeError: Mock object has no attribute 'propagate'

I have noticed this does work when the value is initialized in the class body so for example

class Logger(Filterer):
    name: str
    level: int
    parent: Union[Logger, PlaceHolder]
    propagate: bool = False
    handlers: List[Handler]
    disabled: int

This would not fail with the test in question.

Wondering if this is intended behavior or not or if I am misunderstanding something. I have tested this with Python 3.6.10, 3.8.2, all with the same result.

[efagerberg]

Sorry one small note, the error in the example happens on

logger.propagate = False

and not

assert logger.propagate is False

[tirkarthi]

mock uses dir to iterate through the attributes that needs to be specced [0]. Unless the variable is initialized it's not listed in dir. Below is an example where age is initialized and name is not. name is not present in dir(Person) and hence spec will not be able to detect this. This is similar to https://bugs.python.org/issue36580.

cat /tmp/baz.py
class Person:
name: str
age: int = 10

print(dir(Person))
print(Person.name)

python /tmp/baz.py
['__annotations__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'age']
Traceback (most recent call last):
  File "/tmp/baz.py", line 6, in <module>
    print(Person.name)
AttributeError: type object 'Person' has no attribute 'name'

[0]

cpython/Lib/unittest/mock.py

Line 2647 in e005ead

for entry in dir(spec):

[efagerberg]

Rereading the documentation, I see that a class attribute set to null will return a MagicMock for that attribute. That might be a reasonable workaround. Perhaps the more concrete solution would be that dir lists uninitialized class attributes and if a type hint is present the class attributes uses a spec of that type hint.

[efagerberg]

Reflecting on it more, there should be a sensible way to retrieve the set attributes of the init method of any class without explicitly instantiating it, via the inspect module.