Add async magic method support to unittest.mock.Mock

[brettcannon]

BPO	26467
Nosy	@brettcannon, @rbtcollins, @benjaminp, @voidspace, @berkerpeksag, @dimaqq, @1st1, @yan12125, @lisroach, @butla, @miss-islington, @tirkarthi
PRs	bpo-26467: Adds AsyncMock for asyncio Mock library support #9296 Adding missing acknowledgement #17130 [3.8] Add Ilya Kulakov to Misc/ACKS. (GH-17130) #17138

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/lisroach'
closed_at = <Date 2019-05-20.16:36:26.838>
created_at = <Date 2016-03-01.22:51:46.522>
labels = ['3.8', 'type-feature', 'library']
title = 'Add async magic method support to unittest.mock.Mock'
updated_at = <Date 2019-11-13.02:40:34.212>
user = 'https://github.com/brettcannon'

bugs.python.org fields:

activity = <Date 2019-11-13.02:40:34.212>
actor = 'miss-islington'
assignee = 'lisroach'
closed = True
closed_date = <Date 2019-05-20.16:36:26.838>
closer = 'lisroach'
components = ['Library (Lib)']
creation = <Date 2016-03-01.22:51:46.522>
creator = 'brett.cannon'
dependencies = []
files = []
hgrepos = []
issue_num = 26467
keywords = ['patch']
message_count = 12.0
messages = ['261092', '267338', '267479', '267569', '283657', '293644', '333208', '342926', '342929', '356485', '356511', '356513']
nosy_count = 15.0
nosy_names = ['brett.cannon', 'rbcollins', 'benjamin.peterson', 'michael.foord', 'jmehnle', 'berker.peksag', 'Dima.Tisnek', 'yselivanov', 'txomon', 'yan12125', 'tsutsumi', 'lisroach', 'butla', 'miss-islington', 'xtreak']
pr_nums = ['9296', '17130', '17138']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26467'
versions = ['Python 3.8']

[brettcannon]

Mock doesn't support stuff like __aexit__ which means you can't mock out something like an async context manager.

[tsutsumi]

I'm interested in taking this on. I'm considering for this approach, creating a new MagicProxy class specifically for async, and assigning __aexit__ and __aenter__ to those.

The __call__ method itself has to be detected as asynchronous, and I don't believe it's possible for a __call__ to behave appropriately for both asynchronous and synchronous cases.

[tsutsumi]

Taking a look at the code, this may require a bit more discussion.

Mock's classes create copies of themselves when an attribute is accessed. As such, I can't just add __aexit__ and have it generate a different mock type entirely, unless I know exactly what an async variant of this mock is.

If there was a way to specifically provide a function for asynchronous situations, like __acall__, that would make this a lot easier as well.

Anyway, my proposal now is:

create new classes MockAsync and MagicMockAsync, and have MockAsync and MagicMockAsync implement the __aexit__ and __aenter__ methods.

How is that as an approach?

[brettcannon]

There's also __aiter__, but it's semantics might be changing (issue bpo-27243).

Robert or Michael, any opinions on Yusuke's proposal?

[txomon]

I found this while trying to test an async context manager. This is a critical feature to enable migrations to async code, as the impossibility to test something properly is not acceptable in many environments.

Implementing it in a way that __call__ returns an object capable of being coroutine or normal function would avoid having to implement Async specific Mocks, wouldn't it? I am not too confident, but would it be doable to have an implementation that depends on whether _is_coroutine is accessed or not?

I don't like it, but I really don't like the fact that we need to patch different all the methods depending on whether they are coroutine or not.

Obviously, having the __acall__ method would really help solving this issue.

[butla]

I guess that it's worth noting that the same problem is being talked about in an issue on the asynctest library: Martiusweb/asynctest#29

[dimaqq]

Perhaps it's possible to vendor asynctest mock in the same vein as mock found it's way into unittest?

The real power of asynctest is in constructs like:

with asynctest.mock.patch("module.Class", autospec=True):
    ...

Where mock instances know what methods are async.

[lisroach]

New changeset 77b3b77 by Lisa Roach in branch 'master':
bpo-26467: Adds AsyncMock for asyncio Mock library support (GH-9296)
77b3b77

[lisroach]

Added and AsyncMock class which supports mocking async magic methods.

[brettcannon]

Adding Lisa to potentially add the PR from Ilya.

[benjaminp]

New changeset d6d6e2a by Benjamin Peterson (Ilya Kulakov) in branch 'master':
Add Ilya Kulakov to Misc/ACKS. (GH-17130)
d6d6e2a

[miss-islington]

New changeset e5125f7 by Miss Islington (bot) in branch '3.8':
Add Ilya Kulakov to Misc/ACKS. (GH-17130)
e5125f7