New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MagicMock doesn't implement `__aenter__` or `__aexit__` #29

Closed
txomon opened this Issue Dec 19, 2016 · 42 comments

Comments

Projects
None yet
9 participants
@txomon

txomon commented Dec 19, 2016

When testing asyncio code, the probability of needing to test async context manager is high.

The current code using MagicMock doesn't implement __aenter__ or __aexit__ methods

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Dec 20, 2016

Hi,

Thanks for the report, it's on the todo list!

@Gr1N

This comment has been minimized.

Gr1N commented Dec 20, 2016

+1 it would be really useful to add implementation of __aenter__ and __aexit__ methods.

@txomon

This comment has been minimized.

txomon commented Dec 20, 2016

I add this link as relevant to the conversation http://bugs.python.org/issue26467

@serg666

This comment has been minimized.

serg666 commented Jan 27, 2017

Hi, guys! Any news about this issue?

@serg666

This comment has been minimized.

serg666 commented Jan 27, 2017

I would like to mock aiopg.sa.create_engine with CoroutineMock, but if I use async-with-style it does not work

import asyncio
import aiopg.sa
import asynctest


mock = asynctest.CoroutineMock()

async def create_engine():
    kwargs = {'echo': True}
    return await aiopg.sa.create_engine(
        'dbname=db user=user password=pass host=127.0.0.1',
        **kwargs
   )

async def go(get_engine):
    engine = await get_engine()
    async with engine.acquire() as conn:
        async with conn.begin():
            r = await conn.execute('SELECT version();')
    return await r.fetchone()

row = asyncio.get_event_loop().run_until_complete(go(create_engine))
print(row)

row = asyncio.get_event_loop().run_until_complete(go(mock))
print(row)
python atest.py 
('PostgreSQL 9.6.1 on x86_64-suse-linux-gnu, compiled by gcc (SUSE Linux) 6.2.1 20161209 [gcc-6-branch revision 243481], 64-bit',)
Traceback (most recent call last):
  File "atest.py", line 25, in <module>
    row = asyncio.get_event_loop().run_until_complete(go(mock))
  File "/usr/lib64/python3.5/asyncio/base_events.py", line 337, in run_until_complete
    return future.result()
  File "/usr/lib64/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib64/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "atest.py", line 17, in go
    async with engine.acquire() as conn:
AttributeError: __aexit__

but if I do not use async-with-style all works fine

import asyncio
import aiopg.sa
import asynctest


mock = asynctest.CoroutineMock()

async def create_engine():
    kwargs = {'echo': True}
    return await aiopg.sa.create_engine(
        'dbname=wallet user=wallet password=qazwsx host=127.0.0.1',
        **kwargs
   )

async def go(get_engine):
    engine = await get_engine()
    conn = await engine.acquire()
    trans = await conn.begin()
    r = await conn.execute('SELECT version();')
    await trans.commit()
    engine.release(conn)
    return await r.fetchone()

row = asyncio.get_event_loop().run_until_complete(go(create_engine))
print(row)

row = asyncio.get_event_loop().run_until_complete(go(mock))
print(row)
('PostgreSQL 9.6.1 on x86_64-suse-linux-gnu, compiled by gcc (SUSE Linux) 6.2.1 20161209 [gcc-6-branch revision 243481], 64-bit',)
<CoroutineMock name='mock.......' id='139844439568168'>

@Martiusweb Martiusweb self-assigned this Mar 9, 2017

@Galbar

This comment has been minimized.

Galbar commented Mar 21, 2017

We had the same problem mocking aiopg. We ended up with this workaround:

class AsyncContextManagerMock(MagicMock):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        for key in ('aenter_return', 'aexit_return'):
            setattr(self, key,  kwargs[key] if key in kwargs else MagicMock())

    async def __aenter__(self):
        return self.aenter_return

    async def __aexit__(self, *args):
        return self.aexit_return
@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Mar 28, 2017

Hello,

Some news regarding this issue: I'm working on a solution which seems to be working for the basic needs (in the async_with branch). Some cases are yet to be tested, but if some of you want to try it, feedback is welcome.

@serg666

This comment has been minimized.

serg666 commented Mar 30, 2017

Hello, @Martiusweb !

Do you mean async_magic branch ?

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Mar 30, 2017

@serg666

This comment has been minimized.

serg666 commented Mar 30, 2017

I have just try async_magic branch with code as below

import asyncio
import aiopg.sa
import asynctest


mock = asynctest.CoroutineMock()

async def create_engine():
    kwargs = {'echo': True}
    return await aiopg.sa.create_engine(
        'dbname=db user=dbuser password=qazwsx host=127.0.0.1',
        **kwargs
   )

async def go(get_engine):
    engine = await get_engine()
    async with engine.acquire() as conn:
        async with conn.begin():
            r = await conn.execute('SELECT version();')
    return await r.fetchone()

row = asyncio.get_event_loop().run_until_complete(go(create_engine))
print(row)

row = asyncio.get_event_loop().run_until_complete(go(mock))
print(row)

I have got the following:

python atest.py
('PostgreSQL 9.6.2 on x86_64-suse-linux-gnu, compiled by gcc (SUSE Linux) 6.3.1 20170202 [gcc-6-branch revision 245119], 64-bit',)
Traceback (most recent call last):
File "atest.py", line 25, in
row = asyncio.get_event_loop().run_until_complete(go(mock))
File "/usr/lib64/python3.6/asyncio/base_events.py", line 466, in run_until_complete
return future.result()
File "atest.py", line 17, in go
async with engine.acquire() as conn:
TypeError: object MagicMock can't be used in 'await' expression

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Mar 30, 2017

This is because asynctest has no way to know that engine.acquire() is a coroutine, hence it's mocked as a MagicMock rather than a CoroutineMock.

However, it worked before because CoroutineMock returned an other CoroutineMock as the result of its invocation, which is wrong.

You can fix this by doing something like:

engine_mock = asyncio.MagicMock(Engine())
mock = asyncio.CoroutineMock(return_value=engine_mock)
@serg666

This comment has been minimized.

serg666 commented Mar 30, 2017

import asyncio
import aiopg.sa
import asynctest


mock = asynctest.CoroutineMock(return_value=asynctest.MagicMock())

async def create_engine():
    kwargs = {'echo': True}
    return await aiopg.sa.create_engine(
        'dbname=db user=dbuser password=qazwsx host=127.0.0.1',
        **kwargs
   )

async def go(get_engine):
    engine = await get_engine()
    async with engine.acquire() as conn:
        async with conn.begin():
            r = await conn.execute('SELECT version();')
    return await r.fetchone()

row = asyncio.get_event_loop().run_until_complete(go(create_engine))
print(row)

row = asyncio.get_event_loop().run_until_complete(go(mock))
print(row)

python atest.py
('PostgreSQL 9.6.2 on x86_64-suse-linux-gnu, compiled by gcc (SUSE Linux) 6.3.1 20170202 [gcc-6-branch revision 245119], 64-bit',)
Traceback (most recent call last):
File "atest.py", line 26, in
row = asyncio.get_event_loop().run_until_complete(go(mock))
File "/usr/lib64/python3.6/asyncio/base_events.py", line 466, in run_until_complete
return future.result()
File "atest.py", line 18, in go
async with engine.acquire() as conn:
TypeError: object MagicMock can't be used in 'await' expression

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Mar 30, 2017

That's still the same issue: asynctest can't know that the MagicMock (without spec) returned by mock() must have an aquire attribute which is a Coroutine.

This:

mock = asynctest.CoroutineMock(return_value=asynctest.MagicMock())

is already equivalent to the default behavior, but you can do this:

mock = asynctest.CoroutineMock()
mock().acquire = CoroutineMock()  # which tells that acquire is a coroutine.

(edited: first sentence was incomplete)

@serg666

This comment has been minimized.

serg666 commented Apr 11, 2017

Ok. I just modify code like this

import asyncio
import aiopg.sa
import asynctest


def make_engine(connection):
    return asynctest.CoroutineMock(
        return_value=asynctest.CoroutineMock(
            acquire=asynctest.CoroutineMock(
                return_value=connection
            )
        )
    )

async def create_engine():
    kwargs = {'echo': True}
    return await aiopg.sa.create_engine(
        'dbname=dbname user=dbuser password=dbpassword host=127.0.0.1',
        **kwargs
   )

async def go(get_engine):
    engine = await get_engine()
    print(engine.acquire)
    print(asyncio.iscoroutinefunction(engine.acquire))
    print(asyncio.iscoroutine(engine.acquire()))
    async with engine.acquire() as conn:
        async with conn.begin():
            r = await conn.execute('SELECT version();')
    return await r.fetchone()


#row = asyncio.get_event_loop().run_until_complete(go(create_engine))
#print(row)

row = asyncio.get_event_loop().run_until_complete(go(make_engine(asynctest.CoroutineMock())))
print(row)

so, mock have an acquire attribute which is a Coroutine.

but any way I have got the following:

python atest.py 
<CoroutineMock name='mock.acquire' id='140187495530904'>
True
True
Traceback (most recent call last):
  File "atest.py", line 35, in <module>
    row = asyncio.get_event_loop().run_until_complete(go(make_engine(asynctest.CoroutineMock())))
  File "/usr/lib64/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "atest.py", line 27, in go
    async with engine.acquire() as conn:
AttributeError: __aexit__

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented May 1, 2017

You must use MagicMock instead of CoroutineMock: Engine, _EngineAcquireContextManager, and _ConnectionContextManager are not coroutines.

Anyway, MagicMock is not smart enough to do what you want to achieve: you the mock system to infer the type of the result of engine.acquire() and conn.begin(), and it's not possible in pure Python. It would be the same problem with a non-asynchronous equivalent.

If you want to mock all the chain, you must do something like:

def make_engine():
    connection_mock = asynctest.MagicMock()
    connection_mock.begin.return_value = asynctest.MagicMock(
        aiopg.sa.connection._SAConnectionContextManager)

    acquire_mock = asynctest.MagicMock(
        aiopg.sa.engine._EngineAcquireContextManager)
    acquire_mock.__aenter__.return_value = connection_mock

    engine_mock = asynctest.MagicMock(aiopg.sa.engine.Engine)
    engine_mock.acquire.return_value = acquire_mock

    return asynctest.CoroutineMock(return_value=engine_mock)
@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented May 1, 2017

I implemented aenter, aexit, aiter and anext.

I think there are cases left to test, especially cases dealing with exceptions (eg: when an exception is raised inside an asynchronous context manager, the exception seems to be swallowed and lost).

It's also tedious to mock those magic methods, because the async context manager class must be used as mock spec. Classes like AsyncContextManagerMock and AsyncIteratorMock can be useful to provide a more user friendly API.

@serg666

This comment has been minimized.

serg666 commented May 2, 2017

@Martiusweb , thanks a lot!

The following code works fine

import asyncio
import aiopg.sa
import asynctest


def make_engine():
    connection_mock = asynctest.MagicMock(
        execute=asynctest.CoroutineMock(
            return_value=asynctest.CoroutineMock(
                fetchone=asynctest.CoroutineMock()
            )
        )
    )
    connection_mock.begin.return_value = asynctest.MagicMock(
        aiopg.sa.connection._SAConnectionContextManager)

    acquire_mock = asynctest.MagicMock(
        aiopg.sa.engine._EngineAcquireContextManager)
    acquire_mock.__aenter__.return_value = connection_mock

    engine_mock = asynctest.MagicMock(aiopg.sa.engine.Engine)
    engine_mock.acquire.return_value = acquire_mock

    return asynctest.CoroutineMock(return_value=engine_mock)

async def create_engine():
    kwargs = {'echo': True}
    return await aiopg.sa.create_engine(
        'dbname=dbname user=dbuser password=dbpasssword host=127.0.0.1',
        **kwargs
   )

async def go(get_engine):
    engine = await get_engine()
    async with engine.acquire() as conn:
        async with conn.begin():
            r = await conn.execute('SELECT version();')
    return await r.fetchone()


row = asyncio.get_event_loop().run_until_complete(go(create_engine))
print(row)

row = asyncio.get_event_loop().run_until_complete(go(make_engine()))
print(row)

the result is

python atest.py 
('PostgreSQL 9.6.2 on x86_64-suse-linux-gnu, compiled by gcc (SUSE Linux) 6.3.1 20170202 [gcc-6-branch revision 245119], 64-bit',)
<MagicMock name='mock.fetchone()' id='140316582692120'>
Unclosed connection
connection: <aiopg.connection.Connection object at 0x7f9e0014f470>

but on branch async_magic I have seen message about Unclosed connection when execute

row = asyncio.get_event_loop().run_until_complete(go(create_engine))

Note, on current version from pypi I have not seen this message.

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented May 2, 2017

Hi,

This is likely because somewhere in your code an actual connection is created but never closed.
It might be left open because of the mock, but it's because of the way aiopg and/or your test crafted and it's not directly related to asynctest.

You'll have to track which part of your test uses real code rather than the mock.

@serg666

This comment has been minimized.

serg666 commented May 2, 2017

No no no!!!

the following code does not use MOCK at all

import asyncio
import aiopg.sa
import asynctest

'''
def make_engine():
    connection_mock = asynctest.MagicMock(
        execute=asynctest.CoroutineMock(
            return_value=asynctest.CoroutineMock(
                fetchone=asynctest.CoroutineMock()
            )
        )
    )
    connection_mock.begin.return_value = asynctest.MagicMock(
        aiopg.sa.connection._SAConnectionContextManager)

    acquire_mock = asynctest.MagicMock(
        aiopg.sa.engine._EngineAcquireContextManager)
    acquire_mock.__aenter__.return_value = connection_mock

    engine_mock = asynctest.MagicMock(aiopg.sa.engine.Engine)
    engine_mock.acquire.return_value = acquire_mock

    return asynctest.CoroutineMock(return_value=engine_mock)
'''

async def create_engine():
    kwargs = {'echo': True}
    return await aiopg.sa.create_engine(
        'dbname=db user=user password=passwd host=127.0.0.1',
        **kwargs
   )

async def go(get_engine):
    engine = await get_engine()
    async with engine.acquire() as conn:
        async with conn.begin():
            r = await conn.execute('SELECT version();')
    return await r.fetchone()


row = asyncio.get_event_loop().run_until_complete(go(create_engine))
print(row)

#row = asyncio.get_event_loop().run_until_complete(go(make_engine()))
#print(row)

and when I execute this code on current version from pypi everything is OK, but when I execute this code on the async_magic branch I have seen

('PostgreSQL 9.6.2 on x86_64-suse-linux-gnu, compiled by gcc (SUSE Linux) 6.3.1 20170202 [gcc-6-branch revision 245119], 64-bit',)
Unclosed connection
connection: <aiopg.connection.Connection object at 0x7fd9f28463c8>
@serg666

This comment has been minimized.

serg666 commented May 2, 2017

U can test it yourself. Just switch to current version from pypi and execute the code above. Then switch to async_magic branch and try to execute this code. U will see message about Unclosed connection

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented May 2, 2017

In your example, you don't use asynctest at all. I don't see what I can do for you there.

Since it's not related to the async context managers mocking issue, if you think it's related to asynctest and can provide more details (for instance, use git bisect to find the commit which introduced a bug), please open a new issue.

@serg666

This comment has been minimized.

serg666 commented May 2, 2017

I just want to say, that on current version from pypi and on async_magic result of execution the above code is different. The only difference is asynctest version

@serg666

This comment has been minimized.

serg666 commented May 2, 2017

Executing the above code in both branches master and async_magic leads to message about Unclosed connection. But if I use current version from pypi https://pypi.python.org/pypi/asynctest/0.10.0 everything is ok

@serg666

This comment has been minimized.

serg666 commented May 2, 2017

If I switch to commit a3b7b5e on master (the current version in pypi) everything is OK. If I switch back to master I have got message about Unclosed connection

@serg666

This comment has been minimized.

serg666 commented May 2, 2017

Wow!

If I switch back to one commit from current master

git checkout HEAD~

everything is OK executing the code above. No message about Unclosed connection. If I switch to the HEAD again , I have got message about Unclosed connection. So, something wrong with commit e005798

This is about commit which introduced a bug (strange behavior).

@serg666

This comment has been minimized.

serg666 commented May 2, 2017

For reference: If I comment importing asynctest in the code above everything is OK on master (no message about Unclosed connection).

import asyncio
import aiopg.sa
# import asynctest
...
@pfertyk

This comment has been minimized.

pfertyk commented Jun 13, 2017

@Martiusweb what are the plans for this issue? Are __aenter__ and __aexit__ methods going to be supported by asynctest?

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Jun 19, 2017

Yes, I've been working on this but there are still some parts I need to fix before I can merge the PR:
#42

You can try the branch though, as the most common use cases are working.

@thehesiod

This comment has been minimized.

thehesiod commented Sep 22, 2017

nice work @Martiusweb on a real fix and @Galbar on work-around, this thread inspired us to do a simpler fix:

class MagicMockContext(MagicMock):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        type(self).__aenter__ = CoroutineMock(return_value=MagicMock())
        type(self).__aexit__ = CoroutineMock(return_value=MagicMock())

with usage:

with asynctest.mock.patch('dummy.Dummy', new_callable=MagicMockContext) as mockClientSession:
@argaen

This comment has been minimized.

argaen commented Oct 16, 2017

@Martiusweb any chance the branch could be merged? I've been using it for a while and works for me. If not, what are the missing things? Maybe other people can help there.

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Oct 17, 2017

I'll review my work and list what's missing by the end of the week. Exception raised in the context manager are not propagated correctly yet: I need to write the tests to ensure it works.

I also would like to add an API which helps manipulating those mocks, but I'll probably do that in an other PR.

@argaen

This comment has been minimized.

argaen commented Oct 17, 2017

Cool, once the list is available I'll check if there is something I can help this. I prefer installing things from a released version rather than from specific branch/commit :P

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Oct 23, 2017

This issue has been fixed in the 0.11.0 release which has just been pushed to pypi.

Feedback is more than welcome! If you see anything buggy or that can be improved, open a new bug!

@Martiusweb Martiusweb closed this Oct 23, 2017

@argaen

This comment has been minimized.

argaen commented Oct 23, 2017

Good news, thanks a lot! :)

@x0zzz

This comment has been minimized.

x0zzz commented Oct 30, 2017

With asynctest==0.11.0 this should work, right?:

from asynctest import CoroutineMock
from asyncio import get_event_loop

async def f():
    async with CoroutineMock():
        print('f()')

get_event_loop().run_until_complete(f())

However I get:
AttributeError: __aexit__

Am I doing something wrong?

For completeness:

❯❯❯ pip freeze | ag asynctest                                                    async ✱
asynctest==0.11.0
@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Oct 31, 2017

You must think of CoroutineMock as the mock of a coroutine function, nothing else. In your case, you should use MagicMock, not CoroutineMock.

(Note to self: I really need to work on the guide)

@x0zzz

This comment has been minimized.

x0zzz commented Nov 1, 2017

Sorry for being dense, but this:

from asynctest import MagicMock
from asyncio import get_event_loop

async def f():
    async with MagicMock():
        print('f()')

get_event_loop().run_until_complete(f())

gives this:
TypeError: object MagicMock can't be used in 'await' expression

Can you please help me?

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Nov 2, 2017

You spotted a bug: unless you specify a spec implementing __aenter__ and __aexit__, this will not work. I opened #55 and will work on it today if I can find the time.

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Nov 4, 2017

@x0zzz the bug is fixed in v0.11.1

@x0zzz

This comment has been minimized.

x0zzz commented Nov 4, 2017

Thank you so very much :)

@thehesiod

This comment has been minimized.

thehesiod commented Nov 13, 2017

btw just a q in case anyone knows off the top of their head, is there any quick way to ensure that __aenter__ __enter__ returns "self" by default (same magicmock)?

@Martiusweb

This comment has been minimized.

Owner

Martiusweb commented Nov 14, 2017

Maybe I misunderstood your comment, but this is not the default behavior of unittest:

>>> mo = unittest.mock.MagicMock()
>>> with m as mm:
...   print(id(m), id(mm), m is mm)
... 
140366555841704 140366478933072 False

The simplest way to do this is to set the return_value:

mock = asynctest.mock.MagicMock()
mock.__aenter__.return_value = self
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment