RuntimeError: Event loop is closed as of 0.11.0 in particular test setup

[edaniszewski]

Python 3.7.4

pytest-asyncio==0.11.0
pytest==5.4.1
sanic==19.12.2

I'm not entirely sure how to classify what is happening here, as it is happening in a very particular test setup. I've managed to strip down the tests to the bare minimum to reproduce.

Directory Structure

$ tree
.
├── test_app.py
├── test_example.py
└── tox.ini

File Contents

tox.ini

[tox]
envlist = py3
skipsdist = True

[testenv]
deps =
    pytest
    pytest-asyncio==0.11.0
    sanic
commands =
    pytest -s .

test_app.py

import sanic
from sanic.response import json

app = sanic.Sanic()


@app.route('/')
async def route(request):
    return json({'status': 'ok'})


def test_app():
    resp = app.test_client.get('/', gather_request=False)
    assert resp.status == 200
    assert resp.json == {'status': 'ok'}

test_example.py

import pytest
import asyncio


@pytest.fixture()
async def clear_cache():
    """Test fixture to clear the backend aiocache cache.

    The memory cache implementation shares state in a class member, not
    an instance member, so we must clear between tests to ensure one test
    does not pollute state for another test case.
    """
    yield
    await asyncio.sleep(0.1)


@pytest.mark.asyncio
@pytest.mark.usefixtures('clear_cache')
class TestSomething:

    async def test_something(self):
        await asyncio.sleep(0.1)
        assert True

Test Output

$ tox
py3 recreate: /Users/edaniszewski/dev/tmp/testing/.tox/py3
py3 installdeps: pytest, pytest-asyncio==0.11.0, sanic
py3 installed: aiofiles==0.5.0,attrs==19.3.0,certifi==2020.4.5.1,chardet==3.0.4,h11==0.8.1,h2==3.2.0,hpack==3.0.0,hstspreload==2020.4.24,httptools==0.1.1,httpx==0.9.3,hyperframe==5.2.0,idna==2.9,importlib-metadata==1.6.0,more-itertools==8.2.0,multidict==4.7.5,packaging==20.3,pluggy==0.13.1,py==1.8.1,pyparsing==2.4.7,pytest==5.4.1,pytest-asyncio==0.11.0,rfc3986==1.4.0,sanic==19.12.2,six==1.14.0,sniffio==1.1.0,ujson==2.0.3,uvloop==0.14.0,wcwidth==0.1.9,websockets==8.1,zipp==3.1.0
py3 run-test-pre: PYTHONHASHSEED='3847369090'
py3 run-test: commands[0] | pytest -s .
==================================================== test session starts ====================================================
platform darwin -- Python 3.7.4, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
cachedir: .tox/py3/.pytest_cache
rootdir: /Users/edaniszewski/dev/tmp/testing
plugins: asyncio-0.11.0
collected 2 items                                                                                                           

test_app.py [2020-04-24 11:16:54 -0400] [37301] [INFO] Goin' Fast @ http://127.0.0.1:42101
[2020-04-24 11:16:54 -0400] [37301] [INFO] http://127.0.0.1:42101/
[2020-04-24 11:16:54 -0400] - (sanic.access)[INFO][127.0.0.1:51882]: GET http://127.0.0.1:42101/  200 15
[2020-04-24 11:16:54 -0400] [37301] [INFO] Starting worker [37301]
[2020-04-24 11:16:54 -0400] [37301] [INFO] Stopping worker [37301]
[2020-04-24 11:16:54 -0400] [37301] [INFO] Server Stopped
.
test_example.py EE

========================================================== ERRORS ===========================================================
______________________________________ ERROR at setup of TestSomething.test_something _______________________________________

args = (), kwargs = {}, request = <SubRequest 'clear_cache' for <Function test_something>>
setup = <function pytest_fixture_setup.<locals>.wrapper.<locals>.setup at 0x10dec3cb0>
finalizer = <function pytest_fixture_setup.<locals>.wrapper.<locals>.finalizer at 0x10dec3d40>

    def wrapper(*args, **kwargs):
        request = kwargs['request']
        if strip_request:
            del kwargs['request']
    
        gen_obj = generator(*args, **kwargs)
    
        async def setup():
            res = await gen_obj.__anext__()
            return res
    
        def finalizer():
            """Yield again, to finalize."""
            async def async_finalizer():
                try:
                    await gen_obj.__anext__()
                except StopAsyncIteration:
                    pass
                else:
                    msg = "Async generator fixture didn't stop."
                    msg += "Yield only once."
                    raise ValueError(msg)
            asyncio.get_event_loop().run_until_complete(async_finalizer())
    
        request.addfinalizer(finalizer)
>       return asyncio.get_event_loop().run_until_complete(setup())

.tox/py3/lib/python3.7/site-packages/pytest_asyncio/plugin.py:102: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
uvloop/loop.pyx:1430: in uvloop.loop.Loop.run_until_complete
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   RuntimeError: Event loop is closed

uvloop/loop.pyx:668: RuntimeError
_____________________________________ ERROR at teardown of TestSomething.test_something _____________________________________

    def finalizer():
        """Yield again, to finalize."""
        async def async_finalizer():
            try:
                await gen_obj.__anext__()
            except StopAsyncIteration:
                pass
            else:
                msg = "Async generator fixture didn't stop."
                msg += "Yield only once."
                raise ValueError(msg)
>       asyncio.get_event_loop().run_until_complete(async_finalizer())

.tox/py3/lib/python3.7/site-packages/pytest_asyncio/plugin.py:99: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
uvloop/loop.pyx:1430: in uvloop.loop.Loop.run_until_complete
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   RuntimeError: Event loop is closed

uvloop/loop.pyx:668: RuntimeError
===================================================== warnings summary ======================================================
test_app.py:5
  /Users/edaniszewski/dev/tmp/testing/test_app.py:5: DeprecationWarning: Sanic(name=None) is deprecated and None value support for `name` will be removed in the next release. Please use Sanic(name='your_application_name') instead.
    app = sanic.Sanic()

test_app.py::test_app
  /Users/edaniszewski/dev/tmp/testing/.tox/py3/lib/python3.7/site-packages/httpx/client.py:234: UserWarning: Passing a 'verify' argument when making a request on a client is due to be deprecated. Instantiate a new client instead, passing any 'verify' arguments to the client itself.
    "Passing a 'verify' argument when making a request on a client "

test_example.py::TestSomething::test_something
  /Users/edaniszewski/dev/tmp/testing/.tox/py3/lib/python3.7/site-packages/pytest_asyncio/plugin.py:102: RuntimeWarning: coroutine 'pytest_fixture_setup.<locals>.wrapper.<locals>.setup' was never awaited
    return asyncio.get_event_loop().run_until_complete(setup())

test_example.py::TestSomething::test_something
  /Users/edaniszewski/dev/tmp/testing/.tox/py3/lib/python3.7/site-packages/pytest_asyncio/plugin.py:99: RuntimeWarning: coroutine 'pytest_fixture_setup.<locals>.wrapper.<locals>.finalizer.<locals>.async_finalizer' was never awaited
    asyncio.get_event_loop().run_until_complete(async_finalizer())

-- Docs: https://docs.pytest.org/en/latest/warnings.html
================================================== short test summary info ==================================================
ERROR test_example.py::TestSomething::test_something - RuntimeError: Event loop is closed
ERROR test_example.py::TestSomething::test_something - RuntimeError: Event loop is closed
========================================== 1 passed, 4 warnings, 2 errors in 1.10s ==========================================
ERROR: InvocationError for command /Users/edaniszewski/dev/tmp/testing/.tox/py3/bin/pytest -s . (exited with code 1)
__________________________________________________________ summary __________________________________________________________
ERROR:   py3: commands failed

Notes

• it appears that having the sanic tests (test_app.py) run prior to the example test file is required for this error happen. renaming the file to test_zapp.py so it runs after test_example.py mitigates the issue.
• within the sanic test file, it appears that it is important to use the app test_client. I'm not sure if other things will produce a similar effect, but based on my specific use case, commenting out tests which used the test_app appeared to mitigate the problem.
• it appears to be important that the @pytest.mark.usefixtures decorator is used, and that the fixture is an async function.
  • the order in which the decorators (@pytest.mark.asyncio, @pytest.mark.usefixtures) does not appear to matter
• downgrading to pytest-asyncio==0.10.0 mitigates the issue as well, so this seems to be related to something that changed with the 0.11.0 release.
• I've tested with py3.6, py3.7, and py3.8 and had the same results for each version

[simonfagerholm]

I tested your examples on PR #156 and it resolves the problems you describe

[edaniszewski]

@simonfagerholm Thanks for all your work looking into this and the related issues and getting your PR up. Much appreciated!