Add a warning when setting up async fixtures

[seifertm]

What's the problem this feature will solve?

Pytest cannot evaluate fixture that are async functions or async generators. Users of async pytest plugins may unintentionally annotate those functions with @pytest.fixture rather than the async plugin's fixture function.

The fixture result ends up to be an unawaited async generator or coroutine, which is not what the user expected. Most of the time, the corresponding tests will fail. However, when the user defines an autouse fixture to perform setup or teardown, the fixture can silently fail.

Fixture results in async generator

import pytest

@pytest.fixture
async def async_fixture():
    yield 42

def test_this(async_fixture):
    assert async_fixture == 42  # fails

Fixture results in coroutine

import pytest

@pytest.fixture
async def async_fixture():
    return 42

def test_this(async_fixture):
    assert async_fixture == 42  # fails

Fixture silently does nothing

import pytest

@pytest.fixture(autouse=True)
async def async_fixture():
    yield 42
   # perform teardown

def test_this():
    assert True  # succeeds but should fail

Describe the solution you'd like

Pytest emits the following warning when running async test functions without an async plugin (see #2224):

PytestUnhandledCoroutineWarning: async def functions are not natively supported and have been skipped.
  You need to install a suitable plugin for your async framework, for example:
    - anyio
    - pytest-asyncio
    - pytest-tornasync
    - pytest-trio
    - pytest-twisted

The warning is emitted as part of a trylast hook to pytest_pyfunc_call. That means async pytest plugins can create hook wrappers to synchronize the test function and prevent the warning from being emitted.

I suggest to add the same behavior to the use of @pytest.fixture on async functions or async generators.

Examples

• Async fixture with autouse=True not executing properly in test pytest-asyncio#518
• v0.18.0 doesn't find some of my async fixtures pytest-asyncio#286
• 0.19.0: Fixture misses attribute pytest-asyncio#390
• pytest async fixture seems not work on linux-py3.7 pytest-asyncio#386

Alternative Solutions

#9962 proposes a mechanism to await results of awaitable tests and fixtures. It can potentially address the issue described here.

Additional context

From the above list of async plugins, only anyio, pytest-asyncio, and pytest-trio make use of pytest_fixture_setup.

[seifertm]

This approach may collide with anyio which uses the standard @pytest.fixture decorator for coroutines and async generators.

https://anyio.readthedocs.io/en/stable/testing.html#asynchronous-fixtures