Release 0.13 and 0.14 break standard use of Starlette TestClient

[pquentin]

Steps to reproduce

Write t.py:

import pytest
import starlette
from starlette.testclient import TestClient
from starlette.applications import Starlette
from starlette.routing import Route

app = Starlette(routes=[Route("/", lambda request: "Hello")])


@pytest.fixture(scope="session")
def test_client():
    with TestClient(app) as client:
        yield client


def test_http(test_client):
    pass


@pytest.mark.asyncio
async def test_other():
    pass

Run pip install pytest pytest-asyncio starlette and run pytest t.py.

Expected results

All tests pass, including their teardown

Actual results

=== test session starts ===
platform darwin -- Python 3.7.7, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
rootdir: ...
plugins: asyncio-0.13.0.dev0
collected 2 items

t.py ..E                                                                                                               [100%]

=== ERRORS ===
___ ERROR at teardown of test_other ___
    @pytest.fixture(scope="session")
    def test_client():
        with TestClient(app) as client:
>           yield client

t.py:13:
_ _ _
.../starlette/testclient.py:463: in __exit__
    loop.run_until_complete(self.wait_shutdown())
.../lib/python3.7/asyncio/base_events.py:587: in run_until_complete
    return future.result()
.../starlette/testclient.py:489: in wait_shutdown
    message = await self.send_queue.get()
_ _ _

self = <Queue at 0x7fe05b257cd0 maxsize=0 tasks=1>

    async def get(self):
        """Remove and return an item from the queue.

        If queue is empty, wait until an item is available.
        """
        while self.empty():
            getter = self._loop.create_future()
            self._getters.append(getter)
            try:
>               await getter
E               RuntimeError: Task <Task pending coro=<TestClient.wait_shutdown() running at .../starlette/testclient.py:489> cb=[_run_until_complete_cb() at .../lib/python3.7/asyncio/base_events.py:157]> got Future <Future pending> attached to a different loop

.../lib/python3.7/asyncio/queues.py:159: RuntimeError
--- Captured stdout teardown ---
Task was destroyed but it is pending!
task: <Task pending coro=<TestClient.lifespan() running at .../starlette/testclient.py:468> wait_for=<Future finished result=None>>
=== 2 passed, 1 error in 0.17s ===

It was difficult to get to a reproducer, because it's important to have 1/ one test using pytest-asyncio 2/ one test using starlette test client in a fixture. Using such a fixture is actually recommended in the Starlette docs: https://www.starlette.io/config/#a-full-example and was working with pytest-asyncio 0.12.0.

I bisected, and the first bad commit is f97e900.

This is different from #166 because #166 was happening with 0.12.0.

[Tinche]

Hey @pquentin, thanks for the report.

Looks like the Starlette TestClient actually uses asyncio under the hood, and depends on an event loop. So if you encode that dependency into the fixture, like this:

@pytest.fixture(scope="session")
def my_client(event_loop):
    with TestClient(app) as client:
        yield client

the actual error becomes apparent:

ScopeMismatch: You tried to access the 'function' scoped fixture 'event_loop' with a 'session' scoped request object, involved factories

The fix is to make the event_loop fixture be session-scoped too. That's done by overriding it. Here's a working file:

import asyncio
import pytest
from starlette.testclient import TestClient
from starlette.applications import Starlette
from starlette.routing import Route

app = Starlette(routes=[Route("/", lambda request: "Hello")])


@pytest.fixture(scope="session")
def event_loop(request):
    loop = asyncio.get_event_loop_policy().new_event_loop()
    yield loop
    loop.close()


@pytest.fixture(scope="session")
def my_client(event_loop):
    with TestClient(app) as client:
        yield client


def test_http(my_client):
    pass


@pytest.mark.asyncio
async def test_other():
    pass

So to me this looks OK on the pytest-asyncio side.

[alblasco]

Thanks @pquentin for your feedback and @Tinche for this right analysis.

@pquentin Please see my analysis for #170 and #168, so you can check if any of them can also affect your case.
Please. Let me know your kind feedback.

[pquentin]

@alblasco Honestly I'm fine with saying that this is a Starlette documentation bug, as you've been documenting for years that this isn't supported.

Thanks @Tinche for the explanation! Closing.

[pquentin]

It's not even a Starlette documentation bug, because Starlette uses a function-scoped fixture in its examples.