Skip to content

Clean up pending tasks when event_loop fixture leaves scope #200

@sureshjoshi

Description

@sureshjoshi

I'm not sure if this would be considered a bug or a feature :)

I noticed that in some of my tests, my long running tasks (e.g. while loops) were throwing warnings about tasks being destroyed but were pending. I guess this is happening because the loop is closed at the end of a test, but my while loops are still while'ing

In my projects, I maintain a cleanup function at the end of my main function to keep track of what tasks are still alive, then I cancel them before shutting down the loop. The tasks themselves are running forever intentionally, and they should only shut down when the app shuts down (and I shut down gracefully for cleaner logs).

I was wondering if this functionality should be added to the event_loop fixture (maybe optionally) to silence warnings where they don't really belong? Or, is this the responsibility of the developer via a custom fixture?

Activity

diefans

diefans commented on Jan 28, 2021

@diefans

IMHO you are responsible to cancel your long running tasks before the test exits

seifertm

seifertm commented on Sep 16, 2022

@seifertm
Contributor

There are very few things that need to be solved for the corresponding PR to get merged. Hence I'm marking this as good first issue.

changed the title [-]Should the default event_loop fixture add task cleanup?[/-] [+]Clean up pending tasks when event_loop fixture leaves scope[/+] on Nov 1, 2022
priyanshu-panwar

priyanshu-panwar commented on Jan 10, 2023

@priyanshu-panwar

Can I take this issue and solve it? @seifertm

seifertm

seifertm commented on Jan 10, 2023

@seifertm
Contributor

@priyanshu-panwar Of course, give it a try.

The existing PR is generally good. My main concerns are the use of non-public CPython APIs (i.e. the use of asyncio.runners._cancel_all_tasks) and the explicit call to gc.collect.

samypr100

samypr100 commented on Nov 9, 2023

@samypr100

@seifertm One thought I had is that an alternative, at least on Python 3.11 and above is to use the new asyncio.Runner.

@pytest.fixture(scope="session")
def event_loop() -> Iterator[asyncio.AbstractEventLoop]:
    with asyncio.Runner() as runner:
        yield runner.get_loop()

That would at least help with the cleanup and avoid using non-public API's. A loop factory could be provided for use-cases like using uvloop.

seifertm

seifertm commented on Nov 10, 2023

@seifertm
Contributor

@samypr100 I fully agree that using asyncio.Runner is the way to go for pytest-asyncio in the future.
This is tightly connected to #127.

As we speak, pytest-asyncio v0.23 is in the making which contains substantial changes, such as the deprecation of event_loop fixture overrides. I don't want to include any additional functionality, because the release is big enough as it is. In fact, the v0.22 had to be retracted due to problems connected to the deprecation.

Once the deprecation is in place, we can come back to this issue and investigate replacing existing calls to loop.run_until_complete() with calls to asyncio.Runner. Note that pytest-asyncio still supports Python 3.8, where asyncio.Runner isn't available. So we also need to find ways to backport that functionality.

samypr100

samypr100 commented on Dec 17, 2023

@samypr100

@seifertm In order to attempt help, I took a stab at a working asyncio.Runner backport that works all the way down to Python 3.8 and that its also compatible with uvloop. It preserves contextvars and passes CPython's existing unit tests for asyncio.Runner. If you're interested, here's the package https://pypi.org/project/backports.asyncio.runner/

added this to the v1.1 milestone on Jun 25, 2025
seifertm

seifertm commented on Jun 30, 2025

@seifertm
Contributor

This issue has been addressed by migrating internally to asyncio.Runner. Thanks @samypr100 for providing a backport to older Python versions!

Can anyone confirm that this issue is resolved in the pre-release version pytest-asyncio==1.1.0a1?

samypr100

samypr100 commented on Jul 2, 2025

@samypr100

Thanks @seifertm, I've released 1.2.0 to include an additional bug fix from CPython 3.11 between the time I had originally released this backport and now. Hope it helps 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Participants

      @diefans@sureshjoshi@samypr100@seifertm@priyanshu-panwar

      Issue actions

        Clean up pending tasks when event_loop fixture leaves scope · Issue #200 · pytest-dev/pytest-asyncio