3.11.5 regression: StreamWriter.__del__ fails if event loop is already closed

[bmerry]

Bug report

Bug description:

PR #107650 added a StreamWriter.__del__ that emits a ResourceWarning if a StreamWriter is not closed by the time it is garbage collected, and it has been backported as 3.11.5. However, if the event loop has already been closed by the time this happens, it causes a RuntimeError message to be displayed. It's non-fatal because exceptions raised by __del__ are ignored, but it causes an error message to be displayed to the user (as opposed to ResourceWarning, which is only shown when one opts in).

Code like the following used to run without any visible error, but now prints a traceback (and does not report the ResourceWarning for the unclosed StreamWriter when run with -X dev):

#!/usr/bin/env python3

import asyncio

async def main():
    global writer
    reader, writer = await asyncio.open_connection("127.0.0.1", 22)

asyncio.run(main())

Output in 3.11.5:

Exception ignored in: <function StreamWriter.__del__ at 0x7fd54ee11080>
Traceback (most recent call last):
  File "/usr/lib/python3.11/asyncio/streams.py", line 396, in __del__
  File "/usr/lib/python3.11/asyncio/streams.py", line 344, in close
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 860, in close
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
RuntimeError: Event loop is closed

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Linked PRs

• gh-109538: Catch closed loop runtime error and issue warning #111983
• [3.11] gh-109538: Catch closed loop runtime error and issue warning (GH-111983) #112141
• [3.12] gh-109538: Catch closed loop runtime error and issue warning (GH-111983) #112142

[dpr-0]

Same problem here

[smirnoffs]

I have the same problem with 3.11.6. Thanks for reporting it.

[pfps]

Is there a workaround?

[dpr-0]

Is there a workaround?

Use gc.collect() may help

Here is my workaround.

@pytest.fixture
def event_loop():
    policy = asyncio.get_event_loop_policy()
    loop = policy.new_event_loop()
    loop.set_debug(True)
    yield loop
    gc.collect()
    loop.close()

[pfps]

@dpr-0 Thanks, but just adding that to my program didn't prevent the messages.

[dpr-0]

@dpr-0 Thanks, but just adding that to my program didn't prevent the messages.

Hope you can find the problem T^T

[dtatarkin]

Same problem for python 3.12.0

[gvanrossum]

Sorry for the inconvenience. Can you (or anyone else reading this) submit a PR for the main branch? We will then backport the PR to 3.12 and 3.11.

[bmerry]

Sorry for the inconvenience. Can you (or anyone else reading this) submit a PR for the main branch? We will then backport the PR to 3.12 and 3.11.

What is the desired behaviour if the event loop is already closed when the StreamWriter is destroyed? Should __del__ just become a no-op?

[gvanrossum]

What you expected -- I presume it'd still issue the resource warning, just not the "loop is closed" message.

[dpr-0]

@gvanrossum I would imagine like this

def __del__(self, warnings=warnings):
    if not self._transport.is_closing():
        try:
            self.close()
        except RuntimeError:
            warnings.warn(f"loop is closed", ResourceWarning)
        else:
            warnings.warn(f"unclosed {self!r}", ResourceWarning)

If this one ok, I can submit a PR for this issues

[gvanrossum]

Sure, let’s give that a try as a PR.

Maybe separately it would be nice to gc.collect() in loop.close(). That’s a larger discussion though.

[RoTorEx]

Did anyone find a solution?

[gvanrossum]

Did anyone find a solution?

It's fixed in the main, 3.12 and 3.11 branches. It will be fixed in the next official releases of 3.11 and 3.12 from python.org (within 1-2 months).

[LostInDarkMath]

Did anyone find a solution?

It's fixed in the main, 3.12 and 3.11 branches. It will be fixed in the next official releases of 3.11 and 3.12 from python.org (within 1-2 months).

Is this fix included in 3.11.7?
Because I'm still able to reproduce it with Python 3.11.7

Edit: nevermind, I used 3.11.6 by accident. All good