StreamWriter.wait_closed() can hang indefinitely.

[tomchristie]

BPO	39758
Nosy	@warsaw, @bitdancer, @asvetlov, @tomchristie, @1st1, @JulienPalard, @lysnikolaou, @aeros, @r-owen, @kumaraditya303

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-02-26.10:36:06.728>
labels = ['3.11', 'type-bug', 'expert-asyncio']
title = 'StreamWriter.wait_closed() can hang indefinitely.'
updated_at = <Date 2022-03-03.11:14:26.540>
user = 'https://github.com/tomchristie'

bugs.python.org fields:

activity = <Date 2022-03-03.11:14:26.540>
actor = 'kumaraditya'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2020-02-26.10:36:06.728>
creator = 'tomchristie'
dependencies = []
files = []
hgrepos = []
issue_num = 39758
keywords = []
message_count = 6.0
messages = ['362688', '362738', '387644', '387645', '400863', '414423']
nosy_count = 12.0
nosy_names = ['barry', 'r.david.murray', 'asvetlov', 'tomchristie', 'yselivanov', 'mdk', 'lys.nikolaou', 'aeros', 'Fran\xc3\xa7ois Voron', 'r3owen', 'kumaraditya', 'seer']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39758'
versions = ['Python 3.11']

[tomchristie]

Raising an issue that's impacting us on httpx.

It appears that in some cases SSL unwrapping can cause .wait_closed() to hang indefinately.

Trio are particularly careful to work around this case, and have an extensive comment on it: https://github.com/python-trio/trio/blob/31e2ae866ad549f1927d45ce073d4f0ea9f12419/trio/_ssl.py#L779-L829

Originally raised via encode/httpx#634

Tested on:

• Python 3.7.6
• Python 3.8.1

import asyncio
import ssl
import certifi

hostname = 'login.microsoftonline.com'
context = ssl.create_default_context()
context.load_verify_locations(cafile=certifi.where())

async def main():
    reader, writer = await asyncio.open_connection(hostname, 443, ssl=context)
    print('opened')
    writer.close()
    print('close started')
    await writer.wait_closed()
    print('close completed')

asyncio.run(main())

[JulienPalard]

Can reproduce in:

• 3.8.0
• 3.8.2
• 3.7.6

[r-owen]

I am also seeing this in Python 3.8.6. I am not using SSL, but am simply calling await writer.wait_closed() on an asyncio.StreamWriter. Sometimes it works quickly and sometimes it hangs indefinitely.

[r-owen]

Regarding my previous comment: I have never seen this in Python 3.7 (though I see that this particular bug is listed as being present there) so it may be a different underlying issue.

[seer]

See this issue on 3.9.6
No SSL, but plain sockets.
This seems to appear when writer.write/writer.drain was cancelled, and writer.close/writer.wait_closed called after this.

[kumaraditya303]

On main branch with rewritten SSL implementation #75458 bpo-44011, this raises TimeoutError:

--------------------------------------------------------------------------------
opened
close started

Traceback (most recent call last):
  File "/workspaces/cpython/main.py", line 18, in <module>
    asyncio.run(main())
    ^^^^^^^^^^^^^^^^^^^
  File "/workspaces/cpython/Lib/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/cpython/Lib/asyncio/base_events.py", line 645, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/workspaces/cpython/main.py", line 14, in main
    await writer.wait_closed()
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/cpython/Lib/asyncio/streams.py", line 344, in wait_closed
    await self._protocol._get_close_waiter(self)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: SSL shutdown timed out

---

This issue has been fixed, it can be closed @asvetlov.