websocket.receive_bytes() AssertionError

[hughsw]

We have a FastApi/uvicorn websockets service. During testing we can routinely (but not deterministically) trigger an AssertError from websockets.receive_bytes():

Synchrony: 7.45: Synchrony.edge: Exception: AssertionError: error:
Traceback (most recent call last):
  File "/app/synchrony.py", line 190, in edge
    if await call() is None: break
  File "/app/synchrony.py", line 170, in recv
    payload = await websocket.receive_bytes()
  File "/usr/local/lib/python3.9/site-packages/starlette/websockets.py", line 89, in receive_bytes
    assert self.application_state == WebSocketState.CONNECTED
AssertionError

This happens (sometimes) during testing when we $ kill <clientPID> an active client that is using the service. That is, when the connection to the service is abruptly terminated. When we stand up the same service behind PyPi websockets we do not see such a failure.

For each connection there are multiple async tasks running. Another task is doing await websocket.close(), which triggers a RuntimeError in deep in the starlette async close code, prior to (or concurrent with) the above failure. Documentation is hard to find, so it's not clear that starlette websockets promise an idempotent close() method. Nonetheless, AssertionError, by definition, means there's a code issue...

If there is interest I can post other (large) related stack traces.

Thanks.

[hughsw]

Here's a couple more observations:

• The AssertionError becomes more elusive as I turn on debugging logic -- strong evidence for a race condition
• We have not seen the AssertionError if only a single connection is open -- that is, we've only seen the problem when multiple connections are open, and one or more of them is abruptly terminated (via $ kill <ws client>)

Also, the related, non-AssertionError, stacktraces suggest that PyPi websockets is running deep inside -- so, (my ignorance, ongoing education here) perhaps starlette/uvicorn uses a different asycio backend rather than a different websockets implementation....

[hughsw]

We are also seeing this in the field, sometimes, when a browser-based client closes the websocket -- exactly the same call stack: that is, an AssertionError from await websocket.receive_bytes() in the server when the websocket has been closed by the (Javascript) client.

[lovelydinosaur]

See also #1471

[lovelydinosaur]

Right - this ought to be in a somewhat better state after #1472 - You'll at least be getting RuntimeError rather than an assert - and the messaging will inform you more clearly what ASGI message was expected vs. what it actually got.

[lovelydinosaur]

Given that this is now in a better state since #1472 I'm going to close this off for now.

Happy to dig into it further if anyone's still got issues with it.