Can the recv() non blocking?

[delphikwok]

Hello, can the recv() return a None or empty string when no incoming message pending?

[aaugustin]

With the current implementation, the following function will work:

def recv_nowait(websocket):
        try:
            return websocket.messages.get_nowait()
        except asyncio.queues.QueueEmpty:
            pass

Could you elaborate about your use case?

I'm unsure adding that to the API will encourage users to take advantage of asyncio correctly.

However the precedent of Queue.get / Queue.get_nowait suggests it could be appropriate.

[aaugustin]

I'm going to close this based on the lack of feedback, but I'm open to reconsidering the need for a public API here.

If someone else finds this idea interesting, please comment on this issue and let's discuss.

[djmarcin]

I think I want something like this. In my case, I am ingesting a data feed and only want to evaluate the state of the world once all pending updates have been applied, so I want to know when the queue is empty.

Something like this:

while True:
  process(await ws.recv())
  try:
    while True:
      process(ws.recv_nowait())
  except websocket.QueueEmpty:
    pass

Although it would be nice if this wasn't quite so verbose.

edit -- The point of the above code is to prevent yielding to other coroutines before all queued messages are processed. It's entirely possible that this is already how await ws.recv() in a loop works but I just don't know enough about asyncio to be sure.

edit2 -- It does appear from a simple test that asyncio.Queue works such that if await queue.get() does not block, execution will not yield to other coroutines. I guess this makes sense it just wasn't immediately obvious it had to be true. I guess I don't need recv_nowait() after all... :)

[aaugustin]

Hello @djmarcin, I'm skeptical of the robustness of the code you're describing. It suggests you haven't fully embraced the nature of async code.

What about the situation where some data is being processed by the NIC, or is sitting in the OS buffers, or is in the asyncio buffers, but hasn't been parsed by websockets yet? Can you really consider that the queue is empty if data made it to your computer?

At what point to you want to evaluate the state of the world? After 1 nanosecond without data? 1 microsecond? 1 millisecond? 1 second?

---

Technically, the current implementation of ws.recv() won't yield control if there's a message available in the queue, see https://github.com/aaugustin/websockets/blob/d4a6a24fcf9786b1917668bfaee946e85bc22478/websockets/protocol.py#L294-L298

However it feels weird to rely on await <something> not yielding control. yield is the marker that you can yield control in asyncio-based applications!

If you're willing to rely on implementation details, I think you should hit the ws.messages queue directly:

while True:
    try:
        message = ws.messages.get_nowait()  # private API /!\
    except asyncio.queues.QueueEmpty:
        break
    else:
        process(message)

# queue is empty, evaluate the state of the world here

If something changes in websockets, your code will break loudly.

If you were relying on the current implementation of recv(), it could change and break your code in much more subtle and hard-to-detect ways.

[Randelung]

Hello!

I found this issue after a while of digging and since you asked for input I'll post this here. Since it's a three year old thread I'm sure lots has changed, and if you want me to move to a new issue, I understand.

I'm writing a server that does other things while waiting for unsolicited messages on the websocket. Currently I'm wait_foring for 10ms to see if there is data in the message queue at this very moment and, if not, yield the thread using asyncio.sleep(0). It goes in the same category as expecting None instead of blocking the call.
Since the message queue is there, would it be possible to add a data_available() bool or int to check first? I'd like to avoid the timer if possible, and using private API is not a good idea, as you already mentioned. :)

Also, I'm wondering if returning None or an empty string/list wouldn't be more suitable for an async/await approach, since waiting is exactly the problem async/await is trying to solve - yield the thread if there's nothing to do.
I do realize that that would be a breaking change for most. An optional parameter (blocking = True) would work, though.

Cheers!

[lgrahl]

This can be solved in several ways with asyncio tools:

• Use separate async tasks that run concurrently. One receives, one does whatever other work there is.
• Use a waiting primitive such as asyncio.wait.

[Randelung]

I didn't realize it was that simple. I saw this stackoverflow post and thought I'd have to spawn another thread, anyway. I've refactored to use the async for loop in its own little task and was able to send messages and await other stuff without incoming traffic, so it works as you say. That is of course much more convenient than checking for data each time.

Thanks!