Common Client Connection Keep-alive Pattern

[ssshah86]

Hi, fantastic library! Using Python 3.7.2 and looking ahead to support 3.8 for the community as well.

Problem
The suggested code, while True to keep the websocket client open does not yield back to the main async loop. I believe there are a few folks struggling with that. https://websockets.readthedocs.io/en/stable/intro.html#basic-example

Additional issue threads (mostly closed, in order of relevance)
#83, #398, #527, #565, #534, #559

Solutions (6 of them)

• Use loop.run_forever(); will require calling stop() to cease
• Use serve_forever()
• Run as a server, can still use recv and send commands, use the while True method as in the documentation; can use websockets.serve; this obviously has its weaknesses
• Use Futures.add_callback_done to add a new task, task can check if an unsubscribe property is True
• Schedule two identical tasks, as FIRST_COMPLETED completes, add a new task, seems sloppy. src
• Close from another coroutine/task by getting the loop and cancelling, see How would i close a websockets connection while its stuck awaiting in websocket.recv() #527

Important notes
• 3.7 recommends using asyncio.run() as the primary entrance for the loop, and only calling it once
• using async with as the context manager forces a connection close as exiting the block; is there an elegant way to keep it open outside the block? I don't believe so...but I am not sure
• async for is no longer suggested

Since this issue seems to be popping up repeatedly, I'm happy to contribute to the documentation if we can find an acceptable solution. Thanks for your help in advance!

[mattrunchey]

Not sure if this helps, but I also struggled with using a bare while true from the docs in a class structure. This is the pattern I've been using for keeping a connection alive in a class object is to use ensure_future with while True that awaits the connect. Snippet of code that accomplishes this:

import EventLoop


class AsyncWSClient:

  def __init__(self, url):
    self.con = None
    self.url = url
    self.loop = EventLoop.get_loop()
    asyncio.ensure_future(self.connection_object(), loop=self.loop)
    asyncio.ensure_future(self.handle_message(), loop=self.loop)

  async def connection_object(self):
    self.con = await websockets.connect(self.url)


  async def handle_message(self):
      while not self.con:
          await asyncio.sleep(1)
      try:
          while True:
              message = await self.con.recv()

I use this client in an event loop that is thrown to a background thread (I throw a bunch of other coroutines back there too, so the main thread is unblocked... nice for testing):

class EventLoop:

  def start(cls):
    cls.loop = asyncio.new_event_loop()
    threading.Thread(target=cls._loop_thread_main, name="EventLoopThread", daemon=True)
 
  def get_loop(cls):
    if not cls.loop:
      cls.start()
    return cls.loop()

  def cls._loop_thread_main(cls):
    try:
      cls.loop.run_forever()
    except:
      pass #usually KBinterrupt
    finally:
      cls.stop_loop_clean()

  def cls.stop_loop_clean(cls):
    tasks = asyncio.Task.all_tasks(loop=cls.loop)
    cls.loop.call_soon_threadsafe(cls.loop.create_task, cls._clean_tasks(tasks))
    cls.thread.join() 
    cls.loop.stop()

  async def _clean_tasks(cls, tasks):
    remainder = []
    for task in tasks:
      cls._logger.info("Cancelling task %s" % task)
      task.cancel()
      remainder.append(task)
    await asyncio.gather(*remainder, return_exceptions=True)
    cls.loop.stop()

As long as my main thread invokes and starts the EventLoop correctly (this happens in another part of the program), the client connection gets instantiated and services messages until the main program is finished or closed with Ctrl+C.

[ssshah86]

Thanks! I will try this and get back to this thread. If I understand it correctly I can also add appropriate documentation

[aaugustin]

The suggested code, while True to keep the websocket client open does not yield back to the main async loop.

I don't understand this.

while True:
    await ...

definitely yields to the event loop.

Can you clarify what code sample you're referring to?

[aaugustin]

I read this issue again today and I haven't been able to nail down what it's about.

The ability to keep a connection alive is built-in (and enabled by default [and slightly buggy]) in websockets 7.0.

There's documentation about how to shutdown a server properly here: https://websockets.readthedocs.io/en/stable/deployment.html#graceful-shutdown

[aaugustin]

#534 might be related too.