[Feature request] More specific exception for connection refused

[gnachman]

I would like to retry making a connection when the remote is not listening on the port. This is a common use case for iTerm2's Python API, where scripts would like to launch the app and then connect to it once it has started.

Currently, when I do websockets.connect() and the remote isn't listening on the port it fails with OSError with arguments of Multiple exceptions: [Errno 61] Connect call failed ('::1', 1912, 0, 0), [Errno 61] Connect call failed ('127.0.0.1', 1912)

My current approach is to catch OSError and assume it is a connection failure that should be retried, but I'm sure there are other possible causes of OSError.

I see that websockets defines many exceptions: https://websockets.readthedocs.io/en/stable/api.html#websockets.exceptions.ConnectionClosed

My request is to add websockets.exceptions.ConnectionRefused(remotes), where remotes could hold the list of (ip,port) tuples.

[aaugustin]

OK, I understand the request.

The exception you're seeing bubbles from asyncio; websockets doesn't have access to any additional information. So, whatever websockets could do to tell this particular OSError apart from other causes of OSError, you could also do right now.

I'd like to try to reproduce, examine the exception, and see if there's something I can do. (Unfortunately I'm quite busy right now and haven't been able to make time for websockets for a few weeks.)

Also, just curious — what's the use case for websockets in iTerm2 and how did you end up selecting my library?

[aaugustin]

I forgot to mention I'm a very happy iTerm2 user, thank you :-)

[gnachman]

Hey Aymeric, sorry for being slow to get back to you.

Version 3.3 of iTerm2 (currently in beta) offers a Python API. Programs using the API run in separate processes and communicate over a websocket. General information about the new API is here:
https://iterm2.com/python-api/

To reproduce, ensure you do not have iTerm2 version 3.3 running (since the point is that it will fail to connect), and run this program:

1. Do pip3 install iterm2
2. Run this program:

import iterm2
async def main(): pass
iterm2.run_until_complete(main)

The last call tries to open the websocket. It will fail like this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/iterm2/connection.py", line 263, in run_until_complete
    Connection().run_until_complete(coro)
  File "/usr/local/lib/python3.7/site-packages/iterm2/connection.py", line 101, in run_until_complete
    self.run(False, coro)
  File "/usr/local/lib/python3.7/site-packages/iterm2/connection.py", line 169, in run
    loop.run_until_complete(self.async_connect(async_main))
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/site-packages/iterm2/connection.py", line 248, in async_connect
    async with websockets.connect(_uri(), extra_headers=_headers(), subprotocols=_subprotocols()) as websocket:
  File "/usr/local/lib/python3.7/site-packages/websockets/py35/client.py", line 2, in __aenter__
    return await self
  File "/usr/local/lib/python3.7/site-packages/websockets/py35/client.py", line 12, in __await_impl__
    transport, protocol = await self._creating_connection
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 968, in create_connection
    ', '.join(str(exc) for exc in exceptions)))
OSError: Multiple exceptions: [Errno 61] Connect call failed ('::1', 1912, 0, 0), [Errno 61] Connect call failed ('127.0.0.1', 1912)

I would like to modify the iterm2 to module to discover these failures and retry periodically. This will enable the user to launch the app and then connect after it's running.

I ended up selecting your library because it used asyncio, which I had already decided to use, and that it just worked and made sense and was well documented. Also, it's the top search result for python asyncio websocket :)

[aaugustin]

I investigated and I don't think we'll have a very good solution.

The exception we're discussing here is:

>>> exc
OSError("Multiple exceptions: [Errno 61] Connect call failed ('::1', 1912, 0, 0), [Errno 61] Connect call failed ('127.0.0.1', 1912)")

I'm not finding any way to get to the two underlying exceptions:

>>> dir(exc)
['__cause__', '__class__', '__context__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__suppress_context__', '__traceback__', 'args', 'characters_written', 'errno', 'filename', 'filename2', 'strerror', 'with_traceback']
>>> exc.__cause__
>>> exc.__context__
>>> exc.__dict__
{}
>>> repr(exc)
'OSError("Multiple exceptions: [Errno 61] Connect call failed (\'::1\', 1912, 0, 0), [Errno 61] Connect call failed (\'127.0.0.1\', 1912)")'
>>> str(exc)
"Multiple exceptions: [Errno 61] Connect call failed ('::1', 1912, 0, 0), [Errno 61] Connect call failed ('127.0.0.1', 1912)"
>>> exc.args
("Multiple exceptions: [Errno 61] Connect call failed ('::1', 1912, 0, 0), [Errno 61] Connect call failed ('127.0.0.1', 1912)",)
>>> exc.errno
>>> exc.filename
>>> exc.filename2
>>> exc.strerror
>>> 

This is really https://bugs.python.org/issue29980. As you can see here the underlying exceptions are lost; you just get a string representation.

So the best I have to offer is this:

>>> "Connect call failed" in str(exc)
True
>>> "Errno 61" in str(exc)
True

Obviously I wish there was a better solution but I don't think there is one.

Also I don't think it's reasonable for websockets to paper over this issue. It isn't going to do anything other than reraise an exception and the original exception is clear enough. In such cases I prefer keeping things simple.

[gnachman]

Thanks for looking in to it. I agree that websockets should not try to fix this bug. I think I'll stick with treating all OSError's as retriable. I'm not sure checking strings is safe because it will be brittle when Python changes.

[naikymen]

Hi! I am having the same issue, thanks for sharing. :)

I just worked around it by passing an IPv4 address (0.0.0.0) instead of a name (localhost) to connect. In this way it only fails once.

• Before: OSError: Multiple exceptions: [Errno 111] Connect call failed ('::1', 7125, 0, 0), [Errno 111] Connect call failed ('127.0.0.1', 7125)
• After: ConnectionRefusedError: [Errno 111] Connect call failed ('0.0.0.0', 7125)

Best and thanks again!