AttributeError 'NoneType' object has no attribute 'fileno'.

[povilasb]

We somewhat discussed the issue at https://gitter.im/encode/community .

TL;DR when connection is closed/reset httpcore assumes socket is still available for transport and crashes with

AttributeError 'NoneType' object has no attribute 'fileno'.

Environment

Docker image: python:3.8.6-buster
Python: 3.8.6
httpcore: 0.12.2

Details

• The exact line of the code is here: https://github.com/encode/httpcore/blob/0.12.2/httpcore/_backends/asyncio.py#L181

    def is_readable(self) -> bool:
        transport = self.stream_reader._transport  # type: ignore
        sock: socket.socket = transport.get_extra_info("socket")
        return is_socket_readable(sock.fileno())

• If transport.get_extra_info("socket") returns None, sock.fileno() crashes.
• First they type of sock should probably be Optional[socket.socket].
• The python docs are not very detailed about the guarantees of extra["socket"] existence but in the example code they do test it: https://docs.python.org/3/library/asyncio-protocol.html#asyncio.BaseTransport.get_extra_info
• So as other code e.g. in uvicorn or aiohttp
• I wish I could provide a working example, but this only occasionally fails in my system.
• However I do have a stacktrace capture:

AttributeError 'NoneType' object has no attribute 'fileno'.

  File "httpx/_client.py", line 1371, in request
    response = await self.send(
  File "httpx/_client.py", line 1406, in send
    response = await self._send_handling_auth(
  File "httpx/_client.py", line 1444, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "httpx/_client.py", line 1476, in _send_handling_redirects
    response = await self._send_single_request(request, timeout)
  File "httpx/_client.py", line 1502, in _send_single_request
    (status_code, headers, stream, ext,) = await transport.arequest(
  File "httpcore/_async/http_proxy.py", line 124, in arequest
    return await self._tunnel_request(
  File "httpcore/_async/http_proxy.py", line 190, in _tunnel_request
    connection = await self._get_connection_from_pool(origin)
  File "httpcore/_async/connection_pool.py", line 248, in _get_connection_from_pool
    if connection.is_socket_readable():
  File "httpcore/_async/connection.py", line 169, in is_socket_readable
    return self.connection is not None and self.connection.is_socket_readable()
  File "httpcore/_async/http11.py", line 205, in is_socket_readable
    return self.socket.is_readable()
  File "httpcore/_backends/asyncio.py", line 179, in is_readable
    return is_socket_readable(sock.fileno())

• Was going through asyncio implementation trying to locate where the extra["socket"] gets cleared, but couldn't find anything related 🤔 For reference it is set here

Anyway, even though I do not fully understand asyncio transports, based on other examples I guess it is pretty reasonable to test if get_exra_info("socket") is not None?

[florimondmanca]

I guess we should return True if there's no socket retuned... ?

[povilasb]

I was going to say False :)
meaning if socket not available, connection not readable 🤔

[florimondmanca]

@povilasb Actually that's counter-intuitive, but "socket is readable" is actually a good proxy for "the connection was dropped". Think what happens when we do sock.recv(): it blocks until the server sends some non-empty data (so it's not readable until there's data in the read buffer). When the server has disconnected and the connection is essentially broken, you'll see that sock.recv() returns immediately with b"" — the socket is readable at all times. So we're using readability as a proxy to detect broken connections; and if the transport has no socket, then we definitely should treat this as if the connection was broken, hence as if the socket (if it was there) was readable. Hope that makes sense. :)

[povilasb]

It does, thank you :)

Will make a PR tomorrow.