TestSelector cannot find a previously opened file descriptor.

[toffan]

Hello @Martiusweb

First, thanks a lot for the library. We use it in my company to test our asynchronous library and it is so helpful. I encountered an issue using asynctest 0.13.0 with python 3.7.3. Here is the
description.

Issue

I want to use the same http session during all my tests. To do so I open it in setUpClass and then just keep it in my tests. However it results in a desynchronisation between TestSelector._fd_to_key and TestSelector._selector._fd_to_key because cls.loop._selector is different between setUpClass and a regular test (patched in TestCase._patch_loop).

In my example I use aiohttp to ease the reading but it is not meaningful here because the issue seems to be related to the underlying socket.

Steps to reproduce

The testcase I use.

# test.py
import aiohttp
import asyncio
import asynctest


class TestSSCCE(asynctest.TestCase):

    use_default_loop = True
    forbid_get_event_loop = False

    @classmethod
    def setUpClass(cls):
        cls.loop = asyncio.get_event_loop()
        cls.loop.run_until_complete(cls.initialize())

    @classmethod
    async def initialize(cls):
        cls.session = await aiohttp.ClientSession().__aenter__()
        await cls.session.get("http://172.17.0.2/")

    @classmethod
    def tearDownClass(cls):
        try:
            cls.loop.run_until_complete(
                cls.session.__aexit__(None, None, None)
            )
        finally:
            cls.loop.close()

    async def test(self):
        content = "A" * 256 * 1024
        async with self.session.get(
            "http://172.17.0.2/",
            data=content
        ) as resp:
            print(await resp.read())


if __name__ == "__main__":
    asynctest.main()

Here are the actual steps I use to reproduce the issue. Note that some parts have been eluded (virtualenv creation). If necessary I can provide a more complete list of steps.

$ # First run a counterpart web server
$ sudo docker run -ti --name webserver --rm nginx
$ # Get the IP address of the container. Note that it is the one used in test.py
$ sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}' webserver
172.17.0.2
$ # Install dependencies
$ pip install asynctest==0.13.0 aiohttp==3.5.4
$ python -V           
Python 3.7.3
$ python test.py
[...]
KeyError: '6 is not registered'
[...]
KeyError: '6 (FD 6) is already registered'

Below is the complete stack trace.

$ python test.py
E
======================================================================
ERROR: test (__main__.TestSSCCE)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.7/asyncio/selector_events.py", line 287, in _add_writer
    key = self._selector.get_key(fd)
  File "/usr/lib64/python3.7/selectors.py", line 192, in get_key
    raise KeyError("{!r} is not registered".format(fileobj)) from None
KeyError: '6 is not registered'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/asynctest/case.py", line 297, in run
    self._run_test_method(testMethod)
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/asynctest/case.py", line 354, in _run_test_method
    self.loop.run_until_complete(result)
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/asynctest/case.py", line 224, in wrapper
    return method(*args, **kwargs)
  File "/usr/lib64/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "test.py", line 34, in test
    data=content
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/aiohttp/client.py", line 1005, in __aenter__
    self._resp = await self._coro
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/aiohttp/client.py", line 497, in _request
    await resp.start(conn)
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/aiohttp/client_reqrep.py", line 844, in start
    message, payload = await self._protocol.read()  # type: ignore  # noqa
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/aiohttp/streams.py", line 588, in read
    await self._waiter
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/aiohttp/client_reqrep.py", line 553, in write_bytes
    await self.body.write(writer)
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/aiohttp/payload.py", line 231, in write
    await writer.write(self._value)
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/aiohttp/http_writer.py", line 101, in write
    self._write(chunk)
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/aiohttp/http_writer.py", line 68, in _write
    self._transport.write(chunk)
  File "/usr/lib64/python3.7/asyncio/selector_events.py", line 868, in write
    self._loop._add_writer(self._sock_fd, self._write_ready)
  File "/usr/lib64/python3.7/asyncio/selector_events.py", line 290, in _add_writer
    (None, handle))
  File "/home/toffan/.virtualenvs/test-asynctest/lib/python3.7/site-packages/asynctest/selector.py", line 241, in register
    key = self._selector.register(fileobj, events, data)
  File "/usr/lib64/python3.7/selectors.py", line 352, in register
    key = super().register(fileobj, events, data)
  File "/usr/lib64/python3.7/selectors.py", line 242, in register
    .format(fileobj, key.fd))
KeyError: '6 (FD 6) is already registered'

----------------------------------------------------------------------
Ran 1 test in 0.004s

FAILED (errors=1)

It has also been reproduced with python 3.5.7 or asynctest 0.12.4

Fix

I have taken a look at the issue and my first idea was to copy the _fd_to_key dict in TestSelector.__init__ (cf. below) . However it does not prevent a future desynchronization between the two. Typically, asyncio.BaseSelectorEventLoop._sock_connect defines a callback that unregister the file descriptor from the original selector and would leave TestSelector._fd_to_key untouched.

class TestSelector(selectors._BaseSelectorImpl):
    def __init__(self, selector=None):
        # [...]
        if selector is not None:
            self._fd_to_key = selector._fd_to_key.copy()

So I guess the fix may require some deeper change in the library. Would it be possible for you to take a look at it. Thanks a lot.

[t-soliduslink]

I ran into this issue, too.

It's highly surprising because the existence of use_default_loop suggests that it's a supported use case to make use of the event loop in setUpClass and tearDownClass.

Here is a simplified test case, simply based on a background child process:

#!/usr/bin/python3

import asyncio
import asynctest
import subprocess

IO = asyncio.get_event_loop()

class SomeTest (asynctest.TestCase):
    use_default_loop = True
    @classmethod
    def setUpClass(cls):
        print("setUpClass: starting child process")
        coro = asyncio.create_subprocess_shell(
            """
               while true; do
                 echo Foobar
                 sleep 0.5
                done
            """,
            stdout = subprocess.PIPE,
            stderr = subprocess.STDOUT)
        cls.proc = IO.run_until_complete(coro)


    @classmethod
    def tearDownClass(cls):
        print("tearDownClass: terminating the process")
        cls.proc.terminate()
        print("Reading process output until EOF when process terminated:")
        while True:
            line = IO.run_until_complete(cls.proc.stdout.readline())
            print(f"read: {line}")
            if len(line) == 0:
                break

    def test_it(self):
        print("Test method test_it: NOOP")


if __name__ == '__main__':
    asynctest.main(argv = ["foo_test.py"])

Producing:

setUpClass: starting child process
Test method test_it: NOOP
.tearDownClass: terminating the process
Reading process output until EOF when process terminated:
read: b'Foobar\n'
Exception in callback _UnixReadPipeTransport._read_ready()
handle: <Handle _UnixReadPipeTransport._read_ready()>
Traceback (most recent call last):
  File "/usr/lib/python3.7/asyncio/unix_events.py", line 479, in _read_ready
    data = os.read(self._fileno, self.max_size)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.7/asyncio/unix_events.py", line 483, in _read_ready
    self._fatal_error(exc, 'Fatal read error on pipe transport')
  File "/usr/lib/python3.7/asyncio/unix_events.py", line 526, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'
read: b''

----------------------------------------------------------------------
Ran 1 test in 0.515s

OK

[Martiusweb]

Thanks for the report.

Indeed this is a bug that @toffan's patch will fix (#133).