Application hanging after KeyboardInterrupt

[amackillop]

• uvloop version: 0.14.0
• Python version: 3.7.4
• Platform: Manjaro
• Can you reproduce the bug with PYTHONASYNCIODEBUG in env?: Yes
• Does uvloop behave differently from vanilla asyncio? How?: With vanilla, the application terminates gracefully upon reception of a keyboard interrupt. If I use uvloop, I get a runtime warning and the application hangs until I run a kill -9 on the process id.

Snippet: issue.py

import asyncio
from asyncio import AbstractEventLoop
import os
from typing import Union, Type

import uvloop  # type: ignore
from aiohttp import web
import signal

import aiologger

logger = aiologger.Logger.with_default_handlers()


async def handle_exception(loop: AbstractEventLoop, context):
    # context["message"] will always be there; but context["exception"] may not
    msg = context.get("exception", context["message"])
    await logger.error(f"Caught exception: {msg}")
    await logger.info("Shutting down...")
    asyncio.create_task(shutdown(loop))


async def shutdown(loop: AbstractEventLoop, signal=None):
    """Cleanup tasks tied to the service's shutdown."""
    if signal:
        await logger.info(f"Received exit signal {signal.name}...")

    tasks = [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]

    [task.cancel() for task in tasks]

    await logger.info(f"Cancelling {len(tasks)} outstanding tasks")
    await asyncio.gather(*tasks, return_exceptions=True)
    loop.stop()


async def start(
    app: web.Application, host: str, port: Union[str, int]
) -> web.AppRunner:
    """Start the server"""
    runner = web.AppRunner(app)
    await runner.setup()
    server = web.TCPSite(runner, host, int(port))
    await server.start()
    return runner


def main() -> None:
    """Entrypoint"""
    host = os.environ.get("HOST", "localhost")
    port = os.environ.get("PORT", 8000)
    app = web.Application()
    loop = asyncio.get_event_loop()
    signals = (signal.SIGHUP, signal.SIGTERM, signal.SIGINT)
    for s in signals:
        loop.add_signal_handler(
            s, lambda s=s: asyncio.create_task(shutdown(loop, signal=s))
        )
    loop.set_exception_handler(handle_exception)
    print(
        f"======== Running on http://{host}:{port} ========\n" "(Press CTRL+C to quit)"
    )
    try:
        runner = loop.run_until_complete(start(app, host, port))
        loop.run_forever()
    finally:
        loop.run_until_complete(runner.cleanup())
        loop.close()


if __name__ == "__main__":
    uvloop.install()
    main()

Run the file:
python issue.py

Then hit Ctrl+c on keyboard, output:

======== Running on http://0.0.0.0:8000 ========
(Press CTRL+C to quit)
^Cissue.py:70: RuntimeWarning: coroutine 'handle_exception' was never awaited
Coroutine created at (most recent call last)
  File "issue.py", line 78, in <module>
    main()
  File "issue.py", line 70, in main
    loop.run_forever()
  loop.run_forever()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

Which hangs until running kill -9 <pid>

If you comment out uvloop.install() and follow the same steps, the program terminates as expected. Output:

======== Running on http://0.0.0.0:8000 ========
(Press CTRL+C to quit)
^CReceived exit signal SIGINT...
Cancelling 0 outstanding tasks

[achimnol]

I believe that the exception handler for an asyncio event loop should be a normal function, not a coroutine function. The documentation says it's just a callable.

[achimnol]

Tried running your snippet, and I got many strange behaviors, such as haning or infinite loop of exception handlers, etc., even after changing the exception handler to just print messages as a non-coroutine function.
I think it's primarily because your are mixing loop.stop and asyncio.create_task during a single event loop tick, and the completion of those created tasks are not guaranteed because your code just waits for runner.cleanup().

[achimnol]

If you want to keep using aiologger, I'd suggest to use janus to mediate the synchronous loop exception handler and a separate logger coroutine task.
Maybe you might be interested at my boilerplate wrapper, aiotools.server.