Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ctrl-C support #71

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

dferens
Copy link
Contributor

@dferens dferens commented Nov 20, 2020

Description

Code to replicate issue:

(run and press Ctrl+C)

import asyncio
import time
import aiomultiprocess

async def _main():
    # not reproducible with small number of processes
    async with aiomultiprocess.Pool(20):
        time.sleep(99999)

if __name__ == '__main__':
    try:
        asyncio.run(_main())
    except KeyboardInterrupt:
        pass

You will see a lot of stack traces like:

Traceback (most recent call last):
  File "/Users/user/venv/lib/python3.8/site-packages/aiomultiprocess/core.py", line 143, in run_async
    result: R = loop.run_until_complete(unit.target(*unit.args, **unit.kwargs))
  File "/Users/user/.pyenv/versions/3.8.5/lib/pyaio process 33621 failed
Traceback (most recent call last):
  File "/Users/user/venv/lib/python3.8/site-packages/aiomultiprocess/core.py", line 143, in run_async
    result: R = loop.run_until_complete(unit.target(*unit.args, **unit.kwargs))
  File "/Users/user/.pyenv/versions/3.8.5/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/Users/user/.pyenv/versions/3.8.5/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/Users/user/.pyenv/versions/3.8.5/lib/python3.8/asyncio/base_events.py", line 1823, in _run_once
    event_list = self._selector.select(timeout)
  File "/Users/user/.pyenv/versions/3.8.5/lib/python3.8/selectors.py", line 558, in select
    kev_list = self._selector.control(None, max_ev, timeout)
KeyboardInterrupt
aio process 33627 failed

This PR is supposed to solve this problem.

@amyreese
Copy link
Member

amyreese commented Dec 7, 2020

Keyboard interrupt is an unfortunately complicated and nuanced problem. I think I'd prefer to see this handled using signal handlers to (IMO) ignore keyboard interrupt in the child processes (or at the very least in PoolWorker), and then allow the parent process to catch and gracefully request the child processes be terminated. Eg, for process pools, if the parent process defines a keyboard interrupt handler to gracefully stop their process, the last thing they would want is the child processes to terminate all pending work and abort immediately. As-is, this behavior just causes the parent process to hang forever waiting for results that the child processes will never deliver because they aborted. If you don't have the time to rework this, I have an idea of how I'd like to handle it, and can probably put that together soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants