Refuse handlers if the child watcher has no loop attached

[vxgmichel]

This PR follows up on the discussion from issue #390:

• f348122 fixes the issue
• f7a5259 is a proposal and can easily be discarded

This has been tested against the example from issue #390.

[1st1]

Will push this before b2.

[1st1]

Would very much want this to land in 3.6. Please address the comments.

[1st1]

This is something new... Is there any reason you do this as part of this PR?

[vxgmichel]

Yes, otherwise a warning from f7a5259 is generated. Weirdly enough it only happens with the FastChildWatcher. There might be a better way to address this issue though.

[vxgmichel]

I just had a deeper look and we can get rid of this addition by fixing FastChildWatcher (using some SafeChildWatcher code):

diff --git a/asyncio/unix_events.py b/asyncio/unix_events.py
index 54cfc75..5ffa645 100644
--- a/asyncio/unix_events.py
+++ b/asyncio/unix_events.py
@@ -937,7 +937,15 @@ def _do_waitpid_all(self):
                 pid, status = os.waitpid(-1, os.WNOHANG)
             except ChildProcessError:
                 # No more child processes exist.
-                return
+                if not self._callbacks:
+                    return
+                # Some child processes are already reaped
+                # (may happen if waitpid() is called elsewhere).
+                pid = next(iter(self._callbacks))
+                returncode = 255
+                logger.warning(
+                    "Unknown child process pid %d, will report returncode 255",
+                    pid)
             else:
                 if pid == 0:
                     # A child process is still alive.

This also allows us to get rid of some complexity in test_sigchld_child_reaped_elsewhere:

diff --git a/tests/test_unix_events.py b/tests/test_unix_events.py
index 6cf4417..b4a6d80 100644
--- a/tests/test_unix_events.py
+++ b/tests/test_unix_events.py
@@ -1318,12 +1318,7 @@ def test_sigchld_child_reaped_elsewhere(self, m):
         with self.ignore_warnings:
             self.watcher._sig_chld()

-        if isinstance(self.watcher, asyncio.FastChildWatcher):
-            # here the FastChildWatche enters a deadlock
-            # (there is no way to prevent it)
-            self.assertFalse(callback.called)
-        else:
-            callback.assert_called_once_with(58, 255)
+        callback.assert_called_once_with(58, 255)

     @waitpid_mocks
     def test_sigchld_unknown_pid_during_registration(self, m):

All the tests run OK after those modifications. Let me know which solution you prefer.

[1st1]

Not so sure about this solution. Let's keep the current hack-ish code.

[1st1]

How about using with self.assertWarnsRegexp?

[vxgmichel]

Done, it's much cleaner now!

[1st1]

Can we add a test for this?

[vxgmichel]

Done!

[1st1]

@asvetlov Could you please help me with reviewing this PR?

[asvetlov]

Sorry for a long delay.
LGTM.
if win32 looks tricky but I can live with it.

[asvetlov]

Looks like I should update my pytest fixtures for pytest-aiohttp by explicitly attaching/detaching test loop.
It's really nice catch. By writing fixtures for aiohttp I totally forgot about subprocess quirks on UNIX.
Honestly I almost never run subprocesses in my asyncio-based code but raising an explicit error is much better than just hanging in worst case.
Casual usage without default loop disabling is not affected as I see.

[1st1]

@asvetlov Andrew, I'm on vacation, and I can't commit this myself right now. If you have time, could you please commit this pr (and merge it to cpython 3.5/3.6/default)?

[1st1]

Committed by hand. Thanks, Vincent!

[1st1]

See also http://bugs.python.org/issue28368