With forkserver, Process.exitcode does not get signal number

[pitrou]

BPO	30589
Nosy	@pitrou, @applio
PRs	Fix bpo-30589: improve Process.exitcode with forkserver #1989

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-06-12.13:29:17.846>
created_at = <Date 2017-06-07.15:21:05.658>
labels = ['3.7', 'type-bug', 'library']
title = 'With forkserver, Process.exitcode does not get signal number'
updated_at = <Date 2017-06-30.08:37:05.015>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2017-06-30.08:37:05.015>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2017-06-12.13:29:17.846>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2017-06-07.15:21:05.658>
creator = 'pitrou'
dependencies = []
files = []
hgrepos = []
issue_num = 30589
keywords = []
message_count = 4.0
messages = ['295343', '295778', '295779', '297360']
nosy_count = 3.0
nosy_names = ['pitrou', 'sbt', 'davin']
pr_nums = ['1989']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30589'
versions = ['Python 3.7']

[pitrou]

The documentation for multiprocessing.exitcode says:
"""
The child’s exit code. This will be None if the process has not yet terminated. A negative value -N indicates that the child was terminated by signal N.
"""

This is true for the "fork" method, but not "forkserver" where a child terminated by a signal will get an exitcode of 255. This is because forkserver relies on the child writing its own exit code in a pipe, which obviously doesn't work if it was killed (255 is simply a fallback value).

See forkserver's Popen.poll():

    def poll(self, flag=os.WNOHANG):
        if self.returncode is None:
            from multiprocessing.connection import wait
            timeout = 0 if flag == os.WNOHANG else None
            if not wait([self.sentinel], timeout):
                return None
            try:
                self.returncode = forkserver.read_unsigned(self.sentinel)
            except (OSError, EOFError):
                # The process ended abnormally perhaps because of a signal
                self.returncode = 255
        return self.returncode

[pitrou]

New changeset dfd5f34 by Antoine Pitrou in branch 'master':
Fix bpo-30589: improve Process.exitcode with forkserver (bpo-1989)
dfd5f34

[pitrou]

I've merged a fix for Python 3.7. Since the fix is a bit delicate, I don't want to risk regression by merging it into 3.6 and 3.5. Closing now.

[pitrou]

In the end, I'm glad I added a stress test (test_many_processes) as part of this issue.

It helper uncover a serious reliability issues in CPython's delivery of signals (https://bugs.python.org/issue30703) and then triggered the discovery of a more minor bug in our setitimer() wrapper (https://bugs.python.org/issue30807).

Hopefully signal processing is more reliable in Python now!