Add Linux's signalfd() to the signal module

[gpshead]

BPO	32443
Nosy	@gpshead, @pitrou, @njsmith

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-12-29.01:27:20.548>
labels = ['3.7', '3.8', 'type-feature', 'library']
title = "Add Linux's signalfd() to the signal module"
updated_at = <Date 2018-01-06.07:38:43.164>
user = 'https://github.com/gpshead'

bugs.python.org fields:

activity = <Date 2018-01-06.07:38:43.164>
actor = 'njs'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-12-29.01:27:20.548>
creator = 'gregory.p.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 32443
keywords = []
message_count = 6.0
messages = ['309151', '309152', '309166', '309167', '309196', '309229']
nosy_count = 3.0
nosy_names = ['gregory.p.smith', 'pitrou', 'njs']
pr_nums = []
priority = 'low'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue32443'
versions = ['Python 3.7', 'Python 3.8']

[gpshead]

We should add a wrapper for both signalfd() and a function to read and decode the structure from the fd into a dataclass.

The code we need to build such a thing from appears to exist in BSD & MIT licensed form in:

PyPI contains two extension module wrappers of just the system call:
https://pypi.python.org/pypi/signalfd/
https://pypi.python.org/pypi/python-signalfd

Why add this Linux specific API to the standard library? I believe I could use signalfd() within the subprocess module to get rid of the need to busy-loop-poll to see when children have exited on subprocess.Popen.wait() calls with a timeout. [a proof of concept using the above modules would be a good idea first]

[gpshead]

An example subprocess improvement using sigtimedwait() - #5035 - led me to believe that signalfd() is superior as it sounds like signalfd() does not require pthread_sigmask (sigprocmask) global state manipulation calls. I have not confirmed if that is true.

[pitrou]

See also https://bugs.python.org/issue12304

[pitrou]

Also see how the forkserver module does without it:
https://github.com/python/cpython/blob/master/Lib/multiprocessing/forkserver.py#L146

Reading Jean-Paul's messages in https://bugs.python.org/issue8407, I'm unclear what is required to take advantage of signalfd(). Jean-Paul says """In order to effectively use signalfd(), the signals in question must be blocked, though""". The signalfd man page says:

"""Normally, the set of signals to be received via the file descriptor should be blocked using sigprocmask(2), to prevent the signals being handled according to their default dispositions""".

And it's not clear what is meant by that (what happens if you don't block those signals?). Also:

"""As a consequence of the read(2), the signals are consumed, so that they are no longer pending for the process (i.e., will not be caught by signal handlers, and cannot be accepted using sigwaitinfo(2))."""

But how about the converse? i.e. can a signal be consumed first by a signal handler and the fd not written to at all?

Also see fork() semantics which might (or not) require special handling.

Also see https://bugs.python.org/issue31489 for an issue related to fork(), signals and fds.

[gpshead]

My reading of the Linux signalfd man page may be optimistic. :)

Regardless, it'd be nice to have it available in the stdlib so it could be used if deemed useful. I expect this to only ever be added by someone making use of it in another stdlib module.

As for what multiprocessing.forkserver does, the old manual signal handler and pipe trick is a reasonably well known one. But a forkserver is not safe to be started when threads exist. (unlike subprocess)

Signals are process global state, no thread compatible library can rightfully take ownership of a one.

[pitrou]

Regardless, it'd be nice to have it available in the stdlib so it could be used if deemed useful.

Agreed.

Signals are process global state, no thread compatible library can rightfully take ownership of a one.

But then is the signalfd() idea for subprocess doomed as well?