subprocess - uncaught PermissionError in send_signal can cause hang

[wesinator]

BPO	37091
Nosy	@gpshead, @wesinator, @iritkatriel
PRs	bpo-37091: subprocess - Handle uncaught PermissionError in send_signal #13669
Superseder	bpo-37424: subprocess.run timeout does not function if shell=True and capture_output=True

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gpshead'
closed_at = <Date 2022-01-30.16:51:48.774>
created_at = <Date 2019-05-29.22:52:02.516>
labels = ['3.7', '3.8', 'type-bug', 'library']
title = 'subprocess - uncaught PermissionError in send_signal can cause hang'
updated_at = <Date 2022-01-30.16:51:48.774>
user = 'https://github.com/wesinator'

bugs.python.org fields:

activity = <Date 2022-01-30.16:51:48.774>
actor = 'iritkatriel'
assignee = 'gregory.p.smith'
closed = True
closed_date = <Date 2022-01-30.16:51:48.774>
closer = 'iritkatriel'
components = ['Library (Lib)']
creation = <Date 2019-05-29.22:52:02.516>
creator = 'wesinator'
dependencies = []
files = []
hgrepos = []
issue_num = 37091
keywords = []
message_count = 4.0
messages = ['343939', '344014', '345061', '411015']
nosy_count = 3.0
nosy_names = ['gregory.p.smith', 'wesinator', 'iritkatriel']
pr_nums = ['13669']
priority = 'low'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '37424'
type = 'behavior'
url = 'https://bugs.python.org/issue37091'
versions = ['Python 3.7', 'Python 3.8']

[wesinator]

Python 3.7.3
Ubuntu 18.10 Cosmic

#13669

Encountered a condition where uncaught PermissionError caused a hang
running a subprocess command with sudo -u

Traceback (most recent call last):
  File "/usr/lib/python3.7/subprocess.py", line 474, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/lib/python3.7/subprocess.py", line 939, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.7/subprocess.py", line 1707, in _communicate
    self.wait(timeout=self._remaining_time(endtime))
  File "/usr/lib/python3.7/subprocess.py", line 990, in wait
    return self._wait(timeout=timeout)
  File "/usr/lib/python3.7/subprocess.py", line 1616, in _wait
    raise TimeoutExpired(self.args, timeout)
subprocess.TimeoutExpired: Command '['sudo', '-u', 'chrome', '/snap/bin/chromium', '--headless', '--disable-gpu', '--hide-scrollbars', '--ignore-certificate-errors', '--enable-sandbox', '--incognito', '--mute-audio', '--disable-databases', '--enable-strict-powerful-feature-restrictions', '--no-pings', '--no-referrers', '--timeout=30000', '--window-size=1280,1000', '--screenshot=[REDACTED]_screenshot.png', 'https://[REDACTED]']' timed out after 59.9998986274004 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/subprocess.py", line 476, in run
    process.kill()
  File "/usr/lib/python3.7/subprocess.py", line 1756, in kill
    self.send_signal(signal.SIGKILL)
  File "/usr/lib/python3.7/subprocess.py", line 1746, in send_signal
    os.kill(self.pid, sig)
PermissionError: [Errno 1] Operation not permitted

[gpshead]

Clarifying: The problem is that the TimeoutExpired exception that is desired was replaced with a PermissionError that the caller isn't expecting to catch and handle.

[gpshead]

even with that initially proposed patch, the subprocess.run code is going to hang if the .kill() fails without raising an exception:

The run() TimeoutExpired path calls communicate() after the kill to consume all remaining output:
https://github.com/python/cpython/blob/master/Lib/subprocess.py#L478

and the __exit__ code path calls wait().

both assume that the kill() succeeded because the normal case is not someone launching a child process that they are unable to kill (as happens when sudo or similar is used).

Today's workarounds: Don't use timeout when launching a child you cannot kill. OR launch it via an intermediate process that you can kill (shell=True or equivalent).

To address this issue, catching a PermissionError exception around both of those kill() calls and preventing it from calling either communicate() or wait() as we cannot force the child process to exit makes sense. With some indication that this happened left as a hint to the user on the TimeoutExpired itself.

[iritkatriel]

I think this was fixed in bpo-37424.