subprocess may incorrectly redirect a low fd to stderr if another low fd is closed

[izbyshev]

BPO	32844
Nosy	@gpshead, @izbyshev, @miss-islington
PRs	bpo-32844: subprocess: Fix a potential misredirection of a low fd to stderr #5689 [3.7] bpo-32844: Fix a subprocess misredirection of a low fd (GH5689) #6262 [3.6] bpo-32844: Fix a subprocess misredirection of a low fd (GH5689) #6263

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-03-26.21:03:05.987>
created_at = <Date 2018-02-14.21:10:48.727>
labels = ['extension-modules', 'type-bug', 'library', '3.7']
title = 'subprocess may incorrectly redirect a low fd to stderr if another low fd is closed'
updated_at = <Date 2018-03-26.21:03:05.986>
user = 'https://github.com/izbyshev'

bugs.python.org fields:

activity = <Date 2018-03-26.21:03:05.986>
actor = 'izbyshev'
assignee = 'none'
closed = True
closed_date = <Date 2018-03-26.21:03:05.987>
closer = 'izbyshev'
components = ['Extension Modules', 'Library (Lib)']
creation = <Date 2018-02-14.21:10:48.727>
creator = 'izbyshev'
dependencies = []
files = []
hgrepos = []
issue_num = 32844
keywords = ['patch']
message_count = 5.0
messages = ['312181', '312182', '314478', '314479', '314481']
nosy_count = 3.0
nosy_names = ['gregory.p.smith', 'izbyshev', 'miss-islington']
pr_nums = ['5689', '6262', '6263']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32844'
versions = ['Python 3.6', 'Python 3.7']

[izbyshev]

When redirecting, subprocess attempts to achieve the following state: each fd to be redirected to is less than or equal to the fd it is redirected from, which is necessary because redirection occurs in the ascending order of destination descriptors. It fails to do so if a low fd (< 2) is redirected to stderr and another low fd is closed, which may lead to an incorrect redirection, for example:

$ cat test.py
import os
import subprocess
import sys

os.close(0)

subprocess.call([sys.executable, '-c',
'import sys; print("Hello", file=sys.stderr)'],
stdin=2,
stderr=1)

$ python3 test.py 2>/dev/null
$ python3 test.py >/dev/null
Hello

Expected behavior:
$ python3 test.py >/dev/null
$ python3 test.py 2>/dev/null
Hello

[izbyshev]

Note that the PR doesn't attempt to fix leaking of low dup'ed fds to the child. I'll file a separate report for that in a while.

[gpshead]

New changeset 0e7144b by Gregory P. Smith (Alexey Izbyshev) in branch 'master':
bpo-32844: Fix a subprocess misredirection of a low fd (GH5689)
0e7144b

[miss-islington]

New changeset 0545563 by Miss Islington (bot) in branch '3.7':
bpo-32844: Fix a subprocess misredirection of a low fd (GH5689)
0545563

[miss-islington]

New changeset 57db13e by Miss Islington (bot) in branch '3.6':
bpo-32844: Fix a subprocess misredirection of a low fd (GH5689)
57db13e