Use user shell in subprocess

[JanStuden]

BPO	24919
Nosy	@bitdancer, @eryksun
Superseder	bpo-16255: subrocess.Popen needs /bin/sh but Android only has /system/bin/sh

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-08-23.22:30:27.431>
created_at = <Date 2015-08-23.21:47:42.670>
labels = ['type-bug', 'library']
title = 'Use user shell in subprocess'
updated_at = <Date 2015-08-23.23:16:41.128>
user = 'https://bugs.python.org/JanStuden'

bugs.python.org fields:

activity = <Date 2015-08-23.23:16:41.128>
actor = 'eryksun'
assignee = 'none'
closed = True
closed_date = <Date 2015-08-23.22:30:27.431>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2015-08-23.21:47:42.670>
creator = 'Jan Studen\xc3\xbd'
dependencies = []
files = []
hgrepos = []
issue_num = 24919
keywords = []
message_count = 4.0
messages = ['249023', '249025', '249027', '249028']
nosy_count = 3.0
nosy_names = ['r.david.murray', 'eryksun', 'Jan Studen\xc3\xbd']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '16255'
type = 'behavior'
url = 'https://bugs.python.org/issue24919'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4', 'Python 3.5']

[JanStuden]

According to POSIX specification the pathname of user shell is stored in SHELL (environmental variable, see http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08) so I think that is good idea to use that pathname instead of hardcoded one.
Change is only in one line of subprocess package to use SHELL environmental variable and use hardcoded shell pathname as fallback.

lines 1431-1433

            if shell:
                args = ["/bin/sh", "-c"] + args
                if executable:

to

            if shell:
                args = [os.environ.get("SHELL","/bin/sh"), "-c"] + args
                if executable:

[bitdancer]

Thanks for the suggestion, but that would make programs using subprocess non-portable. There is an open issue to use the *default* shell instead of hard coding it (because the sh-alike is not at /bin/sh on, eg, Android), but using the user shell would break lots of programs. Your suggestion was brought up by a core dev (and rejected) in that issue (bpo-16255)

[eryksun]

I expect Popen's shell=True option to use the same shell as os.system. The POSIX spec for the system function requires running sh, as follows:

execl(<shell path>, "sh", "-c", command, (char *)0);

glibc uses "/bin/sh" for the shell path. Modifying the SHELL environment variable doesn't affect the behavior of os.system. I would be surprised if Popen's shell=True had been designed like that. Thankfully it's too late to change this since it could break existing code. ;-)

The story is different on Windows. The CRT's system() prefers the shell that's set in the ComSpec environment variable. So on Windows Popen uses os.environ.get("COMSPEC", "cmd.exe").

[eryksun]

Here's the rationale given for ignoring SHELL in POSIX system():

One reviewer suggested that an implementation of system() might
want to use an environment variable such as SHELL to determine
which command interpreter to use. The supposed implementation
would use the default command interpreter if the one specified
by the environment variable was not available. This would allow
a user, when using an application that prompts for command lines
to be processed using system(), to specify a different command
interpreter. Such an implementation is discouraged. If the
alternate command interpreter did not follow the command line
syntax specified in the Shell and Utilities volume of
POSIX.1-2008, then changing SHELL would render system()
non-conforming. This would affect applications that expected the
specified behavior from system(), and since the Shell and
Utilities volume of POSIX.1-2008 does not mention that SHELL
affects system(), the application would not know that it needed
to unset SHELL.