File descriptors given to pass_fds should have close-on-exec flag unset

[jstewmon]

Here's a script to reproduce the issue:

import os
import sys

if os.name == 'posix' and sys.version_info[0] < 3:
    import subprocess32 as subprocess
else:
    import subprocess

p1 = subprocess.Popen(['echo', 'hi'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['echo', 'bye'], stdout=subprocess.PIPE)

# uncommenting these lines will produce expected result in python2.7
# subprocess._set_cloexec(p1.stdout.fileno(), False)
# subprocess._set_cloexec(p2.stdout.fileno(), False)

subs = [p1, p2]

fds = [i.stdout.fileno() for i in subs]
args = ['/dev/fd/{}'.format(fd) for fd in fds]

p2 = subprocess.Popen(
    ['cat'] + args,
    pass_fds=fds
)

python 3.5 output:

hi
bye

python 2.7 output:

cat: /dev/fd/3: No such file or directory
cat: /dev/fd/4: No such file or directory

My C isn't very good, but I traced through the _posixsubprocess.c and fileutils.c source on the 2.7 tag of cpython sources on GitHub, and it looks like it should be calling ioctl(fd, FIONCLEX, NULL) for each fd in pass_fds, so I haven't been able to figure out why this isn't working in python 2.7.

[nirs]

I think the suggested behavior makes sense, and it should not be risky since it does not make sense to pass fds with close-on-exec bit set.

This was added to python 3 in:
python/cpython@daf45555#diff-d1725c4e30cb0ce653d5c958a105640bR140

But I think this issue is not very important on python 2.7, since by default all file descriptors are create without the close-on-exec bit set. The example we see here is not common, passing the pipe of unrelated process to another child process.

@jstewmon, can you show real world use case that break because of this?

[th0ma5w]

Here is a non-working example for subprocess32 that works in >3.2 but in 2.7 ... omitting the pass_fds flag works in 2.6 ... make a file with something in it in /tmp/test to print out

from subprocess32 import Popen, PIPE
data = Popen(
    ["cat"],
    stdin=open('/tmp/test', 'rb'),
    stdout=PIPE,
    universal_newlines=True,
    ).stdout
fd_name = '/dev/fd/%d' % data.fileno()
fddata = Popen(
    ["cat",fd_name],
    stdout=PIPE,
    universal_newlines=True,
    pass_fds=(data.fileno(),)).stdout
print(fddata.read())

produces in 2.7:

cat: /dev/fd/4: No such file or directory

with subprocess in 3.2 prints out the contents of /tmp/test
in 2.6.6 subprocess ... without the pass_fds flag prints out the contents of /tmp/test
in 2.7 with subprocess it also can't find the file descriptor

This came up on the Central Ohio Python list.