-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
subprocess closes redirected fds even if they are in pass_fds #76451
Comments
Demonstration: $ cat test.py
import os
import subprocess
import sys fd = os.dup(sys.stdout.fileno())
subprocess.call([sys.executable, '-c',
'import sys;'
'print("Hello stdout");'
'print("Hello fd", file=open(int(sys.argv[1]), "w"))',
str(fd)],
stdout=fd,
pass_fds=[fd]) $ python3 test.py
Hello stdout
Traceback (most recent call last):
File "<string>", line 1, in <module>
OSError: [Errno 9] Bad file descriptor I see two issues here:
Please provide your thoughts. |
Thanks for the report and the investigation!
That sounds like a bug to me, so we should fix it if decently possible.
Same here, it seems like a bug which deserves fixing. Do you want to submit a PR for this? |
Regarding fixing (1), I'm worrying about backward compatibility a bit. Some people who discovered that behavior might rely on such "move" semantics and expect that the redirected descriptor is not leaked into the child. OTOH, since descriptors are non-inheritable by default since 3.4, it might be less of an issue now. What do you think? Otherwise, yes, I'd like to fix this, but fixing is a bit trickier than it may seem because sometimes descriptors are duplicated in _posixsubprocess.c (in case of low fds), so we need to track the ownership properly. I've already performed some work and discovered another issue: in a corner case involving closed standard fds, pipes created by subprocess may leak into the child and/or an incorrect redirection may occur. Since I can't completely fix this issue without affecting a discovered one, I think I'll do the opposite: file a new issue, fix it, and then go back to this one. |
Well, people shouldn't rely on bugs. Otherwise we would never be able to fix bugs, lest someone relies on it. |
This bug stems from: https://github.com/python/cpython/blob/master/Modules/_posixsubprocess.c#L454 When stdout=fd is passed, that results in c2pwrite=fd. The stdin/stdout/stderr pipe fds are closed when > 2 without checking to see if they are listed in pass_fds. https://github.com/python/cpython/blob/master/Lib/subprocess.py#L1306 maps the following: |
Actually I've started to work on this with bpo-32844, but got no feedback. This issue may of course be fixed independently, but the problems with descriptor ownership for fds <= 2 will remain unless all ownership problems are fixed at once. |
An alternative fix is here: izbyshev@b89b52f Feel free to use the test if you don't like the approach :) |
Thank you, Gregory! |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: