New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PermissionError in subprocess.check_output() when an inaccessible directory on the path #69667
Comments
When running subprocess.check_output(['doesnotexist']) as another user, I expect a FileNotFoundError, but a PermissionError is thrown. How to reproduce: [user1 tmp]$ su --login user2
Password:
[user2 ~]$ python
Python 3.5.0 (default, Sep 20 2015, 11:28:25)
[GCC 5.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.check_output(['asdf'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/subprocess.py", line 629, in check_output
**kwargs).stdout
File "/usr/lib/python3.5/subprocess.py", line 696, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.5/subprocess.py", line 950, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.5/subprocess.py", line 1540, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'asdf'
>>> quit()
[user2 ~]$ exit
logout
[user1 tmp]$ su user2
Password:
[user2 tmp]$ python
Python 3.5.0 (default, Sep 20 2015, 11:28:25)
[GCC 5.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.check_output(['asdf'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/subprocess.py", line 629, in check_output
**kwargs).stdout
File "/usr/lib/python3.5/subprocess.py", line 696, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.5/subprocess.py", line 950, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.5/subprocess.py", line 1540, in _execute_child
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied
>>> quit()
[user2 tmp]$ |
It is almost certainly the case that the error message is correct. The most likely explanation given your pasted session is that user2 does not have permission in the CWD. When you use --login, the CWD changes to user2's home, which it of course has permission in. |
user2 in fact has permissions in the CWD. |
On some unix systems you cannot execute files in /tmp, even if x is set. Is that true for yours? Bascially, subprocess is just reporting the error that the OS is reporting to it, which is why I say the messages is almost certainly correct. |
I just tried the same thing in /home/user2 which is the home of user2 and in /. |
I just found that a "chmod o+x /home/user1" fixes the problem in the sense that now a FileNotFoundError is thrown. Could it be that python tries to access the "original" home directory somehow? For example to write to a pipe? |
Narrowing it down further: In $PATH there is a subdirectory of /home/user1. Doing an strace on the example above shows the line If I delete "/home/user1/bin" from $PATH, then the correct FileNotFoundError is thrown. So it seems (just guessing) that subprocess.check_output() somehow throws the older, obsolete error code. Would this be possible? |
What do you mean by older obsolete error code? It sounds like the problem is that user2 doesn't have read (or execute?) permission for that directory in the path. I'd think it would just skip it in that case, though. I don't have time to run tests myself right now...maybe you can take a look at what subprocess is actually doing when that error is generated? (It might be in the C code, though, I don't remember). |
In subprocess.py there's the following code that builds a sequence of potential paths for the executable 1: executable = os.fsencode(executable)
if os.path.dirname(executable):
executable_list = (executable,)
else:
# This matches the behavior of os._execvpe().
executable_list = tuple(
os.path.join(os.fsencode(dir), executable)
for dir in os.get_exec_path(env)) In this case it tries to execute "/home/user1/bin/asdf", which fails with EACCES (to log this using strace, use -f to follow the fork). This occurs in child_exec in _posixsubprocess.c, in the following loop 2: /* This loop matches the Lib/os.py _execvpe()'s PATH search when */
/* given the executable_list generated by Lib/subprocess.py. */
saved_errno = 0;
for (i = 0; exec_array[i] != NULL; ++i) {
const char *executable = exec_array[i];
if (envp) {
execve(executable, argv, envp);
} else {
execv(executable, argv);
}
if (errno != ENOENT && errno != ENOTDIR && saved_errno == 0) {
saved_errno = errno;
}
}
/* Report the first exec error, not the last. */
if (saved_errno)
errno = saved_errno; saved_errno will be set to EACCES and stored back to errno after all attempts to execute potential paths fail. This is then reported back to the parent process, which raises a PermissionError. |
So, two interesting questions: does this in fact match the behavior of os._execvpe, and does it match the behavior of the shell? The latter would appear to be false, and could arguably be claimed to be a bug. If we agree that it is, we need to learn what the behavior of the shell is in a bunch of corner cases (only inaccessible directories on path, only match is accessible but not executable, etc). If this is a bug I'd guess it applies to all supported python versions. Note that it should be possible to reproduce this using a single user, so I've changed the title accordingly. |
Definitely a bug. The path search should silently skip directories it can't On Tue, Oct 27, 2015, 7:05 AM R. David Murray <report@bugs.python.org>
|
I think it's fine. child_exec() tries all paths. It saves the first error that's not ENOENT or ENOTDIR. The saved error gets reported back, else ENOENT or ENOTDIR. This is the same as os._execvpe: for dir in path_list:
fullname = path.join(dir, file)
try:
exec_func(fullname, *argrest)
except (FileNotFoundError, NotADirectoryError) as e:
last_exc = e
except OSError as e:
last_exc = e
if saved_exc is None:
saved_exc = e
if saved_exc is not None:
raise saved_exc
raise last_exc Perhaps the rule for PATH search errors should be documented for os.execvp[e] and subprocess. I think it matches the shell, except, AFAIK, the shell doesn't distinguish ENOENT and ENOTDIR errors. For example, where "/noexec" is a directory that has no execute access: $ /bin/sh -c spam
/bin/sh: 1: spam: not found
$ PATH=/noexec:$PATH /bin/sh -c spam
/bin/sh: 1: spam: Permission denied |
3.11 label could be added as well: and I confirm, this is unexpected behavior not emitted by standard shells: |
Workaround for python bug: - python/cpython#69667 Details: - Taxel#1223 (comment)
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: