importlib: PermissionError during startup if working directory isn't readable

[moreati]

Bug report

Bug description:

On macOS importlib._bootstrap_external.PathFinder._path_importer_cache() raises PermissionError during interpreter startup if '' is included in sys.path and the current working directory is not readable.

Reproduction

Given a CWD that is not readable by fred, the user fred cannot run Python

➜  private su fred -c "whoami; python3.13 -c 'pass'"
Password:
fred
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1262, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1544, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1516, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1495, in _path_importer_cache
PermissionError: [Errno 13] Permission denied

➜  private uname -mv; whoami; pwd; ls -ld                                                        
Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:44 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6000 arm64
alex
/Users/alex/private
drwx------  2 alex  staff  64 25 Feb 10:44 .

➜  private python3.13 -c "import sys;print(sys.version)"           
3.13.0a4+ (heads/main:6550b54813, Feb 25 2024, 10:56:11) [Clang 15.0.0 (clang-1500.1.0.2.5)]

Workaround

Adding -P, prevents '' being added to sys.path, so avoids the exception

➜  private su fred -c "whoami; python3.13 -P -c 'pass'"                                    
Password:
fred

Discussion

On macOS the libc function getcwd() can return EACCES. From the manpage

[EACCES] Read or search permission was denied for a component of the pathname. This is only checked in limited cases, depending on implementation details.

When searching for importable modules PathFinder._path_importer_cache() attempts to determine the cwd by calling os.getcwd(), it handles a FileNotFoundError exception, but not PermissionError. Because PathFinder is used during interpreter startup user code has no opportunity to catch the exception.

Proposed fix

Ignore PermissionError in PathFinder._path_importer_cache(), the same way FileNotFoundError is currently. This would result in imports succeeding, but without getting cached. E.g. applying the below change & rebuilding/reinstalling

➜  private su fred -c "whoami; python3.13 -c 'import sys;print(sys.version)'"
Password:
fred
3.13.0a4+ (heads/main:6550b54813, Feb 25 2024, 10:56:11) [Clang 15.0.0 (clang-1500.1.0.2.5)]

I'm happy to submit a PR with this, and unit tests as deemed suitable

➜  cpython git:(main) ✗ git diff
diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py
index 2a9aef0317..5d1f4f1de0 100644
--- a/Lib/importlib/_bootstrap_external.py
+++ b/Lib/importlib/_bootstrap_external.py
@@ -1493,7 +1493,7 @@ def _path_importer_cache(cls, path):
         if path == '':
             try:
                 path = _os.getcwd()
-            except FileNotFoundError:
+            except (FileNotFoundError, PermissionError):
                 # Don't cache the failure as the cwd can easily change to
                 # a valid directory later on.
                 return None

Other Python Versions

In Python 3.10, 3.11 & 3.12 the same exception can occur when user code imports a non-builtin module (e.g. base64, zlib), but it does not occur during interpreter startup. I presume this is due to a change in which modules are required during interpreter initialisation.

➜  private su fred -c "whoami; python3.10 -c 'import sys;print(sys.version);import zlib'"              
Password:
fred
3.10.13 (main, Aug 24 2023, 12:59:26) [Clang 15.0.0 (clang-1500.1.0.2.5)]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 945, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1439, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1408, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1366, in _path_importer_cache
PermissionError: [Errno 13] Permission denied

➜  private su fred -c "whoami; python3.11 -c 'import sys;print(sys.version);import zlib'"
Password:
fred
3.11.7 (main, Dec  4 2023, 18:10:11) [Clang 15.0.0 (clang-1500.1.0.2.5)]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1138, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1078, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1504, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1473, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1431, in _path_importer_cache
PermissionError: [Errno 13] Permission denied

➜  private su fred -c "whoami; python3.12 -c 'import sys;print(sys.version);import zlib'"
Password:
fred
3.12.1 (main, Dec  7 2023, 20:45:44) [Clang 15.0.0 (clang-1500.1.0.2.5)]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1262, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1524, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1496, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1475, in _path_importer_cache
PermissionError: [Errno 13] Permission denied

CPython versions tested on:

3.10, 3.11, 3.12, CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-115911: Ignore PermissionError during import from cwd #116131

[moreati]

Streamlined reproducer, without su or sudo

#!/usr/bin/env bash
set -o errexit
set -o nounset
set -o pipefail

PYTHON="${PYTHON:-python}"
P="issue115911"

if [ -d "$P" ]; then
    rmdir "$P"
fi

mkdir "$P"          # Create directory with permissive mode
cd "$P"             # Make it our working directory, while the mode allows
chmod ugo-rx .      # Remove read & execute from it, for everyone including us
"$PYTHON" -c "pass" # Try to execute python in it

or as a shell oneliner

(p="issue115911"; ([ -d "$p" ] && rmdir "$p"); mkdir "$p" && cd "$p" && chmod ugo-rx . && ~/src/cpython/python.exe -c "pass")

➜  tmp PYTHON=~/src/cpython/python.exe ~/src/cpython/issue115911.sh
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1262, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1544, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1516, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1495, in _path_importer_cache
PermissionError: [Errno 13] Permission denied

[moreati]

According to https://manpages.debian.org/buster/manpages-dev/getcwd.3.en.html#NOTES (Debian 10.x, released 2019, linux kernel 4.19, glibc 2.28) on Linux with glibc

• getcwd(3) can only fail with EACCESS given both: kernel < 2.1.92 (no getcwd(2) syscall), /proc fs missing
• getcwd(3) may return a relative path (POSIX non-conforming) given both: kernel >= 2.6.36, glibc < 2.27

In the later case glibc >= 2.27 returns ENOENT instead, which is already handled by the try/except in PathFinder._path_importer_cache().