Change in error when sys.path contains a nonexistent folder (importlib)

[takluyver]

BPO	14846
Nosy	@brettcannon, @pitrou, @ericvsmith, @ericsnowcurrently, @takluyver

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-08-20.03:18:39.610>
created_at = <Date 2012-05-17.23:35:08.197>
labels = ['interpreter-core', 'type-bug']
title = 'Change in error when sys.path contains a nonexistent folder (importlib)'
updated_at = <Date 2012-08-20.03:18:39.599>
user = 'https://github.com/takluyver'

bugs.python.org fields:

activity = <Date 2012-08-20.03:18:39.599>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2012-08-20.03:18:39.610>
closer = 'python-dev'
components = ['Interpreter Core']
creation = <Date 2012-05-17.23:35:08.197>
creator = 'takluyver'
dependencies = []
files = []
hgrepos = []
issue_num = 14846
keywords = []
message_count = 4.0
messages = ['161025', '161026', '161027', '168616']
nosy_count = 6.0
nosy_names = ['brett.cannon', 'pitrou', 'eric.smith', 'python-dev', 'eric.snow', 'takluyver']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14846'
versions = ['Python 3.3']

[takluyver]

I've come across a difference from 3.2 to 3.3 while running the IPython test suite. It occurs when a directory on sys.path has been used for an import, then deleted without being removed from sys.path. Previous versions of Python ignore the nonexistent folder, but in 3.3 a FileNotFound error appears.

This might be by design (errors should never pass silently), but I haven't found it mentioned in the What's New for 3.3.

$ cat import.py
import sys, os, shutil
os.mkdir('foo')
with open('foo/bar.py', 'w'):
    pass
sys.path.insert(0, 'foo/')
import bar   # Caches a FileFinder for foo/
shutil.rmtree('foo')
import random   # Comes later on sys.path

$ python3.2 import.py

$ python3.3 import.py
Traceback (most recent call last):
  File "import.py", line 8, in <module>
    import random
  File "<frozen importlib._bootstrap>", line 1162, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1124, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1078, in _find_module
  File "<frozen importlib._bootstrap>", line 927, in find_module
  File "<frozen importlib._bootstrap>", line 973, in find_module
  File "<frozen importlib._bootstrap>", line 1005, in _fill_cache
FileNotFoundError: [Errno 2] No such file or directory: 'foo/'

The last entry in that traceback is calling "_os.listdir(path)".

[ericsnowcurrently]

The caching mechanism is going to check for changes in the directory. So the recommendation is to clear the cache manually:

<quote>
The default finders used by import now utilize a cache of what is contained
within a specific directory. If you create a Python source file or sourceless
bytecode file, make sure to call importlib.invalidate_caches() to clear out
the cache for the finders to notice the new file.
</quote>

(see http://docs.python.org/dev/whatsnew/3.3.html#porting-python-code)

That section doesn't really talk about the directory side of things, but calling importlib.invalidate_caches() works great to eliminate the error in your example.

[ericsnowcurrently]

...rather, the error caused by your example. :)

[python-dev]

New changeset bfd04bfb55c5 by Nick Coghlan in branch 'default':
Close bpo-14846: Handle a sys.path entry going away
http://hg.python.org/cpython/rev/bfd04bfb55c5