Cryptic traceback when sys.path[0] is None

[warsaw]

BPO	16514
Nosy	@warsaw, @brettcannon, @jcea, @jaraco, @ericvsmith, @cjerdonek, @ericsnowcurrently
Files	16514.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/warsaw'
closed_at = <Date 2012-11-20.20:55:30.405>
created_at = <Date 2012-11-20.15:17:15.131>
labels = ['interpreter-core', 'type-bug']
title = 'Cryptic traceback when sys.path[0] is None'
updated_at = <Date 2012-11-22.02:40:01.049>
user = 'https://github.com/warsaw'

bugs.python.org fields:

activity = <Date 2012-11-22.02:40:01.049>
actor = 'jcea'
assignee = 'barry'
closed = True
closed_date = <Date 2012-11-20.20:55:30.405>
closer = 'barry'
components = ['Interpreter Core']
creation = <Date 2012-11-20.15:17:15.131>
creator = 'barry'
dependencies = []
files = ['28051']
hgrepos = []
issue_num = 16514
keywords = ['patch', '3.3regression']
message_count = 7.0
messages = ['175996', '176000', '176001', '176006', '176007', '176021', '176025']
nosy_count = 8.0
nosy_names = ['barry', 'brett.cannon', 'jcea', 'jaraco', 'eric.smith', 'chris.jerdonek', 'python-dev', 'eric.snow']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue16514'
versions = ['Python 3.3', 'Python 3.4']

[warsaw]

When sys.path[0] is None, attempting to import a module produces a cryptic chained traceback. This is a regression from 3.2 where the import would succeed.

The basic issue is that in 3.2's import.c, non-string/bytes items on sys.path are essentially ignored (see the loop in import.c:1708 find_module()) while they are not ignored in 3.3's importlib. This means that because zipimporter.zipimiporter is by default the first thing on sys.path_hooks, zipimporter.zipimporter(None) gets called. This raises a TypeError which starts the chained exception. Note that the fact that zipimporter.zipimporter(None) raises a TypeError is *not* a regression. The regression is that None makes its way to sys.path_hooks at all.

I think this will be relatively easy to fix, if we agree that the current regressive behavior is a bug.

[warsaw]

Patch for 3.3, omitting the requisite importlib.h change.

[warsaw]

Oh, the patch includes the removal of some unused imports from test_path.py, thanks to pyflakes.

[ericvsmith]

I agree it's a bug. And the fix looks good to me.

Having said that: I haven't looked at the import.c version to verify what is happening. Does the test pass under 3.2?

[warsaw]

Oh, btw, I intend to add documentation that makes explicit:

• sys.path entries must be strings or bytes, everything else is ignored
• path importers should expect strings or bytes
• the encoding of bytes is left to the individual path hooks to define, however it it can't decode the bytes it should ignore the path entry or raise an ImportError (effectively ignoring it anyway).

[warsaw]

On Nov 20, 2012, at 04:51 PM, Eric V. Smith wrote:

I agree it's a bug. And the fix looks good to me.

Thanks.

Having said that: I haven't looked at the import.c version to verify what is
happening. Does the test pass under 3.2?

Yep. See below. When you take a look at the path_hooks iteration loop,
you'll see why.

-----snip snip-----

import sys

sys.path.insert(0, None)
sys.path_importer_cache.pop(None, None)
sys.modules.pop('email', None)

import email
print(email)
-----snip snip

---

% python3.2 /tmp/foo.py
<module 'email' from '/usr/lib/python3.2/email/__init__.py'>
% python3.3 /tmp/foo.py
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1271, in _path_importer_cache
KeyError: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/foo.py", line 7, in <module>
    import email
  File "<frozen importlib._bootstrap>", line 1561, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1519, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1473, in _find_module
  File "<frozen importlib._bootstrap>", line 1308, in find_module
  File "<frozen importlib._bootstrap>", line 1284, in _get_loader
  File "<frozen importlib._bootstrap>", line 1273, in _path_importer_cache
  File "<frozen importlib._bootstrap>", line 1254, in _path_hooks
TypeError: 'NoneType' object is not iterable
Error in sys.excepthook:
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1271, in _path_importer_cache
KeyError: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 58, in apport_excepthook
    from cStringIO import StringIO
  File "<frozen importlib._bootstrap>", line 1561, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1519, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1473, in _find_module
  File "<frozen importlib._bootstrap>", line 1308, in find_module
  File "<frozen importlib._bootstrap>", line 1284, in _get_loader
  File "<frozen importlib._bootstrap>", line 1273, in _path_importer_cache
  File "<frozen importlib._bootstrap>", line 1254, in _path_hooks
TypeError: 'NoneType' object is not iterable

Original exception was:
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1271, in _path_importer_cache
KeyError: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/foo.py", line 7, in <module>
    import email
  File "<frozen importlib._bootstrap>", line 1561, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1519, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1473, in _find_module
  File "<frozen importlib._bootstrap>", line 1308, in find_module
  File "<frozen importlib._bootstrap>", line 1284, in _get_loader
  File "<frozen importlib._bootstrap>", line 1273, in _path_importer_cache
  File "<frozen importlib._bootstrap>", line 1254, in _path_hooks
TypeError: 'NoneType' object is not iterable

[python-dev]

New changeset 291406748217 by Barry Warsaw in branch '3.3':

• Issue bpo-16514: Fix regression causing a traceback when sys.path[0] is None
  http://hg.python.org/cpython/rev/291406748217

New changeset a82ee9a1457a by Barry Warsaw in branch 'default':

• Issue bpo-16514: Fix regression causing a traceback when sys.path[0] is None
  http://hg.python.org/cpython/rev/a82ee9a1457a