importlib mixes up '.' and os.getcwd()

[pitrou]

BPO	14052
Nosy	@brettcannon, @ncoghlan, @pitrou, @skrah

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/brettcannon'
closed_at = <Date 2012-02-20.18:29:12.853>
created_at = <Date 2012-02-18.20:38:46.751>
labels = ['invalid', 'library']
title = "importlib mixes up '.' and os.getcwd()"
updated_at = <Date 2012-02-21.23:15:22.976>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2012-02-21.23:15:22.976>
actor = 'skrah'
assignee = 'brett.cannon'
closed = True
closed_date = <Date 2012-02-20.18:29:12.853>
closer = 'brett.cannon'
components = ['Library (Lib)']
creation = <Date 2012-02-18.20:38:46.751>
creator = 'pitrou'
dependencies = []
files = []
hgrepos = []
issue_num = 14052
keywords = []
message_count = 7.0
messages = ['153656', '153663', '153702', '153782', '153788', '153790', '153913']
nosy_count = 5.0
nosy_names = ['brett.cannon', 'ncoghlan', 'pitrou', 'Arfrever', 'skrah']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue14052'
versions = ['Python 3.2', 'Python 3.3']

[pitrou]

An empty sys.path string means the current directory in relative terms:

$ touch foo.py
$ python3 -c "import foo; print(foo.__file__)"
foo.py

But importlib uses os.getcwd() instead in PathFinder._path_importer_cache(). This impacts semantics of path searching, as well as the values of __file__ and __cached__ attributes.

[brettcannon]

It's surprisingly difficult to get right because tests will fail if you keep the path relative in other situations (such as test_cmd_line_script). It was an absolute pain to get it to where it is now. If you can make it keep the relative path and pass the test then that's great, but relative file paths suck for compatibility and I don't think they are that useful to begin with.

[ncoghlan]

Purging __file__ of relative references isn't a problem - they're *supposed* to always be absolute. The import.c version just stuffs it up sometimes (mainly due to the way it handles the empty string in the path).

IOW, while the importlib behaviour *is* different from the old behaviour, it's really a long overdue fix rather than a defect.
(You can still induce weirdness by adding paths involving '.' or '..' to sys.path, but that's more a case of "don't do that". We add the empty string ourselves, so we should handle it more cleanly than we do in import.c)

[pitrou]

There was the following sporadic buildbot failure:

======================================================================
FAIL: test_package___file__ (test.test_imp.PEP3147Tests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_imp.py", line 333, in test_package___file__
    os.sep.join(('.', 'pep3147', '__init__.py')))
AssertionError: 'pep3147/__init__.py' != './pep3147/__init__.py'
- pep3147/__init__.py
+ ./pep3147/__init__.py
? ++

I wonder if the test suite is playing games and sometimes using importlib :-/

[pitrou]

[...]

And I can't reproduce, even when using --randseed.

[brettcannon]

The test suite does use importlib for some imports as finders from importlib for some packages stick around. I brought this up a way back but people thought it was actually a good idea to let importlib handle imports to stress test it.

Anyway, if it can't be reproduced then I don't think this is worth worrying about. As Nick pointed out, everything should be absolute anyway.

[skrah]

It just happened again on the new FreeBSD-9.0 bot:

http://www.python.org/dev/buildbot/all/builders/AMD64%20FreeBSD%208.2%203.x/builds/1845/steps/test/logs/stdio

======================================================================
FAIL: test_package___file__ (test.test_imp.PEP3147Tests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/test_imp.py", line 333, in test_package___file__
    os.sep.join(('.', 'pep3147', '__init__.py')))
AssertionError: 'pep3147/__init__.py' != './pep3147/__init__.py'
- pep3147/__init__.py
+ ./pep3147/__init__.py
? ++