[doc] import silently prefers package over module when both available

[shai]

BPO	17108
Nosy	@terryjreedy, @merwok, @bitdancer, @ericsnowcurrently

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2013-02-02.22:41:08.621>
labels = ['easy', 'type-bug', '3.9', '3.10', '3.11', 'docs']
title = '[doc] import silently prefers package over module when both available'
updated_at = <Date 2021-12-09.23:09:23.109>
user = 'https://bugs.python.org/shai'

bugs.python.org fields:

activity = <Date 2021-12-09.23:09:23.109>
actor = 'iritkatriel'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2013-02-02.22:41:08.621>
creator = 'shai'
dependencies = []
files = []
hgrepos = []
issue_num = 17108
keywords = ['easy']
message_count = 14.0
messages = ['181225', '181232', '181233', '181234', '181236', '181242', '181268', '181269', '181272', '181273', '181687', '181688', '181692', '181710']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'eric.araujo', 'Arfrever', 'r.david.murray', 'docs@python', 'eric.snow', 'shai']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17108'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[shai]

Consider the following directory structure:

a-\
__init__.py
b.py
b-|
__init__.py

Now, in Python (I checked 2.7.3 and 3.2.3, haven't seen the issue mentioned anywhere so I suspect it is also in later Pythons), if you import a.b, you always get the package (that is, the b folder), and the module (b.py) is silently ignored. I tested by putting the line """print("I'm a package")""" in a/b/init.py and """print("I'm a module")""" in a/b.py.

This becomes a real problem with tools which find modules dynamically, like test harnesses.

I'd expect that in such cases, Python should "avoid the temptation to guess", and raise an ImportError.

Thanks, Shai.

[bitdancer]

This is behavior has been true since packages were introduced, and is not going to change. However, I agree that it could be better documented.

In Python3 something that needs to do dynamic module discovery can use importlib to be sure of using the same logic that Python itself uses. (Well, that's completely true only for 3.3+, but it is 99+% true for 3.1 and 3.2 as well).

[shai]

Thanks for the quick response.

If this isn't changing, I'd definitely want better documentation. In particular, the rationale behind this should be explained.

I submitted the bug because a co-worker unintentionally caused a whole suite of tests to be ignored.

Thanks again,
Shai.

[ericsnowcurrently]

Other than the language reference (with its updated info on imports--thanks Barry!), what other documentation would benefit from a note on this? Somwhere in http://docs.python.org/dev/tutorial/modules.html?

[bitdancer]

It was mostly the language reference I was thinking of, since that's where I think one would naturally go to find out about unexpected behavior of the import statement, but a note in the tutorial is probably not a bad idea. I'm not sure if this rises to the level of a FAQ or not, but perhaps it does.

Technically I suppose the importlib documentation should also mention it where appropriate if it does not already, but I wouldn't expect that to be a very discoverable location for it.

By the way, I seem to remember there being some detail about namespace packages that potentially affected this, but I don't remember if that was a proposal or something that wound up in the final implementation. Do you?

[ericsnowcurrently]

By the way,...

Yeah, PEP-420 (implemented in 3.3) introduced namespace packages. The new behavior you're thinking of is where a package doesn't need a __init__.py. So path-based lookup for modules, the order goes like this (for "import spam.eggs"):

1. look for a directory named "spam" with a __init__.py,
2. look for a file named spam.py,
3. look for a directory named "spam" (becomes an namespace package),
4. raise ImportError (used to be step 3).

Once spam gets loaded, spam.eggs gets imported...

[bitdancer]

Ah, yes.

To clarify for Shai, by the way, the reason this stuff can't change (and the reason there is a new step 3 instead of changing the whole algorithm to be something more sensible) is because of the requirement of maintaining backward compatibility.

[shai]

Hi,

the reason this stuff can't change [... is] backward compatibility.

Thanks, but this is still unclear to me. The required fix for code that would break because of the change I propose, is removal of dead code which looks misleadingly alive.

Is the backward-compatibility requirement really that strict?

[bitdancer]

Yes. Imagine you have a deployed application, and there happens to be an xx.py file that is masked by a package in it. You upgrade from pythonX.Y.Z to X.Y.Z+1, and your application is suddenly throwing an error. Yes it is easy to fix, but we prefer not to break things that way.

Now, could we throw an error in Python 3.4? It could be discussed, at least.

[shai]

Oh, sure, this was unclear of me. I thought you were talking about Python 3.4. I wasn't really expecting this to be fixed in the stable branches.

Thanks,
Shai.