__init__.py can be a directory

[abraithwaite]

BPO	21784
Nosy	@birkenfeld, @rhettinger, @vstinner, @ericvsmith, @berkerpeksag

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-10-06.14:38:36.413>
created_at = <Date 2014-06-17.02:48:32.737>
labels = ['type-bug', 'docs']
title = '__init__.py can be a directory'
updated_at = <Date 2014-10-06.14:38:36.412>
user = 'https://bugs.python.org/abraithwaite'

bugs.python.org fields:

activity = <Date 2014-10-06.14:38:36.412>
actor = 'georg.brandl'
assignee = 'docs@python'
closed = True
closed_date = <Date 2014-10-06.14:38:36.413>
closer = 'georg.brandl'
components = ['Documentation']
creation = <Date 2014-06-17.02:48:32.737>
creator = 'abraithwaite'
dependencies = []
files = []
hgrepos = []
issue_num = 21784
keywords = []
message_count = 9.0
messages = ['220787', '220788', '220789', '220802', '220832', '220841', '220842', '221131', '228698']
nosy_count = 7.0
nosy_names = ['georg.brandl', 'rhettinger', 'vstinner', 'eric.smith', 'docs@python', 'berker.peksag', 'abraithwaite']
pr_nums = []
priority = 'low'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21784'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[abraithwaite]

Is this expected? It was very confusing when I cloned a repo that didn't have the __init__.py there when I had just made it, but of course git doesn't track files, only directories. I had accidentaly done mkdir instead of touch.

$ mkdir pkg/__init__.py
$ touch pkg/foobar.py
$ python
Python 3.4.1 (default, May 19 2014, 17:23:49) 
[GCC 4.9.0 20140507 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pkg import foobar
>>> foobar
<module 'pkg.foobar' from '/home/abraithwaite/pkg/foobar.py'>
>>>

[abraithwaite]

but of course git doesn't track files, only directories.

but of course git doesn't track *directories*, only *files*.

[rhettinger]

Is this expected?

It is a little weird but usually problematic. The cause that import checks for the existence of '__init__.py' but doesn't then add isfile() or isdir() check which would be unnecessary 99.9% of the time.

I classify this a harmless oddity.

[berkerpeksag]

I think this is related to PEP-420.

    $ tree pkg/
    pkg/
    ├── foobar.py

    $ python3.2 -c "from pkg import foobar"
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ImportError: No module named pkg

But, with Python 3.3 and newer:

    $ python3.3 -c "from pkg import foobar"
    hello

See https://docs.python.org/3/whatsnew/3.3.html#pep-420-implicit-namespace-packages for the whatsnew entry.

[abraithwaite]

Interesting. I saw the same behavior on 2.7.7 as well:

$ python2
Python 2.7.7 (default, Jun  3 2014, 01:46:20) 
[GCC 4.9.0 20140521 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from pkg import foobar
>>> foobar
<module 'pkg.foobar' from 'pkg/foobar.py'>
>>> 

Anyways, it doesn't bother me too much. I opened a ticket because I couldn't find an already open bug about it or any reference to it on google (although the keywords aren't great for google searches).

Cheers!

[vstinner]

See also the issue bpo-7732.

[vstinner]

I remember that I added an check in Python 3.2 on the file type to explicitly reject directories:
http://hg.python.org/cpython/rev/125887a41a6f

+ if (stat(buf, &statbuf) == 0 && S_ISDIR(statbuf.st_mode))
+ /* it's a directory */
+ fp = NULL;
+ else

I wrote this change in the C code, I didn't report the change to the Python code (importlib).

A huge work was also done in importlib to reduce the number of calls to stat(). So maybe a check was dropped by mistake?

[rhettinger]

So maybe a check was dropped by mistake?

Why do we care? For the most part, Linux treats directories as files (if I do a mkdir twice, the error is "mkdir: tmp: File exists". We don't care about the other flags rwx so why should we care about d?

Python should avoid adding superfluous checks when it doesn't have to.

Anyways, it doesn't bother me too much.

AFAICT, it hasn't bothered anyone.

[birkenfeld]

I agree, closing.