Treat __init__.pyc same as __init__.py in module_list

[avandever]

There are some circumstances where packages on disk could only have a .pyc
version of __init__.py. It would be nice if ipython would autocomplete them as
well. I've been using this in a production environment for several weeks now.

Test plan:

mkdir avtest; touch avtest/__init__.pyc
./ipython.py
from avt<tab>

autocomplete worked. Also tested with __init__.py to verify no regression.

[bfroehle]

Looks good to me, but we might want to make this properly generic. I haven't looked closely at the code. Just based off what's shown in the diff view it'd be something like:

if any(isfile(pjoin(path, p, '__init__' + suffix[0])) for suffix in imp.get_suffixes())

(But we may want to make this a function, or preprocess the list of '__init__' + suffix). Thoughts?

Unfortunately, in the long term this isn't going to be a valid check because of (implicit) namespace packages. I haven't digested that logic, and since it only applies to 3.3 or later, I think we don't need to worry (much) about it yet.

[takluyver]

get_suffixes() also turns up things like .so, and I don't think __init__.so makes a directory importable.

[bfroehle]

Actually, __init__.so does indicate a valid package. You can see the logic for it in ExtensionFileLoader.

To refresh your memory, the list of extension suffixes in Python 3.3:

>>> from importlib._bootstrap import EXTENSION_SUFFIXES
>>> EXTENSION_SUFFIXES
['.cpython-33m.so', '.abi3.so', '.so']

Also, here can be other regular source extensions in some cases, like .pyo and maybe .pyw.

[takluyver]

If I try copying a Python .so module to a init.so file, I can't import
the directory, but maybe changing the name like that isn't valid. It
probably doesn't matter, though - I don't think I've ever seen an
init.so file, and distutils doesn't seem to have an obvious way to
create one.

[rkern]

Extension modules need to have a C function named init<module> if the filename is <module>.so, so copying another .so to a different name will not make a loadable extension module. Having distutils make an __init__.so should be just like making any other extension module with a particular name.

[rkern]

Actually, for Python 3.3.0, the function needs to be named PyInit_<packagename>. I can verify that it works. It does not appear to work for Python 2.7.3.

[takluyver]

That's probably down to the new import system based on importlib. In that
case, let's go with @bfroehle's suggestion.

[avandever]

Oy, sorry, this is my first pull request on github, so feel free to throw pointers at me. I modified with @bfroehle's suggestion, tested, and (I think) updated. Now we're future-proof! ;)

[takluyver]

Thanks Andrew. For future reference, try to avoid merging branches within a pull request. The code change looks fine, though. I'm just running the tests, because the merge seems to have confused travis-ci.

[takluyver]

Test results for commit f441784 merged into master (097ce42)
Platform: linux2

• python2.7: OK (libraries not available: azure oct2py pymongo wx wx.aui)
• python3.2: OK (libraries not available: azure oct2py pymongo wx wx.aui)

Not available for testing: python2.6

[bfroehle]

@rkern: As an aside I think you need an (empty) __init__.py file to trick Python into loading the __init__.so file. It's weird.