pytype can't handle packages

[vadimcn]

Create a Python package and a script that uses it:

foo/__init__.py:

# (empty)

foo/bar.py:

def baz():
    print 'foo.bar.baz'

test.py:

from foo.bar import baz
baz()

Running python test.py outputs foo.bar.baz, as expected.

Running pytype test.py produces an error: File "test.py", line 1, in <module>: Can't find module 'foo.bar'. [import-error]

[martindemello]

right now this does not work; you need to have a foo/bar.pyi file as well (which you can generate by running pytype over bar.py):

$ pytype foo/bar.py -o foo/bar.pyi
$ cat foo/bar.pyi
def baz() -> None: ...

$ pytype test.py -o -
def baz() -> None: ...

we are working on a tool to automate this process.

[lkgarrison]

@martindemello Any update on automating this? I don't see how pytype is particularly usable without this. I get an "import-error" when importing any of my own modules as well as some third party modules.

I often see errors along the lines of "Can't find module ___", where ___ is actually a subpackage within my package.

[martindemello]

pytype now runs in whole-project mode, rather than a file at a time, so this should just work. do you have a repo we can look at that demonstrates the errors?

[lkgarrison]

Yes, I've created a simplified example at https://github.com/lkgarrison/pytype-import-error

I'm simply running pytype in pytype_import_error/ as I have setup.cfg configured to specify a directory for pytype

Note that if you delete pytype_import_error/__init__.py, the import errors vanish. Please let me know if this is a bug or if there is something wrong on my end

[martindemello]

this does look like a bug. reopening the issue till we figure it out.

[martindemello]

it works if you change main.py to read

from . import adjacent_module
from .subpackage import foo
from .subpackage.helpers import helper_fn

[rchen152]

I played around with this a bit, and I think I know what's going on. In Python 2, the names of subpackages are automatically loaded into __init__.py's namespace, which allows you to use absolute imports for them. In Python 3, this is an error. You can see this by trying to import the example code:

$ python2.7
Python 2.7.13 (default, Nov 24 2017, 17:33:09) 
>>> import pytype_import_error.main
>>> 
$ python3.6
Python 3.6.3 (default, Oct  3 2017, 21:16:13) 
>>> import pytype_import_error.main
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/google/home/rechen/lkgarrison_pytype_import_error/pytype_import_error/main.py", line 1, in <module>
    import adjacent_module
ModuleNotFoundError: No module named 'adjacent_module'

Pytype is using the Python 3 behavior in both Python 2 and Python 3. I'm guessing that when __init__.py is removed, pytype stops treating pytype_import_error as a package, so it looks for imports in the current directory as if main.py were a standalone script.

So:

• This is a pytype bug.
• However, there's a workaround (use relative imports as @martindemello shows in the previous comment), which also has the advantage of making the code Python 3-compatible.
• We likely won't have the resources to fix a Python 2-only bug, given that Python 2 is quickly approaching end of life.

Given all this, I'm going to close this issue again, but please reopen it if I've misunderstood anything.