monkeytype/typing.py import error in PyCharm

[johnarnold]

Trying to set up dev environment per instructions in CONTRIBUTING.rst

Getting an import error:
Traceback (most recent call last):
File "/home/johnar/github/MonkeyType/monkeytype/cli.py", line 17, in
from typing import (
File "/home/johnar/github/MonkeyType/monkeytype/typing.py", line 10, in
from typing import (
ImportError: cannot import name 'Any'

In pycharm it actually shows that import _Any and _Union result in unresolved reference, as they're not defined in the typing module.

This is with stdlib typing 3.6.4:
typing 3.6.4

[johnarnold]

This is weird... i can import from python console ok in the venv, just blows up when running cli.py in pycharm.

[carljm]

Sounds like pycharm just isn’t using the venv. I don’t use pycharm so not sure how to make it use the venv, but that will be necessary to use it for monkeytype development.

[johnarnold]

@carljm Maybe another contributer/watcher/user will have the answer?

Pycharm is a very popular IDE, it seems likely that someone else will run into the same issue. In this case, it appears to be caused by monkeytype/typing.py stepping on the namespace for builtin typing, which is confusing pycharm. If i rename typing.py and all references to monkey_typing, the import error goes away.

[carljm]

Sure, I can leave this open for awhile to see if another PyCharm user can help.

Seems like PyCharm is either a) setting up sys.path wrong (with the monkeytype directory itself on sys.path instead of the containing directory; or possibly due to changing the current working directory to the monkeytype directory), which would cause our typing.py to be accessible as top-level typing instead of as monkeytype.typing, or b) somehow getting Py3 import behavior wrong and re-enabling "implicit relative imports" from the bad old Py2 days, which would mean that import typing from monkeytype/whatever.py could import monkeytype/typing.py, even without the monkeytype directory on sys.path.

[johnarnold]

Yep, it does seem like the sys.path is getting messed up:

print(json.dumps(sys.path, indent=2))
[
"/home/johnar/github/MonkeyType/monkeytype", <---
"/snap/pycharm-community/43/helpers/pydev",
"/snap/pycharm-community/43/helpers/pydev",
"/home/johnar/.PyCharmCE2017.3/system/cythonExtensions",
"/home/johnar/.virtualenvs/MonkeyType-sBIHPtkn/lib/python36.zip",
"/home/johnar/.virtualenvs/MonkeyType-sBIHPtkn/lib/python3.6",
"/home/johnar/.virtualenvs/MonkeyType-sBIHPtkn/lib/python3.6/lib-dynload",
"/usr/lib/python3.6",
"/home/johnar/.virtualenvs/MonkeyType-sBIHPtkn/lib/python3.6/site-packages",
"/home/johnar/github/MonkeyType"
]

I did a little digging, at it seems that this is default behavior for python:

"projects/pathproblem" is added to sys.path by Python itself, not by PyCharm. See http://docs.python.org/2/library/sys.html#sys.path : "As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter."

Because i was running /MonkeyType/monkeytype/cli.py as a script, it was adding the monkeytype directory to the path automatically.

For anyone else who may run into this path conflict while developing/debugging, my solution was just to add a run.py file to the top level directory of the project:

#!/usr/bin/env python3

from monkeytype.cli import entry_point_main

if __name__ == '__main__':
    entry_point_main()

[carljm]

Ah, makes sense! FWIW, when developing MonkeyType outside of PyCharm, it gets pip install -e into the virtualenv, which makes setuptools install a monkeytype script entry point wrapper into the venv, and we run that, we don't run monkeytype/cli.py directly. Effectively this is the same wrapper you created manually as run.py.

Thanks for digging and finding the workaround!

[devdoomari3]

@carljm can we just rename monkeytype/typing.py to monkeytype/type_utils.py ?

I wasted 3++ hours due to

from typing import (
   ...
)

inside monkeytype/typing.py importing itself. (though before that 3++ hours, everything was OK - even on pycharm / terminal. And then, some changes on monkeytype source-code caused havoc...)

Also, according to https://stackoverflow.com/questions/6031584/importing-from-builtin-library-when-module-with-same-name-exists , it's better to have not-conflicting module names

(though this seems to be a big problem on python-side: I never experienced this kind of problem in node-js imports / java imports because they have clear distinction on local / global imports...)

[carljm]

Today modern Python no longer has any ambiguity between top-level and local-relative imports, so the only name that needs to not conflict is the top-level name, and our top-level name is "monkeytype", which does not conflict. The only way this can break is if something adds the wrong path to sys.path, so the module that should be monkeytype.typing (no conflict) becomes just typing (conflict!). But if something is adding the wrong path to sys.path, renaming typing.py to type_util.py won't help, it'll just replace one error message with another one, because all the imports of monkeytype.anything will fail.

I'm sorry you lost some time debugging a path issue, but I don't think that a rename of typing.py would actually be relevant to fixing that.