Importing namespace packages crashes

[pylint-bot]

Originally reported by: Thorsten Lockert (BitBucket: tholo, GitHub: @tholo?)

---

It appears that with the pylint 1.2 release one cannot check packages that import from namespace packages anymore.

In some cases previously (particularly since setuptools changed its handling of such packages recently) one would get false positives claiming packages did not exist.

However, with pylint 1.2 one now get "ImportError: No module named x" for the (first) top-level namespace package encountered.

---

• Bitbucket: https://bitbucket.org/logilab/pylint/issue/203

[pylint-bot]

Original comment by Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):

---

Please notice pylint 1.2 just got released last week. If the problem still
arise, would you please provide a way to reproduce it?

[pylint-bot]

Original comment by Thorsten Lockert (BitBucket: tholo, GitHub: @tholo?):

---

Arg, I mis-stated above -- this is with pylint 1.2, with the previous minor issues seeing some false positives with setuptools 3.x (but not with 2.x).

To reproduce, create some code that imports from a namespaced package, e.g. zope.deprecation or paste.script (from PasteScript), and run pylint on it. Poof.

[pylint-bot]

Original comment by Marc Rintsch (BitBucket: blackjack, GitHub: @blackjack?):

---

marrow / marrow.mailer is another example.

from marrow.mailer import Mailer, Message

Leads to

Traceback (most recent call last):
  File "/usr/local/bin/pylint", line 9, in <module>
    load_entry_point('pylint==1.2.0', 'console_scripts', 'pylint')()
  File "/usr/local/lib/python2.7/dist-packages/pylint/__init__.py", line 21, in run_pylint
    Run(sys.argv[1:])
  File "/usr/local/lib/python2.7/dist-packages/pylint/lint.py", line 1047, in __init__
    linter.check(args)
  File "/usr/local/lib/python2.7/dist-packages/pylint/lint.py", line 626, in check
    self.check_astroid_module(astroid, walker, rawcheckers, tokencheckers)
  File "/usr/local/lib/python2.7/dist-packages/pylint/lint.py", line 712, in check_astroid_module
    walker.walk(astroid)
  File "/usr/local/lib/python2.7/dist-packages/pylint/utils.py", line 714, in walk
    self.walk(child)
  File "/usr/local/lib/python2.7/dist-packages/pylint/utils.py", line 711, in walk
    cb(astroid)
  File "/usr/local/lib/python2.7/dist-packages/pylint/checkers/imports.py", line 269, in visit_from
    self._add_imported_module(node, '%s.%s' % (importedmodnode.name, name))
  File "/usr/local/lib/python2.7/dist-packages/pylint/checkers/imports.py", line 302, in _add_imported_module
    importedmodname = get_module_part(importedmodname)
  File "/usr/local/lib/python2.7/dist-packages/logilab/common/modutils.py", line 352, in get_module_part
    path=path, context_file=context_file)
  File "/usr/local/lib/python2.7/dist-packages/logilab/common/modutils.py", line 297, in file_from_modpath
    return _file_from_modpath(modpath, path, context)
  File "/usr/local/lib/python2.7/dist-packages/logilab/common/modutils.py", line 556, in _file_from_modpath
    mtype, mp_filename = _module_file(modpath, path)
  File "/usr/local/lib/python2.7/dist-packages/logilab/common/modutils.py", line 636, in _module_file
    return _search_zip(modpath, pic)[:2]
  File "/usr/local/lib/python2.7/dist-packages/logilab/common/modutils.py", line 577, in _search_zip
    raise ImportError('No module named %s' % '.'.join(modpath))
ImportError: No module named marrow

[pylint-bot]

Original comment by Jeff McGee (BitBucket: jeffamcgee, GitHub: @JeffAMcGee?):

---

I'm running into this problem with zope.interface. Here's a small test case that crashes pylint:

#!sh

#!/bin/bash
mkvirtualenv pylint_zope_test
pip install zope.interface==4.1.1 pylint==1.2.1
echo "import zope.interface">pylint_zope_test.py
pylint pylint_zope_test.py

[pylint-bot]

Original comment by Francois Vanderkelen (BitBucket: sikwan, GitHub: @Sikwan?):

---

I have the exact same issue with one of my company's project. Just found out that when I am running pylint on an empty environement (just containing pylint) and no packages from my namespace, it is running completely.

Thing is all dependencies used in the project from the same namespace cannot be installed beforehand.

I am not quite sure the results are relevant, since it never worked on my project ... I will check that and keep you posted.

[pylint-bot]

Original comment by Isis Lovecruft (BitBucket: isislovecruft, GitHub: @isislovecruft?):

---

I am currently experiencing the same issue, while attempting to run pylint-1.2.1 on one of my work repositories. I am running inside a virtualenv.

For what it's worth, this is the traceback from the crash:

Traceback (most recent call last):
  File "/home/isis/.virtualenvs/bridgedb/bin/pylint", line 9, in <module>
    load_entry_point('pylint==1.2.1', 'console_scripts', 'pylint')()
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/pylint/__init__.py", line 21, in run_pylint
    Run(sys.argv[1:])
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/pylint/lint.py", line 1051, in __init__
    linter.check(args)
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/pylint/lint.py", line 626, in check
    self.check_astroid_module(astroid, walker, rawcheckers, tokencheckers)
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/pylint/lint.py", line 712, in check_astroid_module
    walker.walk(astroid)
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/pylint/utils.py", line 715, in walk
    self.walk(child)
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/pylint/utils.py", line 712, in walk
    cb(astroid)
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/pylint/checkers/imports.py", line 269, in visit_from
    self._add_imported_module(node, '%s.%s' % (importedmodnode.name, name))
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/pylint/checkers/imports.py", line 302, in _add_imported_module
    importedmodname = get_module_part(importedmodname)
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/logilab/common/modutils.py", line 352, in get_module_part
    path=path, context_file=context_file)
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/logilab/common/modutils.py", line 297, in file_from_modpath
    return _file_from_modpath(modpath, path, context)
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/logilab/common/modutils.py", line 556, in _file_from_modpath
    mtype, mp_filename = _module_file(modpath, path)
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/logilab/common/modutils.py", line 636, in _module_file
    return _search_zip(modpath, pic)[:2]
  File "/home/isis/.virtualenvs/bridgedb/local/lib/python2.7/site-packages/logilab/common/modutils.py", line 577, in _search_zip
    raise ImportError('No module named %s' % '.'.join(modpath))
ImportError: No module named zope

Disabling F0401 does not fix it, neither does setting [TYPECHECK] zope=yes in my rcfile. Next, I tried adding the following init-hook to my pylint rcfile:

init-hook='import sys,os; venv = os.environ.get("VIRTUAL_ENV"); v = os.path.join(venv, "lib", "python" + str(sys.version_info.major) + "." + str(sys.version_info.micro), "site-packages") if venv is not None else None; sys.path.insert(0, v);'

which did not fix this issue either. The traceback is always on the zope namespace package.

However, downgrading to pylint-1.1.0 does fix this issue, so it seems that this bug was introduced somewhere between 1.1.0 and 1.2.0.

[pylint-bot]

Original comment by Francois Vanderkelen (BitBucket: sikwan, GitHub: @Sikwan?):

---

Tried to rollback to 1.1.0 to see what how it was going.

The ImportError is indeed not there, but Pylint cannot process some imports.

For example :

(This is just dummy arborescence typed in, hence the missing init files )

• company
  • name
    • user.py
    • acl.py

acl.py

#!python
from user import User

Where User is a class declared in user.py. (Not dynamically, just a standard class)

Results in :

E: 5, 0: No name 'User' in module 'user' (no-name-in-module)

So I changed to relative import with a ".", same error. Tried also absolute path which results in :

E: 5, 0: No name 'module' in module 'company' (no-name-in-module)

Let me know if you want me to put code on Bitbucket so you can run it by yourselves.

[pylint-bot]

Original comment by Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):

---

The ImportError is due to a change in the 'import' checker, I don't think pylint/astroid import capabilities changed otherwise. @Sikwan I'm not able to reproduce your problem:

#!sh

[syt@orion tmp]$ tree company/
company/
├── __init__.py
└── name
    ├── acl.py
    ├── __init__.py
    └── user.py

1 directory, 4 files
[syt@orion tmp]$ cat company/name/user.py 
class User(object):
    pass
[syt@orion tmp]$ cat company/name/acl.py 
from .user import User
from user import User as U2

print User
print U2
[syt@orion tmp]$ pylint -rn --disable=missing-docstring company
************* Module company.name.user
R:  1, 0: Too few public methods (0/2) (too-few-public-methods)
************* Module company.name.acl
W:  2, 0: Relative import 'user', should be 'company.name.user' (relative-import)

[pylint-bot]

Original comment by Sylvain Thénault (BitBucket: sthenault, GitHub: @sthenault?):

---

fix issue #203, we should not let ImportError propagate from the import checker

[pylint-bot]

Original comment by Francois Vanderkelen (BitBucket: sikwan, GitHub: @Sikwan?):

---

I have to admit my example is not working anymore (I destroyed my virtualenv, so I guess the version of setuptools was different ?).

I will try out the fix on my repositories and try to provide an example if this is not working !

Thanks for the fix :)