ImportWarning due to usage of deprecated method

[GergelyKalmar]

Steps to reproduce

It seems that the importer.find_module calls in the _search_zip function emit an ImportWarning when namespace packages are encountered:

https://github.com/PyCQA/astroid/blob/bff51e9d41a3084b9313fa2108ae13f079bc0b58/astroid/interpreter/_import/spec.py#L276-L292

This then breaks pytest-pylint runs in which warnings are turned into errors.

According to the importlib documentation find_module is deprecated and one should use find_spec instead. When patching the library locally this seems to have solved the issue with the namespace packages.

Current behavior

An ImportWarning is raised and the namespace package does not get imported. For example:
ImportWarning: Not importing directory /home/.venv/lib/python3.5/site-packages/google: missing __init__

Expected behavior

The package gets imported and no ImportWarning is issued.

Version

astroid == 2.3.1

[GergelyKalmar]

Hm, spent again an hour fighting with this just to end up finding my own issue from 6 months ago :).

The problem still persists with astroid==2.4.1. For now the only workaround is to turn off filterwarnings in pytest.ini for these errors:

filterwarnings =
    error
    ignore:.*Not importing directory.*:ImportWarning

Still, pylint keeps complaining about missing names when run through pytest-pylint in namespace packages so we need to keep adding # pylint: disable=no-member to every call when we're using Google Cloud packages for example. Could someone try to fix the mentioned deprecated call? :(

[GergelyKalmar]

In order to reproduce the issue one could use a requirements.txt file like this:

google-cloud-secret-manager==1.0.0
pylint==2.5.2
pytest==5.4.2
pytest-pylint==0.17.0

and then an example.py file like this:

from google.cloud import secretmanager

and a pytest.ini:

[pytest]
addopts = --pylint
filterwarnings = error

If one then runs pytest we get an ugly error:

platform linux -- Python 3.8.2, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: /home/gregory/Projects/pylint-error, inifile: pytest.ini
plugins: flake8-1.0.6, pylint-0.17.0
collected 1 item
--------------------------------------------------------------------------------
Linting files
.INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/gregory/.pyorbs/pylint-error/lib/python3.8/site-packages/astroid/manager.py", line 229, in file_from_module_name
INTERNALERROR>     value = self._mod_file_cache[(modname, contextfile)]
INTERNALERROR> KeyError: ('google', None)
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
...
traceback of a deep call stack ending up at an ImportWarning caused by this issue
...
INTERNALERROR>   File "<frozen importlib._bootstrap_external>", line 489, in _find_module_shim
INTERNALERROR> ImportWarning: Not importing directory /home/gregory/.pyorbs/pylint-error/lib/python3.8/site-packages/google: missing __init__

If one removes the filterwarnings line from pytest.ini it works as expected. Calling pylint directly also works fine (pylint example.py succeeds). However, calling pytest with the more strict filterwarnings option is just not possible, also, one gets a No name 'cloud' in module 'google' error with pylint when run through pytest.

[hippo91]

@GergelyKalmar thanks for your report, investigations and associated PR. I'm unable to reproduce the bug.
I apply the recipe you gave and simply run pytest in the directory but i got the following error:

ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --pylint

I'm not so familiar with pytest so excuse me if the solution is trivial.

[GergelyKalmar]

@hippo91 no worries at all! Did you create a virtual environment and installed the packages listed in requirements.txt? You would need to execute pytest inside the virtual environment with pytest-pylint installed, I think that should reproduce the bug.

[GergelyKalmar]

In particular, you may want to execute this in the folder where the files are saved:

python3 -m venv env
source env/bin/activate
pip install wheel
pip install -r requirements.txt
pytest

[hippo91]

@GergelyKalmar thanks for your indications. 👍 I succeeded in reproducing the bug. BTW i answer to the PR.