Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code attempted executed when checking module with name 'platform' #3386

Closed
Tenzer opened this issue Feb 5, 2020 · 7 comments · Fixed by #3396
Closed

Code attempted executed when checking module with name 'platform' #3386

Tenzer opened this issue Feb 5, 2020 · 7 comments · Fixed by #3396
Assignees
Labels
Blocker 🙅 Blocks the next release Bug 🪲 Crash 💥 A bug that makes pylint crash

Comments

@Tenzer
Copy link

Tenzer commented Feb 5, 2020

This might be a problem with Astroid, but I'll try here first as it's easy to reproduce with Pylint:

Steps to reproduce

  1. echo "import non_existing_module" > platform.py
  2. pylint platform.py

Current behavior

Gives the following stack trace:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 263, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/jeppe/.bin/pylint", line 5, in <module>
    from pylint import run_pylint
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/__init__.py", line 13, in <module>
    from pylint.checkers.similar import Run as SimilarRun
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/checkers/__init__.py", line 42, in <module>
    from pylint.checkers.base_checker import BaseChecker, BaseTokenChecker
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/checkers/base_checker.py", line 17, in <module>
    from pylint.config import OptionsProviderMixIn
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/config.py", line 49, in <module>
    from pylint import utils
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/utils/__init__.py", line 44, in <module>
    from pylint.utils.ast_walker import ASTWalker
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 8, in <module>
    from astroid import nodes
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/astroid/__init__.py", line 63, in <module>
    from astroid.nodes import *
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/astroid/nodes.py", line 23, in <module>
    from astroid.node_classes import (
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/astroid/node_classes.py", line 38, in <module>
    from astroid import bases
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/astroid/bases.py", line 32, in <module>
    MANAGER = manager.AstroidManager()
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/astroid/util.py", line 26, in <lambda>
    lambda: importlib.import_module("." + module_name, "astroid")
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/astroid/manager.py", line 25, in <module>
    from astroid import modutils
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/astroid/modutils.py", line 32, in <module>
    import platform
  File "/Users/jeppe/platform.py", line 1, in <module>
    import non_existing_module
ModuleNotFoundError: No module named 'non_existing_module'
Traceback (most recent call last):
  File "/Users/jeppe/.bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/__init__.py", line 23, in run_pylint
    PylintRun(sys.argv[1:])
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/lint.py", line 1731, in __init__
    linter.check(args)
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/lint.py", line 1006, in check
    self._parallel_check(files_or_modules)
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/lint.py", line 1080, in _parallel_check
    for result in self._parallel_task(files_or_modules):
  File "/Users/jeppe/.virtualenvs/sublime-tools/lib/python3.8/site-packages/pylint/lint.py", line 1028, in _parallel_task
    manager = multiprocessing.Manager()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/context.py", line 57, in Manager
    m.start()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/managers.py", line 583, in start
    self._address = reader.recv()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
    buf = self._recv(4)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/connection.py", line 383, in _recv
    raise EOFError
EOFError

Expected behavior

Pylint's checks will run like usual.

pylint --version output

[jeppe@reventon ~]$ pylint --version
pylint 2.4.4
astroid 2.3.3
Python 3.8.1 (default, Jan  7 2020, 10:39:14)
[Clang 11.0.0 (clang-1100.0.33.16)]

I am not able to reproduce this on Python 3.7.6.

I've also been able to reproduce the problem with slightly different stack traces by naming the file the following:

  • __future__.py
  • array.py
  • ast.py
  • astroid.py
  • brain_numpy_utils.py
  • collections.py
  • configparser.py
  • contextlib.py

and there are probably many more. I tried to go for the list of modules imported by Astroid when it is executing.

@PCManticore PCManticore added Bug 🪲 Crash 💥 A bug that makes pylint crash labels Feb 5, 2020
@PCManticore PCManticore pinned this issue Feb 5, 2020
@PCManticore PCManticore added the Blocker 🙅 Blocks the next release label Feb 5, 2020
@PCManticore
Copy link
Contributor

Thanks for the report! We shouldn't import any modules when they are named like components of astroid of pylint. There's probably an import hook or a current path modification that is currently causing this. I'm marking this as a blocker for the 2.5 release since it's rather serious and there are already a handful of similar reports.

@PCManticore PCManticore self-assigned this Feb 5, 2020
PCManticore added a commit that referenced this issue Feb 11, 2020
``python -m pylint ...`` adds the current working directory as the first element
of ``sys.path``. This opens up a potential security hole where ``pylint`` will import
user level code as long as that code resides in modules having the same name as stdlib
or pylint's own modules.

Close #3386
PCManticore added a commit that referenced this issue Feb 11, 2020
``python -m pylint ...`` adds the current working directory as the first element
of ``sys.path``. This opens up a potential security hole where ``pylint`` will import
user level code as long as that code resides in modules having the same name as stdlib
or pylint's own modules.

Close #3386
PCManticore added a commit that referenced this issue Feb 11, 2020
``python -m pylint ...`` adds the current working directory as the first element
of ``sys.path``. This opens up a potential security hole where ``pylint`` will import
user level code as long as that code resides in modules having the same name as stdlib
or pylint's own modules.

Close #3386
@PCManticore
Copy link
Contributor

@Tenzer This should be fixed with the latest master branch. Can you give it a go and let us know if it works for you?

@Tenzer
Copy link
Author

Tenzer commented Feb 11, 2020

The minimal test case I added in the OP (echo "import non_existing_module" > platform.py; pylint platform.py) still fail, although the behaviour and the stack trace is slightly different now:

[jeppe@reventon ~]$ pylint platform.py
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/pylint/lint.py", line 72, in <module>
    import astroid
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/__init__.py", line 63, in <module>
    from astroid.nodes import *
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/nodes.py", line 23, in <module>
    from astroid.node_classes import (
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/node_classes.py", line 38, in <module>
    from astroid import bases
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/bases.py", line 32, in <module>
    MANAGER = manager.AstroidManager()
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/util.py", line 26, in <lambda>
    lambda: importlib.import_module("." + module_name, "astroid")
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/manager.py", line 25, in <module>
    from astroid import modutils
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/modutils.py", line 32, in <module>
    import platform
  File "/Users/jeppe/platform.py", line 1, in <module>
    import non_existing_module
ModuleNotFoundError: No module named 'non_existing_module'
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 513, in _handle_workers
    cls._maintain_pool(ctx, Process, processes, pool, inqueue,
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 337, in _maintain_pool
    Pool._repopulate_pool_static(ctx, Process, processes, pool,
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_io.TextIOWrapper' object
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/pylint/lint.py", line 72, in <module>
    import astroid
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/__init__.py", line 63, in <module>
    from astroid.nodes import *
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/nodes.py", line 23, in <module>
    from astroid.node_classes import (
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/node_classes.py", line 38, in <module>
    from astroid import bases
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/bases.py", line 32, in <module>
    MANAGER = manager.AstroidManager()
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/util.py", line 26, in <lambda>
    lambda: importlib.import_module("." + module_name, "astroid")
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/manager.py", line 25, in <module>
    from astroid import modutils
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/astroid/modutils.py", line 32, in <module>
    import platform
  File "/Users/jeppe/platform.py", line 1, in <module>
    import non_existing_module
ModuleNotFoundError: No module named 'non_existing_module'

At this point the command hangs and only exits after I press Ctrl+C twice.

If I name the file astroid.py the stack trace doesn't involve the Astroid code at all which might be easier to debug:

[jeppe@reventon ~]$ pylint astroid.py
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/pylint/lint.py", line 72, in <module>
    import astroid
  File "/Users/jeppe/astroid.py", line 1, in <module>
    import non_existing_module
ModuleNotFoundError: No module named 'non_existing_module'
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 513, in _handle_workers
    cls._maintain_pool(ctx, Process, processes, pool, inqueue,
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 337, in _maintain_pool
    Pool._repopulate_pool_static(ctx, Process, processes, pool,
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_io.TextIOWrapper' object
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/Users/jeppe/.virtualenvs/tmp-310d6d1bd409ad7/lib/python3.8/site-packages/pylint/lint.py", line 72, in <module>
    import astroid
  File "/Users/jeppe/astroid.py", line 1, in <module>
    import non_existing_module
ModuleNotFoundError: No module named 'non_existing_module'

@PCManticore
Copy link
Contributor

Thanks for confirming! Just to make sure I understand, you are using pylint <...> and multiple jobs? Does it reproduce for you with python -m pylint <...> and no a single job? How about python -m pylint <...> and multiple jobs?

@PCManticore PCManticore reopened this Feb 11, 2020
@Tenzer
Copy link
Author

Tenzer commented Feb 11, 2020

I've just renamed my ~/.pylintrc file to make sure it wasn't being used. This makes a difference!

With the latest released version:

[jeppe@reventon ~]$ pylint --version
pylint 2.4.4
astroid 2.3.3
Python 3.8.1 (default, Jan  7 2020, 10:39:14)
[Clang 11.0.0 (clang-1100.0.33.16)]
[jeppe@reventon ~]$ python -m pylint astroid.py
Traceback (most recent call last):
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 184, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 143, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/Users/jeppe/.virtualenvs/tmp-24b606f49d8419f/lib/python3.8/site-packages/pylint/__init__.py", line 13, in <module>
    from pylint.checkers.similar import Run as SimilarRun
  File "/Users/jeppe/.virtualenvs/tmp-24b606f49d8419f/lib/python3.8/site-packages/pylint/checkers/__init__.py", line 42, in <module>
    from pylint.checkers.base_checker import BaseChecker, BaseTokenChecker
  File "/Users/jeppe/.virtualenvs/tmp-24b606f49d8419f/lib/python3.8/site-packages/pylint/checkers/base_checker.py", line 17, in <module>
    from pylint.config import OptionsProviderMixIn
  File "/Users/jeppe/.virtualenvs/tmp-24b606f49d8419f/lib/python3.8/site-packages/pylint/config.py", line 49, in <module>
    from pylint import utils
  File "/Users/jeppe/.virtualenvs/tmp-24b606f49d8419f/lib/python3.8/site-packages/pylint/utils/__init__.py", line 44, in <module>
    from pylint.utils.ast_walker import ASTWalker
  File "/Users/jeppe/.virtualenvs/tmp-24b606f49d8419f/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 8, in <module>
    from astroid import nodes
  File "/Users/jeppe/astroid.py", line 1, in <module>
    import non_existing_module
ModuleNotFoundError: No module named 'non_existing_module'

With the version from the master branch:

[jeppe@reventon ~]$ pylint --version
pylint 2.5.0-dev1
astroid 2.3.3
Python 3.8.1 (default, Jan  7 2020, 10:39:14)
[Clang 11.0.0 (clang-1100.0.33.16)]
[jeppe@reventon ~]$ python -m pylint astroid.py
************* Module astroid
astroid.py:1:0: C0114: Missing module docstring (missing-module-docstring)
astroid.py:1:0: E0401: Unable to import 'non_existing_module' (import-error)
astroid.py:1:0: W0611: Unused import non_existing_module (unused-import)

----------------------------------------------------------------------
Your code has been rated at -60.00/10 (previous run: -60.00/10, +0.00)

Enabling multiple jobs makes it break again:

[jeppe@reventon ~]$ python -m pylint --jobs=2 astroid.py
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/Users/jeppe/.virtualenvs/tmp-ddb14cab8fa647b/lib/python3.8/site-packages/pylint/lint.py", line 72, in <module>
    import astroid
  File "/Users/jeppe/astroid.py", line 1, in <module>
    import non_existing_module
ModuleNotFoundError: No module named 'non_existing_module'
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 513, in _handle_workers
    cls._maintain_pool(ctx, Process, processes, pool, inqueue,
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 337, in _maintain_pool
    Pool._repopulate_pool_static(ctx, Process, processes, pool,
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_io.TextIOWrapper' object
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/Users/jeppe/.virtualenvs/tmp-ddb14cab8fa647b/lib/python3.8/site-packages/pylint/lint.py", line 72, in <module>
    import astroid
  File "/Users/jeppe/astroid.py", line 1, in <module>
    import non_existing_module
ModuleNotFoundError: No module named 'non_existing_module'

Also, doesn't seem to help on all file names, collections.py for instance still fails with this stack trace:

[jeppe@reventon ~]$ python -m pylint collections.py
Could not import runpy module
Traceback (most recent call last):
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/runpy.py", line 15, in <module>
    import importlib.util
  File "/Users/jeppe/.virtualenvs/tmp-ddb14cab8fa647b/lib/python3.8/importlib/util.py", line 14, in <module>
    from contextlib import contextmanager
  File "/Users/jeppe/.pyenv/versions/3.8.1/lib/python3.8/contextlib.py", line 5, in <module>
    from collections import deque
  File "/Users/jeppe/collections.py", line 1, in <module>
    import non_existing_module
ModuleNotFoundError: No module named 'non_existing_module'

I hope that helps.

@PCManticore PCManticore unpinned this issue Mar 12, 2020
@PCManticore
Copy link
Contributor

This has been fixed with #3411. The only issue that remains is the collections one, but that is caused by a bug in Python so there's not much we can do: https://bugs.python.org/issue33053

@Tenzer
Copy link
Author

Tenzer commented Mar 16, 2020

Great! Thank you for fixing this!

vmiklos added a commit to vmiklos/osm-gimmisn that referenced this issue May 8, 2020
Which means we need to explicitly set PYTHONPATH, otherwise test_foo
won't find module foo, see
<pylint-dev/pylint#3386>.
vmiklos added a commit to vmiklos/osm-gimmisn that referenced this issue May 8, 2020
Which means we need to explicitly set PYTHONPATH, otherwise test_foo
won't find module foo, see
<pylint-dev/pylint#3386>.

Tested-by: Travis
timricese pushed a commit to timricese/orm that referenced this issue Jun 25, 2020
@DanielNoord DanielNoord mentioned this issue Apr 20, 2022
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Blocker 🙅 Blocks the next release Bug 🪲 Crash 💥 A bug that makes pylint crash
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants