"ModuleNotFoundError: No module named 'html5lib'"

[slhck]

Short description

I want to run dephell in a Poetry project, but it fails with a missing module.

Output

Steps to reproduce

Use this pyproject.toml with current Poetry (v1.0.5):

[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = [""]

[tool.poetry.dependencies]
python = "^3.8"

[tool.poetry.dev-dependencies]
dephell = "^0.8.1"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

➜ poetry install
Creating virtualenv test-ZmL0gUDe-py3.8 in /home/werner/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (0.6s)

Writing lock file


Package operations: 40 installs, 0 updates, 0 removals

  - Installing pyparsing (2.4.6)
  - Installing six (1.14.0)
  - Installing packaging (20.3)
  - Installing attrs (19.3.0)
  - Installing certifi (2019.11.28)
  - Installing chardet (3.0.4)
  - Installing dephell-specifier (0.2.1)
  - Installing idna (2.9)
  - Installing multidict (4.7.5)
  - Installing ptyprocess (0.6.0)
  - Installing urllib3 (1.25.8)
  - Installing async-timeout (3.0.1)
  - Installing dephell-pythons (0.1.14)
  - Installing docutils (0.16)
  - Installing markupsafe (1.1.1)
  - Installing mistune (0.8.4)
  - Installing pexpect (4.8.0)
  - Installing requests (2.23.0)
  - Installing ruamel.yaml.clib (0.2.0)
  - Installing shellingham (1.3.2)
  - Installing yarl (1.4.2)
  - Installing aiohttp (3.6.2)
  - Installing cerberus (1.3.2)
  - Installing dephell-archive (0.1.6)
  - Installing dephell-argparse (0.1.2)
  - Installing dephell-changelogs (0.0.1)
  - Installing dephell-discover (0.2.10)
  - Installing dephell-licenses (0.1.6)
  - Installing dephell-links (0.1.5)
  - Installing dephell-markers (1.0.3)
  - Installing dephell-setuptools (0.2.3)
  - Installing dephell-shells (0.1.4)
  - Installing dephell-venvs (0.1.17)
  - Installing dephell-versioning (0.1.1)
  - Installing jinja2 (2.11.1)
  - Installing m2r (0.2.1)
  - Installing ruamel.yaml (0.16.10)
  - Installing tomlkit (0.5.11)
  - Installing yaspin (0.16.0)
  - Installing dephell (0.8.1)

➜ poetry run dephell
Traceback (most recent call last):
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/bin/dephell", line 6, in <module>
    from dephell.cli import entrypoint
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/cli.py", line 11, in <module>
    from .commands import COMMANDS
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/commands/__init__.py", line 2, in <module>
    from .discover import COMMANDS
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/commands/discover.py", line 82, in <module>
    module = import_module('.' + module_name, package=package)
  File "/home/werner/.pyenv/versions/3.8.1/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/commands/deps_add.py", line 7, in <module>
    from ..actions import get_resolver
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/actions/__init__.py", line 7, in <module>
    from ._converting import attach_deps
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/actions/_converting.py", line 7, in <module>
    from ..converters import CONVERTERS
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/converters/__init__.py", line 5, in <module>
    from .base import BaseConverter
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/converters/base.py", line 11, in <module>
    from ..controllers import Graph, Mutator, Resolver
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/controllers/__init__.py", line 3, in <module>
    from ._dependency import DependencyMaker
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/controllers/_dependency.py", line 18, in <module>
    from ..repositories import get_repo
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/repositories/__init__.py", line 3, in <module>
    from ._getter import get_repo
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/repositories/_getter.py", line 7, in <module>
    from ._git.git import GitRepo
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/repositories/_git/git.py", line 17, in <module>
    from .._local import LocalRepo
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/repositories/_local.py", line 11, in <module>
    from ._warehouse import WarehouseLocalRepo
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/repositories/_warehouse/__init__.py", line 5, in <module>
    from ._simple import WarehouseSimpleRepo
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/dephell/repositories/_warehouse/_simple.py", line 28, in <module>
    html5lib = import_module('html5lib')
  File "/home/werner/.pyenv/versions/3.8.1/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'html5lib'

Versions

Dephell: 0.8.1
Python: 3.8.1
Poetry: 1.0.5

[orsinium]

Hmm. Cannot reproduce on Python 3.7. Do you have 3.7 on your system? Could you try it, please? What about DepHell from master branch? What if extra full specified?

[slhck]

It works with Python 3.7.

I could not use full due to the issue in #254, but if I use the workaround (with the prerelease-specification for fissix), I get the same error as above. So full is not the issue.

When I use the master branch of dephell, I get a different error:

➜ poetry run dephell
Traceback (most recent call last):
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/bin/dephell", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    def _initialize_master_working_set():
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3234, in _call_aside
    f(*args, **kwargs)
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3263, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/werner/.cache/pypoetry/virtualenvs/test-ZmL0gUDe-py3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'ruamel.yaml' distribution was not found and is required by dephell

When I added full, I got the same.

[orsinium]

Ah, I see. It is easy to fix. However, I didn't get why it is different for different Pythons. Hmmm...

[slhck]

Awesome! Yeah, I'm not sure either what the problem is, but glad it could be easily solved.