DistributionNotFound for typing-extensions with Python 3.10

[williamjmorenor]

Your Environment

Thank you for taking the time to report an issue.

To more efficiently resolve this issue, we'd like to know some basic information about your system and setup.

1. Your operating system:
   Fedora 32, Windows 10 and the lastes ubuntu version available in github actions

2. Version of python you are running:
   3.10.0-alpha.3

3. How did you install twine? Did you use your operating system's package manager or pip or something else?
   pip install twine

4. Version of twine you have installed (include complete output of):
   twine-3.3.0-py3-none-any.whl

5. Which package repository are you targeting?
   Pip

Running with python 3.10.0-alpha.3 i got:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.0-alpha.3/x64/bin/twine", line 8, in <module>
    sys.exit(main())
  File "/opt/hostedtoolcache/Python/3.10.0-alpha.3/x64/lib/python3.10/site-packages/twine/__main__.py", line 28, in main
    result = cli.dispatch(sys.argv[1:])
  File "/opt/hostedtoolcache/Python/3.10.0-alpha.3/x64/lib/python3.10/site-packages/twine/cli.py", line 80, in dispatch
    main = registered_commands[args.command].load()
  File "/opt/hostedtoolcache/Python/3.10.0-alpha.3/x64/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2461, in load
    self.require(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.0-alpha.3/x64/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2484, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/opt/hostedtoolcache/Python/3.10.0-alpha.3/x64/lib/python3.10/site-packages/pkg_resources/__init__.py", line 787, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'typing-extensions>=3.6.4; python_version < "3.8"' distribution was not found and is required by importlib-metadata

Maybe it's comparing versions wrong and thinking '3.10' < '3.8'

[williamjmorenor]

References:

• python 3.10: 'typing-extensions>=3.6.4; python_version < "3.8"' distribution was not found and is required by importlib-metadata python/typing#771
• 'typing-extensions>=3.6.4; python_version < "3.8"' distribution was not found and is required by importlib-metadata build#201
• DistributionNotFound for typing-extensions with Python 3.10 python/importlib_metadata#271

[sigmavirus24]

pkg_resources.DistributionNotFound: The 'typing-extensions>=3.6.4; python_version < "3.8"' distribution was not found and is required by importlib-metadata

This line indicates quite clearly this is a bug in importlib-metadata which we rely on. Please open an issue there.

Further, if the bug is that something is comparing 3.10 < 3.8 and lexically sorting things, that's still not a bug in Twine.

I'll leave this open in the event someone else runs into it, but it's not a bug in Twine

[layday]

The issue is with pkg_resources. Seeing as twine already has a dependency on importlib-metadata, iter_entry_points in cli.py could be replaced with the equivalent function from importlib-metadata.

[bhrutledge]

I was able to reproduce this building https://github.com/bhrutledge/example-pkg-bhrutledge using a venv created from 3.10-dev installed via pyenv:

$ source /tmp/venv-310/bin/activate

(venv-310)$ pip install --no-cache-dir build twine

(venv-310)$ python3 -m build

$ twine check dist/*
Traceback (most recent call last):
  ...
pkg_resources.DistributionNotFound: The 'importlib-metadata; python_version < "3.8"' distribution was not found and is required by the application

However, noting that pkg_resources is distributed via setuptools, I upgraded setuptools, and that fixed the issue:

(venv-310)$ pip install -U setuptools
  ...
      Successfully uninstalled setuptools-47.1.0
Successfully installed setuptools-51.1.1

(venv-310)$ twine check dist/*
Checking dist/example_pkg_bhrutledge-0.0.4a4-py3-none-any.whl: PASSED
Checking dist/example-pkg-bhrutledge-0.0.4a4.tar.gz: PASSED

I haven't dug in further than that; it'd be nice to find out the specific commit/PR/release in setuptools where this was resolved.

That said, I tend to agree that this is not an issue with Twine, though it does seem like not relying on pkg_resources could mitigate it.

[bhrutledge]

A binary chop helped me discover that this was resolved in setuptools v49.0.0. The only thing that jumped out from the setuptools changelog was "Update vendored packaging in pkg_resources to 19.2" via pypa/setuptools#2180. Sure enough, the packaging changelog for v19.1 points to pypa/packaging#119 dealing w/ 2-digit versions, which was resolved by pypa/packaging#165.

@williamjmorenor I think you can resolve this for your workflow by upgrading setuptools where you upgrade pip, i.e. pip install --upgrade setuptools pip. I think that's a common practice when setting up a new Python environment.

I'm going to leave this open for now to see if anyone wants to offer additional thoughts/suggestions. For example, should we avoid using pkg_resources as @layday suggests, to avoid this confusing error for users who don't upgrade setuptools?

[sigmavirus24]

For example, should we avoid using pkg_resources as @layday suggests, to avoid this confusing error for users who don't upgrade setuptools?

The fastest fix is to not verify dependencies via pkg_resources by calling resolve() instead of require() on the entry point returned (or, on very old setuptools versions calling load(require=False). (The default behaviour is to verify the entry-point's requires list on disk.)

I think importlib-metadata provides a similar (and possibly faster) alternative to iter_entry_points from what I remember.

[bhrutledge]

@jaraco has started work on replacing pkg_resources in #728

[bhrutledge]

With the merge of #728, I think this is resolved. Feel free to reopen if it's not.