>= 0.15 has undeclared dependency on setuptools

[layday]

Describe the bug
pkg_resources, which is part of setuptools, is used to parse the pipx version number and is also used extensively in the venv_metadata_inspector. setuptools should not be assumed to be available in the runtime environment unless explicitly required.

How to reproduce
Install pipx in a virtual environment without setuptools:

$ python -m venv .venv
$ source .venv/bin/activate.fish
$ python -m pip uninstall setuptools
Uninstalling setuptools-41.2.0:
  Would remove:
    .venv/bin/easy_install
    .venv/bin/easy_install-3.8
    .venv/lib/python3.8/site-packages/easy_install.py
    .venv/lib/python3.8/site-packages/pkg_resources/*
    .venv/lib/python3.8/site-packages/setuptools-41.2.0.dist-info/*
    .venv/lib/python3.8/site-packages/setuptools/*
Proceed (y/n)? y
  Successfully uninstalled setuptools-41.2.0
$ python -m pip list
Package Version
------- -------
pip     19.2.3
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ python -m pip install pipx
Collecting pipx
  Downloading https://files.pythonhosted.org/packages/27/75/7a5762963703df84edc13c16ee6aaa3bcf559887a24c38d59d5e8a900ffa/pipx-0.15.1.2-py3-none-any.whl (43kB)
     |████████████████████████████████| 51kB 522kB/s
Collecting argcomplete<2.0,>=1.9.4 (from pipx)
  Downloading https://files.pythonhosted.org/packages/82/7d/455e149c28c320044cb763c23af375bd77d52baca041f611f5c2b4865cf4/argcomplete-1.11.1-py2.py3-none-any.whl
Collecting userpath (from pipx)
  Downloading https://files.pythonhosted.org/packages/8a/72/07927efb668a0aa0cef502dbe4da442ac9598f19344bca9a3eb9bd062ec1/userpath-1.3.0-py2.py3-none-any.whl
Collecting click (from userpath->pipx)
  Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Installing collected packages: argcomplete, click, userpath, pipx
Successfully installed argcomplete-1.11.1 click-7.0 pipx-0.15.1.2 userpath-1.3.0
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ pipx --version
Traceback (most recent call last):
  File ".venv/bin/pipx", line 6, in <module>
    from pipx.main import cli
  File ".venv/lib/python3.8/site-packages/pipx/main.py", line 9, in <module>
    from pkg_resources import parse_version
ModuleNotFoundError: No module named 'pkg_resources'

Expected behavior
As above.

[gaborbernat]

pkg_resources is not really supported going ahead, instead one should prefer the packaging module over pkg_resources.

[layday]

packaging does not interact with packages; to replace pkg_resources you would need to use a combination of importlib.metadata and packaging.

[gaborbernat]

True, depending on what subset of pkg_resources we use

[jaraco]

I've been working to eliminate all of the implicit dependencies on setuptools in my environments, especially in my main environment, where pipx resides, but this change means I'm stuck on the older pipx 0.14 until this issue is resolved.

[jaraco]

Investigating deeper, it seems the issue is pretty straightforward. Although pkg_resources is used in venv_metadata_inspector, its usage is optional (all imports are lenient). Only the usage in main.py is required. Replacing that usage with packaging allows the project to run again.

[itsayellow]

I was trying to find a reference on pkg_resources being deprecated and couldn't find one. @jaraco, can you add a link to something talking about pkg_resources being deprecated?

As for using pkg_resources in venv_metadata_inspector, that will not cause a setuptools dependency in pipx itself. We install setuptools in a separate venv to use for pipx-installed apps.

The new use of setuptools in pipx itself is only due to its usage in main.py, and you are right, that exists in 0.15.0.0.

[layday]

Pardon my prying, but I don't understand how __version_info__ is used. Importing packaging or pkg_resources at start-up does not come cheap; though packaging is definitely better in that regard:

$ python -X importtime -c 'import pkg_resources' 2>| tail -n 1
import time:     17122 |     118926 | pkg_resources
$ python -X importtime -c 'import packaging.version' 2>| tail -n 1
import time:      3299 |      10451 | packaging.version

[cs01]

Pardon my prying, but I don't understand how version_info is used

I don't either.

It was added by @clbarnes in #239, but it doesn't seem to be used.

[cs01]

Reopening since I believe __version_info__ can be removed as well as the packaging requirement.

[clbarnes]

I find __version_info__ pretty convenient when it exists in other packages, because version strings obviously don't necessarily sort correctly. It's metadata for the benefit of dependents more than it for ourselves. We as package contributors know our version string convention, but others don't; in most of my packages I use a simple {major}.{minor}.{patch} so adding __version_info__ doesn't require anything outside of stdlib, but as pipx uses more of the version string spec I thought it more maintainable to go with the semi-official solution.

I'm not precious about it if it's better to remove it!

[uranusjr]

An alternative I find ins some projects (e.g. Django) is to do it the other way around: define __version_info__ (or equivalent) with a tuple literal, and format a string from that. This has other implications (i.e. get_version() in setup.py would be more complex), but nothing insurmountable.

[clbarnes]

Yes, that is my preferred solution too; I don't use it in most places because automated versioning infrastructure like bumpversion needs the string literal to be the source of truth. IMO the best way to make the version string easily accessible to setup.py is to put __version__ (and __version_info__, if it makes it through the night 😉 ) into its own python file in the package. Then setup.py can use runpy.run_path to get it without any side-effects, and everything else can just import it.

[layday]

Consumers can pull in packaging.version if they need to compare versions. How often does this happen for non-library packages in particular?

[itsayellow]

I'm just wondering (I think along with @layday), does any package use pipx as a dependency? Isn't that the main reason to have things like __version_info__? If nobody is really using us as a library, maybe having an easily-parsable version variable is not so important.

[cs01]

If nobody is really using us as a library, maybe having an easily-parsable version variable is not so important.

pipx is like pip in this regard. Internal APIs are subject to breaking changes without semantic version bumps. The only supported API is the CLI.

An alternative I find ins some projects (e.g. Django) is to do it the other way around: define __version_info__(or equivalent) with a tuple literal, and format a string from that. This has other implications (i.e. get_version() in setup.py would be more complex), but nothing insurmountable.

If there is some reason to define a __version_info__ tuple (I don't really see one, but I know people use packages in all sorts of different ways), I like this suggestion. It is easy enough to derive a string from it and define that string a constant in main.py, leaving setup.py's parsing logic unchanged. If anyone wants to put a PR together with this I would be fine with it.

[clbarnes]

This branch has pipx's own version-handling updated that way: there's a version.py in the package which defines __version_info__ as a tuple, and then generates __version__ from that. setup.py gets the version by runpy.run_path-ing that file, and main.py just imports it.

[cs01]

@clbarnes LGTM, if you make a PR I'll approve.

[clbarnes]

I'm working on getting rid of pkg_resources from venv_metadata_inspector too, just having trouble with the record/installed-files.txt. Why can't awscli just be normal 🙄

[jaraco]

This issue hit me again today. I installed pipx and it immediately failed with a ModuleNotFoundError. Maybe consider adding the dependency on setuptools until it's not needed?

[gaborbernat]

@jaraco if you installed it via a recent pip this should not happen

[jaraco]

draft $ python -m venv env                                                                                                                                                
draft $ env/bin/pip install -q -U pip                                                                                                                                        
draft $ env/bin/pip uninstall -q -y setuptools
draft $ env/bin/pip install -q pipx                                                                                                                                       
draft $ env/bin/pipx --help                                                                                                                                               
Traceback (most recent call last):
  File "env/bin/pipx", line 5, in <module>
    from pipx.main import cli
  File "/Users/jaraco/draft/env/lib/python3.8/site-packages/pipx/main.py", line 9, in <module>
    from pkg_resources import parse_version
ModuleNotFoundError: No module named 'pkg_resources'

[layday]

A new version with the fix hasn't been released yet.

[jaraco]

Confirmed.

~ $ pip-run -q git+https://github.com/pipxproject/pipx -- -m pipx --help | head -n 1                                                                                      
usage: __main__.py [-h] [--version]

I probably worked around it by installing from master at the time. Let me rephrase my request.

How about release a version with the fix?

[itsayellow]

Can we close this? (Released) pipx v0.15.4.0 has no dependency on setuptools or pkg_resources. The remaining use of pkg_resources is in the package venvs, which does not depend on the python installation running pipx.

[jaraco]

Sounds good to me.