flake8 >= 2.2.0 fails with pep8 1.5.7 due to pkg_resources.VersionConflict with mysterious empty list

[asottile]

In GitLab by @dolphm on Oct 21, 2014, 06:09

I'm hoping this is a bug in setuptools or something, but I'm stumped. I've nuked my Python install on OS X (using homebrew), re-installed pip, setuptools, flake8, and pep8 from scratch (using brew-installed pip), and this is easily reproduced. It's been ongoing for over a week, perhaps since the release of https://pypi.python.org/pypi/setuptools/6.1 which has a related fix https://bitbucket.org/pypa/setuptools/issue/268 -- but reverting to setuptools 6.0.2 does not resolve the issue (also tried a much older setuptools==5.8 without success).

To reproduce (on my system, anyway):

$ pip install --upgrade setuptools flake8
Requirement already up-to-date: setuptools in /usr/local/lib/python2.7/site-packages
Downloading/unpacking flake8 from https://pypi.python.org/packages/source/f/flake8/flake8-2.2.5.tar.gz#md5=6dea927949b94c9d9495ab24bcdf9cf0
Using download cache from /Users/dolph/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Ff%2Fflake8%2Fflake8-2.2.5.tar.gz
Running setup.py (path:/private/var/folders/n5/s3phyknn2p18ls0ml6s4wm640000gn/T/pip_build_dolph/flake8/setup.py) egg_info for package flake8
    
Requirement already up-to-date: pyflakes>=0.8.1 in /usr/local/lib/python2.7/site-packages (from flake8)
Downloading/unpacking pep8>=1.5.7 (from flake8)
Using download cache from /Users/dolph/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2F3.3%2Fp%2Fpep8%2Fpep8-1.5.7-py2.py3-none-any.whl
Requirement already up-to-date: mccabe>=0.2.1 in /usr/local/lib/python2.7/site-packages (from flake8)
Installing collected packages: flake8, pep8
Found existing installation: flake8 2.1.0
    Uninstalling flake8:
    Successfully uninstalled flake8
Running setup.py install for flake8
    
    Installing flake8 script to /usr/local/bin
Found existing installation: pep8 1.5.6
    Uninstalling pep8:
    Successfully uninstalled pep8
Successfully installed flake8 pep8
Cleaning up...

And the subsequent error:

$ flake8 --version
Traceback (most recent call last):
File "/usr/local/bin/flake8", line 9, in <module>
    load_entry_point('flake8==2.2.5', 'console_scripts', 'flake8')()
File "/usr/local/lib/python2.7/site-packages/flake8/main.py", line 24, in main
    flake8_style = get_style_guide(parse_argv=True, config_file=DEFAULT_CONFIG)
File "/usr/local/lib/python2.7/site-packages/flake8/engine.py", line 102, in get_style_guide
    kwargs['parser'], options_hooks = get_parser()
File "/usr/local/lib/python2.7/site-packages/flake8/engine.py", line 55, in get_parser
    (extensions, parser_hooks, options_hooks) = _register_extensions()
File "/usr/local/lib/python2.7/site-packages/flake8/engine.py", line 28, in _register_extensions
    checker = entry.load()
File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 2182, in load
    self.require(env, installer)
File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 2196, in require
    items = working_set.resolve(reqs, env, installer)
File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 632, in resolve
    raise VersionConflict(tmpl % args)
pkg_resources.VersionConflict: pep8 1.5.7 is installed but pep8==1.5.6 is required by []

(pep8==1.5.6 is required by WHAT?!)

The only workaround I can find is installing flake8==2.1.0 and pep8==1.5.6 (shown below):

$ pip install --upgrade setuptools "flake8==2.1.0" "pep8==1.5.6"
Downloading/unpacking setuptools from https://pypi.python.org/packages/3.4/s/setuptools/setuptools-7.0-py2.py3-none-any.whl#md5=918e7e5ea108507e1ffbbdfccc3496b1
Using download cache from /Users/dolph/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2F3.4%2Fs%2Fsetuptools%2Fsetuptools-7.0-py2.py3-none-any.whl
Downloading/unpacking flake8==2.1.0
Using download cache from /Users/dolph/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Ff%2Fflake8%2Fflake8-2.1.0.tar.gz
Running setup.py (path:/private/var/folders/n5/s3phyknn2p18ls0ml6s4wm640000gn/T/pip_build_dolph/flake8/setup.py) egg_info for package flake8

Downloading/unpacking pep8==1.5.6
Using download cache from /Users/dolph/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2F3.3%2Fp%2Fpep8%2Fpep8-1.5.6-py2.py3-none-any.whl
Requirement already up-to-date: pyflakes>=0.7.3 in /usr/local/lib/python2.7/site-packages (from flake8==2.1.0)
Requirement already up-to-date: mccabe>=0.2.1 in /usr/local/lib/python2.7/site-packages (from flake8==2.1.0)
Installing collected packages: setuptools, flake8, pep8
Found existing installation: setuptools 5.8
    Uninstalling setuptools:
    Successfully uninstalled setuptools
Found existing installation: flake8 2.2.5
    Uninstalling flake8:
    Successfully uninstalled flake8
Running setup.py install for flake8

    Installing flake8 script to /usr/local/bin
Found existing installation: pep8 1.5.7
    Uninstalling pep8:
    Successfully uninstalled pep8
Successfully installed setuptools flake8 pep8
Cleaning up...

This combo works:

$ flake8 --version
2.1.0 (pep8: 1.5.6, pyflakes: 0.8.1, GitCheckCommitTitleBug: 0.0.1, GitCheckCommitTitleLength: 0.0.1, GitCheckCommitTitlePeriodEnding: 0.0.1, hacking.core: 0.0.1, ProxyChecker: 0.0.1, mccabe: 0.2.1) CPython 2.7.8 on Darwin

For funsies:

$ python -c "import setuptools; print(setuptools.__version__)"
7.0

[asottile]

In GitLab by @sigmavirus24 on Oct 21, 2014, 08:13

So I can reproduce this with the following steps:

$ mkvirtualenv --version
1.11.6
$ mktmpenv
$ pip install flake8 hacking
$ curl -O https://bootstrap.pypa.io/get-pip.py
$ flake8 get-pip.py
Traceback (most recent call last):
  File "/Users/ian7708/virtualenv/tmp-362ac240fd778619/bin/flake8", line 9, in <module>
    load_entry_point('flake8==2.2.5', 'console_scripts', 'flake8')()
  File "/Users/ian7708/virtualenv/tmp-362ac240fd778619/lib/python2.7/site-packages/flake8/main.py", line 24, in main
    flake8_style = get_style_guide(parse_argv=True, config_file=DEFAULT_CONFIG)
  File "/Users/ian7708/virtualenv/tmp-362ac240fd778619/lib/python2.7/site-packages/flake8/engine.py", line 102, in get_style_guide
    kwargs['parser'], options_hooks = get_parser()
  File "/Users/ian7708/virtualenv/tmp-362ac240fd778619/lib/python2.7/site-packages/flake8/engine.py", line 55, in get_parser
    (extensions, parser_hooks, options_hooks) = _register_extensions()
  File "/Users/ian7708/virtualenv/tmp-362ac240fd778619/lib/python2.7/site-packages/flake8/engine.py", line 28, in _register_extensions
    checker = entry.load()
  File "/Users/ian7708/virtualenv/tmp-362ac240fd778619/lib/python2.7/site-packages/pkg_resources.py", line 2145, in load
    if require: self.require(env, installer)
  File "/Users/ian7708/virtualenv/tmp-362ac240fd778619/lib/python2.7/site-packages/pkg_resources.py", line 2159, in require
    items = working_set.resolve(reqs, env, installer)
  File "/Users/ian7708/virtualenv/tmp-362ac240fd778619/lib/python2.7/site-packages/pkg_resources.py", line 643, in resolve
    raise VersionConflict(dist, req) # XXX put more info here
pkg_resources.VersionConflict: (pep8 1.5.7 (/Users/ian7708/virtualenv/tmp-362ac240fd778619/lib/python2.7/site-packages), Requirement.parse('pep8==1.5.6'))
$ python -c 'import setuptools; print(setuptools.__version__)'
3.6
$ pip install -U setuptools
$ flake8 get-pip.py
# same traceback
$ pip install -U --force-reinstall hacking
Downloading/unpacking hacking
  Downloading hacking-0.9.2-py2-none-any.whl
Downloading/unpacking pbr>=0.6,!=0.7,<1.0 (from hacking)
  Downloading pbr-0.10.0-py2.py3-none-any.whl (63kB): 63kB downloaded
Downloading/unpacking flake8==2.1.0 (from hacking)
  Downloading flake8-2.1.0.tar.gz
  Running setup.py (path:/Users/ian7708/virtualenv/tmp-362ac240fd778619/build/flake8/setup.py) egg_info for package flake8

Downloading/unpacking pyflakes==0.8.1 (from hacking)
  Downloading pyflakes-0.8.1-py2.py3-none-any.whl
Downloading/unpacking pep8==1.5.6 (from hacking)
  Downloading pep8-1.5.6-py2.py3-none-any.whl
Downloading/unpacking six>=1.6.0 (from hacking)
  Downloading six-1.8.0-py2.py3-none-any.whl
Downloading/unpacking pip (from pbr>=0.6,!=0.7,<1.0->hacking)
  Downloading pip-1.5.6-py2.py3-none-any.whl (1.0MB): 1.0MB downloaded
Downloading/unpacking mccabe>=0.2.1 (from flake8==2.1.0->hacking)
  Downloading mccabe-0.2.1.tar.gz
  Running setup.py (path:/Users/ian7708/virtualenv/tmp-362ac240fd778619/build/mccabe/setup.py) egg_info for package mccabe

Installing collected packages: hacking, pbr, flake8, pyflakes, pep8, six, pip, mccabe
  Found existing installation: hacking 0.9.2
    Uninstalling hacking:
      Successfully uninstalled hacking
  Found existing installation: pbr 0.10.0
    Uninstalling pbr:
      Successfully uninstalled pbr
  Found existing installation: flake8 2.2.5
    Uninstalling flake8:
      Successfully uninstalled flake8
  Running setup.py install for flake8

    Installing flake8 script to /Users/ian7708/virtualenv/tmp-362ac240fd778619/bin
  Found existing installation: pyflakes 0.8.1
    Uninstalling pyflakes:
      Successfully uninstalled pyflakes
  Found existing installation: pep8 1.5.7
    Uninstalling pep8:
      Successfully uninstalled pep8
  Found existing installation: six 1.8.0
    Uninstalling six:
      Successfully uninstalled six
  Found existing installation: pip 1.5.6
    Uninstalling pip:
      Successfully uninstalled pip
  Found existing installation: mccabe 0.2.1
    Uninstalling mccabe:
      Successfully uninstalled mccabe
  Running setup.py install for mccabe

Successfully installed hacking pbr flake8 pyflakes pep8 six pip mccabe
Cleaning up...
$ flake8 get-pip.py
get-pip.py:17420:13: F401 'setuptools' imported but unused

[asottile]

In GitLab by @sigmavirus24 on Oct 21, 2014, 08:18

When I first install flake8 and hacking together I see:

pip install flake8 hacking
Downloading/unpacking flake8
  Downloading flake8-2.2.5.tar.gz
  Running setup.py (path:/Users/ian7708/virtualenv/tmp-362ac240fd778619/build/flake8/setup.py) egg_info for package flake8

Downloading/unpacking hacking
  Downloading hacking-0.9.2-py2-none-any.whl
Downloading/unpacking pyflakes>=0.8.1 (from flake8)
  Downloading pyflakes-0.8.1-py2.py3-none-any.whl
Downloading/unpacking pep8>=1.5.7 (from flake8)
  Downloading pep8-1.5.7-py2.py3-none-any.whl
Downloading/unpacking mccabe>=0.2.1 (from flake8)
  Downloading mccabe-0.2.1.tar.gz
  Running setup.py (path:/Users/ian7708/virtualenv/tmp-362ac240fd778619/build/mccabe/setup.py) egg_info for package mccabe

Downloading/unpacking pbr>=0.6,!=0.7,<1.0 (from hacking)
  Downloading pbr-0.10.0-py2.py3-none-any.whl (63kB): 63kB downloaded
Downloading/unpacking six>=1.6.0 (from hacking)
  Downloading six-1.8.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): pip in ./lib/python2.7/site-packages (from pbr>=0.6,!=0.7,<1.0->hacking)
Installing collected packages: flake8, hacking, pyflakes, pep8, mccabe, pbr, six
  Running setup.py install for flake8

    Installing flake8 script to /Users/ian7708/virtualenv/tmp-362ac240fd778619/bin
  Running setup.py install for mccabe

Successfully installed flake8 hacking pyflakes pep8 mccabe pbr six
Cleaning up...

I'm setting up a new env to see what happens if I uninstall hacking/pbr after receiving the error. The stack trace points to the extension loading code and I'll have to debug into that to see which extension it's failing to load (I suspect, hacking)

[asottile]

In GitLab by @sigmavirus24 on Oct 21, 2014, 08:25

This still happens even if you upgrade to setuptools 7.0 after making a clean tmp environment.

[asottile]

In GitLab by @sigmavirus24 on Oct 21, 2014, 08:27

If you then do pip uninstall -y pbr hacking six flake8 works fine. Since I already tested installing flake8 first and then hacking and found no problems, this seems to be an issue with how hacking is installed.

[asottile]

In GitLab by @sigmavirus24 on Oct 21, 2014, 14:25

This is an unfortunate combination of version pinning and some part of the dependency checking failing. It's not a bug here. If you're using hacking, you should only be using pip install hacking to ensure that you get the appropriate versions of flake8, pep8, etc