New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Undocumented dependency on new-ish pip #940

Closed
jcsp opened this Issue Jan 24, 2017 · 12 comments

Comments

Projects
None yet
6 participants
@jcsp

jcsp commented Jan 24, 2017

As of now, anyone running "pip install --upgrade setuptools" on a CentOS 6 machine will find it fails horribly:

pip install --upgrade setuptools
Downloading/unpacking setuptools from https://pypi.python.org/packages/b1/d1/3df86b914d72b18d37266f964a11576f135eadbb7d2e3c5165377e4263bc/setuptools-34.0.1.zip#md5=9e907950a94d38c0295df0da2507c280
  Downloading setuptools-34.0.1.zip (617kB): 617kB downloaded
  Running setup.py egg_info for package setuptools
    Traceback (most recent call last):
      File "<string>", line 3, in <module>
      File "setuptools/__init__.py", line 10, in <module>
        from six.moves import filter, map
    ImportError: No module named six.moves
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 3, in <module>

  File "setuptools/__init__.py", line 10, in <module>

    from six.moves import filter, map

ImportError: No module named six.moves

I discovered this when a long-functioning build script started failing all of a sudden.

I gather from #937 and #938 that a more recent pip is required, and indeed installing it with pip install --upgrade pip does allow installation of setuptools to proceed.

May I suggest that the README should specify what version of pip is required, as it is a newer version than is present on many systems.

@jaraco

This comment has been minimized.

Member

jaraco commented Jan 24, 2017

I don't think there's a specific version of pip that's required. It probably varies by platform and maybe by Python version. At the very least, whatever pip version is used needs to rely on the wheel format and not the sdist. If you would like to try different versions, I could update the README or the CHANGES to indicate which version to use.

@jcsp

This comment has been minimized.

jcsp commented Jan 24, 2017

I can tell you that pip 1.4.1 does not work, and pip 9.0.1 does work.

If the required version was not known/is not knowable at the point the new dependency was introduced and pushed into the wild, I guess my best suggestion is a big health warning that people should install the latest pip if they're using the latest setuptools.

Alternatively, you could indicate which version of setuptools people should pin to in order to have something that will run on stable distributions that ship an older pip.

@shadowmint

This comment has been minimized.

shadowmint commented Jan 24, 2017

We're seeing this too with pip 1.5.4 on old machines with ubuntu 14.04 LTS and system pip installed.

As a workaround on machines with system pip versions which are too old I suggest:

virutalenv foo
./foo/bin/pip install -U pip
./foo/bin/pip install -r requirements,txt

fwiw, I think this is a breaking change to setuptools, not a minor release version (34.0 -> 34.0.1)

@adambkaplan

This comment has been minimized.

adambkaplan commented Jan 24, 2017

I'm also seeing this issue with our builds on CircleCI, running Ubuntu 12.04 (Precise).

@zroadhouse-rmn

This comment has been minimized.

zroadhouse-rmn commented Jan 24, 2017

We have a build pipeline that packs up all the sdists for our application and its dependencies and installs then using only the local index.

Does the workaround above (upgrade pip first) really depend on installing from the wheel format? I.e. is the sdist broken for setuptools?

@jaraco

This comment has been minimized.

Member

jaraco commented Jan 25, 2017

Upgrading pip first doesn't depend on installing from wheel format. Upgrading pip is necessary to install setuptools from a wheel. If you don't install setuptools from a wheel, you need to install setuptools' dependencies first (currently appdirs, packaging, six).

@zroadhouse-rmn

This comment has been minimized.

zroadhouse-rmn commented Jan 25, 2017

Ok. Very helpful. Perhaps worthy of a release note?

@jaraco

This comment has been minimized.

Member

jaraco commented Jan 25, 2017

@zroadhouse-rmn You mean beyond the release notes already made? The second paragraph of the change for v34.0.0 already mentions "pre-installing the Setuptools dependencies". Perhaps you could suggest some wording that would be clearer?

@jcsp

This comment has been minimized.

jcsp commented Jan 25, 2017

I think the clear wording would be something like (at the top, not in the body of text):
Important: setuptools is no longer installable with older versions of pip. If you are installing setuptools on an older linux distribution, you may need to install the latest pip before attempting to use it to install setuptools. If you seen an error like "ImportError: No module named six.moves", try updating pip before installing setuptools.

Because people will be running into this issue for a long time to come (these older versions of pip aren't going away any time soon), it would probably be prudent to put an ImportError handler in the code that gives people a clue about what's going on -- instead of "ImportError: No module named six.moves", it could say "Dependencies not found, are you trying to install with an older version of pip? "

@jaraco

This comment has been minimized.

Member

jaraco commented Jan 25, 2017

@jcsp Would you be willing to put together a PR?

b-schubert added a commit to FRED-2/ImmunoNodes that referenced this issue Jan 25, 2017

mattbenjamin added a commit to linuxbox2/s3-tests that referenced this issue Jan 25, 2017

bootstrap: force new setuptools
Deal setuptools upgrade problem discussed here:
 pypa/setuptools#940

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
@zroadhouse-rmn

This comment has been minimized.

zroadhouse-rmn commented Jan 25, 2017

@jaraco - I re-read the description in changes:

Users are welcome to contrive other means to install or upgrade Setuptools using other means, such as pre-installing the Setuptools dependencies with pip or a bespoke bootstrap tool, but such usage is not recommended and is not supported.

From this description, my use case is no longer supported (installing/upgrading setuptools from sdist). I'll likely proceed with the workaround you suggested above (installing pyparsing, packaging, appdirs, and six prior to installing setuptools). Longer term, I'll need to rethink our application packaging solution, which relies on sdist vs wheel to produce platform independent packages that fully contain all python dependencies.

mattbenjamin added a commit to linuxbox2/s3-tests that referenced this issue Jan 25, 2017

bootstrap: force new setuptools
Deal setuptools upgrade problem discussed here:
 pypa/setuptools#940

Now follows the same formula as upstream teuthology.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

mattbenjamin added a commit to ceph/s3-tests that referenced this issue Jan 25, 2017

bootstrap: force new setuptools
Deal setuptools upgrade problem discussed here:
 pypa/setuptools#940

Now follows the same formula as upstream teuthology.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

gaul added a commit to gaul/s3proxy that referenced this issue Jan 25, 2017

@marksteward

This comment has been minimized.

marksteward commented Aug 1, 2017

Ultimately fixed with setuptools 36 (see #980).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment