Undocumented dependency on new-ish pip

[jcsp]

As of now, anyone running "pip install --upgrade setuptools" on a CentOS 6 machine will find it fails horribly:

pip install --upgrade setuptools
Downloading/unpacking setuptools from https://pypi.python.org/packages/b1/d1/3df86b914d72b18d37266f964a11576f135eadbb7d2e3c5165377e4263bc/setuptools-34.0.1.zip#md5=9e907950a94d38c0295df0da2507c280
  Downloading setuptools-34.0.1.zip (617kB): 617kB downloaded
  Running setup.py egg_info for package setuptools
    Traceback (most recent call last):
      File "<string>", line 3, in <module>
      File "setuptools/__init__.py", line 10, in <module>
        from six.moves import filter, map
    ImportError: No module named six.moves
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 3, in <module>

  File "setuptools/__init__.py", line 10, in <module>

    from six.moves import filter, map

ImportError: No module named six.moves

I discovered this when a long-functioning build script started failing all of a sudden.

I gather from #937 and #938 that a more recent pip is required, and indeed installing it with pip install --upgrade pip does allow installation of setuptools to proceed.

May I suggest that the README should specify what version of pip is required, as it is a newer version than is present on many systems.

[jaraco]

I don't think there's a specific version of pip that's required. It probably varies by platform and maybe by Python version. At the very least, whatever pip version is used needs to rely on the wheel format and not the sdist. If you would like to try different versions, I could update the README or the CHANGES to indicate which version to use.

[jcsp]

I can tell you that pip 1.4.1 does not work, and pip 9.0.1 does work.

If the required version was not known/is not knowable at the point the new dependency was introduced and pushed into the wild, I guess my best suggestion is a big health warning that people should install the latest pip if they're using the latest setuptools.

Alternatively, you could indicate which version of setuptools people should pin to in order to have something that will run on stable distributions that ship an older pip.

[shadowmint]

We're seeing this too with pip 1.5.4 on old machines with ubuntu 14.04 LTS and system pip installed.

As a workaround on machines with system pip versions which are too old I suggest:

virutalenv foo
./foo/bin/pip install -U pip
./foo/bin/pip install -r requirements,txt

fwiw, I think this is a breaking change to setuptools, not a minor release version (34.0 -> 34.0.1)

[adambkaplan]

I'm also seeing this issue with our builds on CircleCI, running Ubuntu 12.04 (Precise).

[zroadhouse-rmn]

We have a build pipeline that packs up all the sdists for our application and its dependencies and installs then using only the local index.

Does the workaround above (upgrade pip first) really depend on installing from the wheel format? I.e. is the sdist broken for setuptools?

[jaraco]

Upgrading pip first doesn't depend on installing from wheel format. Upgrading pip is necessary to install setuptools from a wheel. If you don't install setuptools from a wheel, you need to install setuptools' dependencies first (currently appdirs, packaging, six).

[zroadhouse-rmn]

Ok. Very helpful. Perhaps worthy of a release note?

[jaraco]

@zroadhouse-rmn You mean beyond the release notes already made? The second paragraph of the change for v34.0.0 already mentions "pre-installing the Setuptools dependencies". Perhaps you could suggest some wording that would be clearer?

[jcsp]

I think the clear wording would be something like (at the top, not in the body of text):
Important: setuptools is no longer installable with older versions of pip. If you are installing setuptools on an older linux distribution, you may need to install the latest pip before attempting to use it to install setuptools. If you seen an error like "ImportError: No module named six.moves", try updating pip before installing setuptools.

Because people will be running into this issue for a long time to come (these older versions of pip aren't going away any time soon), it would probably be prudent to put an ImportError handler in the code that gives people a clue about what's going on -- instead of "ImportError: No module named six.moves", it could say "Dependencies not found, are you trying to install with an older version of pip? "

[jaraco]

@jcsp Would you be willing to put together a PR?

[zroadhouse-rmn]

@jaraco - I re-read the description in changes:

Users are welcome to contrive other means to install or upgrade Setuptools using other means, such as pre-installing the Setuptools dependencies with pip or a bespoke bootstrap tool, but such usage is not recommended and is not supported.

From this description, my use case is no longer supported (installing/upgrading setuptools from sdist). I'll likely proceed with the workaround you suggested above (installing pyparsing, packaging, appdirs, and six prior to installing setuptools). Longer term, I'll need to rethink our application packaging solution, which relies on sdist vs wheel to produce platform independent packages that fully contain all python dependencies.

[marksteward]

Ultimately fixed with setuptools 36 (see #980).