setuptools requires pip

[sagetrac-buck]

In the current master branch (7eb8510), when I ./sage -i setuptools it fails with this message:

****************************************************
patching file setuptools/command/easy_install.py
Hunk #1 succeeded at 1491 (offset 45 lines).
Hunk #2 succeeded at 1529 with fuzz 1 (offset 45 lines).
Hunk #3 succeeded at 1555 (offset 45 lines).
patching file pkg_resources/__init__.py
running install
running bdist_egg
running egg_info
writing requirements to setuptools.egg-info/requires.txt
writing setuptools.egg-info/PKG-INFO
writing top-level names to setuptools.egg-info/top_level.txt
writing dependency_links to setuptools.egg-info/dependency_links.txt
writing entry points to setuptools.egg-info/entry_points.txt
Searching for pip
Reading https://pypi.python.org/simple/pip/
Download error on https://pypi.python.org/simple/pip/: [Errno 110] Connection timed out -- Some packages may not be found!
Couldn't find index page for 'pip' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno 110] Connection timed out -- Some packages may not be found!
No local packages or download links found for pip
error: Could not find suitable distribution for Requirement.parse('pip')

real    4m15.474s
user    0m0.516s
sys     0m0.138s

This timeout is caused by the "poison" proxy:

build/bin/sage-spkg:export http_proxy=http://192.0.2.0:5187/

but I don't know why the process is searching for pip. My workaround was python -m ensurepip, but that's only going to work for python>=2.7.9.

I also tried sage -i pip, but it depends on setuptools, so that's circular.

I'm not sure how you're going to solve this.

Component: packages: standard

Issue created by migration from https://trac.sagemath.org/ticket/18955

[jdemeyer]

comment:2

Works for me:

$ ./sage -i setuptools
Found local metadata for setuptools-12.4
Package setuptools-12.4 is already installed.
Use 'sage -f setuptools' to force a reinstallation.

$ ./sage -f setuptools
Found local metadata for setuptools-12.4
Using cached file /usr/local/src/sage-config/upstream/setuptools-12.4.tar.gz
setuptools-12.4
====================================================
Setting up build directory for setuptools-12.4
Finished set up
****************************************************
Host system:
Linux tamiyo 3.17.7-gentoo #1 SMP PREEMPT Wed Dec 31 20:06:39 CET 2014 x86_64 Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz GenuineIntel GNU/Linux
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=/usr/x86_64-pc-linux-gnu/gcc-bin/4.8.4/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/4.8.4/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-4.8.4/work/gcc-4.8.4/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/4.8.4 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.4/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.8.4 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.8.4/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/4.8.4/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.4/include/g++-v4 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/4.8.4/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 4.8.4 p1.3, pie-0.6.1' --enable-libstdcxx-time --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-altivec --disable-fixed-point --enable-targets=all --disable-libgcj --enable-libgomp --disable-libmudflap --disable-libssp --enable-lto --without-cloog --enable-libsanitizer
Thread model: posix
gcc version 4.8.4 (Gentoo 4.8.4 p1.3, pie-0.6.1) 
****************************************************
patching file setuptools/command/easy_install.py
Hunk #1 succeeded at 1491 (offset 45 lines).
Hunk #2 succeeded at 1529 with fuzz 1 (offset 45 lines).
Hunk #3 succeeded at 1555 (offset 45 lines).
patching file pkg_resources/__init__.py
running install
running bdist_egg
running egg_info
writing requirements to setuptools.egg-info/requires.txt
writing setuptools.egg-info/PKG-INFO
writing top-level names to setuptools.egg-info/top_level.txt
writing dependency_links to setuptools.egg-info/dependency_links.txt
writing entry points to setuptools.egg-info/entry_points.txt
reading manifest file 'setuptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'setuptools.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
[...]
Successfully installed setuptools-12.4
[...]

[jdemeyer]

comment:3

Perhaps it would be good to mention exactly which commands you executed.

[sagetrac-buck]

comment:4

The "setuptools is already installed" message makes me strongly suspect that pip is also already installed. You won't be able to reproduce this if you're in an old checkout where pip is installed. I too can get the setuptools spkg to install if I (manually) ensure pip is already installed, which I mentioned in my OP workaround.

To test, python -m pip should fail with "no such module".

I ran make, from a fresh clone.

Demo session:

./sage -i -f setuptools
Successfully installed setuptools-12.4


./local/bin/python -m pip
(pip's help is printed)

# uninstalls from local/lib/python2.7/site-packages/pip-6.1.1-py2.7.egg
./local/bin/python -m pip uninstall --yes pip
Uninstalling pip-6.1.1:
  Successfully uninstalled pip-6.1.1

# uninstalls from local/lib/python2.7/site-packages/pip/
./local/bin/python -m pip uninstall --yes pip                                                            Uninstalling pip:
  Successfully uninstalled pip

# shows that pip is truly uninstalled now
./local/bin/python -m pip uninstall --yes pip
/home/buck/trees/mine/sage/local/bin/python: No module named pip


./sage -i -f setuptools
...
Searching for pip
Reading https://pypi.python.org/simple/pip/

(stall for ~120 seconds)

Download error on https://pypi.python.org/simple/pip/: [Errno 110] Connection timed out -- Some packages may not be found!
Couldn't find index page for 'pip' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/

(stall for ~120 seconds)

Download error on https://pypi.python.org/simple/: [Errno 110] Connection timed out -- Some packages may not be found!
No local packages or download links found for pip
error: Could not find suitable distribution for Requirement.parse('pip')

real	4m15.108s
user	0m0.569s
sys	0m0.130s
************************************************************************
Error installing package setuptools-12.4
************************************************************************

I've also attached the unadulterated install log.

[sagetrac-buck]

setuptools install log

[jhpalmieri]

comment:5

Attachment: setuptools-12.4.log

If I install from a fresh version 6.8 tarball (from http://files.sagemath.org/src/index.html), I don't see this problem. setuptools is installed before pip, and the string "pip" is not found anywhere in the setuptools log file. Could you have some environment variable which is interfering? Or some Python configuration file?

[sagetrac-buck]

comment:6

jhpalmieri: You didn't show results of my pre-condition test:

# shows that pip is truly uninstalled now
./local/bin/python -m pip
/home/buck/trees/mine/sage/local/bin/python: No module named pip

I'll try to trace through the code and pinpoint the issue.

[sagetrac-buck]

comment:7

So. I traced through it. I believe the sequents of events are this:

• we run: python setup.py install
• which runs: setup.py egg_info
• even though it's not installed yet, this uses setuptools' egg_info.run
• which iterates through all "egg_info.writers" entrypoints (whatever that may be)
• one of which is ~/.local/.../pbr (this is why you can't repro =/)
• pbr apparently requires pip
• setuptools ensures entrypoint requirements are met
• so it tries to install pip
• splosion

In summary, NOTABUG. Possibly it's a pbr or setuptools bug, but probably not.

That said, sage --env could prevent this issue in future by setting PYTHONNOUSERSITE=true.

[sagetrac-buck]

comment:8

In case anyone cares to know, I got into this state by running pip install --user virtualenvwrapper, which is not entirely bad practice.

[jhpalmieri]

comment:9

So is a patch like this worthwhile?

diff --git a/build/bin/sage-spkg b/build/bin/sage-spkg
index f4b085b..0de067a 100755
--- a/build/bin/sage-spkg
+++ b/build/bin/sage-spkg
@@ -148,6 +148,9 @@ if [ -n "$PYTHONPATH" ]; then
     fi
 fi
 
+# Set PYTHONNOUSERSITE=yes to avoid trouble with setuptools.
+export PYTHONNOUSERSITE=yes
+
 ##################################################################
 # Handle special command-line options
 ##################################################################

This should set PYTHONNOUSERSITE while installing packages but not while running Sage.

[sagetrac-buck]

comment:10

I would set it while running sage as well, and change the comment to "prevent ~/.local python packages from interfering with sage".

As far as I've seen, the sage philosophy is that sage dependencies are essentially separate from any system software.

Replying to @jhpalmieri:

So is a patch like this worthwhile?

diff --git a/build/bin/sage-spkg b/build/bin/sage-spkg
index f4b085b..0de067a 100755
--- a/build/bin/sage-spkg
+++ b/build/bin/sage-spkg
@@ -148,6 +148,9 @@ if [ -n "$PYTHONPATH" ]; then
     fi
 fi
 
+# Set PYTHONNOUSERSITE=yes to avoid trouble with setuptools.
+export PYTHONNOUSERSITE=yes
+
 ##################################################################
 # Handle special command-line options
 ##################################################################

This should set PYTHONNOUSERSITE while installing packages but not while running Sage.

[jhpalmieri]

comment:11

Okay, see #14243.

[jhpalmieri]

comment:12

I propose closing this as duplicate/invalid/wontfix.