Pip 10 issues #106

lboxell · 2018-05-22T18:21:44Z

I seem to be having an issue with the pip.main() call in the python script when pip 10.0.1 is installed.

Levi1$ easy_install --upgrade pip
Searching for pip
Reading https://pypi.python.org/simple/pip/
Best match: pip 10.0.1
Processing pip-10.0.1-py2.7.egg
pip 10.0.1 is already the active version in easy-install.pth
Installing pip script to /Users/Levi1/anaconda/bin
Installing pip2.7 script to /Users/Levi1/anaconda/bin
Installing pip2 script to /Users/Levi1/anaconda/bin

Using /Users/Levi1/anaconda/lib/python2.7/site-packages/pip-10.0.1-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip
DN0a232986:brazil_census Levi1$ python
Python 2.7.13 |Anaconda 2.2.0 (x86_64)| (default, Dec 20 2016, 23:05:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
import pip
pip.main()
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'module' object has no attribute 'main'

I switched to 9.0.2 and things worked for me again.

Levi1$ sudo pip uninstall pip
Password:
The directory '/Users/Levi1/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Uninstalling pip-10.0.1:
Would remove:
/Users/Levi1/anaconda/bin/pip
/Users/Levi1/anaconda/bin/pip2
/Users/Levi1/anaconda/bin/pip2.7
/Users/Levi1/anaconda/lib/python2.7/site-packages/pip-10.0.1.dist-info/*
/Users/Levi1/anaconda/lib/python2.7/site-packages/pip/*
Proceed (y/n)? y
Successfully uninstalled pip-10.0.1
The directory '/Users/Levi1/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
You are using pip version 8.1.0, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
DN0a232986:brazil_census Levi1$ easy_install pip==9.0.2
Searching for pip==9.0.2
Best match: pip 9.0.2
Processing pip-9.0.2-py2.7.egg
Removing pip 10.0.1 from easy-install.pth file
Adding pip 9.0.2 to easy-install.pth file
Installing pip script to /Users/Levi1/anaconda/bin
Installing pip2.7 script to /Users/Levi1/anaconda/bin
Installing pip2 script to /Users/Levi1/anaconda/bin

Using /Users/Levi1/anaconda/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg
Processing dependencies for pip==9.0.2
Finished processing dependencies for pip==9.0.2
DN0a232986:brazil_census Levi1$ python
Python 2.7.13 |Anaconda 2.2.0 (x86_64)| (default, Dec 20 2016, 23:05:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org

import pip
pip>>> pip.version
'9.0.2'
pip.main()

Usage:
pip [options]

This impacts the python configuration code in the template. I'm not sure if this is idiosyncratic to me, or a broader issue, so I'm flagging here for you guys.

qlquanle · 2018-05-22T18:42:26Z

@lboxell I had the same issue recently. It's related to Anaconda I think. It was on my personal laptop so my resolution was to re-install conda and to not upgrade pip. I haven't looked further as to how conda installations can live with upgraded pip.

lboxell · 2018-05-22T18:44:07Z

The other thing I'll note is that it may be nice to have the scons.zip file not stored in git-lfs. It's pretty small and keeping it as a non-pointer would allow people to just do the "Download ZIP" option instead of forcing the cloning with git-lfs already setup.

lboxell · 2018-05-22T18:47:43Z

@stanfordquan,

Gotcha. I got a solution working on my end as well, but if its a broader pip 10 issue (which seems like others are having similar problems here), then warning people in the readme may be helpful.

RoyalTS · 2018-05-27T23:38:34Z

Ran into this as well. Pip 10 does not expose pip.main() anymore and does this on purpose (pypa/pip#5149).

Can't you just replace config_python.py with a pip requirements.txt file with the right package names and versions and bash script that calls pip on that file with the right command line arguments?

qlquanle · 2018-06-11T23:51:28Z

I will explore the above solution this week.

qlquanle · 2018-06-18T23:14:14Z

Pull request opened in #107. I implemented @RoyalTS's solution above with a requirements.txt. I maintain the workflow of running python config/config_python.py and stuck a subprocess call into the config_python.py file because that seems more intuitive for the less-sophisticated user. The reason that it had to be a subprocess call was because pip gets really mad about people trying to call it from within their program (see [here]).

@RoyalTS and @lboxell , can you please let me know if this solution works for you?

P.s. check out this incredible exchange pip devs had. Let GSLab RA's aspire to never be this stuck-up about improving user accessibility for gslab_python

…t for config_python.py (#107) Template 4.1.2 Release Notes Bugs Pip v.10 update broke our installation method in template. We changed our suggested installation method by removing config_python.py and suggesting user call pip install -r config/requirements.txt directly . (#106 #107) * #106 switch to using requirements.txt * #106 change the installation of gslab python * #106 fixed typo * #106 Use only requirements.txt * #106 revise README * #106 change compatibility to gslab_python v4.1.1

qlquanle · 2018-07-11T01:51:54Z

@RoyalTS @lboxell, gslab_python@v4.1.1 and template@v4.1.2 should solve this issue. Please let me know if the fix works for you guys. Thanks! 🚒

lboxell · 2018-07-11T17:44:30Z

@stanfordquan,

I was able to get things to run using the above two versions you link to.

However, I had an issue with matplotlib described here: https://stackoverflow.com/questions/31373163/anaconda-runtime-error-python-is-not-installed-as-a-framework/41433353, but the conda install matplotlib fixed things. I think its because my default python is pointing to the anaconda installation and the requirements.txt installation of matplotlib didn't update things for the anaconda environment. You can see this below.

Installing collected packages: matplotlib
Successfully installed matplotlib-2.2.2
You are using pip version 9.0.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
DN0a1e67ee:template Levi1$ cd analysis/
DN0a1e67ee:analysis Levi1$ python run.py
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
build_python(["build/descriptive/plot.eps", "build/descriptive/table.txt"], ["source/descriptive/descriptive.py", "build/prepare_data/data.txt"])
Error: Python did not run successfully. Please check that the executable, source, and target files are correctly specified. Check build/descriptive/sconscript_descriptive.log and sconstruct.log for errors.
Command tried: python source/descriptive/descriptive.py > build/descriptive/sconscript_descriptive.log
Traceback (most recent call last):
File "source/descriptive/descriptive.py", line 3, in
import matplotlib.pyplot as plt
File "/Users/Levi1/anaconda/lib/python2.7/site-packages/matplotlib/pyplot.py", line 115, in
_backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
File "/Users/Levi1/anaconda/lib/python2.7/site-packages/matplotlib/backends/init.py", line 62, in pylab_setup
[backend_name], 0)
File "/Users/Levi1/anaconda/lib/python2.7/site-packages/matplotlib/backends/backend_macosx.py", line 17, in
from matplotlib.backends import _macosx
RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.

scons: *** [build/descriptive/plot.eps] ExecCallError :
Traceback (most recent call last):
File "/Users/Levi1/Dropbox/Research/template/config/scons/scons-local-3.0.1/SCons/Action.py", line 1197, in execute
result = self.execfunction(target=target, source=rsources, env=env)
File "/Users/Levi1/anaconda/lib/python2.7/site-packages/gslab_scons/builders/build_python.py", line 26, in build_python
builder.execute_system_call()
File "/Users/Levi1/anaconda/lib/python2.7/site-packages/gslab_scons/builders/gslab_builder.py", line 119, in execute_system_call
self.do_call()
File "/Users/Levi1/anaconda/lib/python2.7/site-packages/gslab_scons/builders/gslab_builder.py", line 150, in do_call
self.raise_system_call_exception(traceback = ex.output)
File "/Users/Levi1/anaconda/lib/python2.7/site-packages/gslab_scons/builders/gslab_builder.py", line 166, in raise_system_call_exception
raise ExecCallError(message)
ExecCallError
scons: building terminated because of errors.
DN0a1e67ee:analysis Levi1$ conda install matplotlib
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /Users/Levi1/anaconda:

The following NEW packages will be INSTALLED:

backports:                     1.0-py27hb4f9756_1              
backports.functools_lru_cache: 1.5-py27_1                      
cycler:                        0.10.0-py27hfc73c78_0           
functools32:                   3.2.3.2-py27h8ceab06_1          
kiwisolver:                    1.0.1-py27h9856860_0            
libcxx:                        4.0.1-h579ed51_0                
libcxxabi:                     4.0.1-hebd6815_0                
olefile:                       0.45.1-py27_0                   
subprocess32:                  3.5.2-py27h1de35cc_0            
xz:                            5.2.4-h1de35cc_4

The following packages will be UPDATED:

conda:                         4.3.30-py27h407ed3a_0            --> 4.5.5-py27_0        
freetype:                      2.5.2-0                <unknown> --> 2.8-h12048fb_1      
jpeg:                          8d-1                   <unknown> --> 9b-he5867d9_2       
libpng:                        1.5.13-1               <unknown> --> 1.6.34-he12f830_0   
libtiff:                       4.0.2-1                <unknown> --> 4.0.9-hcb84e12_1    
matplotlib:                    1.4.3-np19py27_1       <unknown> --> 2.2.2-py27ha7267d0_0
pillow:                        2.7.0-py27_1           <unknown> --> 4.2.1-py27h2cf1d5f_0
pycosat:                       0.6.1-py27_0           <unknown> --> 0.6.3-py27h6c51c7e_0
zlib:                          1.2.8-0                <unknown> --> 1.2.11-hf3cbc9b_2

The following packages will be SUPERSEDED by a higher-priority channel:

conda-env:                     2.6.0-0                          --> 2.6.0-h36134e3_0

Proceed ([y]/n)? y

conda-env-2.6. 100% |############################################################################################| Time: 0:00:00 2.08 MB/s
jpeg-9b-he5867 100% |############################################################################################| Time: 0:00:00 1.19 MB/s
libcxxabi-4.0. 100% |############################################################################################| Time: 0:00:00 1.37 MB/s
xz-5.2.4-h1de3 100% |############################################################################################| Time: 0:00:00 1.30 MB/s
zlib-1.2.11-hf 100% |############################################################################################| Time: 0:00:00 1.45 MB/s
libcxx-4.0.1-h 100% |############################################################################################| Time: 0:00:00 1.23 MB/s
libpng-1.6.34- 100% |############################################################################################| Time: 0:00:00 1.18 MB/s
backports-1.0- 100% |############################################################################################| Time: 0:00:00 3.27 MB/s
freetype-2.8-h 100% |############################################################################################| Time: 0:00:00 1.23 MB/s
functools32-3. 100% |############################################################################################| Time: 0:00:00 3.63 MB/s
kiwisolver-1.0 100% |############################################################################################| Time: 0:00:00 1.21 MB/s
libtiff-4.0.9- 100% |############################################################################################| Time: 0:00:00 1.26 MB/s
olefile-0.45.1 100% |############################################################################################| Time: 0:00:00 1.86 MB/s
pycosat-0.6.3- 100% |############################################################################################| Time: 0:00:00 1.46 MB/s
subprocess32-3 100% |############################################################################################| Time: 0:00:00 2.01 MB/s
backports.func 100% |############################################################################################| Time: 0:00:00 7.89 MB/s
cycler-0.10.0- 100% |############################################################################################| Time: 0:00:00 10.51 MB/s
pillow-4.2.1-p 100% |############################################################################################| Time: 0:00:00 1.54 MB/s
matplotlib-2.2 100% |############################################################################################| Time: 0:00:05 1.20 MB/s
conda-4.5.5-py 100% |############################################################################################| Time: 0:00:00 1.20 MB/s
DN0a1e67ee:analysis Levi1$ python run.py
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
build_python(["build/descriptive/plot.eps", "build/descriptive/table.txt"], ["source/descriptive/descriptive.py", "build/prepare_data/data.txt"])
Install directory: "build/descriptive" as "release/descriptive"
scons: done building targets.

qlquanle · 2018-07-11T17:46:03Z

I had the same issue and assumed it was idiosyncratic (🙅‍♂️ matplotlib). Glad it was resolved.

qlquanle · 2018-07-11T17:46:20Z

I'm going to chalk that one up to individuals' matplotlib installations.

RoyalTS · 2018-07-11T17:53:12Z

@lboxell What did your (root) conda environment look like after pip installing the requirements.txt but before the conda install matplotlib? Does conda perhaps not install a package via pip if it's already installed via anaconda?

lboxell · 2018-07-11T18:02:11Z

@RoyalTS,

The pip install requirements.txt said that matplotlib 2.2 was successfully installed (and appears to be checking/installing in the correct anaconda location):

Requirement already satisfied: setuptools in /Users/Levi1/anaconda/lib/python2.7/site-packages (from kiwisolver>=1.0.1->matplotlib>=2.2.2->-r config/requirements.txt (line 3))
Installing collected packages: matplotlib
Successfully installed matplotlib-2.2.2

However, the output from conda install matplotlib suggests that only matplotlib 1.4.3 was recognized:

Package plan for installation in environment /Users/Levi1/anaconda:

The following NEW packages will be INSTALLED:

backports: 1.0-py27hb4f9756_1
backports.functools_lru_cache: 1.5-py27_1
cycler: 0.10.0-py27hfc73c78_0
functools32: 3.2.3.2-py27h8ceab06_1
kiwisolver: 1.0.1-py27h9856860_0
libcxx: 4.0.1-h579ed51_0
libcxxabi: 4.0.1-hebd6815_0
olefile: 0.45.1-py27_0
subprocess32: 3.5.2-py27h1de35cc_0
xz: 5.2.4-h1de35cc_4
The following packages will be UPDATED:

conda: 4.3.30-py27h407ed3a_0 --> 4.5.5-py27_0
freetype: 2.5.2-0 --> 2.8-h12048fb_1
jpeg: 8d-1 --> 9b-he5867d9_2
libpng: 1.5.13-1 --> 1.6.34-he12f830_0
libtiff: 4.0.2-1 --> 4.0.9-hcb84e12_1
matplotlib: 1.4.3-np19py27_1 --> 2.2.2-py27ha7267d0_0
pillow: 2.7.0-py27_1 --> 4.2.1-py27h2cf1d5f_0
pycosat: 0.6.1-py27_0 --> 0.6.3-py27h6c51c7e_0
zlib: 1.2.8-0 --> 1.2.11-hf3cbc9b_2
The following packages will be SUPERSEDED by a higher-priority channel:

conda-env: 2.6.0-0 --> 2.6.0-h36134e3_0

So I don't fully understand what exactly the issue was.

RoyalTS · 2018-07-11T18:10:00Z

@lboxell Weird.

What does conda list say after the pip install requirements.txt but before the conda install matplotlib?

lboxell · 2018-07-11T19:19:17Z

Not sure how to check. I don't really use python environments.

RoyalTS · 2018-07-11T19:46:17Z

Too late to check now then but conda list would tell you whether the matplotlib that's linked to your conda root environment comes from conda or from pip (I'm guessing the former) and what version is being used.

You guys don't use virtualenvs either?

lboxell · 2018-07-11T20:01:35Z

Gotcha. No, gslab doesn't use them at this point...

DavidRitzwoller · 2018-07-16T16:56:04Z

Closing this issue. Template is now compatible with Pip 10 after the changes made here #107.

lboxell assigned arosenbe and qlquanle May 22, 2018

qlquanle pushed a commit that referenced this issue Jun 18, 2018

#106 switch to using requirements.txt

ee95b9c

qlquanle mentioned this issue Jun 18, 2018

Pull request for #106: Pip 10 issues, switch to using requirements.txt for config_python.py #107

Merged

DavidRitzwoller self-assigned this Jul 6, 2018

DavidRitzwoller added a commit that referenced this issue Jul 6, 2018

#106 change the installation of gslab python

ee45fd0

DavidRitzwoller added a commit that referenced this issue Jul 7, 2018

#106 fixed typo

25598ee

DavidRitzwoller added a commit that referenced this issue Jul 9, 2018

#106 Use only requirements.txt

cd0fef4

DavidRitzwoller added a commit that referenced this issue Jul 11, 2018

#106 revise README

c66cc5c

qlquanle pushed a commit that referenced this issue Jul 11, 2018

#106 change compatibility to gslab_python v4.1.1

503507e

DavidRitzwoller closed this as completed Jul 16, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pip 10 issues #106

Pip 10 issues #106

lboxell commented May 22, 2018

qlquanle commented May 22, 2018

lboxell commented May 22, 2018

lboxell commented May 22, 2018

RoyalTS commented May 27, 2018

qlquanle commented Jun 11, 2018

qlquanle commented Jun 18, 2018

qlquanle commented Jul 11, 2018

lboxell commented Jul 11, 2018

qlquanle commented Jul 11, 2018

qlquanle commented Jul 11, 2018

RoyalTS commented Jul 11, 2018

lboxell commented Jul 11, 2018

RoyalTS commented Jul 11, 2018

lboxell commented Jul 11, 2018

RoyalTS commented Jul 11, 2018

lboxell commented Jul 11, 2018

DavidRitzwoller commented Jul 16, 2018

Pip 10 issues #106

Pip 10 issues #106

Comments

lboxell commented May 22, 2018

qlquanle commented May 22, 2018

lboxell commented May 22, 2018

lboxell commented May 22, 2018

RoyalTS commented May 27, 2018

qlquanle commented Jun 11, 2018

qlquanle commented Jun 18, 2018

qlquanle commented Jul 11, 2018

lboxell commented Jul 11, 2018

qlquanle commented Jul 11, 2018

qlquanle commented Jul 11, 2018

RoyalTS commented Jul 11, 2018

lboxell commented Jul 11, 2018

RoyalTS commented Jul 11, 2018

lboxell commented Jul 11, 2018

RoyalTS commented Jul 11, 2018

lboxell commented Jul 11, 2018

DavidRitzwoller commented Jul 16, 2018