Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

numpy installation thru install_requires directive issue (Trac #1841) #2434

Closed
numpy-gitbot opened this issue Oct 19, 2012 · 39 comments
Closed

Comments

@numpy-gitbot
Copy link

Original ticket http://projects.scipy.org/numpy/ticket/1841 on 2011-05-25 by trac user ohe, assigned to unknown.

Hello there,

In a virtualenv, with python v2.7.1, i can't install numpy thru the install_requires directive of setuptools/distribute.

from setuptools import setup
setup(...
      install_requires=['numpy'],
      ...)

A python setup.py install gives me the following traceback :

running install
running bdist_egg
running egg_info
writing requirements to UNKNOWN.egg-info/requires.txt
writing UNKNOWN.egg-info/PKG-INFO
writing top-level names to UNKNOWN.egg-info/top_level.txt
writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.6-intel/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to install

creating build/bdist.macosx-10.6-intel
creating build/bdist.macosx-10.6-intel/egg
creating build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/PKG-INFO -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/SOURCES.txt -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/dependency_links.txt -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/requires.txt -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
copying UNKNOWN.egg-info/top_level.txt -> build/bdist.macosx-10.6-intel/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/UNKNOWN-0.0.0-py2.7.egg' and adding 'build/bdist.macosx-10.6-intel/egg' to it
removing 'build/bdist.macosx-10.6-intel/egg' (and everything under it)
Processing UNKNOWN-0.0.0-py2.7.egg
Copying UNKNOWN-0.0.0-py2.7.egg to /Users/olivier/tc/karmatest/lib/python2.7/site-packages
Adding UNKNOWN 0.0.0 to easy-install.pth file

Installed /Users/olivier/tc/karmatest/lib/python2.7/site-packages/UNKNOWN-0.0.0-py2.7.egg
Processing dependencies for UNKNOWN==0.0.0
Searching for numpy
Reading http://pypi.python.org/simple/numpy/
Reading http://numpy.scipy.org
Reading http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=175103
Reading http://numeric.scipy.org
Best match: numpy 1.6.0
Downloading http://sourceforge.net/projects/numpy/files/NumPy/1.6.0/numpy-1.6.0.tar.gz/download
Processing download
Running numpy-1.6.0/setup.py -q bdist_egg --dist-dir /var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/egg-dist-tmp-Zr2lMr
Running from numpy source directory.Warning: distutils distribution has been initialized, it may be too late to add a subpackage commandWarning: distutils distribution has been initialized, it may be too late to add a subpackage fcompilernon-existing path in '/private/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils': 'site.cfg'
Warning: distutils distribution has been initialized, it may be too late to add a subpackage distutilsWarning: distutils distribution has been initialized, it may be too late to add a subpackage testingWarning: distutils distribution has been initialized, it may be too late to add a subpackage f2pyWarning: distutils distribution has been initialized, it may be too late to add an extension _sortWarning: distutils distribution has been initialized, it may be too late to add an extension multiarrayWarning: distutils distribution has been initialized, it may be too late to add an extension umathWarning: distutils distribution has been initialized, it may be too late to add an extension scalarmathWarning: distutils distribution has been initialized, it may be too late to add an extension _dotblasWarning: distutils distribution has been initialized, it may be too late to add an extension umath_testsWarning: distutils distribution has been initialized, it may be too late to add an extension multiarray_testsWarning: distutils distribution has been initialized, it may be too late to add a subpackage coreWarning: distutils distribution has been initialized, it may be too late to add an extension _compiled_baseWarning: distutils distribution has been initialized, it may be too late to add a subpackage libWarning: distutils distribution has been initialized, it may be too late to add a subpackage oldnumericWarning: distutils distribution has been initialized, it may be too late to add an extension _capiWarning: distutils distribution has been initialized, it may be too late to add a subpackage numarrayWarning: distutils distribution has been initialized, it may be too late to add an extension fftpack_liteWarning: distutils distribution has been initialized, it may be too late to add a subpackage fftWarning: distutils distribution has been initialized, it may be too late to add an extension lapack_liteWarning: distutils distribution has been initialized, it may be too late to add a subpackage linalgWarning: distutils distribution has been initialized, it may be too late to add an extension mtrandWarning: distutils distribution has been initialized, it may be too late to add a subpackage randomWarning: distutils distribution has been initialized, it may be too late to add a subpackage maWarning: distutils distribution has been initialized, it may be too late to add a subpackage matrixlibWarning: distutils distribution has been initialized, it may be too late to add a subpackage compatWarning: distutils distribution has been initialized, it may be too late to add a subpackage polynomialWarning: distutils distribution has been initialized, it may be too late to add a subpackage docWarning: distutils distribution has been initialized, it may be too late to add a subpackage numpyCould not locate executable f95
Could not locate executable f90
Could not locate executable f77
Could not locate executable xlf90
Could not locate executable xlf
Could not locate executable ifort
Could not locate executable ifc
Could not locate executable g77
Found executable /usr/local/bin/gfortran
_configtest.c:1: warning: conflicting types for built-in function ‘exp’
_configtest.c:1: warning: conflicting types for built-in function ‘exp’
_configtest.c:1:20: error: Python.h: No such file or directory
_configtest.c:1:20: error: Python.h: No such file or directory
lipo: can't figure out the architecture type of: /var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-//ccLgEqLk.out
_configtest.c:1:20: error: Python.h: No such file or directory
_configtest.c:1:20: error: Python.h: No such file or directory
lipo: can't figure out the architecture type of: /var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-//ccLgEqLk.out
Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    install_requires=['numpy']
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/install.py", line 76, in run
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/install.py", line 104, in do_egg_install
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 211, in run
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 427, in easy_install
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 478, in install_item
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 519, in process_distribution
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 563, in resolve
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 799, in best_match
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 811, in obtain
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 446, in easy_install
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 476, in install_item
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 655, in install_eggs
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 930, in build_and_install
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/easy_install.py", line 919, in run_setup
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/sandbox.py", line 62, in run_setup
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/sandbox.py", line 105, in run
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/sandbox.py", line 64, in <lambda>
  File "setup.py", line 196, in <module>

  File "setup.py", line 189, in setup_package

  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/core.py", line 186, in setup
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/olivier/tc/karmatest/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/setuptools/command/bdist_egg.py", line 167, in run
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/egg_info.py", line 8, in run
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/build_src.py", line 152, in run
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/build_src.py", line 169, in build_sources
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/build_src.py", line 328, in build_extension_sources
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/command/build_src.py", line 385, in generate_sources
  File "/private/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/core/setup.py", line 410, in generate_config_h

  File "/private/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/core/setup.py", line 41, in check_types

  File "/private/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/core/setup.py", line 271, in check_types

SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.
/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/misc_util.py:251: RuntimeWarning: Parent module 'numpy.distutils' not found while handling absolute import
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/misc_util.py", line 251, in clean_up_temporary_directory
ImportError: No module named numpy.distutils
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/var/folders/Iy/Iyse1OVUE38-IcPmyTb65E+++TI/-Tmp-/easy_install-W5KwbN/numpy-1.6.0/numpy/distutils/misc_util.py", line 251, in clean_up_temporary_directory
ImportError: No module named numpy.distutils

I have no probleme to install numpy with pip install numpy or easy_install numpy inside or outside any virtualenv.

OS: Mac OS 10.6.7

@numpy-gitbot
Copy link
Author

@rgommers wrote on 2011-05-27

I'm not sure exactly what the problem is, but can you get around it by just running an install script that first installs numpy and then your package? Relying on these badly-documented setuptools features is not a good idea if you can avoid it.

If I'd have to guess, I'd say calling setuptools.setup() for the first time invokes a bunch of magic that screws up something for numpy.distutils, hence:

Running from numpy source directory.Warning: distutils distribution has been
initialized, it may be too late to add a subpackage commandWarning: distutils
distribution has been initialized, it may be too late to add a subpackage

That warning comes from distutils/misc_util.py.

@numpy-gitbot
Copy link
Author

trac user ijstokes wrote on 2011-06-08

I just had this same error and for me it seems it is caused by some dependency problems. If I re-try the easy_install with the other modules, they work the second time 'round. My best guess is that some dependency is only "visible" after the easy_install session ends.

Ian

@prologic
Copy link

Is it possible to get this fixed at some point? This causes workarounds where there should be non IHMO. I'd really like to get to a point where numpy can be specified in the install_requires option of a setup.py.

@rgommers
Copy link
Member

It's possible of course, if someone invests time to really understand the issue and submits a clean fix. It's not a prio for me personally - install_requires should be avoided imho (at least until pip and easy_install stop upgrading dependencies when upgrading a package at least).

@njsmith
Copy link
Member

njsmith commented Nov 28, 2013

Can you confirm that the same problem still occurs with modern distutils/setuptools/pip?

@prologic
Copy link

It does.

Sent from my iPad

On 29 Nov 2013, at 7:22, njsmith notifications@github.com wrote:

Can you confirm that the same problem still occurs with modern
distutils/setuptools/pip?


Reply to this email directly or view it on
GitHubhttps://github.com//issues/2434#issuecomment-29485806
.

@svisser
Copy link

svisser commented Apr 1, 2014

Just ran across this issue so can confirm it's there. The option install_requires should be preferred over having to write a custom installation script when it's not really needed.

OS: Mac 10.8.3

@calmrat
Copy link

calmrat commented Jun 16, 2014

Hitting this issue to. Not a show stopped, but definitely annoying. Especially because it's not in any way obvious that it's a bug rather than some user/env issue.

@rgommers
Copy link
Member

I've just tried reproducing this by installing pandas (which uses install_requires) in a 2.7 virtualenv created like:

$ virtualenv --no-site-packages nonumpy
$ source nonumpy/bin/activate
$ cd $PANDAS_DIR
$ pip install cython # needed for pandas
$ python setup.py install

If someone can give an easy way to reproduce this, that would help.

@adam-zethraeus
Copy link

I also have this issue in a 2.7 virtual env. os ubuntu wheezy

@rgommers
Copy link
Member

@zethraeus could you provide a set of commands to reproduce this (like in my comment above)? It's hard to fix if we can't reproduce the issue.

@calmrat
Copy link

calmrat commented Jul 16, 2014

Hi! Sorry it took so long, but here's a reproducer.

See: https://gist.github.com/kejbaly2/71517b08536776399198

@rgommers
Copy link
Member

Thanks for the example @kejbaly2, I can reproduce it now.

@necrolyte2
Copy link

Having a very similar issue with trying to use install_requires and numpy

Here is how I can reproduce the error on Red Hat 6.5
https://gist.github.com/necrolyte2/530e9b39dcb0259b69d4

@tbekolay
Copy link

tbekolay commented Dec 2, 2014

As an FYI for anyone experiencing this issue, with the latest version of setuptools, a workaround is to add NumPy to setup_requires. I still kept it in install_requires, so my setup.py looks like:

setup(
...
    setup_requires=["numpy"],
    install_requires=["numpy"],
)

This works, and only installs NumPy once, on both Mac OS X and Linux (not tested on Windows).

As for why I have no clue.

dhalperi added a commit to uwescience/myria-python that referenced this issue Dec 20, 2014
See #16 and
http://stackoverflow.com/questions/27497470/setuptools-finds-wrong-package-during-install

Multiple hours of 100's of Google searches eventually took me to this
numpy thread:
    numpy/numpy#2434 (comment)
for which they had to add numpy to  AND  to fix a similar issue. Doing
this with requests seems to have worked. However, I am somewhat
skeptical as posters on SO indicated they did not see our problem in the
first place.

I tried on my Mac as well as on Ubuntu (dbserver02), and am able to
install from a clean virtualenv on both systems.

Fix #16 for now.
dhalperi added a commit to uwescience/myria-python that referenced this issue Dec 20, 2014
See #16 and
http://stackoverflow.com/questions/27497470/setuptools-finds-wrong-package-during-install

Multiple hours of 100's of Google searches eventually took me to this
numpy thread:
    numpy/numpy#2434 (comment)
for which they had to add numpy to `setup_requires` AND
`install_requires` to fix a similar issue. Doing this with requests
seems to have worked. However, I am somewhat skeptical as posters on SO
indicated they did not see our problem in the first place. I may just be
getting lucky.

I tried on my Mac as well as on Ubuntu (dbserver02), and am able to
install from a clean virtualenv on both systems.

Fix #16 for now.
miku added a commit to miku/siskin that referenced this issue Jan 4, 2015
kianho added a commit to kianho/pconpy that referenced this issue Mar 8, 2015
Removed the long list of dependencies since there is a known issue with
including numpy in this list (see
numpy/numpy#2434)
@embray
Copy link
Contributor

embray commented Mar 9, 2015

I've been trying to track down a similar issue that can occur sometimes when Numpy's build is being run by way of setup_requires=['Numpy'] in Astropy, when Astropy itself is being installed as a setup_requires of another pacakge (I call this 'setupception').

I've tracked it down to this ancient (but in principle still relevant, at least slightly?) bit of code:

def get_distribution(always=False):

In most cases when running Numpy's setup.py, it shoves a function called configuration into the kwargs for numpy.distutils.core.setup. numpy.distutils.core.setup in turn pops the 'configuration' key off the setup kwargs, then sets up some "magic" global variables in distutils.core, and then calls itself recursively. The purpose of this recursive call is to use the distutils machinery to check the command-line arguments for errors/--help.

It then calls the configuration function which kicks off the process of configuring all the Numpy subpackages, before once again calling back into the distutils setup() function with all the packages now fully configured.

Where this all runs afoul is that the Configuration manager occasionally needs to call the get_distribution utility in order to attach stuff to the Distribution object. The need for this is debatable. In fact, while there are some parts of the code in Configuration that think it should be able to do this, there are other parts that think that if there is already a Distribution instance attached to distutils.core._setup_distribution then something is probably wrong (hence the "distutils distribution has been initialized, it may be too late to add a subpackage" warnings that some people have reported in this thread).

To back up a bit and hopefully clarify--when running the setup() function from distutils (or setuptools) it initializes a Distribution object from the setup() kwargs, and then assigns the resulting Distribution to the global variable distutils.core._setup_distribution. At no point does it actually clean up this variable because it assumes it is the only setup that's going to run in the current Python interpreter. It was not designed for setupception. (pip gets around this and similar issues when installing multiple packages by installing each package from its own subprocess). However, in other cases, such as setup_requires, there can already be a Distribution instance from the root package hanging around in distutils.core._setup_distribution. So when Numpy's Configuration class tries to attach stuff to the Distribution object for Numpy itself it can end up messing around with the wrong Distribution in some cases. This results in a mangled Numpy build since some bits are missing.

There are a number of possible workarounds to this, both from setuptools end and/or the Numpy end. I haven't decided what the best approach would be, but the whole thing is definitely a mess :)

@embray
Copy link
Contributor

embray commented Mar 9, 2015

For what it's worth, one possible workaround from the setuptools end would be for setuptools.sandbox.run_setup, the function that setuptools uses to achieve setupception, to save/restore all distutils.core global variables before/after running the setup script in question.

As it happens, I believe the current version of setuptools may already fix that by hiding away the original distutils.core (removing it from sys.modules) before calling the child setup.py script. That should take care of it. But I haven't double-checked this.

Update: Confirmed, the issue I described above is worked around (somewhat by happy accident) in setuptools >= 10.0.

@rgommers
Copy link
Member

rgommers commented Mar 9, 2015

@embray that's a nice bit of detective work! (and it hurts my head a little:) )

You're right that easy_install - and hence get_distribution() I assume - is still relevant enough that we shouldn't be breaking it.

@embray
Copy link
Contributor

embray commented Mar 9, 2015

@rgommers The thing is, that workaround for easy_install was added a very long time ago, and I don't believe it to be entirely necessary anymore (not only for the fact that easy_install itself is effectively deprecated, though I know there are still a handful of good reasons people have to use it). I'll see if I can come up with a patch that addresses the original issue while not being a problem in other contexts.

@rgommers
Copy link
Member

rgommers commented Mar 9, 2015

@embray sounds good. If you get a patch ready and it doesn't get reviewed within ~2 weeks, please do kick me (gently)

@rgommers
Copy link
Member

Just in case the gist in #2434 (comment) disappears at some point, here's a copy of how to reproduce this issue:

cd /tmp

cat > ./setup.py <<DELIM
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Author: "Chris Ward" <cward@redhat.com>

from setuptools import setup

default_setup = dict(
    name='numpy_install_test',
    description='https://github.com/numpy/numpy/issues/2434',
    provides=['numpy_install_test'],
    install_requires=['pandas'],
    requires=['pandas'],
    license='GPLv3',
    author='Chris Ward',
    author_email='cward@redhat.com',
)   

setup(**default_setup)

DELIM


virtualenv --no-site-packages ~/virtenvs/nonumpy
source ~/virtenvs/nonumpy/bin/activate
python setup.py install

parmitam pushed a commit to uwescience/myria-python that referenced this issue May 10, 2017
See #16 and
http://stackoverflow.com/questions/27497470/setuptools-finds-wrong-package-during-install

Multiple hours of 100's of Google searches eventually took me to this
numpy thread:
    numpy/numpy#2434 (comment)
for which they had to add numpy to `setup_requires` AND
`install_requires` to fix a similar issue. Doing this with requests
seems to have worked. However, I am somewhat skeptical as posters on SO
indicated they did not see our problem in the first place. I may just be
getting lucky.

I tried on my Mac as well as on Ubuntu (dbserver02), and am able to
install from a clean virtualenv on both systems.

Fix #16 for now.
ccsplit added a commit to ccsplit/VHostScan that referenced this issue Nov 8, 2017
Numpy sometimes fails to properly install when just within the install_requires (see
[#2434](numpy/numpy#2434)) so I also added it to setup_requires.

Also changed from using scripts => entry_points:{'console_scripts'} and the command to `VHostScan`
within the README for consistency.
tnation14 added a commit to stac-utils/pystac that referenced this issue Jan 4, 2018
Add rasterio to setup_requires and install_requires To fix
numpy/numpy#2434
tnation14 added a commit to stac-utils/pystac that referenced this issue Jan 5, 2018
Rasterio depends on numpy, which must be present in the virtualenv
before installation. See numpy/numpy#2434.
tnation14 added a commit to stac-utils/pystac that referenced this issue Jan 5, 2018
Rasterio depends on numpy, which must be present in the virtualenv
before installation. See numpy/numpy#2434.
tnation14 added a commit to stac-utils/pystac that referenced this issue Jan 5, 2018
Rasterio depends on numpy, which must be present in the virtualenv
before installation. See numpy/numpy#2434.
marcbonnici added a commit to marcbonnici/workload-automation that referenced this issue Feb 14, 2018
For some reason the automatic install of pandas will fail compilation
due to numpy. A workaround for this issue is to specify numpy in
`setup_requires` as mentioned in
numpy/numpy#2434
marcbonnici added a commit to marcbonnici/workload-automation that referenced this issue Feb 14, 2018
For some reason the automatic install of pandas will fail compilation
due to numpy. A workaround for this issue is to specify numpy in
`setup_requires` as mentioned in
numpy/numpy#2434
marcbonnici added a commit to marcbonnici/workload-automation that referenced this issue Feb 14, 2018
For some reason the automatic install of pandas will fail compilation
due to numpy. A workaround for this issue is to specify numpy in
`setup_requires` as mentioned in
numpy/numpy#2434
marcbonnici added a commit to marcbonnici/workload-automation that referenced this issue Feb 14, 2018
For some reason the automatic install of pandas will fail compilation
due to an issue with numpy. A workaround for this issue is to specify
numpy in `setup_requires` as mentioned in numpy/numpy#2434
setrofim pushed a commit to ARM-software/workload-automation that referenced this issue Feb 14, 2018
For some reason the automatic install of pandas will fail compilation
due to an issue with numpy. A workaround for this issue is to specify
numpy in `setup_requires` as mentioned in numpy/numpy#2434
Rahlubenru pushed a commit to Rahlubenru/workload-automation that referenced this issue Mar 23, 2018
For some reason the automatic install of pandas will fail compilation
due to an issue with numpy. A workaround for this issue is to specify
numpy in `setup_requires` as mentioned in numpy/numpy#2434
@omeryounus
Copy link

having hard time installing pandas in python2.7.16 using pip install -R requirements.txt

https://files.pythonhosted.org/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz (11.8MB)
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-WznqHc/pandas/setup.py", line 746, in
**setuptools_kwargs)
File "/usr/local/lib/python2.7/site-packages/setuptools/init.py", line 144, in setup
_install_setup_requires(attrs)
File "/usr/local/lib/python2.7/site-packages/setuptools/init.py", line 139, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 724, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python2.7/site-packages/pkg_resources/init.py", line 782, in resolve
replace_conflicting=replace_conflicting
File "/usr/local/lib/python2.7/site-packages/pkg_resources/init.py", line 1065, in best_match
return self.obtain(req, installer)
File "/usr/local/lib/python2.7/site-packages/pkg_resources/init.py", line 1077, in obtain
return installer(requirement)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 791, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 679, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 705, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1144, in run_setup
run_setup(setup_script, args)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
raise
File "/usr/local/lib/python2.7/contextlib.py", line 35, in exit
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python2.7/contextlib.py", line 35, in exit
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
saved_exc.resume()
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
six.reraise(type, exc, self._tb)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
_execfile(setup_script, ns)
File "/usr/local/lib/python2.7/site-packages/setuptools/sandbox.py", line 45, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-2M8B7z/numpy-1.17.0rc1/setup.py", line 31, in

**RuntimeError: Python version >= 3.5 required.**

Not sure why it's look for numpy-1.17.0rc1 which is not support for python2

Any help would be appreciated, thanks

@mattip
Copy link
Member

mattip commented Jul 2, 2019

Please share your requirements.txt

@omeryounus
Copy link

Requirement file contains:

flask==1.0.2
flask-jwt==0.3.2
flask-script==2.0.6
flask-scrypt==0.1.3.6
PyJWT<1.5.0
blinker==1.4
phonenumbers==8.9.12
flanker==0.9.0
bugsnag==3.4.3
pickledb==0.7.2
requests==2.19.1
python-dateutil==2.8.0
gunicorn==19.9.0
gevent==1.3.6
webargs==4.0.0
ply==3.10
bson==0.5.7
celery==4.2.1
redis==3.2.1
hiredis
numpy==1.16.4

Finally manage it to work keeping the numpy inside requirement file and later installing pandas

Run command in docker:

RUN pip install https://github.com/pydata/pandas/releases/download/v0.24.2/pandas-0.24.2.tar.gz

However, now there is another issue in compiling it's say you need to install cython to compile pandas.

Error:

Exception: Cython-generated file 'pandas/io/sas/sas.c' not found.
Cython is required to compile pandas from a development branch.
Please install Cython or download a release package of pandas.

Any idea?

@mattip
Copy link
Member

mattip commented Jul 2, 2019

Not connected to this issue which concerns using install_requires inside setup.py. Please open a new issue. Do you really need to install pandas from source? Why not pip install pandas==0.24.2 ?

@omeryounus
Copy link

Yes I will open this issue in pandas library. However, the post that I mentioned earlier thats related to numPy since you can not install pandas directly numPy is properly installed on Python2.7 under docker env.
Why not pip install pandas==0.24.2 ? Because when I run it under docker it says.

Exception: Cython-generated file 'pandas/msgpack.c' not found.
Cython is required to compile pandas from a development branch.
Please install Cython or download a release package of pandas.

So I need to use release version of pandas to be installed from tarball

@mattip
Copy link
Member

mattip commented Aug 28, 2019

Closing. I think the python world is moving away from setup_requires and more towards requirements.txt or pyproject.toml. The work-around is below for those who still want to use setup_requires.

setup(
...
    setup_requires=["numpy"],
    install_requires=["numpy"],
)

As for @omeryounus trying to install numpy on python2, that is a separate issue not connected to this one. Some users have reported that upgrading pip and setuptools to the most recent version helped with that problem.

Please reopen if there is more to discuss around setup_requires.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests