New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pandas installation problems with pip version 10 #20775

Open
jorisvandenbossche opened this Issue Apr 21, 2018 · 14 comments

Comments

Projects
None yet
6 participants
@jorisvandenbossche
Copy link
Member

jorisvandenbossche commented Apr 21, 2018

The release of pip version 10 and the presence of a pyproject.toml in our latest released versions gives problems for installing pandas in the following cases:

Problem with installing pandas for Python 3.4

  • You get the error "Double requirement given: numpy ..."
  • The reason for this error is that pandas 0.21-0.22 no longer supports Python 3.4, and thus no longer distributes binary wheels for Python 3.4. Therefore, pip tries to install pandas from source, and this is what now started to fail with Pip version 10.
  • Solution:
    • Install an older version of pandas

      pip install pandas<0.21
      
    • Workaround for package maintainers to specify the pandas version in install_requires: #20697 (comment)

  • The fact that those version are downloaded for Python 3.4 (instead of an older compatible version) was a bug in our setup.py, and this will be fixed in future versions of pandas: #20698

Problem with installing pandas on "special" platforms (32bit, PyPy, ARM, ...) (building from source):

  • You get the error "No matching distribution found for Cython"
  • The reason for this error is that we do not provide wheels for such python versions or platforms. In such a case, pip will try to build pandas from source, but this started to fail with pip version 10. This is because their limited support for PEP518, and they require that all build dependencies of pandas are installed from wheels (pypa/pip#5229), which are often also not available on those python versions / platforms.
  • Solution
    • Use an older version of pip
    • Use pip install pandas --no-build-isolation
  • See eg #20666, #20771
@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Apr 21, 2018

Do we know if there's any way to fix the 0.21.x and 0.22.x releases for python 3.4, or does this fall entirely on users? e.g. if we were to upload a 0.21.2 and 0.22.1 with just the python_requires fix, I assume pip install pandas would still pull down 0.22.0, right?

@jorisvandenbossche

This comment has been minimized.

Copy link
Member

jorisvandenbossche commented Apr 21, 2018

Yes, I don't think we can fix it, because even if we would do a 0.22.1 with the correction, I suppose as well that pip will find the existing 0.22.0 as the next in line that it can install.

@jreback

This comment has been minimized.

Copy link
Contributor

jreback commented Apr 21, 2018

the way to fix this is to re-issue the existing wheels (with this one change) on 0.21 and 0.22. but that is 'banned' by PyPi. Its equivalent in conda of bumping the version number, but AFAIK no way to do this on PyPi.

so maybe ask on the pip tracker on what is possible?

maybe could just put an extra waring in the docs.

@jreback jreback added the Build label Apr 21, 2018

@jreback jreback added this to the 0.23.0 milestone Apr 21, 2018

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Apr 21, 2018

@jorisvandenbossche

This comment has been minimized.

Copy link
Member

jorisvandenbossche commented Apr 24, 2018

so maybe ask on the pip tracker on what is possible?

Well, I already commented on some of the issues, and I suggested an option is for them to disable the new (partial) PEP518 support by default, but I don't have the impression that they will do that.

I don't think there is anything to do for 0.23 (I mainly opened this issue as a point-to issue with a summary of the situation, to point to in case other duplicate issues are opened).
I think the only thing we can do for 0.23.0 is to update documentation on how to use pip install pandas in case of 'special' platforms.

@maximveksler

This comment has been minimized.

Copy link
Contributor

maximveksler commented Apr 29, 2018

adding 2 cents:

I'm building pandas from source on Windows as part our development process, it started failing on dev machines and the work around was to add the --no-build-isolation.

From a pandas user perspective "something that worked before break now, without me changing anything".

So I don't think it's just an issue with the "special" platforms, it also applies to users of windows that are building the library from source.

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Apr 29, 2018

@jorisvandenbossche

This comment has been minimized.

Copy link
Member

jorisvandenbossche commented Apr 29, 2018

Yes, the reason I put "special platforms" is because there you automatically build from source with pip, while for "common" platforms the default is just installing from wheels.

From a pandas user perspective "something that worked before break now, without me changing anything".

You are completely right, but unfortunately, there is nothing we can do about it. We partly caused it by prematurely adding a pyproject.toml, but now it is only pip who can fix it.

@rahulwa

This comment has been minimized.

Copy link

rahulwa commented May 6, 2018

How can i make it work with pipenv?

I am trying like this but this doesn't work
export PIP_NO_BUILD_ISOLATION=true && pipenv install pandas


Edited: Nevermind, PIP_NO_BUILD_ISOLATION=false worked.

c-mertes added a commit to c-mertes/autoCorrection that referenced this issue May 8, 2018

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented May 14, 2018

@jorisvandenbossche do want to include anything in the 0.23 docs about this? Since this is (hopefully) temporary, I'm OK with people searching out this issue rather than codifying anything in the docs.

@jorisvandenbossche

This comment has been minimized.

Copy link
Member

jorisvandenbossche commented May 14, 2018

Or maybe a very brief note with something like "if having installation problems with pip when installing from source (eg for platforms for which no wheels are provided), see " ?
Then at least we can keep the information here easier up to date.

@jorisvandenbossche

This comment has been minimized.

Copy link
Member

jorisvandenbossche commented May 14, 2018

Although, it will not be the case anymore for 0.23.0 once released, so it doesn't really make sense to put it in the whatsnew note there. So yes, let's just keep this issue then?

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented May 14, 2018

Although, it will not be the case anymore for 0.23.0 once released, so it doesn't really make sense to put it in the whatsnew note there.

That's my thoughts.

(moving this off the 0.23.0 milestone).

@TomAugspurger TomAugspurger modified the milestones: 0.23.0, No action May 14, 2018

@UnitedMarsupials

This comment has been minimized.

Copy link

UnitedMarsupials commented May 15, 2018

FWIW, I'm getting this error with python-3.6.5 -- and pip-10.0.1. The work-around of adding PIP_NO_BUILD_ISOLATION=false to the environment seems to help.

blooper05 added a commit to blooper05/cloudmapper that referenced this issue Jun 19, 2018

Add PIP_NO_BUILD_ISOLATION=false to Dockerfile
`docker-compose build` occurs an known installation error of `pandas`.
refs. pandas-dev/pandas#20775

blooper05 added a commit to blooper05/cloudmapper that referenced this issue Jun 20, 2018

Add PIP_NO_BUILD_ISOLATION=false to Dockerfile
`docker-compose build` occurs an known installation error of `pandas`.
refs. pandas-dev/pandas#20775

blooper05 added a commit to blooper05/cloudmapper that referenced this issue Jun 20, 2018

Add PIP_NO_BUILD_ISOLATION=false to Dockerfile
`docker-compose build` occurs a known installation error of `pandas`.
refs. pandas-dev/pandas#20775

blooper05 added a commit to blooper05/cloudmapper that referenced this issue Jun 20, 2018

Add PIP_NO_BUILD_ISOLATION=false to Dockerfile
`docker-compose build` occurs a known installation error of `pandas`.
refs. pandas-dev/pandas#20775

@Juanlu001 Juanlu001 referenced this issue Jun 28, 2018

Merged

Build updates #69

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment