New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pandas-0.19.0rc1: 'global name tempita is not defined' #14204

Closed
jgerardsimcock opened this Issue Sep 11, 2016 · 15 comments

Comments

Projects
None yet
6 participants
@jgerardsimcock

screen shot 2016-09-11 at 4 27 54 pm

I am running a Travis CI build in a python 2.7 min environment. Not sure why it is pulling from an unreleased version of pandas but it looks like the tempita needs to be imported.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Sep 11, 2016

Contributor

you must have a really really old version of cython (or maybe cython is not installed). you are building from source so you need to have the development requirements. http://pandas.pydata.org/pandas-docs/stable/install.html#installing-from-source

This is not recommended unless you are developing pandas. there are wheels and conda packages available.

Contributor

jreback commented Sep 11, 2016

you must have a really really old version of cython (or maybe cython is not installed). you are building from source so you need to have the development requirements. http://pandas.pydata.org/pandas-docs/stable/install.html#installing-from-source

This is not recommended unless you are developing pandas. there are wheels and conda packages available.

@jreback jreback closed this Sep 11, 2016

@jreback jreback added this to the 0.19.0 milestone Sep 11, 2016

@jreback jreback removed the Usage Question label Sep 11, 2016

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Sep 11, 2016

Contributor

acutally this should be a better reported error (meaning setup should fail)

when cython is NOT installed (as is required), we cannot run tempita.
So need to put a check in.

[Sun Sep 11 19:47:16 ~/pandas]$ rm pandas/src/algos_groupby_helper.pxi
[Sun Sep 11 19:47:36 ~/pandas]$ make
python setup.py build_ext --inplace
running build_ext
Traceback (most recent call last):
  File "setup.py", line 94, in <module>
    if cython:
  File "/Users/jreback/miniconda/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/Users/jreback/miniconda/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Users/jreback/miniconda/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/jreback/miniconda/lib/python2.7/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "setup.py", line 368, in build_extensions
    build_ext.build_extensions(self)
  File "setup.py", line 140, in build_extensions
    pyxcontent = tempita.sub(tmpl)
NameError: global name 'tempita' is not defined
make: *** [tseries] Error 1
Contributor

jreback commented Sep 11, 2016

acutally this should be a better reported error (meaning setup should fail)

when cython is NOT installed (as is required), we cannot run tempita.
So need to put a check in.

[Sun Sep 11 19:47:16 ~/pandas]$ rm pandas/src/algos_groupby_helper.pxi
[Sun Sep 11 19:47:36 ~/pandas]$ make
python setup.py build_ext --inplace
running build_ext
Traceback (most recent call last):
  File "setup.py", line 94, in <module>
    if cython:
  File "/Users/jreback/miniconda/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/Users/jreback/miniconda/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Users/jreback/miniconda/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/jreback/miniconda/lib/python2.7/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "setup.py", line 368, in build_extensions
    build_ext.build_extensions(self)
  File "setup.py", line 140, in build_extensions
    pyxcontent = tempita.sub(tmpl)
NameError: global name 'tempita' is not defined
make: *** [tseries] Error 1

@jreback jreback reopened this Sep 11, 2016

@jreback jreback closed this in 461e0e9 Sep 12, 2016

@TomSmithCGAT TomSmithCGAT referenced this issue Sep 18, 2016

Closed

Improve non-conda setup #46

5 of 5 tasks complete
@fungi

This comment has been minimized.

Show comment
Hide comment
@fungi

fungi Oct 4, 2016

It's unclear to me why this was implemented with the equivalent of an assertion in setup.py rather than adding cython to the setup_requires list. Pandas 0.19.0 upgrades via pip on Linux (due to lack of Linux wheels support on PyPI) collapse catastrophically, requiring someone to manually pip install cython into the env first.

fungi commented Oct 4, 2016

It's unclear to me why this was implemented with the equivalent of an assertion in setup.py rather than adding cython to the setup_requires list. Pandas 0.19.0 upgrades via pip on Linux (due to lack of Linux wheels support on PyPI) collapse catastrophically, requiring someone to manually pip install cython into the env first.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Oct 4, 2016

Contributor

@fungi not sure what you are taking about

this issue is fixed on 0.19.0
and wheels are supoorted on linux

Contributor

jreback commented Oct 4, 2016

@fungi not sure what you are taking about

this issue is fixed on 0.19.0
and wheels are supoorted on linux

@fungi

This comment has been minimized.

Show comment
Hide comment
@fungi

fungi Oct 4, 2016

Wheels are supported on Linux, but Linux wheels (the manylinux1 platform PEP aside) are not supported on PyPI. Thus, pip install pandas on Linux throws "ImportError: Building pandas requires cython" instead of installing cython automatically. I'm in the process of opening a separate issue now for this and will gladly submit a PR updating setup.py to use setup_requires for cython unless there's a good reason it shouldn't.

fungi commented Oct 4, 2016

Wheels are supported on Linux, but Linux wheels (the manylinux1 platform PEP aside) are not supported on PyPI. Thus, pip install pandas on Linux throws "ImportError: Building pandas requires cython" instead of installing cython automatically. I'm in the process of opening a separate issue now for this and will gladly submit a PR updating setup.py to use setup_requires for cython unless there's a good reason it shouldn't.

@jorisvandenbossche

This comment has been minimized.

Show comment
Hide comment
@jorisvandenbossche

jorisvandenbossche Oct 4, 2016

Member

@fungi Have a look at PyPI, the linux wheels are definitely there: https://pypi.python.org/pypi/pandas. Maybe you need to update pip to download them?

Member

jorisvandenbossche commented Oct 4, 2016

@fungi Have a look at PyPI, the linux wheels are definitely there: https://pypi.python.org/pypi/pandas. Maybe you need to update pip to download them?

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Oct 4, 2016

Contributor

@fungi before you open an issue
please show an authoritative source that says setup_requires is a good idea -
and is a standard way to do this

development building has always required cython

Contributor

jreback commented Oct 4, 2016

@fungi before you open an issue
please show an authoritative source that says setup_requires is a good idea -
and is a standard way to do this

development building has always required cython

@fungi

This comment has been minimized.

Show comment
Hide comment
@fungi

fungi Oct 4, 2016

Those are the manylinux1 platform wheels to which I was referring. Support for manylinux1 was first added in pip 8.1.0 (March 2016), so yes this should work for anyone using a pip release from the past 7 months. I'm just curious why this was not added as a setup_requires so that it can work on a broader set of platforms which either may not automatically retrieve manylinux1 wheels or need to operate on sdists for other reasons. In contrast, you already have a setup_requires declared on numpy rather than simply aborting setup.py when you don't see it available in the environment.

fungi commented Oct 4, 2016

Those are the manylinux1 platform wheels to which I was referring. Support for manylinux1 was first added in pip 8.1.0 (March 2016), so yes this should work for anyone using a pip release from the past 7 months. I'm just curious why this was not added as a setup_requires so that it can work on a broader set of platforms which either may not automatically retrieve manylinux1 wheels or need to operate on sdists for other reasons. In contrast, you already have a setup_requires declared on numpy rather than simply aborting setup.py when you don't see it available in the environment.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Oct 4, 2016

Contributor

hmm, we are using setup_requires already. ok, then guess would take a PR for adding cython there. Not really sure why we don't have that already, though we do have a slightly different process than some other libraries. IOW, we don't checkin in .c and instead build them in the dev version.

Contributor

jreback commented Oct 4, 2016

hmm, we are using setup_requires already. ok, then guess would take a PR for adding cython there. Not really sure why we don't have that already, though we do have a slightly different process than some other libraries. IOW, we don't checkin in .c and instead build them in the dev version.

@fungi

This comment has been minimized.

Show comment
Hide comment
@fungi

fungi Oct 4, 2016

As to "development building has always required cython," prior to 0.19.0 development building must not have included installs from sdist since pip install 'pandas<0.19.0' on a system with, e.g., pip 7.1.0 successfully retrieves pandas-0.18.1.tar.gz from PyPI and installs it into the environment.

fungi commented Oct 4, 2016

As to "development building has always required cython," prior to 0.19.0 development building must not have included installs from sdist since pip install 'pandas<0.19.0' on a system with, e.g., pip 7.1.0 successfully retrieves pandas-0.18.1.tar.gz from PyPI and installs it into the environment.

@lmeyerov

This comment has been minimized.

Show comment
Hide comment
@lmeyerov

lmeyerov Oct 10, 2016

FWIW, we're hitting this in our various builds and are not sure how to treat. We're seeing the same error for both our internal jenkins & public usages, e.g., https://readthedocs.org/projects/pygraphistry/builds/4425847/ .

FWIW, we're hitting this in our various builds and are not sure how to treat. We're seeing the same error for both our internal jenkins & public usages, e.g., https://readthedocs.org/projects/pygraphistry/builds/4425847/ .

@jorisvandenbossche

This comment has been minimized.

Show comment
Hide comment
@jorisvandenbossche

jorisvandenbossche Oct 10, 2016

Member

The error message can be improved (but that is fixed in 0.19.0, which is released in the meantime), but, as explained in the above comments, this just means that you are trying to build pandas from source, which requires cython to be installed. And this has not changed in comparison to previous versions, this has always been the case.

However, normally there should be wheels available. Not sure why readthedocs is downloading the zip source instead of the binary wheels.

Member

jorisvandenbossche commented Oct 10, 2016

The error message can be improved (but that is fixed in 0.19.0, which is released in the meantime), but, as explained in the above comments, this just means that you are trying to build pandas from source, which requires cython to be installed. And this has not changed in comparison to previous versions, this has always been the case.

However, normally there should be wheels available. Not sure why readthedocs is downloading the zip source instead of the binary wheels.

@fungi

This comment has been minimized.

Show comment
Hide comment
@fungi

fungi Oct 19, 2016

I think there's probably still some confusion here. The "fix" in 0.19.0 makes it impossible to install the pandas sdist (.tar.gz or .zip) packages published on PyPI without manually preinstalling cython in the environment. The sdists of prior releases up to and including 0.18.1 could be installed without cython present. Digging deeper, it seems this worked because you were (and still are, in fact) shipping the cython-generated files in your sdists, which made them installable without cython present. So yes, creating the sdist does require cython (and this is what I assume you mean by "building from source," e.g. setup.py install or setup.py sdist or similar), but installing the created sdist did not require cython until the additional raise ImportError was added in 461e0e9. I get that it made "installs from source" fail with a slightly more obvious error, but it also broke "installs from sdist" at the same time.

fungi commented Oct 19, 2016

I think there's probably still some confusion here. The "fix" in 0.19.0 makes it impossible to install the pandas sdist (.tar.gz or .zip) packages published on PyPI without manually preinstalling cython in the environment. The sdists of prior releases up to and including 0.18.1 could be installed without cython present. Digging deeper, it seems this worked because you were (and still are, in fact) shipping the cython-generated files in your sdists, which made them installable without cython present. So yes, creating the sdist does require cython (and this is what I assume you mean by "building from source," e.g. setup.py install or setup.py sdist or similar), but installing the created sdist did not require cython until the additional raise ImportError was added in 461e0e9. I get that it made "installs from source" fail with a slightly more obvious error, but it also broke "installs from sdist" at the same time.

frol added a commit to frol/docker-alpine-python-machinelearning that referenced this issue Oct 20, 2016

@frol

This comment has been minimized.

Show comment
Hide comment
@frol

frol Oct 20, 2016

There is no wheel for Alpine Linux, so installing Pandas 0.19.0 from PyPi without Cython installed manually, the exception is raised and the installation breaks: https://hub.docker.com/r/frolvlad/alpine-python-machinelearning/builds/bfih8p7usyqibefhnsjk5be/

frol commented Oct 20, 2016

There is no wheel for Alpine Linux, so installing Pandas 0.19.0 from PyPi without Cython installed manually, the exception is raised and the installation breaks: https://hub.docker.com/r/frolvlad/alpine-python-machinelearning/builds/bfih8p7usyqibefhnsjk5be/

@jorisvandenbossche

This comment has been minimized.

Show comment
Hide comment
@jorisvandenbossche

jorisvandenbossche Oct 20, 2016

Member

@fungi Aha, thanks, that clarifies the issue. We should try to solve that for 0.19.1

Member

jorisvandenbossche commented Oct 20, 2016

@fungi Aha, thanks, that clarifies the issue. We should try to solve that for 0.19.1

@jorisvandenbossche jorisvandenbossche modified the milestones: 0.19.1, 0.19.0 Oct 20, 2016

@chris-b1 chris-b1 referenced this issue Oct 22, 2016

Closed

BLD: don't require cython on sdist install #14475

4 of 4 tasks complete

@jreback jreback closed this in 192b1cd Oct 24, 2016

jorisvandenbossche added a commit to jorisvandenbossche/pandas that referenced this issue Nov 2, 2016

amolkahat added a commit to amolkahat/pandas that referenced this issue Nov 26, 2016

frol added a commit to frol/docker-alpine-python-machinelearning that referenced this issue May 26, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment