pip install pyarrow does not work on python 3.7 #1125
Comments
|
We probably aren't going to publish Python 3.7 wheels until 3.7.0 final is out. There is already https://issues.apache.org/jira/browse/ARROW-1535 which would enable source installs to work, would you be able to help with that? |
|
I am new here and a bit confused. I read #1099 and saw that you argue against source builds from "pip" because it involves a lot of work when libraries aren't already installed on the system. Are you still planning to have source builds be available, but just with the caveat that you have to install library requirements yourself ahead of time? I know that's common for e.g. MySQL-python. |
|
Yes, that's what ARROW-1535 is about, so hopefully the 0.8.0 will be able to build from source from PyPI. We are building .deb and .rpm packages for the C++ libraries so installing these on most Linux systems without building some source should not be too bad |
|
ARROW-1535 is resolved so the sdist artifact should be able to be installed. If you could help kick the tires with Python 3.7 before Arrow 0.8.0 is released (end of October / early November) that would be helpful in case we missed some detail |
|
This is the situation for me: pascal@archbook:~$ pip3.7 install --no-cache pyarrow
Collecting pyarrow
Downloading pyarrow-0.3.0.tar.gz (78kB)It is using 0.3.0 and not 0.7.1. I also tried copying the manylinux wheel and renaming it, so it should work with 3.7, but it misses the C++ bindings. Somehow the C++ bindings cannot be found even though 3.6 is using them. |
|
@kootenpv you will have to test using a tarball built from trunk (i.e. |
|
It sounded like a really cool way to do it! I wasn't able to do it though :(. The best I can do is explain what I've tried. On that command, I get output ending with: What am I doing wrong? EDIT: doing make (after cmake), I reached 91% of building, until: Strange, the error only seems to be found here: https://pastebin.com/L43rc3j0 which specifically mentions py3.7. was using |
|
I created https://issues.apache.org/jira/browse/ARROW-1661 to track fixing the libarrow_python build. Note that you must also install the libraries someplace ( |
|
Doing It happens around 40% of the pyenv shell 3.7-dev # <-- installing pyenv, and installing py3.7 with it, you can test it yourself |
|
I guess I forgot to mention I'm using Arch Linux, running the latest stuff does have a downside in this case. I did 2 things and it got a lot further in the and in Now this is the case: When I use |
|
Full script so far: Result: |
|
Can we move this discussion to JIRA? It seems there are multiple layers of issues here, including problems due to bleeding edge gcc. We don't have a lot of developers using bleeding-edge-everything so we will need to create a bunch of issues to tackle each problem in due course |
|
Seems like there are a lot of things that may interfere, it could also be that we detect some things like |
|
I'm out of ideas, but I'm willing to try stuff. Should the discussion continue at https://issues.apache.org/jira/browse/ARROW-1661? |
|
Seems like this is a problem with something of your system configuration or Python 3.7. Using the latest ArchLinux version, Python 3.6, GCC 7.2 and the manual from https://arrow.apache.org/docs/python/development.html#developing-on-linux-and-macos I got Arrow & Parquet successfully built. All tests except those for plasma succeed. Maybe building with an sdist is changing some behaviour. |
|
@xhochy I have it working under Python 3.6 on the same machine (not building from source but from pip, haven't tried from source). It's specifically Python 3.7 and building it from source that seems to be the problem for my system. |
|
@kootenpv would you be up for helping create a Dockerfile for a Python 3.7 build/test? Since 3.7.0 final is not slated until June 2018, this would be worth the effort to be able to reproducibly test until we have a full conda-based stack available for 3.7 |
|
Closing in favor of tracking on https://issues.apache.org/jira/browse/ARROW-1661 |
|
still no Python-3.7 wheel on pypi ... |
|
@stonebig would you like to lend a hand? |
|
You should probably add a warning on "https://arrow.apache.org/install/" saying that you currently cannot install with pip. The current text is misleading in that it tells me that it should work. I tried compiling/installing arrow/cpp from source (OSX / python 3.7) and then installing pyarrow==0.10.* with pip and it still did not work. Got a strange error `Collecting pyarrow INFOCompiler id: Clang Failed building wheel for pyarrow Command "/usr/local/opt/python/bin/python3.7 -u -c "import setuptools, tokenize;file='/private/var/folders/yy/wvq3f61531s70xcq2fyzh7mc0000gn/T/pip-install-24lpih95/pyarrow/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /private/var/folders/yy/wvq3f61531s70xcq2fyzh7mc0000gn/T/pip-record-7m129y7b/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/yy/wvq3f61531s70xcq2fyzh7mc0000gn/T/pip-install-24lpih95/pyarrow/` |
It does work for Python<3.7. For Python 3.7, you need to have installed the Arrow C++ packages via different means. |
|
Locking this as the discussions here go into various directions and should continue as separate issues on https://issues.apache.org/jira/projects/ARROW |
Even after installing Cython before, manually.
The text was updated successfully, but these errors were encountered: