pip install pyarrow==0.9.0.post1 doesn't work on docker Alpine 3.5 or linux

[namank5x]

This version of pyarrow only works on while installing on Mac, doesn't work on linux machines or docker alpine images

Also, simply installing the latest version of pyarrow throws a cython not installed error.
Is there an alternative version works?

Thanks

[xhochy]

pyarrow==0.9.0.post1 is a mac-only version, all other system are released as pyarrow==0.9.0. As the initial 0.9.0 upload was broken for Mac and you cannot reupload the same version again on PyPI we had to add the prefix.

Also note that Alpine Linux is not manylinux1 compatible as it uses a different libc (musl) thus you will need to build pyarrow from sources: https://arrow.apache.org/docs/python/development.html#development

[naveenarumugam]

I am receiving the error as below when trying to use pyarrow 0.9 after installation
ImportError: dlopen(pyarrow/lib.so, 2): Symbol not found: _os_unfair_lock_lock

python -c "import pyarrow as pa; print pa.__version__" Traceback (most recent call last): File "<string>", line 1, in <module> File "pyarrow/__init__.py", line 47, in <module> from pyarrow.lib import cpu_count, set_cpu_count ImportError: dlopen(pyarrow/lib.so, 2): Symbol not found: _os_unfair_lock_lock Referenced from: /usr/local/lib/python2.7/site-packages/pyarrow/libarrow.0.dylib (which was built for Mac OS X 10.12) Expected in: /usr/lib/libSystem.B.dylib in /usr/local/lib/python2.7/site-packages/pyarrow/libarrow.0.dylib

• OS: Mac
• Successfully installed pyarrow-0.9.0.post1
• pip freeze | grep pyarrow ==> pyarrow==0.9.0.post1
• Installation log

Collecting pyarrow Downloading https://files.pythonhosted.org/packages/38/5d/ee1f7ef8d8803ab74eefe633e227d19a53f12b04e4ac4cc4bdd20664b956/pyarrow-0.9.0.post1-cp27-cp27m-macosx_10_6_intel.whl (9.5MB) 100% |████████████████████████████████| 9.5MB 4.2MB/s Requirement already satisfied: futures in /usr/local/lib/python2.7/site-packages (from pyarrow) (3.2.0) Requirement already satisfied: six>=1.0.0 in /usr/local/lib/python2.7/site-packages (from pyarrow) (1.11.0) Requirement already satisfied: numpy>=1.10 in /usr/local/lib/python2.7/site-packages (from pyarrow) (1.14.2) google-cloud-dataflow 2.2.0 requires apache-beam[gcp]==2.2.0, which is not installed. quandl 3.2.0 has requirement requests<2.18,>=2.7.0, but you'll have requests 2.18.3 which is incompatible. pyasn1-modules 0.0.11 has requirement pyasn1==0.3.2, but you'll have pyasn1 0.4.2 which is incompatible. google-cloud-vision 0.29.0 has requirement requests<3.0dev,>=2.18.4, but you'll have requests 2.18.3 which is incompatible. grpcio 1.8.4 has requirement protobuf>=3.5.0.post1, but you'll have protobuf 3.3.0 which is incompatible. google-cloud-spanner 0.29.0 has requirement google-auth>=1.1.0, but you'll have google-auth 1.0.2 which is incompatible. google-cloud-spanner 0.29.0 has requirement requests<3.0dev,>=2.18.4, but you'll have requests 2.18.3 which is incompatible. Installing collected packages: pyarrow Successfully installed pyarrow-0.9.0.post1

[mdeland]

I'm seeing the same error on import after successful install of pyarrow-0.9.0.post1

Symbol not found: _os_unfair_lock_lock

[xhochy]

@naveenarumugam This is totally different issue than the initial one posted. Please open an issue on https://issues.apache.org/jira/projects/ARROW including the version of Mac OS you're using.

[xhochy]

@naveenarumugam @mdeland Captured your problem in https://issues.apache.org/jira/browse/ARROW-2553 This should be easily solved in the next release.

[kadrach]

@xhochy It looks like this approach breaks "locking" of dependencies as is done by pipenv.

When Pipenv does locking, it needs to (well, tries to) make the lock file cross-platform. With a wildcard installation (pyarrow = "*"), it needs to find a latest version for all platforms—but there isn’t one! pyarrow 0.9.0.post1 only works on macOS, and 0.9.0 does not work on macOS (they deleted the broken macOS wheels).

I have little knowledge of the internals of python packaging, it sounds like it is expected that the "latest version" should be compatible with all platforms.