Issue in Pyarrow installation in Docker file

[srinathtcs1993]

I am trying to add pyarrow to my docker file as a dependency. I am getting an issue as below

cmake --build . --config release --
[ 1%] Compiling Cython CXX source for lib...
[91m
Error compiling Cython file:
...

cdef extern from "Python.h":
int PySlice_Check(object)

cdef int check_status(const CStatus& status) nogil except -1

pyarrow/lib.pxd:67:60: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.

Error compiling Cython file:
...
ArrowIOError = IOError

This function could be written directly in C++ if we didn't
define Arrow-specific subclasses (ArrowInvalid etc.)
cdef int check_status(const CStatus& status) nogil except -1:

pyarrow/error.pxi:82:60: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.

Error compiling Cython file:
...
raise ArrowException(message)

This is an API function for C++ PyArrow
cdef api int pyarrow_internal_check_status(const CStatus& status)
nogil except -1:

pyarrow/error.pxi:143:23: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.

[0m[91mmake[2]: *** [CMakeFiles/lib_pyx.dir/build.make:71: CMakeFiles/lib_pyx] Error 1
[0m[91mmake[1]: *** [CMakeFiles/Makefile2:140: CMakeFiles/lib_pyx.dir/all] Error 2
[0m[91mmake: *** [Makefile:136: all] Error 2
[0m[91merror: command 'cmake' failed with exit status 2
[0mThe command '/bin/sh -c mkdir /arrow && wget -q https://github.com/apache/arrow/archive/apache-arrow-${ARROW_VERSION}.tar.gz -O /tmp/apache-arrow.tar.gz && echo "${ARROW_SHA1} *apache-arrow.tar.gz" | sha1sum /tmp/apache-arrow.tar.gz && tar -xvf /tmp/apache-arrow.tar.gz -C /arrow --strip-components 1 && mkdir -p /arrow/cpp/build && cd /arrow/cpp/build && cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE -DOPENSSL_ROOT_DIR=/usr/local/ssl -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=$ARROW_HOME -DARROW_WITH_BZ2=ON -DARROW_WITH_ZLIB=ON -DARROW_WITH_ZSTD=ON -DARROW_WITH_LZ4=ON -DARROW_WITH_SNAPPY=ON -DARROW_PARQUET=ON -DARROW_PYTHON=ON -DARROW_PLASMA=ON -DARROW_BUILD_TESTS=OFF .. && make -j$(nproc) && make install && cd /arrow/python && python setup.py build_ext --build-type=$ARROW_BUILD_TYPE --with-parquet && python setup.py install && rm -rf /arrow /tmp/apache-arrow.tar.gz' returned a non-zero code: 1

This is my docker build file code

FROM python:3.8-alpine3.16

RUN apk update
&& apk upgrade
&& apk add --no-cache build-base
autoconf
thrift-dev
bash
bison
boost-dev
cmake
flex
libressl-dev
zlib-dev
RUN apk upgrade expat

RUN pip install --no-cache-dir six numpy cython

ARG ARROW_VERSION=11.0.0
ARG ARROW_SHA1=c1fed962cddfab1966a0e03461376ebb28cf17d3
ARG ARROW_BUILD_TYPE=release

ENV ARROW_HOME=/usr/local
PARQUET_HOME=/usr/local

#Download and build apache-arrow
RUN mkdir /arrow
&& wget -q https://github.com/apache/arrow/archive/apache-arrow-${ARROW_VERSION}.tar.gz -O /tmp/apache-arrow.tar.gz
&& echo "${ARROW_SHA1} *apache-arrow.tar.gz" | sha1sum /tmp/apache-arrow.tar.gz
&& tar -xvf /tmp/apache-arrow.tar.gz -C /arrow --strip-components 1
&& mkdir -p /arrow/cpp/build
&& cd /arrow/cpp/build
&& cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE
-DOPENSSL_ROOT_DIR=/usr/local/ssl
-DCMAKE_INSTALL_LIBDIR=lib
-DCMAKE_INSTALL_PREFIX=$ARROW_HOME
-DARROW_WITH_BZ2=ON
-DARROW_WITH_ZLIB=ON
-DARROW_WITH_ZSTD=ON
-DARROW_WITH_LZ4=ON
-DARROW_WITH_SNAPPY=ON
-DARROW_PARQUET=ON
-DARROW_PYTHON=ON
-DARROW_PLASMA=ON
-DARROW_BUILD_TESTS=OFF
..
&& make -j$(nproc)
&& make install
&& cd /arrow/python
&& python setup.py build_ext --build-type=$ARROW_BUILD_TYPE --with-parquet
&& python setup.py install
&& rm -rf /arrow /tmp/apache-arrow.tar.gz

kindly let me know what changes should I do to add pyarrow as a dependency to docker file

Component(s)

Python

[raulcd]

Hi,

What version of Cython it is installing? There was a major Cython release some days ago where we are not yet compatible. Can you try to pin Cython with:

RUN pip install --no-cache-dir six numpy "cython<3"

[srinathtcs1993]

tried with cython version 3.0.0, but still same issue exist

[raulcd]

We do not support Cython 3.0 yet. Could you share the log of the Cython version installed, please?

[srinathtcs1993]

tried with this version Cython==0.29.36 and it is working now

[raulcd]

Thanks for sharing! I'll close this issue as the issue to be compatible with Cython >= 3.0 is tracked here: #36730