import rtree leads to OSError: Could not load libspatialindex_c library on aarch64

[thomasaarholt]

Hiya!
This is possibly related to #244.

The following is an issue when pip-installing rtree on linux on arm architecture (aarch64 is the technical name). I do so in a docker container on an M1 (arm) Mac. The issue is not present on Mac.

I recently resumed some work involving geopandas, and with only updating / reinstalling the packages I was using I am now getting the following error. This is quite possibly the same as #244.

conda create -n test python
conda activate test
pip install rtree
python

# With python 3.10 on linux
>>> import rtree
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/rtree/__init__.py", line 9, in <module>
    from .index import Index, Rtree  # noqa
  File "/usr/local/lib/python3.9/site-packages/rtree/index.py", line 17, in <module>
    from . import core
  File "/usr/local/lib/python3.9/site-packages/rtree/core.py", line 74, in <module>
    rt = finder.load()
  File "/usr/local/lib/python3.9/site-packages/rtree/finder.py", line 118, in load
    raise OSError("Could not load libspatialindex_c library")
OSError: Could not load libspatialindex_c library

Edit: I realise now that I might have used pygeos instead of rtree in the past - in order to get spatial joins in geopandas. So rtree may not have previously worked on linux aarch64.

[hobu]

I have an m1 mac. if you can add a Dockerfile and its build command that demonstrates the issue, I would be much more likely to attempt to address it.

These kinds of errors have been so hard to address. if you're using conda, you should use the conda-forge Rtree package instead of pip-installing the wheel. Maybe things are getting confused there.

[thomasaarholt]

A very reasonable request!
Here's a minimum working example of the issue. I suspect it will run fine if run on x86 architecture, but I have no way to confirm:

Dockerfile contents (example taken from @here for anyone who is new to this - the filename is Dockerfile.)

FROM python:3.9-slim-buster

RUN pip install rtree

CMD ["sh", "-c", "python -c 'import rtree'"]

Run the following commands from the directory containing the dockerfile

docker build -t rtree_error .
docker run rtree

The error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/rtree/__init__.py", line 9, in <module>
    from .index import Index, Rtree  # noqa
  File "/usr/local/lib/python3.9/site-packages/rtree/index.py", line 17, in <module>
    from . import core
  File "/usr/local/lib/python3.9/site-packages/rtree/core.py", line 74, in <module>
    rt = finder.load()
  File "/usr/local/lib/python3.9/site-packages/rtree/finder.py", line 118, in load
    raise OSError("Could not load libspatialindex_c library")
OSError: Could not load libspatialindex_c library

[thomasaarholt]

These kinds of errors have been so hard to address. if you're using conda, you should use the conda-forge Rtree package instead of pip-installing the wheel. Maybe things are getting confused there.

I didn't fully address this - I would actually much prefer the pip-solution to work if possible. The only reason I created a conda environment is that I find the environment-functionality very convenient there (and I feel that venv / asdf aren't quite as user-friendly). The reason is that I'm using geopandas to build a geographical calendar at work, and at work everyone is using linux-in-docker via pip for reproducibility.

But I totally understand if this level of debugging is outside your scope! I can probably find a different way.

[hobu]

pip install Rtree in this case only installed the source for the Python library, not the libspatialindex library that's also included with the wheels. That's because none of the wheels on https://pypi.org/project/Rtree/#files match the sys_tags.

In short, the wheel didn't get installed.

python  -c "import packaging.tags; print(list(packaging.tags.sys_tags()))"

In this case, just do apt-get install libspatialindex-c5 -y to pull that in and things should work.

[simaoafonso-pwt]

@hobu The point of wheels is that all necessary libraries are included, right? At the very least, if the library is not found, the installation should fail, not only at runtime.

[hobu]

@simaoafonso-pwt Please stop spamming the repository. If you need 32bit wheels you will have to build them yourself.