Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WARNING:root:Failed to get options via gdal-config (M1 mac) #55

Closed
HugoHeneault opened this issue Nov 11, 2021 · 13 comments
Closed

WARNING:root:Failed to get options via gdal-config (M1 mac) #55

HugoHeneault opened this issue Nov 11, 2021 · 13 comments

Comments

@HugoHeneault
Copy link
Contributor

Hi there,

I'm running make build on macOS (m1) 12.0.1 and encounter this issue:

#11 6.510 Collecting rasterio==1.2.9
#11 6.553   Downloading rasterio-1.2.9.tar.gz (2.3 MB)
#11 7.162   Installing build dependencies: started
#11 13.24   Installing build dependencies: finished with status 'done'
#11 13.24   Getting requirements to build wheel: started
#11 13.45   Getting requirements to build wheel: finished with status 'error'
#11 13.45   ERROR: Command errored out with exit status 1:
#11 13.45    command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_b8p17k_
#11 13.45        cwd: /tmp/pip-install-rusm7ufj/rasterio_d6e3703bd238492fb428a60f0b4d1a47
#11 13.45   Complete output (2 lines):
#11 13.45   WARNING:root:Failed to get options via gdal-config: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'
#11 13.45   ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable.
#11 13.45   ----------------------------------------
#11 13.45 WARNING: Discarding https://files.pythonhosted.org/packages/1f/70/4e7a789f4988955e4c381de80923e184f912683bbe6fc4a3a00c91efdf59/rasterio-1.2.9.tar.gz#sha256=012a4964d8db365be4fae0af9cbeba00e683e5904d5031e8ba42ccb6040cc887 (from https://pypi.org/simple/rasterio/) (requires-python:>=3.6). Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_b8p17k_ Check the logs for full command output.
#11 13.45 ERROR: Could not find a version that satisfies the requirement rasterio==1.2.9 (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.5.1, 0.6, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.9, 0.10, 0.10.1, 0.11, 0.11.1, 0.12, 0.12.1, 0.13, 0.13.1, 0.13.2, 0.14, 0.14.1, 0.15, 0.15.1, 0.16, 0.17, 0.17.1, 0.18, 0.19.0, 0.19.1, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.24.1, 0.25.0, 0.26.0, 0.27.0, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.32.0.post1, 0.33.0, 0.34.0, 0.35.0, 0.35.0.post1, 0.35.1, 0.36.0, 1.0a1, 1.0a2, 1.0a3, 1.0a4, 1.0a6, 1.0a7, 1.0a8, 1.0a9, 1.0a10, 1.0a11, 1.0a12, 1.0b1, 1.0b2, 1.0b3, 1.0b4, 1.0rc1, 1.0rc2, 1.0rc3, 1.0rc4, 1.0rc5, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.3.post1, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.8, 1.0.9, 1.0.10, 1.0.11, 1.0.12, 1.0.13, 1.0.14, 1.0.15, 1.0.16, 1.0.17, 1.0.18, 1.0.20, 1.0.21, 1.0.22, 1.0.23, 1.0.24, 1.0.25, 1.0.26, 1.0.27, 1.0.28, 1.1b1, 1.1b2, 1.1b3, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.2a1, 1.2b1, 1.2b2, 1.2b3, 1.2b4, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.3a1, 1.3a2)
#11 13.46 ERROR: No matching distribution found for rasterio==1.2.9
------
executor failed running [/bin/sh -c pip install         --no-index         --no-cache-dir         --disable-pip-version-check         --find-links=/root/wheels         uwsgi regex &&     pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt &&         rm -rf /root/.cache/pip/* &&         rm root/wheels/* &&         rm /app/requirements.txt]: exit code: 1
make: *** [build] Error 1
Full logs
$ make build
docker build --tag opentopodata:`cat VERSION` --file docker/Dockerfile .
[+] Building 18.1s (11/14)
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                                                                                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 37B                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        0.0s
 => => transferring context: 34B                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         0.0s
 => [internal] load metadata for docker.io/library/python:3.7.12-slim-buster                                                                                                                                                                                                                                                                                                                                                                                                                                             4.3s
 => [internal] load build context                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        0.0s
 => => transferring context: 5.39kB                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      0.0s
 => [builder 1/3] FROM docker.io/library/python:3.7.12-slim-buster@sha256:866b51113519f9022e2f432869d3e7d97b090b9853f4765dbf46394ac28d5a03                                                                                                                                                                                                                                                                                                                                                                               0.0s
 => CACHED [stage-1 2/8] RUN set -e &&     apt-get update &&     apt-get install -y --no-install-recommends         nginx         memcached         supervisor &&     rm -rf /var/lib/apt/lists/*                                                                                                                                                                                                                                                                                                                        0.0s
 => CACHED [builder 2/3] RUN set -e &&     apt-get update &&     apt-get install -y --no-install-recommends         gcc         python3.7-dev                                                                                                                                                                                                                                                                                                                                                                            0.0s
 => CACHED [builder 3/3] RUN pip wheel --wheel-dir=/root/wheels uwsgi==2.0.19.1 &&     pip wheel --wheel-dir=/root/wheels regex==2021.9.30                                                                                                                                                                                                                                                                                                                                                                               0.0s
 => CACHED [stage-1 3/8] COPY --from=builder /root/wheels /root/wheels                                                                                                                                                                                                                                                                                                                                                                                                                                                   0.0s
 => CACHED [stage-1 4/8] COPY requirements.txt /app/requirements.txt                                                                                                                                                                                                                                                                                                                                                                                                                                                     0.0s
 => ERROR [stage-1 5/8] RUN pip install         --no-index         --no-cache-dir         --disable-pip-version-check         --find-links=/root/wheels         uwsgi regex &&     pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt &&         rm -rf /root/.cache/pip/* &&         rm root/wheels/* &&         rm /app/requirements.txt                                                                                                                                                 13.6s
------
 > [stage-1 5/8] RUN pip install         --no-index         --no-cache-dir         --disable-pip-version-check         --find-links=/root/wheels         uwsgi regex &&     pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt &&         rm -rf /root/.cache/pip/* &&         rm root/wheels/* &&         rm /app/requirements.txt:
#11 1.033 Looking in links: /root/wheels
#11 1.040 Processing /root/wheels/uWSGI-2.0.19.1-cp37-cp37m-linux_aarch64.whl
#11 1.044 Processing /root/wheels/regex-2021.9.30-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
#11 1.066 Installing collected packages: uwsgi, regex
#11 1.157 Successfully installed regex-2021.9.30 uwsgi-2.0.19.1
#11 1.157 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#11 1.536 Collecting affine==2.3.0
#11 1.706   Downloading affine-2.3.0-py2.py3-none-any.whl (15 kB)
#11 1.746 Collecting attrs==21.2.0
#11 1.773   Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
#11 1.839 Collecting black==21.9b0
#11 1.868   Downloading black-21.9b0-py3-none-any.whl (148 kB)
#11 1.944 Collecting certifi==2021.5.30
#11 1.980   Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
#11 2.051 Collecting charset-normalizer==2.0.6
#11 2.082   Downloading charset_normalizer-2.0.6-py3-none-any.whl (37 kB)
#11 2.121 Collecting click==8.0.1
#11 2.147   Downloading click-8.0.1-py3-none-any.whl (97 kB)
#11 2.189 Collecting click-plugins==1.1.1
#11 2.215   Downloading click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB)
#11 2.266 Collecting cligj==0.7.2
#11 2.290   Downloading cligj-0.7.2-py3-none-any.whl (7.1 kB)
#11 2.632 Collecting coverage[toml]==6.0.1
#11 2.661   Downloading coverage-6.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (243 kB)
#11 2.726 Collecting flask==2.0.2
#11 2.764   Downloading Flask-2.0.2-py3-none-any.whl (95 kB)
#11 2.811 Collecting flask-caching==1.10.1
#11 2.836   Downloading Flask_Caching-1.10.1-py3-none-any.whl (34 kB)
#11 2.873 Collecting geographiclib==1.52
#11 2.898   Downloading geographiclib-1.52-py3-none-any.whl (38 kB)
#11 2.938 Collecting idna==3.2
#11 2.964   Downloading idna-3.2-py3-none-any.whl (59 kB)
#11 3.027 Collecting importlib-metadata==4.8.1
#11 3.052   Downloading importlib_metadata-4.8.1-py3-none-any.whl (17 kB)
#11 3.083 Collecting iniconfig==1.1.1
#11 3.128   Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
#11 3.160 Collecting itsdangerous==2.0.1
#11 3.200   Downloading itsdangerous-2.0.1-py3-none-any.whl (18 kB)
#11 3.241 Collecting jinja2==3.0.2
#11 3.282   Downloading Jinja2-3.0.2-py3-none-any.whl (133 kB)
#11 3.361 Collecting markupsafe==2.0.1
#11 3.384   Downloading MarkupSafe-2.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (26 kB)
#11 3.409 Collecting mypy-extensions==0.4.3
#11 3.441   Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
#11 3.736 Collecting numpy==1.21.2
#11 3.770   Downloading numpy-1.21.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.0 MB)
#11 4.474 Collecting packaging==21.0
#11 4.500   Downloading packaging-21.0-py3-none-any.whl (40 kB)
#11 4.544 Collecting pathspec==0.9.0
#11 4.575   Downloading pathspec-0.9.0-py2.py3-none-any.whl (31 kB)
#11 4.606 Collecting pep517==0.11.0
#11 4.640   Downloading pep517-0.11.0-py2.py3-none-any.whl (19 kB)
#11 4.702 Collecting pip-tools==6.3.0
#11 4.727   Downloading pip_tools-6.3.0-py3-none-any.whl (47 kB)
#11 4.762 Collecting platformdirs==2.4.0
#11 4.793   Downloading platformdirs-2.4.0-py3-none-any.whl (14 kB)
#11 4.843 Collecting pluggy==1.0.0
#11 4.871   Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
#11 4.898 Collecting polyline==1.4.0
#11 4.934   Downloading polyline-1.4.0-py2.py3-none-any.whl (4.4 kB)
#11 4.986 Collecting py==1.10.0
#11 5.015   Downloading py-1.10.0-py2.py3-none-any.whl (97 kB)
#11 5.055 Collecting pylibmc==1.6.1
#11 5.081   Downloading pylibmc-1.6.1.tar.gz (64 kB)
#11 5.651 Collecting pyparsing==2.4.7
#11 5.684   Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
#11 5.785 Collecting pyproj==3.2.1
#11 5.822   Downloading pyproj-3.2.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (6.1 MB)
#11 6.103 Collecting pytest==6.2.5
#11 6.146   Downloading pytest-6.2.5-py3-none-any.whl (280 kB)
#11 6.205 Collecting pytest-cov==3.0.0
#11 6.243   Downloading pytest_cov-3.0.0-py3-none-any.whl (20 kB)
#11 6.313 Collecting pyyaml==5.4.1
#11 6.347   Downloading PyYAML-5.4.1-cp37-cp37m-manylinux2014_aarch64.whl (716 kB)
#11 6.510 Collecting rasterio==1.2.9
#11 6.553   Downloading rasterio-1.2.9.tar.gz (2.3 MB)
#11 7.162   Installing build dependencies: started
#11 13.24   Installing build dependencies: finished with status 'done'
#11 13.24   Getting requirements to build wheel: started
#11 13.45   Getting requirements to build wheel: finished with status 'error'
#11 13.45   ERROR: Command errored out with exit status 1:
#11 13.45    command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_b8p17k_
#11 13.45        cwd: /tmp/pip-install-rusm7ufj/rasterio_d6e3703bd238492fb428a60f0b4d1a47
#11 13.45   Complete output (2 lines):
#11 13.45   WARNING:root:Failed to get options via gdal-config: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'
#11 13.45   ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable.
#11 13.45   ----------------------------------------
#11 13.45 WARNING: Discarding https://files.pythonhosted.org/packages/1f/70/4e7a789f4988955e4c381de80923e184f912683bbe6fc4a3a00c91efdf59/rasterio-1.2.9.tar.gz#sha256=012a4964d8db365be4fae0af9cbeba00e683e5904d5031e8ba42ccb6040cc887 (from https://pypi.org/simple/rasterio/) (requires-python:>=3.6). Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_b8p17k_ Check the logs for full command output.
#11 13.45 ERROR: Could not find a version that satisfies the requirement rasterio==1.2.9 (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.5.1, 0.6, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.9, 0.10, 0.10.1, 0.11, 0.11.1, 0.12, 0.12.1, 0.13, 0.13.1, 0.13.2, 0.14, 0.14.1, 0.15, 0.15.1, 0.16, 0.17, 0.17.1, 0.18, 0.19.0, 0.19.1, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.24.1, 0.25.0, 0.26.0, 0.27.0, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.32.0.post1, 0.33.0, 0.34.0, 0.35.0, 0.35.0.post1, 0.35.1, 0.36.0, 1.0a1, 1.0a2, 1.0a3, 1.0a4, 1.0a6, 1.0a7, 1.0a8, 1.0a9, 1.0a10, 1.0a11, 1.0a12, 1.0b1, 1.0b2, 1.0b3, 1.0b4, 1.0rc1, 1.0rc2, 1.0rc3, 1.0rc4, 1.0rc5, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.3.post1, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.8, 1.0.9, 1.0.10, 1.0.11, 1.0.12, 1.0.13, 1.0.14, 1.0.15, 1.0.16, 1.0.17, 1.0.18, 1.0.20, 1.0.21, 1.0.22, 1.0.23, 1.0.24, 1.0.25, 1.0.26, 1.0.27, 1.0.28, 1.1b1, 1.1b2, 1.1b3, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.2a1, 1.2b1, 1.2b2, 1.2b3, 1.2b4, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.3a1, 1.3a2)
#11 13.46 ERROR: No matching distribution found for rasterio==1.2.9
------
executor failed running [/bin/sh -c pip install         --no-index         --no-cache-dir         --disable-pip-version-check         --find-links=/root/wheels         uwsgi regex &&     pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt &&         rm -rf /root/.cache/pip/* &&         rm root/wheels/* &&         rm /app/requirements.txt]: exit code: 1
make: *** [build] Error 1

Any idea of what's going on? How could I help you help me fix this? ;-)

Thanks!

@ajnisbet
Copy link
Owner

Hmm, this is strange: it's trying to build rasterio from source, instead of just downloading the wheel. This will fail as the docker image doesn't have the dependencies (like gdal) to build rasterio.

There is a wheel for rasterio 1.2.9 so that should get preferred. When I run the build from scratch, I get

...
Collecting pytest-cov==3.0.0
  Downloading pytest_cov-3.0.0-py3-none-any.whl (20 kB)
Collecting pyyaml==5.4.1
  Downloading PyYAML-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (636 kB)
Collecting rasterio==1.2.9
  Downloading rasterio-1.2.9-cp37-cp37m-manylinux1_x86_64.whl (19.3 MB)
Requirement already satisfied: regex==2021.9.30 in /usr/local/lib/python3.7/site-packages (from -r /app/requirements.txt (line 101)) (2021.9.30)
Collecting requests==2.26.0
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting six==1.16.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting snuggs==1.4.7
...

A workaround would be to download the wheel first then install it manually.

But first could you try building with docker build --tag opentopodata:1.7.0 --file docker/Dockerfile --no-cache .

This will force rebuilding of all layers, perhaps some bad data got cached.

@HugoHeneault
Copy link
Contributor Author

Thanks for your extra-fast reply!

But I'm still having the same issue. This might be due to macOS m1 architecture?

I tried adding the wheel by adding pip wheel --wheel-dir=/root/wheels rasterio==1.2.9 but it is also failing.

@HugoHeneault
Copy link
Contributor Author

Here's the error I have when adding wheel with pip wheel:

make build
docker build --tag opentopodata:`cat VERSION` --file docker/Dockerfile .
[+] Building 8.7s (9/15)
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                                                                      0.0s
 => => transferring dockerfile: 37B                                                                                                                                                                                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                                         0.0s
 => => transferring context: 34B                                                                                                                                                                                                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/python:3.7.12-slim-buster                                                                                                                                                                                                                                                              1.2s
 => [builder 1/3] FROM docker.io/library/python:3.7.12-slim-buster@sha256:866b51113519f9022e2f432869d3e7d97b090b9853f4765dbf46394ac28d5a03                                                                                                                                                                                                0.0s
 => [internal] load build context                                                                                                                                                                                                                                                                                                         0.0s
 => => transferring context: 17.73kB                                                                                                                                                                                                                                                                                                      0.0s
 => CACHED [builder 2/3] RUN set -e &&     apt-get update &&     apt-get install -y --no-install-recommends         gcc         python3.7-dev                                                                                                                                                                                             0.0s
 => CANCELED [builder 3/3] RUN pip wheel --wheel-dir=/root/wheels uwsgi==2.0.19.1 &&     pip wheel --wheel-dir=/root/wheels regex==2021.9.30 &&     pip wheel --wheel-dir=/root/wheels rasterio==1.2.9                                                                                                                                    7.4s
 => CACHED [stage-1 2/9] RUN set -e &&     apt-get update &&     apt-get install -y --no-install-recommends         nginx         memcached         supervisor &&     rm -rf /var/lib/apt/lists/*                                                                                                                                         0.0s
 => ERROR [stage-1 3/9] RUN pip wheel --wheel-dir=/root/wheels rasterio==1.2.9                                                                                                                                                                                                                                                            7.4s
------
 > [stage-1 3/9] RUN pip wheel --wheel-dir=/root/wheels rasterio==1.2.9:
#6 1.252 Collecting rasterio==1.2.9
#6 1.357   Downloading rasterio-1.2.9.tar.gz (2.3 MB)
#6 2.014   Installing build dependencies: started
#6 6.754   Installing build dependencies: finished with status 'done'
#6 6.762   Getting requirements to build wheel: started
#6 7.195   Getting requirements to build wheel: finished with status 'error'
#6 7.195   ERROR: Command errored out with exit status 1:
#6 7.195    command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_m9xu3wj
#6 7.195        cwd: /tmp/pip-wheel-2xm3n6f8/rasterio_94950f13d5dc46e4ba3635ec82daf4b5
#6 7.195   Complete output (2 lines):
#6 7.195   WARNING:root:Failed to get options via gdal-config: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'
#6 7.195   ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable.
#6 7.195   ----------------------------------------
#6 7.196 WARNING: Discarding https://files.pythonhosted.org/packages/1f/70/4e7a789f4988955e4c381de80923e184f912683bbe6fc4a3a00c91efdf59/rasterio-1.2.9.tar.gz#sha256=012a4964d8db365be4fae0af9cbeba00e683e5904d5031e8ba42ccb6040cc887 (from https://pypi.org/simple/rasterio/) (requires-python:>=3.6). Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_m9xu3wj Check the logs for full command output.
#6 7.197 ERROR: Could not find a version that satisfies the requirement rasterio==1.2.9 (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.5.1, 0.6, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.9, 0.10, 0.10.1, 0.11, 0.11.1, 0.12, 0.12.1, 0.13, 0.13.1, 0.13.2, 0.14, 0.14.1, 0.15, 0.15.1, 0.16, 0.17, 0.17.1, 0.18, 0.19.0, 0.19.1, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.24.1, 0.25.0, 0.26.0, 0.27.0, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.32.0.post1, 0.33.0, 0.34.0, 0.35.0, 0.35.0.post1, 0.35.1, 0.36.0, 1.0a1, 1.0a2, 1.0a3, 1.0a4, 1.0a6, 1.0a7, 1.0a8, 1.0a9, 1.0a10, 1.0a11, 1.0a12, 1.0b1, 1.0b2, 1.0b3, 1.0b4, 1.0rc1, 1.0rc2, 1.0rc3, 1.0rc4, 1.0rc5, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.3.post1, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.8, 1.0.9, 1.0.10, 1.0.11, 1.0.12, 1.0.13, 1.0.14, 1.0.15, 1.0.16, 1.0.17, 1.0.18, 1.0.20, 1.0.21, 1.0.22, 1.0.23, 1.0.24, 1.0.25, 1.0.26, 1.0.27, 1.0.28, 1.1b1, 1.1b2, 1.1b3, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.2a1, 1.2b1, 1.2b2, 1.2b3, 1.2b4, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.3a1, 1.3a2)
#6 7.198 ERROR: No matching distribution found for rasterio==1.2.9
#6 7.204 WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
#6 7.204 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
------
executor failed running [/bin/sh -c pip wheel --wheel-dir=/root/wheels rasterio==1.2.9]: exit code: 1
make: *** [build] Error 1

@HugoHeneault
Copy link
Contributor Author

I also tried running make build in rosetta mode, still the same issue.

@ajnisbet ajnisbet changed the title WARNING:root:Failed to get options via gdal-config WARNING:root:Failed to get options via gdal-config (M1 mac) Nov 11, 2021
@HugoHeneault
Copy link
Contributor Author

I'm now looking for a way to build gdal which would allow us to build rasterio from sources...

@ajnisbet
Copy link
Owner

Ah, I missed that you're on an M1 mac, I imagine that is the issue.

I don't have a macbook to test on, and I don't quite understand how M1, docker and rosetta all work together. I'm surprised rosetta didn't work.

I know rasterio doesn't have wheels for M1s (rasterio/rasterio-wheels#69). Even on linux installing gdal and proj is a headache, which is why I went the rasterio route.

This Dockerfile works for me on linux (it builds and the tests pass): on my linux machine it pulls the rasterio wheel of course, but if the base image works on m1 that should have the dependencies needed to build rasterio

FROM osgeo/gdal:ubuntu-full-3.4.0
RUN set -e && \
    apt-get update && \
    apt-get install -y --no-install-recommends \
        nginx \
        memcached \
        python3-pip \
        gcc \
        g++ \
        supervisor \
        libmemcached-dev \
        python3.8-dev && \
    rm -rf /var/lib/apt/lists/*

COPY requirements.txt /app/requirements.txt
RUN pip install \
        --no-cache-dir \
        --disable-pip-version-check \
        uwsgi regex && \
    pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt && \
        rm -rf /root/.cache/pip/* && \
        rm /app/requirements.txt

WORKDIR /app
COPY . /app/

RUN echo > /etc/nginx/sites-available/default && \
    cp /app/docker/nginx.conf /etc/nginx/conf.d/nginx.conf && \
    cp /app/docker/supervisord.conf /etc/supervisor/conf.d/supervisord.conf

CMD ["sh", "/app/docker/run.sh"]
EXPOSE 5000

ENV CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
ENV GDAL_DISABLE_READDIR_ON_OPEN=TRUE
ENV GDAL_NUM_THREADS=ALL_CPUS
ENV GDAL_CACHEMAX=512

Otherwise you could try running without docker using homebrew and/or conda.

@HugoHeneault
Copy link
Contributor Author

This Dockerfile script had ended without error. Awesome! Thanks a lot for your help! 🎉

I'll report back of my m1 experience.

I just read this article - which might need a bit of updates. Hope I won't get into nightmares on other projects. 🤷‍♂️

@ajnisbet
Copy link
Owner

Ok fantastic! The one I posted above? Let me know if you have any issues, otherwise I can add that file as M1.Dockerfile for others.

@HugoHeneault
Copy link
Contributor Author

I had to change one more thing to get it working. I think python's path is different so I simplified it:

I replaced command=/usr/local/bin/python /app/docker/warm_cache.py by command=python /app/docker/warm_cache.py:

[program:warm_cache]
user=www-data
command=python /app/docker/warm_cache.py
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
autorestart=false

Server is now running fine and I get elevation data 👏

Do you want me to add a note on README for M1 owners & do the supervisor conf change?

Thanks again!

@HugoHeneault
Copy link
Contributor Author

I can also confirm that using your Dockerfile posted above works without Rosetta. 🎉

@ajnisbet
Copy link
Owner

@HugoHeneault could you make PR for the supervisor conf change? I can confirm that works with the standard Dockerfile too.

I'll update the docs.

Thank you!

@HugoHeneault
Copy link
Contributor Author

@ajnisbet Yup just did it in #56 :-)

Have a great day!

@ajnisbet
Copy link
Owner

Thanks for your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants