Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing build dependencies libpython3-dev and libgdal-dev #587

Merged

Conversation

totycro
Copy link
Contributor

@totycro totycro commented Dec 7, 2020

I'm not sure why these are required now, apparently some update changed some dependencies. At first glance it doesn't seem directly related to #584, but the temporal proximity is suspicious.

These were the error message I got while building:

copying src/gevent/tests/tests_that_dont_do_leakchecks.txt -> build/lib.linux-x86_64-3.9/gevent/tests
  copying src/gevent/tests/server.key -> build/lib.linux-x86_64-3.9/gevent/tests
  copying src/gevent/tests/test_server.key -> build/lib.linux-x86_64-3.9/gevent/tests
  creating build/lib.linux-x86_64-3.9/gevent/testing/coveragesite
  copying src/gevent/testing/coveragesite/sitecustomize.py -> build/lib.linux-x86_64-3.9/gevent/testing/coveragesite
  running build_ext
  generating cffi module 'build/temp.linux-x86_64-3.9/gevent.libuv._corecffi.c'
  creating build/temp.linux-x86_64-3.9
  Running '(cd  "/tmp/pip-install-doios8nv/gevent_5935bf4ce4024a1d89216659b4333548/deps/libev"  && sh ./configure -C > configure-output.txt )' in /tmp/pip-install-doios8nv/gevent_5935bf4ce4024a1d89216659b4333548
  ./configure: line 6475: /usr/bin/file: No such file or directory
  generating cffi module 'build/temp.linux-x86_64-3.9/gevent.libev._corecffi.c'
  Not configuring libev, 'config.h' already exists
  Not configuring libev, 'config.h' already exists
  building 'gevent.libev.corecext' extension
  creating build/temp.linux-x86_64-3.9/src
  creating build/temp.linux-x86_64-3.9/src/gevent
  creating build/temp.linux-x86_64-3.9/src/gevent/libev
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DLIBEV_EMBED=1 -DEV_COMMON= -DEV_CLEANUP_ENABLE=0 -DEV_EMBED_ENABLE=0 -DEV_PERIODIC_ENABLE=0 -Isrc/gevent/libev -I/usr/include/python3.9 -I/tmp/pip-install-doios8nv/gevent_5935bf4ce4024a1d89216659b4333548/deps -I/tmp/pip-install-doios8nv/gevent_5935bf4ce4024a1d89216659b4333548/src/gevent/libev -I/tmp/pip-install-doios8nv/gevent_5935bf4ce4024a1d89216659b4333548/deps/libev -Isrc/gevent -Isrc/gevent/libev -Isrc/gevent/resolver -I. -I/usr/include/python3.9 -c src/gevent/libev/callbacks.c -o build/temp.linux-x86_64-3.9/src/gevent/libev/callbacks.o
  src/gevent/libev/callbacks.c:3:10: fatal error: Python.h: No such file or directory
      3 | #include "Python.h"
        |          ^~~~~~~~~~
  compilation terminated.
  error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for gevent
Collecting rasterio
  Downloading rasterio-1.1.8.tar.gz (2.1 MB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpmvy0iiij
       cwd: /tmp/pip-install-esormpv9/rasterio_fe3dcb767161460ca67807864fd71f76
  Complete output (2 lines):
  WARNING:root:Failed to get options via gdal-config: [Errno 2] No such file or directory: 'gdal-config'
  ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable.

I'm not sure why these are required now, apparently some update changed
some dependencies.
@justb4
Copy link
Member

justb4 commented Dec 7, 2020

Possibly also related to #585. What I observed was that installing GDAL Python bindings pulled in a complete Python 3.9 and when build deps are removed at the end, it removed Python 3.9 with GDAL Python...At least there is a confusing situation. IMHO the root problem is the use of Debian Bullseye Slim as base package, as literally it is a moving target.

So my proposal is to completely revise the Dockerfile. Possibly using Ubuntu 20.4 Slim with UbuntuGIS PPA and do more pinning of versions to have reproducible Docker builds.

@dsusviela
Copy link

First time commenting in this repo, was using pygeoapi with docker with no problems until recently that decided to delete some images, including this one.

I tried the proposed changes in this PR, but had an issue when running the image related to this bug inside the gevent dependency caused by python upgrading to its newer version 3.9. Bumping the gevent version to 20.9.0 and greenlet to 0.4.17 together with the changes here did it for me.

But still, like @justb4 said, you may want to revise the whole thing, as these bumps in versions may break other things.

Bumping the gevent version to 20.9.0 and greenlet to 0.4.17

See geopython#587 (comment)
The libgdal27 doesn't seem to be available any more in the current
debian docker image.
@totycro
Copy link
Contributor Author

totycro commented Dec 14, 2020

Thanks @dsusviela, I added your suggestions here!

It seems that by now, libgdal27 is not available any more in debian, so i had to upgrade to libgdal28. I'm not sure if this breaks anything, but at least the docker image builds again.

Do you think we could merge this PR soon? Currently I'm not really able to deploy any hotfix because of this, and it's also a bit tedious to develop.
I do agree that #585 could improve things, but if it's going to take more time, a stop-gap solution would be highly appreciated.

@alpha-beta-soup alpha-beta-soup merged commit 4678622 into geopython:master Dec 20, 2020
pvgenuchten pushed a commit to pvgenuchten/pygeoapi that referenced this pull request Jan 18, 2021
Bumping the gevent version to 20.9.0 and greenlet to 0.4.17

See geopython#587 (comment)
pvgenuchten pushed a commit to pvgenuchten/pygeoapi that referenced this pull request Jan 18, 2021
…honh-missing

Add missing build dependencies libpython3-dev and libgdal-dev
francbartoli pushed a commit to francbartoli/pygeoapi that referenced this pull request Jul 8, 2021
Bumping the gevent version to 20.9.0 and greenlet to 0.4.17

See geopython#587 (comment)
francbartoli pushed a commit to francbartoli/pygeoapi that referenced this pull request Jul 8, 2021
…honh-missing

Add missing build dependencies libpython3-dev and libgdal-dev
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants