docker: cleanup alpine Dockerfile and drop Dockerfile_alpine_latest

[neteler]

• drop Dockerfile_alpine_latest as being unsed
• cleanup Dockerfile

(not to be backported)

[mmacata]

Thanks for simplification :)

[mmacata]

I just realized that these files were added in a previous stage, so removing them has no effect in image size, just more organized content of image. So I would still vote for removing them here.

[neteler]

I checked the docker image size, with respect to the biggest included packages:

for i in $(apk info  | sort) ; do apk info -s $i ; done > log

# packages in KiB size, sort by size:
cat log | tr '\n' ' ' | sed 's+B  +B|+g' | tr -s '|' '\n' | grep KiB | sort -n -t' ' -k 4 | tail -n 50
fftw-long-double-libs-3.3.10-r1 installed size: 940 KiB
libxcb-1.15-r0 installed size: 972 KiB
sqlite-libs-3.40.1-r0 installed size: 1008 KiB
cairo-1.17.6-r3 installed size: 1020 KiB
subversion-1.14.2-r4 installed size: 1040 KiB
libkml-1.3.0-r4 installed size: 1064 KiB
libjpeg-turbo-2.1.4-r0 installed size: 1136 KiB
p11-kit-0.24.1-r1 installed size: 1152 KiB
zstd-1.5.2-r9 installed size: 1152 KiB
libxml2-2.10.3-r1 installed size: 1188 KiB
sqlite-3.40.1-r0 installed size: 1196 KiB
netcdf-4.8.1-r4 installed size: 1328 KiB
bash-5.2.15-r0 installed size: 1400 KiB
gdal-dev-3.5.3-r2 installed size: 1404 KiB
cfitsio-4.2.0-r0 installed size: 1452 KiB
bison-3.8.2-r0 installed size: 1468 KiB
libpq-dev-15.2-r0 installed size: 1468 KiB
gettext-0.21.1-r1 installed size: 1660 KiB
libunistring-1.1-r0 installed size: 1712 KiB
pdal-dev-2.4.3-r0 installed size: 1884 KiB
gnutls-3.7.8-r3 installed size: 1936 KiB
krb5-libs-1.20.1-r0 installed size: 1968 KiB
fftw-double-libs-3.3.10-r1 installed size: 2072 KiB
isl25-0.25-r0 installed size: 2088 KiB
fftw-single-libs-3.3.10-r1 installed size: 2112 KiB
librasterlite2-1.1.0b1-r3 installed size: 2180 KiB
openssl-dev-3.0.8-r3 installed size: 2228 KiB
libstdc++-12.2.1_git20220924-r4 installed size: 2356 KiB
qhull-2020.2-r2 installed size: 2660 KiB
icu-data-en-72.1-r1 installed size: 2676 KiB
mpfr4-4.1.0-r0 installed size: 2684 KiB
py3-pillow-9.3.0-r0 installed size: 2732 KiB
postgresql15-client-15.2-r0 installed size: 2780 KiB
libgfortran-12.2.1_git20220924-r4 installed size: 2952 KiB
xerces-c-3.2.4-r1 installed size: 3040 KiB
tzdata-2023c-r0 installed size: 3068 KiB
poppler-22.11.0-r0 installed size: 3156 KiB
libx11-1.8.4-r0 installed size: 3172 KiB
geos-3.11.1-r0 installed size: 3336 KiB
hdf5-1.12.2-r1 installed size: 3540 KiB
subversion-libs-1.14.2-r4 installed size: 3628 KiB
geos-dev-3.11.1-r0 installed size: 3956 KiB
libcrypto3-3.0.8-r3 installed size: 4108 KiB
openexr-3.1.5-r3 installed size: 4284 KiB
icu-libs-72.1-r1 installed size: 4552 KiB
py3-gdal-3.5.3-r2 installed size: 4940 KiB
pdal-2.4.3-r0 installed size: 6408 KiB
aom-libs-3.5.0-r0 installed size: 7632 KiB
libmagic-5.43-r0 installed size: 7920 KiB
binutils-2.39-r2 installed size: 8740 KiB


# packages in MiB size:
cat log | tr '\n' ' ' | sed 's+B  +B|+g' | tr -s '|' '\n' | grep MiB
g++-12.2.1_git20220924-r4 installed size: 35 MiB
gcc-12.2.1_git20220924-r4 installed size: 118 MiB
gdal-3.5.3-r2 installed size: 22 MiB
libspatialite-5.0.1-r5 installed size: 13 MiB
libstdc++-dev-12.2.1_git20220924-r4 installed size: 48 MiB
musl-dev-1.2.3-r4 installed size: 9 MiB
openblas-0.3.21-r0 installed size: 28 MiB
postgresql15-15.2-r0 installed size: 14 MiB
proj-9.1.0-r0 installed size: 10 MiB
py3-numpy-1.23.4-r0 installed size: 30 MiB
python3-3.10.10-r0 installed size: 47 MiB
x265-libs-3.5-r3 installed size: 18 MiB

Candidates to be removed:

• gettext
• pdal-dev
• x265-libs: video coded, but it is a dependency for GDAL in alpine

Done in 52b4f17.

Current size: 684 MB.

I don't see much potential to further shrink the docker image size but suggestions are welcome.

[tmszi]

I tried grouping some layers in a Docker file, something like this:

FROM common as grass

# GRASS GIS specific
# allow work with MAPSETs that are not owned by current user
ENV GRASSBIN="/usr/local/bin/grass" \
    GRASS_SKIP_MAPSET_OWNER_CHECK=1 \
    SHELL="/bin/bash" \
    # https://proj.org/usage/environmentvars.html#envvar-PROJ_NETWORK
    PROJ_NETWORK=ON \
    GRASSBIN=grass \
    LC_ALL="en_US.UTF-8"

# Copy GRASS GIS from build image
COPY --from=build /usr/local/bin/grass /usr/local/bin/grass
COPY --from=build /usr/local/grass* /usr/local/grass/
# run simple LAZ test
COPY docker/testdata/simple.laz /tmp/
# COPY docker/testdata/test_grass_session.py docker/alpine/grass_tests.sh /scripts/
COPY docker/testdata/test_grass_session.py /scripts/

# install external Python API
RUN pip3 install --upgrade pip six grass-session --ignore-installed six; \
    ln -sf /usr/local/grass $(grass --config path); \
    # run some tests and cleanup
    # $SHELL /scripts/grass_tests.sh \
    # && rm -f /scripts/grass_tests.sh /tmp/simple.laz /scripts/test_grass_session.py; \
    # delete unused packages
    apk del --no-cache gettext pdal-dev; \
    # show installed version
    grass --tmp-location XY --exec g.version -rge \
    && pdal --version \
    && python3 --version

# Data workdir
WORKDIR /grassdb
VOLUME /grassdb

CMD ["$GRASSBIN", "--version"]

It reduced the image size to 669MB. It's not too much.

grass-alpine                       latest                                  1e2bd7aed584   7 minutes ago    669MB

Note:

This file docker/alpine/grass_tests.sh doesn't exist in the source code (build image fail) , I added comment inside Dockerfile.

# COPY docker/testdata/test_grass_session.py docker/alpine/grass_tests.sh /scripts/
...
    # run some tests and cleanup
    # /bin/bash /scripts/grass_tests.sh \
    # && rm -f /scripts/grass_tests.sh /tmp/simple.laz /scripts/test_grass_session.py; \

[tmszi]

Remove dev packages from GRASS_RUN_PACKAGES ENV and append them (missing is gdal-dev package) into GRASS_BUILD_PACKAGES ENV. I think that dev packages (gdal-dev, geos-dev, libpq-dev, pdal-dev) is no needed for GRASS GIS runtime or Add-Ons compilation.

ENV GRASS_RUN_PACKAGES="\
      attr \
      build-base \
      bash \
      bison \
      bzip2 \
      cairo \
      curl \
      fftw \
      flex \
      freetype \
      g++ \
      gcc \
      gdal \
      gdal-tools \
      gettext \
      geos \
      gnutls \
      jsoncpp \
      laszip \
      libbz2 \
      libgeotiff \
      libjpeg-turbo \
      libpng \
      libunwind \
      make \
      musl \
      musl-utils \
      ncurses \
      openjpeg \
      openblas \
      py3-numpy \
      py3-pillow \
      py3-six \
      pdal \
      postgresql \
      proj-util \
      sqlite \
      sqlite-libs \
      subversion \
      tiff \
      zstd \
      zstd-libs \
    "

ENV GRASS_BUILD_PACKAGES="\
      build-base \
      bzip2-dev \
      cairo-dev \
      fftw-dev \
      freetype-dev \
      geos-dev \
      git \
      gnutls-dev \
      libc6-compat \
      libjpeg-turbo-dev \
      libpng-dev \
      libpq-dev \
      openjpeg-dev \
      openblas-dev \
      pdal \
      pdal-dev \
      proj-dev \
      python3-dev \
      py3-numpy-dev \
      sqlite-dev \
      tar \
      tiff-dev \
      unzip \
      vim \
      wget \
      zip \
      zstd-dev \
      gdal-dev \
    "

Group layers:

RUN echo "Install Python";\
    apk add --no-cache $PYTHONBIN && \
    $PYTHONBIN -m ensurepip && \
    rm -r /usr/lib/python*/ensurepip && \
    pip$PYTHON_VERSION install --upgrade pip setuptools && \
    if [ ! -e /usr/bin/pip ]; then ln -s pip$PYTHON_VERSION /usr/bin/pip ; fi && \
    if [ ! -e /usr/bin/python ]; then ln -sf /usr/bin/$PYTHONBIN /usr/bin/python; fi && \
    rm -r /root/.cache; \
    # Add the packages
    echo "Install main packages";\
    apk update; \
    apk add --no-cache $GRASS_RUN_PACKAGES

It reduced the image size from 669MB to 659MB.

grass-alpine                       latest                                  f2f54d5d00dc   About an hour ago   659MB

[neteler]

I think that dev packages (gdal-dev, geos-dev, libpq-dev, pdal-dev) is no needed for GRASS GIS runtime or Add-Ons compilation.

@tmszi: This surprises me - it was exactly the reason to add them to run-time but I am no expert here.
Please consider to edit the PR directly:

So, using:

gh pr checkout 2912

Updates in this PR would trigger the tests which I recently added in docker/alpine/grass_tests.sh (showing the need of the -dev packages).

[neteler]

This file docker/alpine/grass_tests.sh doesn't exist in the source code (build image fail) ,

Oh sorry, added in e1f2fd1.
The new tests are essential.

[neteler]

Added git in 4bca1e5 as being needed for g.extension, see e.g.:
https://github.com/OSGeo/grass/actions/runs/4718450552/jobs/8368064252#step:8:9512

[tmszi]

Updates in this PR would trigger the tests which I recently added in docker/alpine/grass_tests.sh (showing the need of the -dev packages).

Yes your are right. These dev packages is required for v.centerpoint addon compilation.

Fetching <v.centerpoint> from GRASS GIS Addons repository (be patient)...
Compiling...
In file included from /usr/local/grass83/include/grass/vect/digit.h:3,
                 from /usr/local/grass83/include/grass/vector.h:4,
                 from areas.c:5:
/usr/local/grass83/include/grass/vect/dig_structs.h:27:10: fatal error: ogr_api.h: No such file or directory
   27 | #include <ogr_api.h>
      |          ^~~~~~~~~~~
compilation terminated.
make: *** [/usr/local/grass83/include/Make/Compile.make:32: OBJ.x86_64-pc-linux-musl/areas.o] Error 1
ERROR: Compilation failed, sorry. Please check above error messages.

[tmszi]

I replaced postgresql package for GRASS GIS runtime with postgresql15-client wich is sufficient for connection to PostgreSQL DB. We don't need launch PostgreSQL DB server under Docker container right?

[neteler]

We don't need launch PostgreSQL DB server under Docker container right?

Confirmed, we don' need a PG server here.