Consider installing build dependencies on the fly

[ottok]

All the Docker images install the build dependencies by first getting the existing build dependencies and then adding some manually defined ones on top. These are created on the fly then the Docker images update. Example: https://github.com/MariaDB/mariadb.org-tools/blob/master/buildbot.mariadb.org/dockerfiles/ubuntu-1804.dockerfile#L16-L26

Instead of maintaining the dependencies manually, consider doing something similar I do my Debian test images:

RUN curl -skO https://salsa.debian.org/mariadb-team/mariadb-10.1/raw/stretch/debian/control
RUN DEBIAN_FRONTEND=noninteractive mk-build-deps -r -i control \
    -t 'apt-get -y -o Debug::pkgProblemResolver=yes --no-install-recommends'

This downloads the debian/control file, and then automatically installs whatever was defined in it. I am not sure if it always makes sense to use this, but I thought I'd share the tip with you in case you want to use it.

[an3l]

Great tip @ottok, thanks. Will notify Vlad about.

[ottok]

Should this perhaps be assigned to @vladbogo?

[fauust]

@ottok, last month, I have worked on building Buildbot containers in GitHub CI. Those are now pushed on https://quay.io/repository/mariadb-foundation/bb-worker?tab=tags and we will start using them (in replacement of the old Dockerfiles that where build directly on BB workers).

In the new Dockerfiles, I install dependencies on the fly as you suggested. So I think that this issue can be closed.

But please let me know if you see anything that could be improved in the Debian/Ubuntu Dockerfiles (above all with respect to the CI matrix and the branch of the debian/control file that is used, see: https://github.com/MariaDB/mariadb.org-tools/blob/master/.github/workflows/bb_containers.yml#L35-L65). For instance, we may have to consider (or need to) create different containers tag to build all the branches if dependencies differs too much between 10.5, 10.6, 10.7 and future branches.

[ottok]

Good, I can see this concept is now used in https://github.com/fauust/mariadb.org-tools/blob/cc495a42fd92430de2ef3b90a4394fb72904daa0/buildbot.mariadb.org/ci_build_images/debian.Dockerfile#L25-L28

It is however followed by excess steps that are probably not needed, but anyway this issue can be closed as the point was just to disseminate the use of mk-build-deps and get rid of manually maintained dependency lists.

[fauust]

In the new Dockerfiles, I install dependencies on the fly as you suggested.

It is indeed ^^ 😄

It is however followed by excess steps that are probably not needed

Happy to resolve this if you can tell me more about my excesses 😄

Thanks for closing anyway.