Add more build dependencies to the Docker build image

[embray]

I noticed that the Docker build-from-clean jobs still build packages like zlib even though it can be avoided since the work done so far on#27330 so long as the correct system headers installed in the build container.

On one hand, while I think it's good to test full builds of Sage-the-distribution with all SPKGs enabled, I think that's more a job for the Buildbot fleet, whereas for Docker images we want to avoid as much overhead as possible.

For now the savings are small (just a few small libraries like zlib and libffi). But as we continue to make progress (e.g. with #27212) the savings will become more significant.

Thoughts?

CC: @saraedum @dimpase @dkrenn

Component: build

Issue created by migration from https://trac.sagemath.org/ticket/27677

[saraedum]

comment:1

build-from-clean is only called very rarely so it's not that important when looking at runtime. I guess that using system provided libraries might make the resulting image a bit smaller.

In any case, I agree that building everything from scratch is what the buildbots are good for. So removing things from build-from-clean appears to be a good idea.

[dkrenn]

Replying to @embray:

I noticed that the Docker build-from-clean jobs still build packages like zlib even though it can be avoided since the work done so far on#27330 so long as the correct system headers installed in the build container.

On one hand, while I think it's good to test full builds of Sage-the-distribution with all SPKGs enabled, I think that's more a job for the Buildbot fleet, whereas for Docker images we want to avoid as much overhead as possible.

Agreed.

For now the savings are small (just a few small libraries like zlib and libffi). But as we continue to make progress (e.g. with #27212) the savings will become more significant.

Yes, small(er) docker images are preferable, in particular, if this is kind of easily possible by using system libraries.

[embray]

comment:3

Not that rarely though, and it will still save significant time in the long term.

I don't think it will make images much smaller, but certainly a little bit.

[embray]

comment:4

As the Sage-8.8 release milestone is pending, we should delete the sage-8.8 milestone for tickets that are not actively being worked on or that still require significant work to move forward. If you feel that this ticket should be included in the next Sage release at the soonest please set its milestone to the next release milestone (sage-8.9).

[embray]

comment:5

If I understand correctly one problem we're having in the docker builds is that when running make build in the sagemath/sagemath-dev image more and more stuff over time has been getting rebuilt.

This seems to be directly related to #27330, as the packages getting rebuilt in the docker image are those that can now be used from the system. So it seems like, possibly, there is a bug (either in the build system, or in the Dockerfiles) causing these packages to be rebuilt (even if they were already built rather than using the system copies). Strange.

I think probably just using the system packages in the first place would avoid this, but that's not entirely clear either.

[saraedum]

comment:6

embray: Last time I checked it was not packages being rebuilt but only documentation (rebuilds though no dependency appears to have changed.)

[mkoeppe]

comment:9

Moving to 9.4, as 9.3 has been released.