Docker image size

[colinleach]

Background

The repo is here. The Dockerfile is here. In the last 3 years this has had few changes: dependabot merges and a sync of org-wide files (code of conduct, etc.)

There was a push in 2023 to reduce image sizes. At that time, Julia was a large but not crazy 765 MB. For comparison, the Python test runner was 145 MB then and is 90.6 MB now. Honestly, I never expect Julia to match that (we're like Python + NumPy + Pandas).

Building the current Dockerfile locally now gives a image of a startling 1.34 GB, which is costing Exercism money, paid regularly to AWS.

Shrinking it would be good! Obviously, it is vital to make sure any changes don't break solutions, and don't slow runtime performance.

Base images

The library image julia:1.11.5 is based on debian:bookworm-slim.

To get I feel for the various official Julia 1.11.5 images, I built a few locally, plus debian:bookworm-slim. The default julia:alpine failed repeatedly.

Image	Size
current test runner	1.34 GB
julia:1.11.5	1.11 GB
julia:1.11.5-bullseye	1.11 GB
julia:alpine3.22	500 error
julia:alpine3.21	1.04 GB
debian:bookworm-slim	74.8 MB

So all the Julia images are >1 GB, nearly 15x the size of the base Debian image. Even alpine isn't much smaller (and we don't know if it will even work in the test runner).

Maybe we could slightly shrink the stuff we add in julia-test-runner, but it won't solve the size problem.

Julia installation

Downloading julia-1.11.5-linux-x86_64.tar.gz via curl, as the official julia:1.11.5 Dockerfile does, gives an installation which is 285.8 MB compressed and 988 MB uncompressed. This points to the main issue!

There has been discussion about the ever-growing installation sizes, for example here.

To quote:

It seems that during the build process stdlibs are compiled with the 2 differents configurations:
1/ -g2 -O3
2/ -g2 -O3 --check-bounds=yes

and:

there’s definitely a case for a light version…And doubling the size for testing packages with checkounds=yes, seems like something that could be served by a dev version.
But with all of these things, someone needs to care and put time into it.

Exercism has a stated preference for official images, and I don't feel competent to get deep into messing with this.

Unfortunately, it seems likely that Julia 1.12 will be even bigger when released in the next few months.

[colinleach]

I did a bit more experimentation, to see if changing our Dockerfile, Project.toml and Manifest.toml would make any difference. Not much!

The Dockerfile installs clang and git. Matching best practices, I edited the RUN command to do apt-get autoremove and rm -rf /var/lib/apt/lists/* for post-install cleanup. It made zero difference.

I wasn't sure if some of the packages in the TOML files might be bloating things, so I removed deps.LinearAlgebra and deps.OpenBLAS_jll. This reduced the image size by a paltry 0.995 MB (0.0744%), with a risk of breaking existing solutions. I don't want to do this.

If installed packages are such a rounding error, relative to the base language, I'd like to investigate adding TimeZones.jl. This would enable some existing Exercism practice exercises, and Erik seems keen to add more (perhaps showing off how good his C# track is for this sort of thing!).

Incidentally, the usual docker image ls is no use in all this: it rounds all the Julia images to 1.34 GB. Get the Size field from docker inspect <image-name> for an exact figure.

[colinleach]

I'd like to investigate adding TimeZones.jl

Also Combinatorics.jl, which some Exercism practice exercises would benefit from.

[BethanyG]

Are there any Alpine docker implementations? When Python bit the bullet and went to Alpine, it saved a bunch of space. We were only using ubuntu because originally, Alpine caused speed issues with CPython. Once that was fixed, we happily switched.

I am seeing Alpine images here might be worth an attempt.

[colinleach]

See top of this post. The killer is Julia itself, the OS makes minimal difference.

Also, I kept getting 500 errors when I tried to build the Alpine image.

[BethanyG]

Ok. I can't experiment today, but I am curious as to why this is so bloated - so I might take a crack at some point. But also? This doesn't have to be teeny-tiny. Do what you can ... but don't think you HAVE to do this. Much more important to get the syllabus and other exercises done. your test runner is not the biggest out there. I think that honor belongs to either Haskell or Rust.

[colinleach]

I think you can tell I haven't made this my top priority.

We seem pretty screwed, if the Julia core team don't prioritize installation size. I guess getting teraflop performance on big clusters is more important for them (not entirely without justification, given the use cases for this language).

[colinleach]

Also Combinatorics.jl, which some Exercism practice exercises would benefit from.

I hesitate to mention Zebra Puzzle...