Self-contained docker build with ARM64 publishing #3962

keynmol · 2025-11-18T14:15:03Z

At the moment, the scala-cli docker image is built using a combination of mill tasks, scripts, and dockerfiles that require externally built binary. What's more, it's only built for x86.

With ARM64 runners on GHA being GA, I propose a simplification - a self-contained multi-stage dockerfile, with a custom Github Workflow that merges the images into a single manifest, meaning that docker will pull the correct image no matter the target platform.

Manifest merging and pushing is a complicated step, and Docker have been amending their docs with the example, which is what this workflow is based on.

I've been using this setup in multiple apps, e.g. Mimalyzer.

There are some aspects of the build I don't really understand – static, mostly static images, customisation for Linux x86, etc.

This proposal attempts to do the simplest possible thing, so that building a docker image is just docker build . -t VirtusLab/scala-cli.

Currently, it only publishes to ghcr.io to test out the workflows without disturbing the main image on docker hub. But additional publishing steps are easily added, of course

keynmol · 2025-11-19T09:40:23Z

Dockerfile

+RUN ./mill -i copyTo --task 'cli[].base-image.nativeImage' --dest "./docker-out/scala-cli" 1>&2
+
+FROM debian:stable-slim
+RUN apt update && apt install build-essential libz-dev clang procps -y


Note that I'm not sure if installing those dependencies in a base image is a good idea, or better left to downstream images. Even though I see the appeal of making native and native-image stuff work out of the box..

Main concern is the size

I mean, we definitely want Scala Native to build nicely with it, so clang is a must.
I'd like to keep them, I think.

Gedochao

@tgodzik @zielinsky can you take a look as well? I think it'd be good to have another pair of eyes on this (or two)

Gedochao · 2025-11-19T12:02:53Z

.github/workflows/publish-docker.yml

+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+# Configures this workflow to run every time a change is pushed to the branch called `release`.


...you mean like, v* tags and main, right?

.github/workflows/publish-docker.yml

Gedochao · 2025-11-19T12:07:48Z

Dockerfile

+RUN ./mill -i copyTo --task 'cli[].base-image.nativeImage' --dest "./docker-out/scala-cli" 1>&2
+
+FROM debian:stable-slim
+RUN apt update && apt install build-essential libz-dev clang procps -y


I mean, we definitely want Scala Native to build nicely with it, so clang is a must.
I'd like to keep them, I think.

Gedochao · 2025-11-19T12:19:32Z

There are some aspects of the build I don't really understand – static, mostly static images, customisation for Linux x86, etc.

Static and mostly static images are in the understanding of GraalVM, as per this doc: https://www.graalvm.org/21.3/reference-manual/native-image/StaticImages/index.html
The way they're built is based on the mill-native-image plugin: https://github.com/alexarchambault/mill-native-image

(...) customisation for Linux x86 (...)

What particular customisation for Linux x86 did you mean?

(...) etc.

Shoot away, I'll try to answer, or at least direct you in the direction of an answer.

What's there in this area was initially coded by @alexarchambault (who's also the author of mill-native-image). While I've tinkered with this here and there, most of it has been lying untouched for a long, long time, and I'm not all that familiar with it either.

tgodzik

I don't know much about this, I only built basic docker images. Overall look ok as long as it work 😅

zielinsky

A few comments from me 😅

.github/workflows/publish-docker.yml

zielinsky · 2025-11-20T22:58:26Z

.github/workflows/publish-docker.yml

+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}


Suggested change

username: ${{ github.actor }}

password: ${{ secrets.GITHUB_TOKEN }}

username: ${{ env.REGISTRY_LOGIN }}

password: ${{ env.REGISTRY_PASSWORD }}

@keynmol I reverted to unresolved, you changed it in only one place

.github/workflows/publish-docker.yml

keynmol · 2025-11-21T09:31:26Z

@Gedochao

What particular customisation for Linux x86 did you mean?

This line threw me off

https://github.com/keynmol/scala-cli/blob/93079619648e0b475f874ee2676387b488d7a637/project/settings/package.mill.scala#L406

Gedochao · 2025-11-21T09:58:14Z

Just to make sure.

We currently have the following images, which are updated with each Scala CLI release:

@keynmol your intention is to create a third one, yes?
smth like virtuslab/scala-cli-platform-native?

Gedochao · 2025-11-21T09:58:40Z

Also, how can we safely test this?

keynmol · 2025-11-21T10:04:23Z

Here's my plan:

Merge this workflow as is, make sure it publishes a working ghcr.io/virtuslab/scala-cli image (similar to existing DockerHub virtuslab/scala-cli)
Modify the workflow in a separate PR to build the slim image in the same way (similar to existing DockerHub virtuslab/scala-cli-slim)
Add DockerHub publishing steps to both images
Remove old docker publishing logic

Up to step 3, the existing docker images on DockerHub are safe and won't be touched, as we only publish to ghcr.io, using it as our testing area.

Steps 1-3 can be done in separate PRs as they won't affect existing dockerhub publishing steps.

In the end, we should have virtuslab/scala-cli and virtuslab/scala-cli-slim that have a manifest merging two platforms, so users automatically get the right image (see screenshot)

Gedochao · 2025-11-21T10:58:05Z

Okay... sounds reasonable.
Let's get the CI green and do it one step at a time.

Gedochao · 2025-11-24T06:21:29Z

The CI is green.
@keynmol the workflow is still on Ubuntu 22.04, while the other workflows already rely on 24.04 (I left comments earlier).
Otherwise, we seem to be good to merge here (might want to rebase on main while you're at it)

keynmol · 2025-11-24T07:46:34Z

~~I have responded to the Ubuntu version comment: #3962 (comment)~~

Actually scratch that, in this instance it won't have any effect, I will update

Gedochao

LGTM
@keynmol ready to merge?

Gedochao · 2025-11-25T08:47:12Z

@zielinsky I will wait for your okay before clicking merge.

keynmol · 2025-11-25T08:47:36Z

I think so, no doubt it will require more iterations when the work flow actually runs :)

zielinsky

One question from me before approval.

zielinsky · 2025-11-25T11:34:46Z

.github/workflows/publish-docker.yml

+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository }}


Doesn't the name of the image have to be in lowercase letters? Currently github.repository will return "VirtusLab/scala-cli" (docs).

keynmol added 3 commits November 18, 2025 13:40

wip

4d9fdec

WIP

b9b37b3

Docker publishing workflow

5a2da94

keynmol commented Nov 19, 2025

View reviewed changes

Gedochao reviewed Nov 19, 2025

View reviewed changes

Gedochao requested review from tgodzik and zielinsky November 19, 2025 12:08

tgodzik reviewed Nov 20, 2025

View reviewed changes

zielinsky reviewed Nov 20, 2025

View reviewed changes

Update versions

9307961

use env registry creds

2d07268

keynmol added 2 commits November 24, 2025 10:16

use ubuntu 24

2b30965

Merge branch 'main' into self-contained-docker-build

ff819ec

Gedochao approved these changes Nov 25, 2025

View reviewed changes

zielinsky reviewed Nov 25, 2025

View reviewed changes

		username: ${{ github.actor }}
		password: ${{ secrets.GITHUB_TOKEN }}

Self-contained docker build with ARM64 publishing #3962

Are you sure you want to change the base?

Self-contained docker build with ARM64 publishing #3962

Conversation

keynmol commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

keynmol Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Gedochao Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Gedochao left a comment

Choose a reason for hiding this comment

Uh oh!

Gedochao Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Gedochao Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Gedochao commented Nov 19, 2025

Uh oh!

tgodzik left a comment

Choose a reason for hiding this comment

Uh oh!

zielinsky left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zielinsky Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

zielinsky Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

keynmol commented Nov 21, 2025

Uh oh!

Gedochao commented Nov 21, 2025

Uh oh!

Gedochao commented Nov 21, 2025

Uh oh!

keynmol commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Gedochao commented Nov 21, 2025

Uh oh!

Gedochao commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

keynmol commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Gedochao left a comment

Choose a reason for hiding this comment

Uh oh!

Gedochao commented Nov 25, 2025

Uh oh!

keynmol commented Nov 25, 2025

Uh oh!

zielinsky left a comment

Choose a reason for hiding this comment

Uh oh!

zielinsky Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

keynmol commented Nov 18, 2025 •

edited

Loading

keynmol commented Nov 21, 2025 •

edited

Loading

Gedochao commented Nov 24, 2025 •

edited

Loading

keynmol commented Nov 24, 2025 •

edited

Loading