Building a proper Dockerfile and natively supporting ARM images

[jmuchovej]

👋 Thanks for building JATOS!

I was looking around for a Dockerfile, only to find it defined in Scala. 😕 (If it were built in a traditional Dockerfile, I could've submitted a PR that adds support for ARMv8.)

https://github.com/JATOS/JATOS/blob/4466ce747ed585f56bfc298b4cb4442622ae8e71/build.sbt

I've never used Scala, but imagine it has templating to support variable changes (to reduce need for manually updating the Dockerfile).

I'm happy to contribute a Dockerfile/whatnot to support this – but wanted to open an issue to check if that's something JATOS is open to. 🙂

[kristian-lange-tw]

I vaguely remember that once I did a release with 'mutli-arch' docker images. Something like described in this blog post: https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/. It's still available on docker hub: docker pull jatos/jatos:multi-arch. It was a one-time thing and I can't really remember how I did it - but if I did it once I can probably do it again (and make it part of every release).
EDIT: Is this 'multi-arch' approach something that suits your needs?

[jmuchovej]

EDIT: Is this 'multi-arch' approach something that suits your needs?

Do you mean this particular multi-arch image? If so, then I don't think so, as it's not up-to-date.

If you're asking "would a multi-arch image suit my needs?" Then yes, that's exactly what I'm looking for.

---

Some naïve perusing of the build.sbt file I linked above gives me the impression that there's nothing variable in the Dockerfile? If so, I can probably knock this out tonight. (Not my first time doing a multi-arch Docker image.)

[kristian-lange-tw]

The latter one, in general :)
And you are right: there is nothing variable in the Dockerfile. Please knock it out 👍

[jmuchovej]

Is the build process for JATOS documented somewhere? (I see a reference to ./opt/ in the Dockerfile-relevant sections, which I presume is generated during the build, but I don't actually know where to find and/or how to source that.)

Furthermore, are you open to using GitHub Workflows to build/push these? (I've set these up in the past, but since I don't see a .github/workflows folder, I'm guessing you either manually build or use Jenkins/equivalent.)

(If you don't wanna use GitHub Workflows, should I write-up a [bash] script that allows for building/tagging/pushing?)

[kristian-lange-tw]

So far we only use sbt for the jatos.zip and then I bundle them by hand with Java and upload the zips by hand to GitHub.

For the docker images I use sbt-native-packager and push them to Docker Hub. I always wanted to further automate the build process but never found the time.

Is the build process for JATOS documented somewhere?
Nope. So far it's really just a couple of sbt commands and the build config in the build.sbt. But I can describe it more in detail.

I guess GitHub Actions would be fine. But somehow I think a simple shell script would do the same job (and keep it simple).

[jmuchovej]

Okay. Cool. Do you mind walking me through the build process and/or linking some resources on how to build them? (I've never used Scala before, so I'm not sure what to do here. 😅)

[kristian-lange-tw]

Sorry for the late reply. I tried for the last couple hours to change the build.sbt to automatically build docker in multi-arch ... and failed. But this is just an inconvenience. The docker building works and the multi-arch building can be done in a separate step.

I used the following command to push a multi-arch image to Docker Hub (it's a pre-release - don't use it in production).

docker buildx build --platform=linux/arm64,linux/amd64 --push -t jatos/jatos:test .

My docker multi-arch experience is pretty non-existent. Do you agree with this command? Can you try this image on an arm processor? Do you think we need a special build for armv8, the one you mentioned before?

[jmuchovej]

Do you agree with this command?

Looks correct at first glance.

Can you try this image on an arm processor?

Just did. It works! 🙂

Do you think we need a special build for armv8, the one you mentioned before?

I don't think so. I believe that linux/arm/v8 is just the latest release of the ARM instruction set. But I'm honestly not sure. I mostly know "how to build" multi-platform images, not necessarily "why this version and not that other one". 😅

[kristian-lange-tw]

Nice. So I'll do a multi-arch docker image from now on 🎉

And since we are talking about docker. What do you think about this Dockerfile? I'm using it since a while now. Maybe you see something we can improve?

FROM openjdk:11-jre
MAINTAINER Kristian Lange
WORKDIR /opt/docker
ADD opt /opt
EXPOSE 9000 9443
RUN apt update -y && apt install vim -y
RUN ["mkdir", "-p", "/opt/docker/logs"]
RUN ["chown", "-R", "daemon:daemon", "."]
RUN ["chmod", "u+x", "loader.sh"]
VOLUME /opt/docker/logs
RUN bash -l -c 'echo export JATOS_SECRET=$(LC_ALL=C tr -cd '[:alnum:]' < /dev/urandom | fold -w128 | head -n1) >> /etc/bash.bashrc'
USER daemon
ENTRYPOINT ["./loader.sh", "start"]

[jmuchovej]

👋 sorry for the slow reply! I forgot that I didn't answer you. 😅

So, I'm curious why the ... export JATOS_SECRET=... line exists in the Dockerfile. How is that any different from this line in ./loader.sh?

JATOS/loader.sh

Line 33 in b5fe354

secret="$(LC_ALL=C tr -cd '[:alnum:]' < /dev/urandom 2>/dev/null | dd bs=64 count=1 2>/dev/null)"

I don't see any functional reference to JATOS_SECRET throughout JATOS.

I saw this:

JATOS/conf/application.conf

Lines 192 to 204 in 9f261be

	# Play's application secret
	# The loader.sh/.bat generate a secret the first time JATOS starts. So
	# usually there is no need to set it manually. Anyway, it can be set
	# via environment variable JATOS_SECRET, or in another conf file
	# like production.conf, or via a -Dplay.http.secret.key parameter,
	# or by env variable JATOS_SECRET.
	# This default secret here is only for development and must not be
	# used in production.
	# ~~~~~
	play.http.secret {
	key = "zj97lcqp896QDatUyDicMfZdAsGeTivm72pq3p52nLfDxMTDCfuKjGoc4Rj"
	key = ${?JATOS_SECRET}
	}

but it's unclear if declaring JATOS_SECRET in the Docker container will override the secret=$(...) statement in ./loader.sh. Furthermore – is it ideal to have all Docker containers specify the same secret? Anyone who pulls, say, the image tagged 3.7.3 will have the same secret (to my knowledge). That seems like a security risk. (This would be because JATOS_SECRET is generated at build-time, rather than runtime. The latter is what I would think the intended behavior is.

Though, of course, I concede most of these points to you, since you're the primary maintainer of this. 😅

[kristian-lange-tw]

So, I'm curious why the ... export JATOS_SECRET=... line exists in the Dockerfile. How is that any different from this line in ./loader.sh?

You are right. It's not needed in the Dockerfile. The one in loader.sh (and loader.bat) do the job. I can't remember why I put it there - maybe I was experimenting with starting JATOS without the loader.sh. Anyway I'll remove it. Thanks for pointing this out 👍

but it's unclear if declaring JATOS_SECRET in the Docker container will override the secret=$(...) statement in ./loader.sh.

No, it wont't. The one in the loader.sh has precedence over the one in the Dockerfile.

is it ideal to have all Docker containers specify the same secret? Anyone who pulls, say, the image tagged 3.7.3 will have the same secret (to my knowledge). That seems like a security risk. (This would be because JATOS_SECRET is generated at build-time, rather than runtime. The latter is what I would think the intended behavior is.

You are correct, this would not be a good idea and a security risk. The secret is used by the Play Framework for things like the session cookie (https://www.playframework.com/documentation/2.8.x/ApplicationSecret). Luckily it got never used, only the one in loader.sh is used.

[kristian-lange]

JATOS has multi-arch Docker images since v3.7.4, with amd64 and arm64 (https://hub.docker.com/r/jatos/jatos/tags).