Provide official container images for CI

[umarcor]

Describe the project you are working on:

Not relevant. This request is usable for any project that needs to be built automatically and periodically.

Describe the problem or limitation you are having in your project:

Currently, there is no official container repository/image that I am aware of. As a result, any user willing to use Godot in CI needs to retrieve and extract both the (headless) tool and the templates. Doing so requires ~1min.

In some distributions, Godot is available through the package manager. However, on the one hand, templates might not be available. On the other hand, the version of Godot might be outdated.

Currently, there are several several users providing Dockerfiles and publishing images. Some examples:

• https://github.com/aBARICHELLO/godot-ci/blob/master/Dockerfile
  • https://hub.docker.com/r/barichello/godot-ci
• https://hub.docker.com/r/ioribranford/godot-docker/dockerfile
  • https://hub.docker.com/r/ioribranford/godot-docker
• https://github.com/umarcor/hwstudio/blob/godot/root/.github/actions/godot/godot.dockerfile
  • https://hub.docker.com/r/hwstudio/godot
• See 150+ results in https://hub.docker.com/search?q=godot&type=image

Unfortunately, there are two main caveats:

• Each of the maintainers of those images needs to "manually" keep track of new Godot releases, in order to update their Dockerfiles.
• There is an unnecessary competition, because most of those projects are almost completely equivalent. I.e., there are no explicit incompatibilities that prevent them from being merged into a single solution.
  • Yet, because of how template installation is managed, there might be small differences that make the workflows not exactly switchable.

Furthermore, since Godot is already using CI, the tool itself is already being built and published in an automated workflow. Moreover, in the org/project, Docker images are already being used for building Godot. For reference, see:

• https://github.com/godotengine/godot/tree/master/.github/workflows
• https://github.com/godotengine/build-containers "These containers should help you build Godot for all platforms supported on any machine that can run Docker containers".

Describe the feature / enhancement and how it helps to overcome the problem or limitation:

The request is the following:

• Create a godot organization in Docker Hub.
  • Alternatively, use GitHub's Package Registry (to avoid handling new credentials).
• Add one or more Dockerfiles to the main repo, based on the references from aBARICHELLO, ioribranford or myself (see above).
• On each tagged commit (release), build the Docker image (which should contain, at least, the headless version and the templates), tag it godot/export:<VERSION> as well as godot/export:latest, and push it to the registry.

Then, users can extend the "official" "base" image with other features (say tools for publishing to itch). But there would be a common reference that is updated automatically together with the official releases.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

See previous point.

My proposal would be to start with a single image based on ubuntu:bionic, and add Godot_v${GODOT_VERSION}-stable_linux_headless.64.zip and Godot_v${GODOT_VERSION}-stable_export_templates.tpz only. To do so, wget|curl and unzip are required. Leaving these in the final image is optional. That alone would be (re)usable by any of the most popular actions/workflows.

There is a point that needs to be addressed, though: where to locate templates. By default, godot expects templates to be located at ~/.local/share/.... That is unfortunate, because when building the docker image the user is typically root, but in CI workflows/runs, it might not be. E.g., in GitHub Actions, HOME is /github/home.

@abarichello addressed it by copying the content in the workflow: https://github.com/aBARICHELLO/godot-ci/blob/master/.github/workflows/godot-ci.yml#L19-L20. I was not convinced with that approach and, instead, I used a script as the entrypoint to a container Action: https://github.com/umarcor/hwstudio/blob/godot/root/.github/actions/godot/gexport.sh. Moreover, in the image, templates are located in /usr/local/share/..., instead of ~/.local/share/.... For the purpose of this request, that difference is trivial.

Nevertheless, this arises the question about the convenience of Godot search for templates in the user home only. I believe it would be sensible for Godot to look at /usr/local/share/... (or any other global location) too. Should that be possible, neither @abarichello's solution (fixes/hacks) nor mine would be required at all.

NOTE: I put the templates in /usr/local/share because godot is installed in /usr/local/bin. But I'd expect /usr/share and /usr/bin, or /bin and /share to be valid too.

If this enhancement will not be used often, can it be worked around with a few lines of script?:

This is not a requirement, but an enhancement to avoid unneccesary fragmentation in the community. Don't take me wrong: fragmentation in open source is frequently good; but I think that not in this specific case.

Is there a reason why this should be core and not an add-on in the asset library?:

This question does not apply.

/cc @abarichello @josephbmanley @croconut

[yeslayla]

I am totally onboard. The lack of an official image has lead to a lot of people redoing work.

Really good point mentioning the gotcha related to the GitHub actions. In my build action, I use @abarichello's container, but have to manually move the default export templates at runtime. Solving this by implementing a global export directory would make building and configuring a CI container pretty elegant.

I also think it's important to note as this is not an integration into the engine itself, people will still have the freedom to build their own image.

[umarcor]

The lack of an official image has lead to a lot of people redoing work.

Just for clarification, it might be one or multiple images. My use case is to export for Linux, Windows, Mac and HTML5 only, without Mono or any other dependency. Hence, I'm ok with a small image containing headless godot and templates only. Nevertheless, I understand that other users might need images with other tools. Hence, it would make sense to provide a few. Yet, I cannot set a number. 3? 5? 8?

Solving this by implementing a global export directory

I might be ignorant and such an option might already exist. That's why I didn't open a specific issue about it. Hope some maintainer can clarify.

I also think it's important to note as this is not an integration into the engine itself, people will still have the freedom to build their own image.

Good point! This is just plumbing/chores. Actually:

• Users can make their images based on the official ones. This is the most expected use.
• Users can clone/fork/copy the repo and create custom images that use the same building logic/scripts as the official ones.
• Users can keep building their own images from scratch.

[Calinou]

We should probably have two Dockerfiles:

• One that builds the headless editor with export templates included.
• One that builds a server (without editor functionality) for hosting dedicated game servers. The user will have to provide the PCK containing the game data themselves somehow.

[yeslayla]

I mean it would definitely be nice to have an android and mono specific containers as well, but it would end up being more to maintain.

[umarcor]

The user will have to provide the PCK containing the game data themselves somehow.

@Calinou, FTR:

• Approach 1: provide it as a runtime argument, docker run -v /path/to/file.pck:/path/to/predefined/location godot/server. The predefined location in the image would need to be documented.

• Approach 2: explain users how to "customize" the official server image.

docker build -t my/server - <<EOF
FROM godot/server
COPY file.pck /path/to/predefined/location
EOF

docker run my/server

The predefined location in the image would need to be documented.

• Approach 3: explain users how to "customize" the image, and provide their custom entrypoint.

docker build -t my/server - <<EOF
FROM godot/server
COPY file.pck /path/to/predefined/location
ENTRYPOINT ["my", "--custom", "command"]
EOF

docker run my/server addition_args

I mean it would definitely be nice to have an android and mono specific containers as well, but it would end up being more to maintain.

@josephbmanley I'd say: one step at a time. After clarifying/fixing the issue about the location of the templates, I believe that @abarichello's Dockerfile is the best to start with. Making that official (probably with minimal enhancements) will trigger an implicit round of updates in multiple repos/actions. We can wait for that to settle down before focusing on adding features or additional containers for more specific use cases.

I believe that godot/export:VERSION, godot/export:VERSION-mono, godot/export:VERSION-android and godot/export:server is a nice set of images to provide. *-mono and *.android can be declared as additional stages in the same Dockerfile as godot/export. It would be two Dockerfiles to maintain. Unfortunately, I am familiar with neither Mono nor Android, so I cannot make an specific proposal.

[jonbonazza]

@Calinou I provide crude versions of these under the summerplay dockerhub. Currently, they don't build and instead just use the official binaries, but Ive been meaning to change these to build from scratch. I definitely agree that at the minimum we should have both headless and server containers. I'd be happy to transfer what I have yo a godotengine dockerhub and help make them proper.

Let me know what you think.

[umarcor]

@jonbonazza, find some enhancements to the summerplay headless image in https://github.com/umarcor/hwstudio/blob/godot/root/.github/actions/godot/godot.dockerfile. Nonetheless, I believe the complexity is not in writing the Dockerfile(s). The main point is that some maintainer of Godot needs to create a Docker user, register the organization in Docker Hub and add the credentials as secrets in the repo.

[jonbonazza]

@umarcor I agree. I was just pointing out that both headless and server images from 3.1 onward exist already in some capacity and Ive been updating them every version since, so once godot maintainers do create the account id be happy to transfer them.

[Bloodyaugust]

@Calinou it has been 2 years since any discussion on this one, and it looks like just as long since https://github.com/Calinou/godot/tree/add-install-export-templates-cli-argument was touched. Is there any time to tackle this sometime soon?

[Calinou]

@Calinou it has been 2 years since any discussion on this one, and it looks like just as long since Calinou/godot@add-install-export-templates-cli-argument was touched. Is there any time to tackle this sometime soon?

I don't have time to work on this anymore; someone else will have to work on this feature. The export templates install dialog has changed a lot since I worked on my branch, so it's likely more efficient to redo it from scratch (but feel free to look at the branch for inspiration).

Also, this discussion should be in #1277.

[dzil123]

I think 1277 is unrelated to this proposal. It would be good if there was a container like abarichello/godot-ci that was under the godotengine organization, so that it can be updated in lockstep with Godot releases.