(feat) Build and push container images

[aledegano]

Create CD job to build and publish the container images for acceptance-tests, init-realm and certificates when they change any time a commit is merged in the master branch.

This CD job allows to deploy the Renku chart directly from this Git repository (e.g. skipping the chart repository) for those instances that want to deploy the tip of the master branch of Renku (e.g. dev.renku.ch).

Part of swissdatasciencecenter/terraform-renku#139.

[rokroskar]

so this basically avoids a chartpress for deploying master?

[rokroskar]

Note that chartpress will actually check which images are already in the registry, so this will just speed up all builds and we don't really need to change the deploy actions.

[aledegano]

Yes, this is to avoid chartpress. Specifically the reason I want to bypass chartpress is that it does not allow to specify latest as tag (or any other non-semver tag), thus once the images are published under that mandatory (semver) tag I would need to update the Helm chart by either pushing it back here in the repo (with the values file updated) or publishing the chart.
IMO both solution creates a lot of noise and cruft for little value.
Note that the actions I have implemented here also do not re-build/re-push the images that have not changed (I have tested this earlier). Finally once Flux will be responsible of deploying on dev.renku.ch we will be able to remove entirely the workflow "Deploy and Test", so the final amount of workflow should stay the same.

[rokroskar]

Specifically the reason I want to bypass chartpress is that it does not allow to specify latest as tag (or any other non-semver tag), thus once the images are published under that mandatory (semver) tag I would need to update the Helm chart by either pushing it back here in the repo (with the values file updated) or publishing the chart.

Using latest is begging for (difficult to debug) trouble, the most obvious problem being "concurrency" issues (e.g. overlapping deploys pushing the same tag). In any case, you can set any tag you want with chartpress. You can always leave the chart as is and override the image tag values by using --set with helm.

Finally once Flux will be responsible of deploying on dev.renku.ch we will be able to remove entirely the workflow "Deploy and Test", so the final amount of workflow should stay the same.

Definitely not the test part - we want still to be testing the deployed versions.

[ableuler]

Using latest is begging for (difficult to debug) trouble, the most obvious problem being "concurrency" issues (e.g. overlapping deploys pushing the same tag

I don't see this as a huge problem since these changes only affect our CD deployment at dev.renku.ch but not the CI deployments, but I guess there could be scenarios where this might happen. @rokroskar would you rather just publish a dev version of the renku chart after every merge to master and deploy that one?

[rokroskar]

That's what we used to do in the past. There is really no obvious downside to that imho except that the list of published charts at https://swissdatasciencecenter.github.io/helm-charts/ gets very long. The problem with "latest" is that you just don't really know what it is - what if a number of deploys fail before the image publish step and you are trying to debug the CD deployment - you'll need to find the last commit for which the pipeline passed to have an idea of what is actually running, and you might not even realize it. Since we use dev.renku.ch for testing the next user-facing version of the platform, we need some more robustness there. What's wrong with using the deploy script that the action is using? That is made exactly for this reason (to easily deploy from any mixture of versions). The alternative is just to override the image tags - that also doesn't seem too difficult.

[aledegano]

That's what we used to do in the past. There is really no obvious downside to that imho except that the list of published charts at https://swissdatasciencecenter.github.io/helm-charts/ gets very long. The problem with "latest" is that you just don't really know what it is - what if a number of deploys fail before the image publish step and you are trying to debug the CD deployment - you'll need to find the last commit for which the pipeline passed to have an idea of what is actually running, and you might not even realize it. Since we use dev.renku.ch for testing the next user-facing version of the platform, we need some more robustness there. What's wrong with using the deploy script that the action is using? That is made exactly for this reason (to easily deploy from any mixture of versions). The alternative is just to override the image tags - that also doesn't seem too difficult.

My only concern was exactly that the list of releases gets very long, but if you don't think that's a problem I will go with that. Overriding the image tags is easy, it's hard to use that tag in the workflow that we use to have Flux deploy the charts. At that point is a lot easier to publish a chart for every merge and deploy all of those charts in dev.renku.ch as they become available.
I'm closing this PR.

[rokroskar]

I've pruned the charts manually maybe a handful of times in the past few years so I wouldn't say it's a huge deal - one could certainly design some automation around this. But we have relatively few merges to master now in comparison to before when every merge to master in all the other repos also updated master on renku.