fix: Disable rolling upgrades for Caddy Kubernetes deployment

[fghaas]

Caddy (as used by Tutor) is a single-instance stateful application, in that it uses a PersistentVolume that can only ever be mounted to one Pod. As such, it is unsuitable for rolling upgrades. In the course of an upgrade, we first need to terminate the existing Pod, then bring up its replacement (just like the other applications configured with PersistentVolumeClaims, like MySQL and MongoDB).

Streamline the caddy deployment with all others that use PVCs, and set its strategy to the Recreate type.

Reference: https://kubernetes.io/docs/tasks/run-application/run-single-instance-stateful-application/

[regisb]

I don't understand what issue this PR is addressing (again, please forgive my lack of k8s expertise 😓). Can you please describe more precisely which scenario we are trying to avoid here? Is this somehow related to this other conversation?

[fghaas]

This is really a cosmetic issue ("explicit is better than implicit").

Right now, if I run tutor k8s start (with RUN_CADDY: true), Tutor deploys a pod from the docker.io/caddy:2.3.0 image, with its StrategyType set to RollingUpdate (this is the default).

However, that Deployment also maps a Docker Volume, /data, from a Kubernetes Persistent Volume (PV).

If I were to now set DOCKER_IMAGE_CADDY: docker.io/caddy:2 (which, at this time, maps to version 2.4.6), and then run tutor config save followed by tutor k8s start, Kubernetes would do a RollingUpdate replacement if it were not for the PV mapping. But because there is a PV mapping, that can't work, so it falls back to a Recreate strategy in which it throws away the old pod before spinning up a new one.

So, might as well specify the Recreate strategy explicitly, like we do on all the other services that use PVs.

[fghaas]

Marking this one as a Draft. I might have a thinko here.

[fghaas]

In view of #533, please disregard this PR.

• Once fix: Stop creating immutable resource label that breaks "tutor k8s" on Tutor version changes #533 lands, upgrading from one Tutor version to the next causes an update to the Annotations of the caddy Deployment's single Pod.
• If its Strategy is set to RollingUpdate as it is now, the service is unavailable for a bit less than when set to Recreate (at least on the Kubernetes cluster I'm working on).

So, let's please drop this one. Sorry for the noise!

[fghaas]

Just so we have a cross-reference: there's a somewhat related patch in #660.