fix: Stop creating immutable resource label that breaks "tutor k8s" o…

…n Tutor version changes

Through the commonLabels directive in kustomization.yml, all resources
get a label named "app.kubernetes.io/version", which is being set to
the Tutor version at the time of initial deployment.

When the user then subsequently progresses to a new Tutor version,
Kubernetes attempts to update this label — but for Deployment,
ReplicaSet, and DaemonSet resources, this is no longer allowed as of
kubernetes/kubernetes#50808. This causes
"tutor k8s start" (at the "kubectl apply --kustomize" step) to break
with errors such as:

Deployment.apps "redis" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"openedx-JIONBLbtByCGUYgHgr4tDWu1", "app.kubernetes.io/managed-by":"tutor", "app.kubernetes.io/name":"redis", "app.kubernetes.io/part-of":"openedx", "app.kubernetes.io/version":"12.1.7"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

Simply removing the app.kubernetes.io/version label from
kustomization.yml will permanently fix this issue for newly created
Kubernetes deployments, which will "survive" any future Tutor version
changes thereafter.

However, *existing* production Open edX deployments will need to throw
the affected Deployments away, and re-create them.

Also, add the Tutor version as a resource annotation instead, using
the `commonAnnotations` directive.

See also:
kubernetes/client-go#508
https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/commonlabels/
https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/commonannotations/

tutor/templates/kustomization.yml

@@ -12,11 +12,16 @@ resources:
 # namespace to deploy all Resources to
 namespace: {{ K8S_NAMESPACE }}
 
-# labels added to all Resources
+# annotations added to all Resources
+# https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/commonannotations/
+commonAnnotations:
+  app.kubernetes.io/version: {{ TUTOR_VERSION }}
+
+# labels (and label selectors) added to all Resources
 # https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/
+# https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/commonlabels/
 commonLabels:
   app.kubernetes.io/instance: openedx-{{ ID }}
-  app.kubernetes.io/version: {{ TUTOR_VERSION }}
   app.kubernetes.io/part-of: openedx
   app.kubernetes.io/managed-by: tutor
   {{ patch("kustomization-commonlabels")|indent(2) }}