CRD update during helm upgrade

[vivekzhere]

kubernetes/kubernetes#94295

Our operator was using a crd.v1beta1.apiextensions.k8s.io and recently updated to crd.v1.apiextensions.k8s.io. In v1beta1 definition spec.preserveUnknownFields set as true. In the v1 version some fields have the x-kubernetes-preserve-unknown-fields set as true. When we do a helm upgrade(we use helm3 with crds in the crd folder), the crd version gets updated to crd.v1.apiextensions.k8s.io, but spec.preserveUnknownFields is still set an true and the x-kubernetes-preserve-unknown-fields field is not at all present in the definition. Our operator copies the crds to another cluster and this fails with

"CustomResourceDefinition.apiextensions.k8s.io \"sfplans.osb.servicefabrik.io\" is invalid: spec.preserveUnknownFields: Invalid value: true: cannot set to true, set x-preserve-unknown-fields to true in spec.versions[*].schema instead"

Output of helm version: version.BuildInfo{Version:"v3.3", GitCommit:"", GitTreeState:"", GoVersion:"go1.15"}

Output of kubectl version: tested on k8s 1.16.12 and 1.18.5

Cloud Provider/Platform (AKS, GKE, Minikube etc.): tested on aws and gcp using gardener

[mattfarina]

CRDs are a difficult problem. A Helm CRD document has been started to highlight the reasons for the current design. You can see the PR at helm/community#138 or read the full doc at https://github.com/helm/community/blob/f9e06c16d89ccea1bea77c01a6a96ae3b309f823/architecture/crds.md.

CRDs are hard with lots of opportunity for problems to arise.

The process to install CRDs is described at https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#install-a-crd-declaration-before-using-the-resource. If you're chart doesn't install custom resources just the CRDs... you can put the CRDs right into the templates directory.

Just be careful. If you uninstall the chart and remove the CRDs all the custom resource in the cluster will be deleted. If two people try to install the same chart in the same cluster with different versions (or default version set) in the cluster it could create problems. I would read the first document to understand the issues.

[github-actions]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.