Multiple CRD APIVersions and migration

[tatsuhiro-t]

Is this a BUG REPORT or FEATURE REQUEST?:

/kind feature

What happened:

I defined a CRD whose APIVersion is v1alpha1. Now things went well, I want to change APIVersion to v1. But CRD.Spec.Version is immutable, and I could not change it. CRD requires its name must match plural, and group, I could not add another CRD with the different version.
Backup custom objects, and delete current CRD, and create new CRD, then importing custom objects will work, but I want easier migration.

What you expected to happen:

CRD can have multiple versions.
A way to migrate between versions (just like we can get Deployment from both apps/v1beta1 and apps/v1beta2) is highly desirable.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others:

[tatsuhiro-t]

/sig api-machinery

[nikhita]

/area custom-resources

/cc @sttts @deads2k @enisoc

[sttts]

We cannot support multiple versions of a CRD because there is no mechanism that can dynamically convert between them. There is a discussion going on in sig-apimachinery whether multiple versions are in scope or not, and if yes, how to add conversions for CRDs.

For the use-case here though the question is whether we can at least allow to change the (one and only) version of CRD without loosing any data. Right now this is not possible. Technically, this could certainly be done. The big question is whether we want to offer that.

Semantically, this would mean that all existing CRs are "suddenly" returned with the updated version string, but the old, unchanged schema. In other words, the consumers of the CR have to make sure that all CRs are migrated (trivial if the migration is a Noop).

/cc @nikhita @caesarxuchao @deads2k @enisoc

[yliaog]

/cc @yliaog

[mbohlool]

/sub

[nikhita]

Just for the record, relevant sig-api-machinery mailing list topics:

• https://groups.google.com/forum/#!topic/kubernetes-sig-api-machinery/07JKqCzQKsc
• https://groups.google.com/forum/#!topic/kubernetes-sig-api-machinery/rX99p_YWObA

[nikhita]

CRD versioning feature issue: kubernetes/enhancements#544.

[nikhita]

WIP is in #60113.

[nikhita]

xref #63518 - CRD Versioning with no-op Conversion

[nikhita]

Just for the record: no-conversion versioning has been implemented in #63830. It will be present in 1.11.

[liggitt]

addressed by no-op conversion, released in 1.11

/close