- Release Signoff Checklist
- Summary
- Motivation
- Proposal
- Design Details
- Production Readiness Review Questionnaire
- Implementation History
- Drawbacks
- Alternatives
- Infrastructure Needed (Optional)
Items marked with (R) are required prior to targeting to a milestone / release.
- (R) Enhancement issue in release milestone, which links to KEP dir in kubernetes/enhancements (not the initial KEP PR)
- (R) KEP approvers have approved the KEP status as
implementable - (R) Design details are appropriately documented
- (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
- e2e Tests for all Beta API Operations (endpoints)
- (R) Ensure GA e2e tests for meet requirements for Conformance Tests
- (R) Minimum Two Week Window for GA e2e tests to prove flake free
- (R) Graduation criteria is in place
- (R) all GA Endpoints must be hit by Conformance Tests
- (R) Production readiness review completed
- (R) Production readiness review approved
- "Implementation History" section is up-to-date for milestone
- User-facing documentation has been created in kubernetes/website, for publication to kubernetes.io
- Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
From the Kubernetes release where this change is introduced, and onwards, beta APIs will not be enabled in clusters by default. Existing beta APIs and new versions of existing beta APIs, will continue to be enabled by default: if v1beta.some.group is currently enabled by default and we create v1beta2.some.group, v1beta2.some.group will still be enabled by default.
Beta APIs are not considered stable and reliance upon APIs in this state leads to exposure to bugs, guaranteed migration pain for users when the APIs move to stable, and the risk that dependencies will grow around unfinished APIs. Enabling beta APIs by default, exacerbates these problems by making them on in nearly every cluster. We observed these problems as we removed long-standing beta APIs and the PRR survey tells us that over 90% of cluster-admins leave production clusters with these APIs enabled. Unsuitability for production use is documented at https://kubernetes.io/docs/reference/using-api/#api-versioning ("The software is not recommended for production uses"), but defaulting on means they are present in nearly every production cluster. By disabling beta APIs by default, a cluster-admin can opt-in for specific APIs without having every incomplete API present in the cluster. This is now practical to do since conformance no longer relies on non-stable APIs.
- Disable new beta APIs by default.
- Continue enabling existing beta APIs and new version of existing beta APIs by default: if v1beta.some.group is currently enabled by default and we create v1beta2.some.group, v1beta2.some.group will still be enabled by default.
- Allow enabling specific resources in beta. Enable coolnewjobtype.v1beta1.batch.k8s.io without enabling other-neat-job.v1beta1.batch.k8s.io
- Change feature gate defaults. Feature gates control new features (not just new APIs) and they are on by default for beta features. This KEP is not changing the lifecycle flow for feature gates. It is currently alpha=off-by-default, beta=on-by-default, stable=locked-to-on.
New beta APIs will be placed into the DisableVersions stanza instead of the EnableVersions stanza (see DefaultAPIResourceConfigSource).
The --runtime-config flag will be extended to allow group/version/resource=true, to enable specific resources.
To enable a beta API, a cluster-admin will have to add the appropriate --runtime-config flags.
As a cluster-admin I want to enable the coolnewjobtype.v1beta1.batch.k8s.io API in my cluster.
To do this I call kube-apiserver --runtime-config=batch.k8s.io/v1beta1/coolnewjobtype.
As a cluster-admin I want to enable all beta APIs as in past releases.
To do this I call kube-apiserver --runtime-config=api/beta=true.
This already exists and will continue to function.
Installers, utilities, controllers, etc that need to know if a certain beta API is present can continue to use the
existing discovery mechanisms (example: kubectl's api-resources sub command or the /api/apps/v1 REST API).
Adoption of beta features will slow. Given how kubernetes is now treated, this is a good thing, not a bad thing. Those users that want to move quickly and get new features can do so by enabling all beta feature or just enabling those that are important for their workload. The PRR survey shows that over 30% of cluster-admins have enabled alpha features on at least some production clusters, so cluster-admins are willing and able to enable features that are not on by default when they are desired.
If two or more APIs are tightly coupled together, it will now be possible to enable them independently. This can lead to unanticipated failure modes, but should only impact beta APIs with beta dependencies. While this is a risk, it is not very common and components should fail safe as a general principle.
If beta APIs are off by default, it's possible that fewer clients will use them and provide feedback. This is a risk, but early adopters are able to enable these features and have a history of enabling alpha features. When moving from beta to GA, it will be important for sigs to explicitly seek feedback. We will address this by extending the PRR questionnaire to include a GA-targeted question to validate that the feature was reasonably validated in production use-cases.
If beta APIs are off by default, it is possible that sigs don't treat taking an API as an indication of a "mostly-baked" API. If this happens, then more transformation may be required. Keeping our beta API rules consistent and continuing to enforce easy to use APIs seems to be the best option.
Integration tests will be written to ensure that no new beta APIs are enabled in the kube-apiserver by default. Unit tests will be written to ensure that the new flag functionality works as expected.
This KEP is a policy KEP, not a feature KEP. It will start as GA.
- Integration and unit tests from above.
- updating the enablement docs for beta
- https://kubernetes.io/docs/reference/using-api/#api-versioning
- https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/#using-a-feature Even though that is talking about feature gates, it is likely worth calling out there that new beta REST APIs are no longer enabled by default)
- email to dev@kubernetes.io to explain the new policy
- blog post explaining change in time for 1.24 release
- CI configuration updated to have a testing mode that enables beta APIs, likely set using
kube-apiserver --runtime-config=api/beta=true - extend the PRR questionnaire to include a GA-targeted question to validate that the feature was reasonably validated in production use-cases.
The additional command line flag format for --runtime-config will not be recognized on older levels of kubernetes.
This means that when downgrading, cluster-admins will have to adjust their CLI arguments if they opted into a new beta API.
This is congruent to flag handling for new features today.
Because this only impacts new beta APIs, there is no behavior change for existing APIs on upgrade.
Because this only impacts new beta APIs, there is no novel skew risk.
Not applicable because this is a policy KEP.