Kubernetes 1.23 Major Themes

[cici37]

Creating a space to submit major themes entries in release notes. We'd like to have folks submit the content over the release cycle instead of having them piled up at the end of the release cycle.

The discussion in previous release cycle.

[xing-yang]

We want to add the following deprecation message in the release notes for sig-storage:

Deprecation of Flexvolume: Flexvolume is deprecated. Out-of-tree CSI driver is the recommended way to write volume drivers in Kubernetes. See this doc here for more information. Maintainers of Flexvolume drivers should implement a CSI Driver and move users of Flexvolume to CSI. Users of Flexvolume should move their workloads to CSI Driver.

CC @saad-ali @msau42 @jsafrane

[sftim]

nit: I think the correct capitalization is “FlexVolume” and “driver”.

[cici37]

@xing-yang Hi Xing, Is there a reference doc link missing here?

[sacgoel]

@xing-yang pl provide the doc link for this. Also, pl share which release will target to remove the support for flexvolume completely ?

[cici37]

Major Theme Candidate: Expression language validation for CRD is available in alpha kep
Will add more details later.

[cici37]

The feature gate CustomResourceValidationExpressions is available in 1.23 as an alpha feature. It allows to validate customer resource through expression language validation rules defined in CustomerResourceDefinition.

[cici37]

Major Theme Candidate: OpenAPIv3
Hi @Jefftree , openAPIv3 could be a great Major Theme Candidate, would you like to write few sentences about it if you like it to be included in 1.23 Major Theme? Thank you 😊

[damemi]

The kube-scheduler is adding a new, simplified config field for Plugins to allow multiple extension points to be enabled in one spot. kubernetes/website#30442

The new multiPoint plugin field in the scheduler's configuration is intended to simplify most scheduler setups for administrators. Plugins that are enabled via multiPoint will automatically be registered for each individual extension point that they implement. For example, a plugin that implements Score and Filter extensions can be simultaneously enabled for both. This means entire plugins can be enabled and disabled without having to manually edit individual extension point settings. These extension points can now be abstracted away due to their irrelevance for most users.

[cici37]

Hi @damemi , Thank you for the input. Would you mind to write few sentences about it so we could use as part of the release Major Theme section?

[damemi]

Sure, updated with a couple sentences

[serathius]

Major Theme Candidate: Deprecation of klog specific flags in Kubernetes components. Features like: writing to files, header and stream manipulation are planned to be removed.

KEP kubernetes/enhancements#2845
cc @pohly

[cici37]

Thanks for adding the Major Theme topic. Would you mind to write few sentences about it so we could use as part of the release Major Theme announcement? Features like: writing to files, header and stream manipulation are planned to be removed. sounds a little bit confusing to the users(is it removed now?)

[pohly]

Let me try to rephrase the same topic. Note that the link currently doesn't have information about the deprecation. That will be there once 1.23 is released.

To simplify the code base, several logging flags got marked as deprecated in Kubernetes 1.23. The code which implements them will be removed in a future release, so users of those need to start replacing the deprecated flags with some alternative solutions.

[pohly]

If you prefer a more stable link (the documentation also will be removed in a future release), then use the KEP.

[xing-yang]

Major Themes from SIG-Storage:

• The generic ephemeral volume feature moved to GA in 1.23. This feature allows any existing storage driver that supports dynamic provisioning to be used as an ephemeral volume with the volume’s lifecycle bound to the Pod. All StorageClass parameters for volume provisioning and all features supported with PersistentVolumeClaims are supported.
• The feature to configure volume permission and ownership change policy for Pods moved to GA in 1.23. This allows users to skip recursive permission changes on mount and speeds up the pod start up time.
• The feature to allow CSI Drivers to declare support for fsGroup based permissions also moved to GA in 1.23.

cc @saad-ali @msau42 @jsafrane

[cici37]

Hi @xing-yang Thanks for adding the Major Themes topic. Would you mind to write few more sentences on each of the topic so users wold have a better understanding? Thank you

[cici37]

Hi @xing-yang , for the features graduate to GA, is that ConfigurableFSGroupPolicy and CSIVolumeFSGroupPolicy? Thank you

[pohly]

Another major theme candidate from SIG Instrumentation/WG Structured Logging:

Structured logging reached its beta milestone. Most log messages from kubelet and kube-scheduler have been converted. Users are encouraged to try out JSON output or parsing of the structured text format and provide feedback on possible solutions for some of the open issues, such as handling of multi-line strings in log values.

[reylejano]

Major theme candidate: IPv4/IPv6 Dual-stack Networking is GA (SIG Network)

IPv4/IPv6 dual-stack networking graduates to GA. Since 1.21, Kubernetes clusters are enabled to support dual-stack networking by default. In 1.23, the IPv6DualStack feature gate is removed. The use of dual-stack networking is not mandatory. Although clusters are enabled to support dual-stack networking, Pods and Services continue to default to single-stack. To use dual-stack networking: Kubernetes nodes have routable IPv4/IPv6 network interfaces, a dual-stack capable CNI network plugin is used, Pods are configured to be dual-stack and Services have their.spec.ipFamilyPolicy field set to either PreferDualStack or RequireDualStack.

cc @bridgetkromhout , ptal, I would like IPv4/IPv6 dual stack to be a Major Theme for 1.23, please make any suggestions

[reylejano]

Major theme candidate: PodSecurity graduates to Beta (SIG Auth)

PodSecurity moves to Beta. PodSecurity replaces the deprecated PodSecurityPolicy admission controller. PodSecurity is an admission controller that enforces Pod Security Standards on Pods in a Namespace based on specific namespace labels that set the enforcement level. In 1.23, the PodSecurity feature gate is enabled by default.

cc @liggitt @tallclair , ptal, I think PodSecurity admission graduating to beta would be a good Major Theme for 1.23. Please make any suggestions

[saschagrunert]

Proposal: Adding support for Container Runtime Interface (CRI) v1

The Kubelet now supports the CRI v1 API, which is now the project-wide default. If a container runtime does not support the v1 API, Kubernetes will fall back to the v1alpha2 implementation. There is no intermediate action required by end-users, because v1 and v1alpha2 do not differ in their implementation. It is likely that v1alpha2 will be removed in one of the future Kubernetes releases to be able to develop v1.

cc @mrunalp

[sftim]

This is a graduation, right?

[cici37]

@saschagrunert Would you mind to help to clarify if this feature is an existing feature graduating to GA or it is just extra support added? The description sounds like a GA graduation. :)

[saschagrunert]

Yes, it's a graduation.

The CRI stands historically for its own but I think it explains as best the situation.

[xing-yang]

One more thing from SIG-Storage:
Status update on CSI Migration in 1.23:
CSI Migration enables the replacement of existing in-tree storage plugins such as kubernetes.io/gce-pd or kubernetes.io/aws-ebs with a corresponding CSI driver. If CSI Migration is working properly, Kubernetes end users shouldn’t notice a difference. After migration, Kubernetes users may continue to rely on all the functionality of in-tree storage plugins using the existing interface.

• CSI Migration feature is turned on by default but stays in Beta for GCE PD, AWS EBS, and Azure Disk in 1.23.
• CSI Migration is introduced as an Alpha feature for Ceph RBD and Portworx in 1.23.
  cc @gnufied @msau42 @saad-ali @jsafrane

[cici37]

Hi @xing-yang Thanks for adding the CSI Migration into Major Themes. Would you mind to write few sentences about CSI Migration so the users would have a better idea on it? Thank you

[xing-yang]

Hi @cici37, I updated the message above. Does it look okay now?

[kevindelgado]

Major Theme Candidate: Server Side Field Validation is alpha.

If the ServerSideFieldValidation feature gate is enabled starting 1.23, users will receive warnings from the server when they send kubernetes objects in the request that contain unknown or duplicate fields. Previously unknown fields and all but the last duplicate fields would be dropped by the server.

With the feature gate enabled we also introduce the fieldValidation query parameter so that users can specify the desired behavior of the server on a per request basis. Valid values for the fieldValidation query parameter are:

• Ignore (default when feature gate is disabled, same as pre-1.23 behavior of dropping/ignoring unkonwn fields)
• Warn (default when feature gate is enabled).
• Strict (this will fail the request with an Invalid Request error)

[josephburnett]

The HPA v2beta2 API is being deprecated. And there is a new HPA v2 (stable) API (mostly just a promotion of beta).

[sftim]

In the full release notes we should make it clear what the story looks like for the v1 HPA API. Are we deprecating HPA v1 in this release / are we not? Readers are likely to want to know. Also, when does the HPA v2beta2 get phased out (should be covered in https://kubernetes.io/docs/reference/using-api/deprecation-guide/ post-release)? If there's room to mention that in the major themes section, I would.

[liggitt]

No, v1 is not deprecated. v2beta2 is announced deprecated in 1.23 and removed in 1.26.

[Jefftree]

Major Theme Candidate: OpenAPI v3 is alpha.

If the OpenAPIV3 feature gate is enabled starting 1.23, users will be able to request the OpenAPI v3.0 spec for all kubernetes types. OpenAPI v3 aims to be fully transparent and includes support for a set of fields that are dropped when publishing OpenAPI v2: default, nullable, oneOf, anyOf. A separate spec is published per Kubernetes group version (at the $cluster/openapi/v3/apis/<group>/<version> endpoint) for improved performance and discovery for all group versions can be found at the $cluster/openapi/v3 path.

cc @apelisse @sttts

[puerco]

Major Theme Candidate: SLSA Level 1 Compliance in the Kubernetes Release Process.

Kubernetes releases are now generating provenance attestation files describing the staging and release phases of the release process and artifacts are verified as they are handed over from one phase to the next. This final piece completes the work needed to comply with Level 1 of the SLSA security framework (Supply-chain Levels for Software Artifacts).

[puerco]

cc @kubernetes/sig-release-leads @kubernetes/release-engineering