Drop support for Shoots with K8s < 1.20 #6987

dimitar-kostadinov · 2022-11-10T09:06:05Z

How to categorize this PR?

/area open-source dev-productivity
/kind cleanup

What this PR does / why we need it:
Drop support for Shoots with K8s < 1.20

Which issue(s) this PR fixes:
Part of #6911

Special notes for your reviewer:

Release note:

⚠️ Gardener does no longer support shoot clusters with Kubernetes versions < 1.20. Make sure to upgrade all existing clusters before upgrading to this Gardener version.

The Shoot kubeconfig Secret in the Project namespace no longer contains the `token` field. The token can be still fetched from the kubeconfig that is present in the `kubeconfig` field.

gardener-prow · 2022-11-10T09:06:09Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

dimityrmirchev

There are multiple occurrences of version 1.19 checks in this repository. Should we also drop them with this PR?

Here is an example

shafeeqes · 2022-11-15T04:13:48Z

/assign

shafeeqes · 2022-11-15T04:54:28Z

In line with #6987 (review), Can you please check the usages of these constraints

gardener/pkg/utils/version/version.go

Lines 26 to 39 in d98cf02

    
           // ConstraintK8sGreaterEqual118 is a version constraint for versions >= 1.18. 
        
           ConstraintK8sGreaterEqual118 *semver.Constraints 
        
           // ConstraintK8sEqual118 is a version constraint for versions == 1.18. 
        
           ConstraintK8sEqual118 *semver.Constraints 
        
           // ConstraintK8sGreaterEqual119 is a version constraint for versions >= 1.19. 
        
           ConstraintK8sGreaterEqual119 *semver.Constraints 
        
           // ConstraintK8sLess119 is a version constraint for versions < 1.19. 
        
           ConstraintK8sLess119 *semver.Constraints 
        
           // ConstraintK8sLess120 is a version constraint for versions < 1.20. 
        
           ConstraintK8sLess120 *semver.Constraints 
        
           // ConstraintK8sEqual119 is a version constraint for versions == 1.19. 
        
           ConstraintK8sEqual119 *semver.Constraints 
        
           // ConstraintK8sGreaterEqual120 is a version constraint for versions >= 1.20. 
        
           ConstraintK8sGreaterEqual120 *semver.Constraints

and remove them as we will only support Shoots >= 1.20 now.

ialidzhikov

Final suggestions, otherwise lgtm

pkg/operation/botanist/component/kubeapiserver/secrets.go

gardener-prow · 2022-11-28T11:42:38Z

@dimitar-kostadinov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-gardener-apidiff	`8370a23`	link	false	`/test pull-gardener-apidiff`
pull-gardener-e2e-kind-ha-single-zone	`8370a23`	link	false	`/test pull-gardener-e2e-kind-ha-single-zone`

Full PR test history. Your PR dashboard. Command help for this repository.
Please help us cut down on flakes by linking this test failure to an open flake report or filing a new flake report if you can't find an existing one. Also see our testing guideline for how to avoid and hunt flakes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

ialidzhikov

/lgtm
/approve

gardener-prow · 2022-11-28T11:59:25Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ialidzhikov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ialidzhikov]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ialidzhikov · 2022-11-28T12:00:18Z

/test pull-gardener-integration
/test pull-gardener-e2e-kind-ha-single-zone

After the removal of support for Kubernetes < 1.20 Shoot clusters (ref #6987), the kubeconfig Secret no longer has the `token` field. Basic auth cannot be enabled for K8s 1.19+ clusters, hence the kubeconfig Secret cannot contain the `username`/`password` fields anymore.

After the removal of support for Kubernetes < 1.20 Shoot clusters (ref gardener#6987), the kubeconfig Secret no longer has the `token` field. Basic auth cannot be enabled for K8s 1.19+ clusters, hence the kubeconfig Secret cannot contain the `username`/`password` fields anymore.

* Allow instantiating v1.26 Kubernetes clients * Update `README.md` and `docs/usage/supported_k8s_versions.md` for the K8s 1.26 * Maintain Kubernetes feature gates $ ./hack/compare-k8s-feature-gates.sh 1.25 1.26 Feature gates added in 1.26 compared to 1.25: APISelfSubjectReview AggregatedDiscoveryEndpoint ConsistentHTTPGetHandlers CrossNamespaceVolumeDataSource DynamicResourceAllocation EventedPLEG LegacyServiceAccountTokenTracking MinimizeIPTablesRestore PDBUnhealthyPodEvictionPolicy PodSchedulingReadiness StatefulSetStartOrdinal TopologyManagerPolicyAlphaOptions TopologyManagerPolicyBetaOptions TopologyManagerPolicyOptions ValidatingAdmissionPolicy WindowsHostNetwork Feature gates removed in 1.26 compared to 1.25: CSIMigrationOpenStack CSRDuration DefaultPodTopologySpread DynamicKubeletConfig IndexedJob NonPreemptingPriority PodAffinityNamespaceSelector PodOverhead PreferNominatedNode ServiceLBNodePortControl ServiceLoadBalancerClass SuspendJob Feature gates locked to default in 1.26 compared to 1.25: CPUManager CSIMigrationvSphere DelegateFSGroupToCSIDriver DevicePlugins DryRun EndpointSliceTerminatingCondition JobTrackingWithFinalizers KubeletCredentialProviders MixedProtocolLBService ServerSideApply ServiceIPStaticSubrange ServiceInternalTrafficPolicy WindowsHostProcessContainers * Maintain `kube-apiserver` admission plugins $ ./hack/compare-k8s-admission-plugins.sh 1.25 1.26 Admission plugins added in 1.26 compared to 1.25: ValidatingAdmissionPolicy Admission plugins removed in 1.26 compared to 1.25: * Maintain `ServiceAccount` names for the controllers part of `kube-controller-manager` $ ./hack/compare-k8s-controllers.sh 1.25 1.26 kube-controller-manager controllers added in 1.26 compared to 1.25: resource-claim-controller kube-controller-manager controllers removed in 1.26 compared to 1.25: * Use 1.26 for local shoot and local e2e test * Deprecate the `podEvictionTimeout` field in favor of newly introduced kube-apiserver fields The kube-controller-manager flag `--pod-eviction-timeout` is deprecated in favor of the kube-apiserver flags `--default-not-ready-toleration-seconds` and `--default-unreachable-toleration-seconds`. The `--pod-eviction-timeout` flag does not have effect when the taint besed eviction is enabled. The taint based eviction is beta (enabled by default) since Kubernetes 1.13 and GA since Kubernetes 1.18. For more details, see kubernetes/kubernetes#74651. This commit allows configuring the kube-apiserver flags `--default-not-ready-toleration-seconds` and `--default-unreachable-toleration-seconds`. The `podEvictionTimeout` field is deprecated in favor of the newly introduced fields. gardener-apiserver no longer defaults the `podEvictionTimeout` field. gardener-apiserver also returns a warning when the `podEvictionTimeout` field is set. * Adapt to the renaming of `etcd_db_total_size_in_bytes` metric to `apiserver_storage_db_total_size_in_bytes` The metric `etcd_db_total_size_in_bytes` is renamed to `apiserver_storage_db_total_size_in_bytes`. Ref kubernetes/kubernetes#113310. * Fix the Pod spec in `simple-load-deployment.yaml.tpl` Test runs of the integration test that uses this template prints the following warning about the issue in the template: ``` {"level":"info","ts":"2022-12-28T19:36:29.043+0200","logger":"KubeAPIWarningLogger","msg":"unknown field \"spec.template.spec.containers[0].nodeName\""} ``` * Update `docs/usage/shoot_credentials_rotation.md` After the removal of support for Kubernetes < 1.20 Shoot clusters (ref #6987), the kubeconfig Secret no longer has the `token` field. Basic auth cannot be enabled for K8s 1.19+ clusters, hence the kubeconfig Secret cannot contain the `username`/`password` fields anymore. * Default `enableStaticTokenKubeconfig` to false for Shoots with K8s version >= 1.26 This commit also adapts most of the testmachinery integration tests to use the `shoots/adminkubeconfig` subresource instead of the static kubeconfig. The Shoot creation intergration is still using the static kubeconfig and it is downloading it to `$TM_KUBECONFIG_PATH/shoot.config`. This commit sets `enableStaticTokenKubeconfig=true` until we figure out which tests/components are using this downloaded kubeconfig. * Add constraint for K8s version < 1.26 The constraint `ConstraintK8sLess126` is currently not used by gardener/gardener but it is introduced for usage from the extensions. * Address review comments * Update `new-kubernetes-version.md` guide with details about the `hyperkube` image * Update `supported_k8s_versions.md` for K8s 1.26 * Update kube-scheduler component's unit tests for K8s 1.26 * Revert the K8s versions used for e2e tests For the reasoning, see #7275 (comment)

gardener-prow bot requested review from ialidzhikov and rfranzke November 10, 2022 09:06

dimitar-kostadinov force-pushed the cleanup/k8s-117-119 branch 4 times, most recently from 85c6303 to f13e7bd Compare November 10, 2022 15:24

gardener-prow bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 10, 2022

dimitar-kostadinov marked this pull request as ready for review November 14, 2022 12:38

gardener-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 14, 2022

gardener-prow bot requested review from timebertt and timuthy November 14, 2022 12:39

dimityrmirchev suggested changes Nov 14, 2022

View reviewed changes

gardener-prow bot assigned dimityrmirchev Nov 14, 2022

gardener-prow bot assigned shafeeqes Nov 15, 2022

dimitar-kostadinov force-pushed the cleanup/k8s-117-119 branch from f13e7bd to ead8d08 Compare November 17, 2022 14:21

gardener-prow bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 17, 2022

dimitar-kostadinov force-pushed the cleanup/k8s-117-119 branch from ead8d08 to 6eb1c10 Compare November 18, 2022 07:38

gardener-prow bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 18, 2022

gardener-prow bot added the lgtm Indicates that a PR is ready to be merged. label Nov 24, 2022

ialidzhikov reviewed Nov 28, 2022

View reviewed changes

pkg/operation/botanist/component/kubeapiserver/secrets.go Show resolved Hide resolved

pkg/operation/botanist/component/kubeapiserver/secrets.go Show resolved Hide resolved

gardener-prow bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 28, 2022

ialidzhikov mentioned this pull request Nov 28, 2022

☂️ Drop support for Shoots with K8s < 1.20 #6911

Closed

11 tasks

dimitar-kostadinov force-pushed the cleanup/k8s-117-119 branch from f4358ee to c919d9e Compare November 28, 2022 10:57

gardener-prow bot removed the lgtm Indicates that a PR is ready to be merged. label Nov 28, 2022

Drop support for Shoots with K8s < 1.20

8370a23

dimitar-kostadinov force-pushed the cleanup/k8s-117-119 branch from c919d9e to 8370a23 Compare November 28, 2022 11:06

gardener-prow bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 28, 2022

ialidzhikov reviewed Nov 28, 2022

View reviewed changes

gardener-prow bot added the lgtm Indicates that a PR is ready to be merged. label Nov 28, 2022

gardener-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 28, 2022

gardener-prow bot merged commit 5a69fa6 into gardener:master Nov 28, 2022

rfranzke mentioned this pull request Jan 19, 2023

Drop rootcapublisher controller in gardener-resource-manager #7367

Merged

ialidzhikov mentioned this pull request Feb 21, 2023

Prepare for the removal of the enableBasicAuthentication field #7534

Merged

rfranzke mentioned this pull request May 3, 2023

🧹 Cleanup a few TODOs #7868

Merged

dimitar-kostadinov mentioned this pull request May 5, 2023

Remove .spec.kubernetes.kubeAPIServer.enableBasicAuthentication field from the Shoot API #7886

Merged

dimitar-kostadinov deleted the cleanup/k8s-117-119 branch May 22, 2023 08:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop support for Shoots with K8s < 1.20 #6987

Drop support for Shoots with K8s < 1.20 #6987

dimitar-kostadinov commented Nov 10, 2022 •

edited by ialidzhikov

gardener-prow bot commented Nov 10, 2022

dimityrmirchev left a comment

shafeeqes commented Nov 15, 2022

shafeeqes commented Nov 15, 2022

ialidzhikov left a comment

gardener-prow bot commented Nov 28, 2022 •

edited

ialidzhikov left a comment

gardener-prow bot commented Nov 28, 2022

ialidzhikov commented Nov 28, 2022

Drop support for Shoots with K8s < 1.20 #6987

Drop support for Shoots with K8s < 1.20 #6987

Conversation

dimitar-kostadinov commented Nov 10, 2022 • edited by ialidzhikov

gardener-prow bot commented Nov 10, 2022

dimityrmirchev left a comment

Choose a reason for hiding this comment

shafeeqes commented Nov 15, 2022

shafeeqes commented Nov 15, 2022

ialidzhikov left a comment

Choose a reason for hiding this comment

gardener-prow bot commented Nov 28, 2022 • edited

ialidzhikov left a comment

Choose a reason for hiding this comment

gardener-prow bot commented Nov 28, 2022

ialidzhikov commented Nov 28, 2022

dimitar-kostadinov commented Nov 10, 2022 •

edited by ialidzhikov

gardener-prow bot commented Nov 28, 2022 •

edited