Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce a new autoscaling mode (VPAAndHPA) for Shoot Kubernetes API servers #9678

Merged
merged 10 commits into from
May 15, 2024
Merged

Introduce a new autoscaling mode (VPAAndHPA) for Shoot Kubernetes API servers #9678

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
041b5b8
Add VPAAndHPAForAPIServer feature gate
ialidzhikov Apr 25, 2024
f4d6c59
Introduce consts for API server autoscaling modes
ialidzhikov Apr 25, 2024
946789d
Add VPAAndHPA autoscaling mode
ialidzhikov Apr 25, 2024
e3e96bb
Add docs for kube-apiserver autoscaling
ialidzhikov Apr 26, 2024
2caae51
Address review comments from voelzmo
ialidzhikov May 8, 2024
f205c8f
Address review comments from rfranzke
ialidzhikov May 8, 2024
d6c7852
Address review comments from voelzmo (2)
ialidzhikov May 9, 2024
756e37e
Minor nits
ialidzhikov May 10, 2024
1433c5f
Address review comments from voelzmo (3)
ialidzhikov May 14, 2024
5da9401
Address review comments from plkokanov
ialidzhikov May 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

* Components
* [Gardener API server](concepts/apiserver.md)
* [In-Tree admission plugins](concepts/apiserver_admission_plugins.md)
* [In-Tree admission plugins](concepts/apiserver-admission-plugins.md)
* [Gardener Controller Manager](concepts/controller-manager.md)
* [Gardener Scheduler](concepts/scheduler.md)
* [Gardener Admission Controller](concepts/admission-controller.md)
Expand Down
0 docs/concepts/apiserver_admission_plugins.md → docs/concepts/apiserver-admission-plugins.md
View file
Open in desktop
File renamed without changes.
8 changes: 0 additions & 8 deletions docs/concepts/etcd.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,6 @@ When a [`gardenlet`](gardenlet.md) reconciles a `Shoot` resource or a [`gardener
`etcd-druid` needs to manage the lifecycle of the desired etcd instance (today `main` or `events`).
Likewise, when the `Shoot` or `Garden` is deleted, `gardenlet` or `gardener-operator` deletes the `Etcd` resources and [etcd Druid](https://github.com/gardener/etcd-druid/) takes care of cleaning up all related objects, e.g. the backing `StatefulSet`s.

## Autoscaling

Gardenlet maintains [`HVPA`](https://github.com/gardener/hvpa-controller/blob/master/config/samples/autoscaling_v1alpha1_hvpa.yaml) objects for etcd `StatefulSet`s if the corresponding [feature gate](../deployment/feature_gates.md) is enabled.
This enables a vertical scaling for etcd.
Downscaling is handled more pessimistically to prevent many subsequent etcd restarts.
Thus, for `production` and `infrastructure` shoot clusters (or all garden clusters), downscaling is deactivated for the main etcd.
For all other shoot clusters, lower advertised requests/limits are only applied during a shoot's maintenance time window.

## Backup

If `Seed`s specify backups for etcd ([example](../../example/50-seed.yaml)), then Gardener and the respective [provider extensions](../extensions/overview.md) are responsible for creating a bucket on the cloud provider's side (modelled through a [BackupBucket resource](../extensions/backupbucket.md)).
Expand Down
26 changes: 14 additions & 12 deletions docs/deployment/feature_gates.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ The following tables are a summary of the feature gates that you can set on diff
| UseNamespacedCloudProfile | `false` | `Alpha` | `1.92` | |
| ShootManagedIssuer | `false` | `Alpha` | `1.93` | |
| VPAForETCD | `false` | `Alpha` | `1.94` | |
| VPAAndHPAForAPIServer | `false` | `Alpha` | `1.95` | |

## Feature Gates for Graduated or Deprecated Features

Expand Down Expand Up @@ -190,15 +191,16 @@ A *General Availability* (GA) feature is also referred to as a *stable* feature.

## List of Feature Gates

| Feature | Relevant Components | Description |
|------------------------------------|-----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| HVPA | `gardenlet`, `gardener-operator` | Enables simultaneous horizontal and vertical scaling in garden or seed clusters. |
| HVPAForShootedSeed | `gardenlet` | Enables simultaneous horizontal and vertical scaling in managed seed (aka "shooted seed") clusters. |
| DefaultSeccompProfile | `gardenlet`, `gardener-operator` | Enables the defaulting of the seccomp profile for Gardener managed workload in the garden or seed to `RuntimeDefault`. |
| CoreDNSQueryRewriting | `gardenlet` | Enables automatic DNS query rewriting in shoot cluster's CoreDNS to shortcut name resolution of fully qualified in-cluster and out-of-cluster names, which follow a user-defined pattern. Details can be found in [DNS Search Path Optimization](../usage/dns-search-path-optimization.md). |
| IPv6SingleStack | `gardener-apiserver`, `gardenlet` | Allows creating seed and shoot clusters with [IPv6 single-stack networking](../usage/ipv6.md) enabled in their spec ([GEP-21](../proposals/21-ipv6-singlestack-local.md)). If enabled in gardenlet, the default behavior is unchanged, but setting `ipFamilies=[IPv6]` in the `seedConfig` is allowed. Only if the `ipFamilies` setting is changed, gardenlet behaves differently. |
| MutableShootSpecNetworkingNodes | `gardener-apiserver` | Allows updating the field `spec.networking.nodes`. The validity of the values has to be checked in the provider extensions. Only enable this feature gate when your system runs provider extensions which have implemented the validation. |
| ShootForceDeletion | `gardener-apiserver` | Allows forceful deletion of Shoots by annotating them with the `confirmation.gardener.cloud/force-deletion` annotation. |
| UseNamespacedCloudProfile | `gardener-apiserver` | Enables usage of `NamespacedCloudProfile`s in `Shoot`s. |
| ShootManagedIssuer | `gardenlet` | Enables the shoot managed issuer functionality described in GEP 24. |
| VPAForETCD | `gardenlet`, `gardener-operator` | Enables VPA for `etcd-main` and `etcd-events`, regardless of HVPA enablement. |
| Feature | Relevant Components | Description |
|---------------------------------|-----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| HVPA | `gardenlet`, `gardener-operator` | Enables simultaneous horizontal and vertical scaling in garden or seed clusters. |
| HVPAForShootedSeed | `gardenlet` | Enables simultaneous horizontal and vertical scaling in managed seed (aka "shooted seed") clusters. |
| DefaultSeccompProfile | `gardenlet`, `gardener-operator` | Enables the defaulting of the seccomp profile for Gardener managed workload in the garden or seed to `RuntimeDefault`. |
| CoreDNSQueryRewriting | `gardenlet` | Enables automatic DNS query rewriting in shoot cluster's CoreDNS to shortcut name resolution of fully qualified in-cluster and out-of-cluster names, which follow a user-defined pattern. Details can be found in [DNS Search Path Optimization](../usage/dns-search-path-optimization.md). |
| IPv6SingleStack | `gardener-apiserver`, `gardenlet` | Allows creating seed and shoot clusters with [IPv6 single-stack networking](../usage/ipv6.md) enabled in their spec ([GEP-21](../proposals/21-ipv6-singlestack-local.md)). If enabled in gardenlet, the default behavior is unchanged, but setting `ipFamilies=[IPv6]` in the `seedConfig` is allowed. Only if the `ipFamilies` setting is changed, gardenlet behaves differently. |
| MutableShootSpecNetworkingNodes | `gardener-apiserver` | Allows updating the field `spec.networking.nodes`. The validity of the values has to be checked in the provider extensions. Only enable this feature gate when your system runs provider extensions which have implemented the validation. |
| ShootForceDeletion | `gardener-apiserver` | Allows forceful deletion of Shoots by annotating them with the `confirmation.gardener.cloud/force-deletion` annotation. |
| UseNamespacedCloudProfile | `gardener-apiserver` | Enables usage of `NamespacedCloudProfile`s in `Shoot`s. |
| ShootManagedIssuer | `gardenlet` | Enables the shoot managed issuer functionality described in GEP 24. |
| VPAForETCD | `gardenlet`, `gardener-operator` | Enables VPA for `etcd-main` and `etcd-events`, regardless of HVPA enablement. |
| VPAAndHPAForAPIServer | `gardenlet` | Enables an autoscaling mechanism for shoot kube-apiserver where it is scaled simultaneously by VPA and HPA on the same metric (CPU and memory usage). The pod-trashing cycle between VPA and HPA scaling on the same metric is avoided by configuring the HPA to scale on average usage (not on average utilization) and by picking the target average utilization values in sync with VPA's allowed maximums. The feature gate takes precedence over the `HVPA` feature gate when they are both enabled. |
Loading