Skip to content

v0.33.0
Compare
Choose a tag to compare
@gardener-robot-ci-2 gardener-robot-ci-2 released this 11 Dec 11:15
v0.33.0
2b33ad5

[gardener]

Action Required

  • [OPERATOR] With this PR we incorporate a major architectural change, namely, the introduction of a new Gardener component: the gardenlet. (#1601, @rfranzke)
    • With previous versions of Gardener we were running the control loops for all shoot clusters and all seed clusters centrally in the garden cluster (gardener-controller-manager).
    • Now, we have split the gardener-controller-manager and factored out the control loops that are involving communication with seed and shoot clusters into the new gardenlet component.
    • The motivation was twofold, mainly to enable true scalability (beyond the capacity of a single and central controller-manager), but secondly also to allow running seed and shoot clusters in isolated networks.
    • With the gardenlet, we distribute the shoot reconciliation (mainly, but also others) control loops into all seed clusters, effectively reducing the load and responsibility of a single gardenlet.
    • Gardener's architecture is now even more comparable with the Kubernetes architecture: The Gardener control plane consists out of the gardener-apiserver, gardener-controller-manager, and gardener-scheduler, while the gardenlet is the primary agent running in every seed cluster. Take a look at this comparison diagram.
    • Unlike the kubelet, the gardenlet allows to control more than one seed cluster (although, we don't recommend this setup for production). Basically, you can even run a single gardenlet in the garden cluster controlling all the seed clusters, resulting in the same Gardener v0 architecture. The landscape operator is responsible for designing its landscape, though, for the mentioned reasons we recommend running one gardenlet per seed.
    • Please find a more detailed description here.
    • Migration from previous Gardener versions:
    • ⚠️ Be aware that the gardener Helm chart is now split into two separate Helm charts: controlplane and gardenlet. Also, some keys in the chart values have been moved around!
    • Please find the migration instructions here.
    • Removals and notable changes
    • The SeedAvailable condition does no longer exist and has been replaced by Bootstrapped and GardenletReady.
    • The spec.secretRef field in the Seed resource is now optional. It is only required in case the Seed is controlled by a Gardenlet that runs outside of the seed cluster itself.
    • The Logging and HVPA feature gates have been moved from the gardener-controller-manager to the gardenlet.
    • The Seed status does now contain a new kubernetesVersion field into which the gardenlet reports the Kubernetes version of the seed cluster.
    • The printer columns for kubectl get seeds have been reworked.
    • The gardener-controller-manager features two new controllers:
    • The seed lifecycle controller. Its main task is to set the GardenletReady condition to Unknown for Seed resources which don't receive heartbeats from the gardenlet anymore.
    • The CSR auto-approval controller watches CertificateSigningRequests and auto-approves them in case they were filed by a gardenlet.
  • [OPERATOR] All garden.sapcloud.io:... RBAC resources have been renamed to gardener.cloud:.... (#1601, @rfranzke)
  • [DEVELOPER] Developers should re-run ./hack/dev-setup-register-gardener in order to register the new core.gardener.cloud/v1beta1 API group. (#1681, @rfranzke)
  • [DEVELOPER] Developers need to run make dev-setup again, and make start-gardenlet in order to start the Gardenlet. Please find here more instructions for how to setup the local development environment. (#1601, @rfranzke)

Most notable changes

  • [USER] Every shoot cluster will now feature a shoot-info configmap in its kube-system namespace. This configmap contains some important information about the shoot cluster itself, e.g., maintenance time window, project name, etc. (#1690, @rfranzke)
  • [USER] As preparation for the final removal of the already deprecated garden.sapcloud.io/v1beta1 API group, all resources (except ShootState) available in core.gardener.cloud/v1alpha1 are now promoted to core.gardener.cloud/v1beta1 with the following changes: (#1681, @rfranzke)
    • The .spec.seed field in core.gardener.cloud/v1alpha1.BackupBucket has been renamed to .spec.seedName in core.gardener.cloud/v1beta1.BackupBucket.
    • The .spec.seed field in core.gardener.cloud/v1alpha1.BackupEntry has been renamed to .spec.seedName in core.gardener.cloud/v1beta1.BackupEntry.
    • The .spec.blockCIDRs field core.gardener.cloud/v1alpha1.Seed has been moved to .spec.networks.blockCIDRs in core.gardener.cloud/v1beta1.Seed.
    • The .spec.addons.kubernetes-dashboard field core.gardener.cloud/v1alpha1.Shoot has been renamed to .spec.addons.kubernetesDashboard in core.gardener.cloud/v1beta1.Shoot.
    • The .spec.addons.nginx-ingress field core.gardener.cloud/v1alpha1.Shoot has been renamed to .spec.addons.nginxIngress in core.gardener.cloud/v1beta1.Shoot.
    • The .status.seed field core.gardener.cloud/v1alpha1.Shoot has been renamed to .status.seedName in core.gardener.cloud/v1beta1.Shoot.
    • The .status.lastError field core.gardener.cloud/v1alpha1.Shoot does no longer exist in core.gardener.cloud/v1beta1.Shoot (in favour of .status.lastErrors).
  • [USER] It is now possible to instruct Gardener to skip certain cleanup tasks when deleting a Shoot cluster by annotating it with shoot.gardener.cloud/skip-cleanup=true. Please be careful using this as it might leave orphaned infrastructure resources. Services (of type load balancer) as well as persistent volume resources are still deleted even if this annotation is set. (#1679, @rfranzke)
  • [OPERATOR] Added dependency-watchdog as a bootstrap component. Both for watching etcd and kube-apiserver endpoints as well as for probing the shoot kube-apiserver for loadbalancer issues. (#1641, @amshuman-kr)
    • This replaces the existing dependency-watchdog in every shoot control-plane in the seed clusters. Bumped dependency-watchdog to v0.3.0.
    • Also, increased node monitor grace period to give the dependency watchdog probe a change to scale down the kube-controller-manager before the nodes are marked as NotReady.
  • [OPERATOR] It is now possible to taint seed clusters with the seed.gardener.cloud/disable-dns taint. This will cause all shoot clusters assigned to this seed to not use any DNS records for the kube-apiservers. Instead, the load balancer IP/hostname is used directly in all kubeconfigs for communication. (#1617, @rfranzke)
  • [OPERATOR] When a load balancer service does now output both .status.ingress[].hostname and .status.ingress[].ip then the provided hostname is now taken instead of the IP address. (#1617, @rfranzke)
  • [DEVELOPER] The exported variable pkg/operation/common.CalicoTyphaDeploymentName has been removed (#1712, @vpnachev)
  • [DEVELOPER] All Gardener components are still working with core.gardener.cloud/v1alpha1 but will soon switch to core.gardener.cloud/v1beta1. That means that the extensions.gardener.cloud/v1alpha1.Cluster resource will then contain the core.gardener.cloud/v1beta1 resources only. Extension controllers should be prepared to be able to work with both the v1alpha1 and the v1beta1 version. (#1681, @rfranzke)
  • [DEVELOPER] The local development setup is now easier if all seeds are tainted with the seed.gardener.cloud/disable-dns taint. No internal or default domain secrets are required in this case. (#1617, @rfranzke)
  • [DEVELOPER] The base image version for all Gardener Docker image is now alpine:3.10. (#1601, @rfranzke)

Improvements

  • [USER] An issue has been resolved which caused Shoots to be displayed as reconciled successfully instead of showing an error, in case the specified DNS provider secret is missing and the Shoot could not be reconciled or deleted. (#1706, @tim-ebert)
  • [USER] It is now possible to configure the external traffic policy for the load balancer service exposing the nginx-ingress addon by setting .spec.addons.nginx-ingress.externalTrafficPolicy. It defaults to Cluster and valid values are {Cluster,Local}. (#1701, @rfranzke)
  • [USER] The version of the nginx-ingress addon has been bumped from v0.22.0 to v0.26.1. (#1701, @rfranzke)
  • [USER] Patch strategy and patch merge keys are added to the public APIs. This allows for effective usage of kubectl patch command. (#1694, @mvladev)
  • [OPERATOR] The system-component health check for calico-typha has been removed. (#1712, @vpnachev)
  • [OPERATOR] Gardener resources, e.g. Shoots, Seeds, ControllerRegistrations, etc. can now be encrypted when the API server writes them to etcd. If you want to enable encryption for certain resources, an EncryptionConfiguration must be passed via the --encryption-provider-config flag to the Gardener-Apiserver. This is based on the Kubernetes standard encryption option which is already supported for the Kube-Apiserver (https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/). (#1707, @timuthy)
  • [OPERATOR] An issue in the shoot care controller has been fixed which caused the Gardener-Controller-Manager to crash. (#1689, @timuthy)
  • [OPERATOR] Gardener now deploys network policies into the kube-system namespace of the shoot to guarantee that the blackbox-exporter component can communicate with the control plane. (#1688, @wyb1)
  • [OPERATOR] An issue preventing Shoot with .spec.kubernetes.allowPrivilegedContainers=false to be created is now fixed. (#1686, @ialidzhikov)
  • [OPERATOR] Server certificates for Grafana (operator / user), Prometheus and Kibana endpoints of Shoots are now created with a validity of 2 years (unlike 10 years previously). (#1685, @timuthy)
  • [OPERATOR] A bug has been fixed which caused the Grafana ingresses of shoots to serve the Kubernetes Fake Certificate instead of a certificate signed by the cluster CA. (#1685, @timuthy)
  • [OPERATOR] Adds error handling when HVPA CRD is not already installed, but a delete operation is attempted (#1678, @ggaurav10)
  • [OPERATOR] The legacy storageclasses ManagedResource is no longer deleted during DeployManagedResources step. (#1677, @ialidzhikov)
  • [OPERATOR] ControllerInstallation controller does now uses ManagedResources. (#1646, @ialidzhikov)
  • [OPERATOR] Added ShootState resource which is used to save the state of Shoot control plane resources necessary for control plane migration. (#1634, @plkokanov)
  • [DEVELOPER] sigs.k8s.io/controller-runtime is updated to v0.2.2. (#1700, @ialidzhikov)

[dependency-watchdog]

Most notable changes

  • [USER] Fixed the command-line incompatibility for the root command introduced in the release 0.3.0. (gardener/dependency-watchdog#8, @amshuman-kr)
  • [USER] Introduced cobra commands. (gardener/dependency-watchdog#7, @amshuman-kr)
      1. The root command works exactly like before.
    • I.e. it watches the endpoint objects and kicks the
    • dependant pods in CrashloopBackoff.
      1. The probe sub-command probes kube-apiservers
    • using internal and external IP kubeconfigs and
    • scales the dependant scale subresources up and down.
    • Both the root command and the probe sub-command support
    • managing a single namespace as well as all namespaces.

[gardener-resource-manager]

Improvements

Docker Images

gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v0.33.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v0.33.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v0.33.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v0.33.0

Assets 3