v1.2.1

Check out the changelog for the v1.2.0 release for more information about what changes were introduced in the 1.2 release.

Changed

Bug Fixes

• Install cri-tools (crictl) on Amazon Linux 2. This fixes the issue with provisioning Kubernetes and Amazon EKS-D clusters on Amazon Linux 2 (#1282)

Checksums

SHA256 checksums can be found in the kubeone_1.2.0-rc.1_checksums.txt file.

v1.2.0

KubeOne v1.2.0

The latest KubeOne 1.2.0 release is focused on community driven improvements and paving the way for future releases that incorporate even more features. We have been adding quite some alpha level features that will be improved and graduated in the future releases.

Please work through the Attention Needed section carefully and for more details read the v1.2.0 changelog.

Major Highlights

Add support for Kubernetes 1.20

Ongoing support for the latest Kubernetes version will give users access to the latest features and improvements. The Kubernetes 1.20 brings many new features, improvements, and fixes. We recommend checking out the official announcement for more details about the latest release.

containerd support

As of Kubernetes 1.20, Dockershim — a component that connects Kubelet and Docker, is deprecated. Starting with Kubernetes 1.23, it’ll not be possible to use Docker on Kubernetes nodes. Instead, a Container Runtime Interface (CRI) compatible container runtime must be used.

Containerd is one of CRI-compatible runtimes, and starting with this KubeOne release, it’s possible to provision clusters using containerd. We’ll also provide a migration path for existing clusters created by KubeOne in one of the upcoming releases.

Please check the Attention Needed section for more details about the upcoming changes related to the container runtimes.

Attention Needed

• [BREAKING/ACTION REQUIRED] Starting with the KubeOne 1.3 release, the kubeone reset command will require an explicit confirmation like the apply command
  • Running the reset command will require typing yes to confirm the intention to unprovision/reset the cluster
  • The command can be automatically approved by using the --auto-approve flag
  • The --auto-approve flag has been already implemented as a no-op flag in this release
  • Starting with this release, running kubeone reset will show a warning about this change each time the reset command is used
• [BREAKING/ACTION REQUIRED] Disallow and deprecate the PodPresets feature
  • If you're upgrading a cluster that uses the PodPresets feature from Kubernetes 1.19 to 1.20, you have to disable the PodPresets feature in the KubeOne configuration manifest
  • The PodPresets feature has been removed from Kubernetes 1.20 with no built-in replacement
  • It's not possible to use the PodPresets feature starting with Kubernetes 1.20, however, it currently remains possible to use it for older Kubernetes versions
  • The PodPresets feature will be removed from the KubeOneCluster API once Kubernetes 1.19 reaches End-of-Life (EOL)
  • As an alternative to the PodPresets feature, Kubernetes recommends using the MutatingAdmissionWebhooks.
• [BREAKING/ACTION REQUIRED] Support for CoreOS has been removed from KubeOne and machine-controller
  • CoreOS has reached End-of-Life on May 26, 2020
  • As an alternative to CoreOS, KubeOne supports Flatcar Linux
  • We recommend migrating your CoreOS clusters to the Flatcar Linux or other supported operating system
• [BREAKING/ACTION REQUIRED] Default values for OpenIDConnect has been corrected to match what's advised by the example configuration
  • Previously, there were no default values for the OpenIDConnect fields
  • This might only affect users using the OpenIDConnect feature
• Kubernetes has announced deprecation of the Docker (dockershim) support in the Kubernetes 1.20 release. It's expected that Docker support will be removed in Kubernetes 1.22 or 1.23
  • All newly created clusters running Kubernetes 1.21+ will be provisioned with containerd instead of Docker
  • Automated migration from Docker to containerd is currently not available, but is planned for one of the upcoming KubeOne releases
  • We highly recommend using containerd instead of Docker for all newly created clusters. You can opt-in to use containerd instead of Docker by adding containerRuntime configuration to your KubeOne configuration manifest:

    containerRuntime:
      containerd: {}

    For the configuration file reference, run kubeone config print --full.

Known Issues

• Provisioning a Kubernetes or Amazon EKS-D cluster on Amazon Linux 2 will fail due to missing crictl binary. This bug has been fixed in the v1.2.1 release.
• Upgrading an Amazon EKS-D cluster will fail due to kubeadm preflight checks failing. We're investigating the issue and you can follow the progress by checking the issue #1284.

Changelog since v1.2.0-rc.1

There have been no changes since the KubeOne v1.2.0-rc.1 release.

Changelog since v1.1.0

For the complete changelog since the KubeOne v1.1.0 release, please check the CHANGELOG document.

Checksums

SHA256 checksums can be found in the kubeone_1.2.0_checksums.txt file.

v1.2.0-rc.1

Changed

General

• Build KubeOne using Go 1.16.1 (#1268, #1267)

Checksums

SHA256 checksums can be found in the kubeone_1.2.0-rc.1_checksums.txt file.

v1.2.0-rc.0

Attention Needed

• [BREAKING/ACTION REQUIRED] Starting with the KubeOne 1.3 release, the kubeone reset command will require an explicit confirmation like the apply command
  • Running the reset command will require typing yes to confirm the intention to unprovision/reset the cluster
  • The command can be automatically approved by using the --auto-approve flag
  • The --auto-approve flag has been already implemented as a no-op flag in this release
  • Starting with this release, running kubeone reset will show a warning about this change each time the reset command is used

Changed

General

• Warn about kubeone reset requiring explicit confirmation starting with KubeOne 1.3 (#1252)

Checksums

SHA256 checksums can be found in the kubeone_1.2.0-rc.0_checksums.txt file.

v1.2.0-beta.1

Attention Needed

• [Breaking] Support for CoreOS has been removed from KubeOne and machine-controller
  • CoreOS has reached End-of-Life on May 26, 2020
  • As an alternative to CoreOS, KubeOne supports Flatcar Linux
  • We recommend migrating your CoreOS clusters to the Flatcar Linux or other supported operating system
• [Breaking] Default values for OpenIDConnect has been corrected to match what's advised by the example configuration
  • Previously, there were no default values for the OpenIDConnect fields
  • This might only affect users using the OpenIDConnect feature
• [Breaking] Disallow and deprecate the PodPresets feature
  • [Action Required] If you're upgrading a cluster that uses the PodPresets feature from Kubernetes 1.19 to 1.20, you have to disable the PodPresets feature in the KubeOne configuration manifest
  • The PodPresets feature has been removed from Kubernetes 1.20 with no built-in replacement
  • It's not possible to use the PodPresets feature starting with Kubernetes 1.20, however, it currently remains possible to use it for older Kubernetes versions
  • The PodPresets feature will be removed from the KubeOneCluster API once Kubernetes 1.19 reaches End-of-Life (EOL)
  • As an alternative to the PodPresets feature, Kubernetes recommends using the MutatingAdmissionWebhooks.

Added

• Add support for Kubernetes 1.20
  • Previously, we've shared that there is an issue affecting newly created clusters where the first control plane node is unhealthy/broken for the first 5-10 minutes. We've investigated the issue and found out that the issue can be successfully mitigated by restarting the first API server. We've implemented a task that automatically restarts the API server if it's affected by the issue (#1243, #1245)
• Add support for Debian on control plane and static worker nodes (#1233)
  • Debian is currently not supported by machine-controller, so it's not possible to use it on worker nodes managed by machine-controller

Changed

API Changes

• [Breaking] Default values for OpenIDConnect has been corrected to match what's advised by the example configuration (#1235)
  • Previously, there were no default values for the OpenIDConnect fields
  • This might only affect users using the OpenIDConnect feature
• [Breaking] Disallow and deprecate the PodPresets feature (#1236)
  • [Action Required] If you're upgrading a cluster that uses the PodPresets feature from Kubernetes 1.19 to 1.20, you have to disable the PodPresets feature in the KubeOne configuration manifest
  • The PodPresets feature has been removed from Kubernetes 1.20 with no built-in replacement
  • It's not possible to use the PodPresets feature starting with Kubernetes 1.20, however, it currently remains possible to use it for older Kubernetes versions
  • The PodPresets feature will be removed from the KubeOneCluster API once Kubernetes 1.19 reaches End-of-Life (EOL)
  • As an alternative to the PodPresets feature, Kubernetes recommends using the MutatingAdmissionWebhooks.

General

• Add rsync on CentOS and Amazon Linux (#1240)

Bug Fixes

• Drop mounting Flexvolume plugins into the OpenStack CCM. This fixes the issue with deploying the OpenStack CCM on the clusters running Flatcar Linux (#1234)
• Ensure all credentials are available to be used in addons. This fixes the issue with the Backups addon not working on non-AWS providers (#1248)

Updated

• Update machine-controller to v1.25.0 (#1238)

Removed

• [Breaking] Support for CoreOS has been removed from KubeOne and machine-controller (#1232)
  • CoreOS has reached End-of-Life on May 26, 2020
  • As an alternative to CoreOS, KubeOne supports Flatcar Linux
  • We recommend migrating your CoreOS clusters to the Flatcar Linux or other supported operating system

Checksums

SHA256 checksums can be found in the kubeone_1.2.0-beta.1_checksums.txt file.

v1.2.0-beta.0

Attention Needed

• Kubernetes has announced deprecation of the Docker (dockershim) support in the Kubernetes 1.20 release. It's expected that Docker support will be removed in Kubernetes 1.22
  • All newly created clusters running Kubernetes 1.21+ will be provisioned with containerd instead of Docker
  • Automated migration from Docker to containerd is currently not available, but is planned for one of the upcoming KubeOne releases
  • We highly recommend using containerd instead of Docker for all newly created clusters. You can opt-in to use containerd instead of Docker by adding containerRuntime configuration to your KubeOne configuration manifest:

    containerRuntime:
      containerd: {}

    For the configuration file reference, run kubeone config print --full.

Known Issues

• Provisioning Kubernetes 1.20 clusters results with one of the control plane nodes being unhealthy/broken for the first 5-10 minutes after provisioning the cluster. This causes KubeOne to fail to create MachineDeployment objects because the machine-controller-webhook service can't be found. Also, one of the NodeLocalDNS pods might get stuck in the crash loop.
  • KubeOne currently still doesn't support Kubernetes 1.20. We do not recommend provisioning 1.20 clusters or upgrading existing clusters to Kubernetes 1.20
  • We're currently investigating the issue. You can follow the progress in the issue #1222

Added

• Add support for containerd container runtime (#1180, #1188, #1190, #1205, #1227, #1229)
  • Kubernetes has announced deprecation of the Docker (dockershim) support in the Kubernetes 1.20 release. It's expected that Docker support will be removed in Kubernetes 1.22
  • All newly created clusters running Kubernetes 1.21+ will default to containerd instead of Docker
  • Automated migration from Docker to containerd is currently not available, but is planned for one of the upcoming KubeOne releases

Changed

Bug Fixes

• Fix wrong legacy Docker version on RPM systems (#1191)

Terraform Configs

• Replace GoBetween load-balancer in vSphere Terraform example by keepalived (#1217)

Addons

• Fix DNS resolution issues for the Backups addon (#1179)

Checksums

SHA256 checksums can be found in the kubeone_1.2.0-beta.0_checksums.txt file.

v1.2.0-alpha.0

Added

• Add support for Amazon Linux 2 (#1167, #1173, #1175, #1176)
  • Support for Amazon Linux 2 is currently in alpha.
  • Currently, all Kubernetes packages are installed by downloading binaries instead of using packages. Therefore, users are required to provide URLs using the new AssetConfiguration API to the CNI tarball, the Kubernetes Node binaries tarball (can be found in the Kubernetes CHANGELOG), and to the kubectl binary. Support for packages is planned for the future.
• Add the AssetConfiguration API (#1170, #1171)
  • The AssetConfiguration API controls how assets are pulled.
  • You can use it to specify custom images for containers or custom URLs for binaries.
  • Currently-supported assets are CNI, Kubelet and Kubeadm (by providing a node binaries tarball), Kubectl, the control plane images, and the metrics-server image.
  • Changing the binary assets (CNI, Kubelet, Kubeadm and Kubectl) currently works only on Amazon Linux 2. Changing the image assets works on all supported operating systems.
• Add Annotations field to the ProviderSpec API used to add annotations to MachineDeployment objects (#1174)
• Support for defining Static Worker nodes in Terraform (#1166)
• Add scrape Prometheus headless service for NodeLocalDNS (#1165)

Changed

General

• Stop Kubelet and reload systemd when removing binaries on CoreOS/Flatcar (#1176)

Updated

• Update Calico CNI to v3.16.5 (#1163)

Checksums

SHA256 checksums can be found in the kubeone_1.2.0-alpha.0_checksums.txt file.

v1.1.0

Attention Needed

• [Breaking] Use Ubuntu 20.04 (Focal) in the example Hetzner Terraform config (#1102)
  • It's highly recommended to bind the image by setting var.image to the image you're currently using to prevent the instances from being recreated the next time you run Terraform!

Changelog since v1.0.0-rc.1

The changelog since the v1.0.5 release is available in the CHANGELOG.md file.

Added

General

• Improvements to the RegistryConfiguration API
  • Add support for insecure Docker registries (#1155)
  • Overwrite machine-controller Hyperkube and Kubelet image repositories (#1157)
  • Overwrite pause image on worker nodes (#1161)
• Add external cloud controller manager support for VMware vSphere clusters (#1159)

Changed

Updated

• Update WeaveNet to v2.7.0 (#1153)
• Update Hetzner Cloud Controller Manager (CCM) to v1.8.1 (#1149)
  • This CCM release includes support for external LoadBalacners backed by Hetzner LoadBalancers

Checksums

SHA256 checksums can be found in the kubeone_1.1.0_checksums.txt file.

v1.1.0-rc.0

Changelog since v1.0.5.

Attention Needed

• [Breaking] Use Ubuntu 20.04 (Focal) in the example Hetzner Terraform config (#1102)
  • It's highly recommended to bind the image by setting var.image to the image you're currently using to prevent the instances from being recreated the next time you run Terraform!

Added

General

• Implement the OverwriteRegistry functionality (#1145)
  • This PR adds a new top-level API field registryConfiguration which controls how images used for components deployed by KubeOne and kubeadm are pulled from an image registry.
  • The registryConfiguration.overwriteRegisty field can be used to specify a custom Docker registry to be used instead of the default one.
  • For example, if registryConfiguration.overwriteRegisty is set to 127.0.0.1:5000, image called k8s.gcr.io/kube-apiserver:v1.19.3 would become 127.0.0.1:5000/kube-apiserver:v1.19.3.
  • Setting registryConfiguration.overwriteRegisty applies to all images deployed by KubeOne and kubeadm, including addons deployed by KubeOne.
  • You can run kubeone config print -f for more details regarding the RegistryConfiguration API.

Addons

• Add the cluster-autoscaler addon (#1103)

Changed

Bug Fixes

• Explicitly restart Kubelet when upgrading clusters running on Ubuntu (#1098)
• Merge CloudProvider structs instead of overriding them when the cloud provider is defined via Terraform (#1108)

Updated

• Update Flannel to v0.13.0 (#1135)
• Update Hetzner Cloud Controller Manager (CCM) to v1.7.0 (#1068)
  • This CCM release includes support for external LoadBalacners backed by Hetzner LoadBalancers

Terraform Configs

• [Breaking] Use Ubuntu 20.04 (Focal) in the example Hetzner Terraform config (#1102)
  • It's highly recommended to bind the image by setting var.image to the image you're currently using to prevent the instances from being recreated the next time you run Terraform!
• Ensure the example Hetzner Terraform config support both Terraform v0.12 and v0.13 (#1102)
• Update Azure example Terraform config to work with the latest versions of the Azure provider (#1059)
• Use Hetzner Load Balancers instead of GoBetween in the example Hetzner Terraform config (#1066)

v1.0.5

Changed

Updated

• Update machine-controller to v1.19.0 (#1141)
  • This machine-controller release uses the Hyperkube Kubelet image for Flatcar worker nodes running Kubernetes 1.18, as the Poseidon Kubelet image repository doesn't publish 1.18 images any longer. This change ensures that you can provision or upgrade to Kubernetes 1.18.8+ on Flatcar.

Checksums

SHA256 checksums can be found in the kubeone_1.0.5_checksums.txt file.