Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add blog post for in-tree cloud providers going beta
- Loading branch information
Showing
1 changed file
with
101 additions
and
0 deletions.
There are no files selected for viewing
101 changes: 101 additions & 0 deletions
101
content/en/blog/_posts/2023-12-20-disabling-in-tree-cloud-provider-goes-beta.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| --- | ||
| layout: blog | ||
| title: "Removing In-Tree Cloud Providers Graduates to Beta Status" | ||
| date: 2023-12-20 | ||
| slug: removing-in-tree-cloud-providers-beta-status | ||
| --- | ||
|
|
||
| **Authors:** Michael McCune, Andrew Sy Kim | ||
|
|
||
| In 2018, the Kubernetes community agreed to form the Cloud Provider Special Interest Group (SIG), with a mission to externalize all cloud provider integrations and remove all the existing in-tree cloud provider integrations. In January 2019, the Kubernetes community approved the initial draft of [KEP-2395: Removing In-Tree Cloud Provider Code][kep2395]. This KEP defines a process by which we can remove cloud provider specific code from the core Kubernetes source tree. From the KEP: | ||
|
|
||
| > Motiviation behind this effort is to allow cloud providers to develop and make releases independent from the core Kubernetes release cycle. The de-coupling of cloud provider code allows for separation of concern between "Kubernetes core" and the cloud providers within the ecosystem. In addition, this ensures all cloud providers in the ecosystem are integrating with Kubernetes in a consistent and extendable way. | ||
| After many years of development and collaboration across many contributors, SIG Cloud Provider is happy to announce that as of the Kubernetes 1.29 release, [KEP-2395][kep2395] will graduate to Beta status. The graduation to Beta is a big milestone as we’ve had to work through the complexities that come with migrating a core component in a distributed system like Kubernetes. With the graduation to Beta, all Kubernetes core components will have cloud provider integrations disabled by default. This brings us one step closer to its final removal which will happen when the KEP graduates to GA. | ||
|
|
||
| Although the new defaults will affect all Kubernetes users, they will require changes for users who are currently using the in-tree providers for Azure, GCE, and vSphere. | ||
|
|
||
| ## What is changing? | ||
|
|
||
| At the most basic level, two [feature gates][fg] are changing their default value from false to true. Those feature gates, `DisableCloudProviders` and `DisableKubeletCloudCredentialProviders`, control the way that the [kube-apiserver][kapi], [kube-controller-manager][kcm], and [kubelet][kubelet] invoke the cloud provider related code that is included in those components. With these feature gates defaulting to true, the former behavior that was provided through the `--cloud-provider` command line flag is changing to only recognize the `external` value. | ||
|
|
||
| Let's see what the [official Kubernetes documentation][fg] says about these feature gates: | ||
|
|
||
| > `DisableCloudProviders`: Disables any functionality in `kube-apiserver`, `kube-controller-manager` and `kubelet` related to the `--cloud-provider` component flag. | ||
| > `DisableKubeletCloudCredentialProviders` Disable the in-tree functionality in kubelet to authenticate to a cloud provider container registry for image pull credentials. | ||
| ## What do you need to do? | ||
|
|
||
| If you are upgrading from Kubernetes 1.28+ and are not on Azure, GCE, or vSphere then you are in luck, there are no changes you will need to make. If you **are** on Azure, GCE, or vSphere, or you are upgrading from a version older than 1.28, then read on. | ||
|
|
||
| Historically, Kubernetes has included code for a set of cloud providers that included AWS, Azure, GCE, OpenStack, and vSphere. Since the inception of [KEP-2395][kep2395] the community has been moving towards removal of that cloud provider code. The OpenStack provider code was removed in version 1.26, and the AWS provider code was removed in version 1.27. This means that users who are upgradging from one of the affected cloud providers and versions will need to modify their deployments. | ||
|
|
||
| ### Upgrading on Azure, GCE, or vSphere | ||
|
|
||
| There are two options for upgrading in this configuration: migrate to external cloud controller managers, or continue using the in-tree provider code. Although migrating to external cloud controller managers is recommended, there are scenarios where continuing with the current behavior is desired. Please choose the best option for your needs. | ||
|
|
||
| #### Migrate to external cloud controller managers | ||
|
|
||
| Migrating to use external cloud controller managers is the recommended upgrade path as it aligns with the future of Kubernetes. To do this you will need to enable the `--cloud-provider=external` command line flag for the `kube-apiserver`, `kube-controller-manager`, and `kubelet` components. In addition you will need to deploy a cloud controller manager for your provider. | ||
|
|
||
| Installing and running cloud controller managers is a larger topic than this post can address, if you would like more information on this process please read the documentation for [Cloud Controller Manager Administration][ccmadmin] and [Migrate Replicated Control Plane To Use Cloud Controller Manager][ccmha]. See below for links to specific cloud provider implementations. | ||
|
|
||
| #### Continue using in-tree provide code | ||
|
|
||
| If you wish to continue using Kubernetes with the in-tree cloud provider code, you will need to modify the command line parameters for `kube-apiserver`, `kube-controller-manager`, and `kubelet` to disable the feature gates for `DisableCloudProviders` and `DisableKubeletCloudCredentialProviders`. To do this, add the following command line flag to the arguments for the previously listed commands: | ||
|
|
||
| ``` | ||
| --feature-gates=DisableCloudProviders=false,DisableKubeletCloudCredentialProviders=false | ||
| ``` | ||
|
|
||
| _Please note that if you have other feature gate modifications on the command line, they will need to include these 2 feature gates._ | ||
|
|
||
| ### Upgrading on other providers | ||
|
|
||
| For providers other than Azure, GCE, or vSphere, good news, the external cloud controller manager should be already be in use. The code for AWS and OpenStack providers has been removed from Kubernetes before version 1.27 was released. Other providers beyond the 5 listed in this post were never included in Kubernetes and as such they began their life as external cloud controller managers. | ||
|
|
||
| ### Upgrading from older Kubernetes versions | ||
|
|
||
| If you are upgrading from a Kubernetes release older than 1.26, and you are on AWS, Azure, GCE, OpenStack, or vSphere then you will need to enable the `--cloud-provider=external` flag, and follow the advice for installing and running a cloud controller manager for your provider. | ||
|
|
||
| Please read the documentation for [Cloud Controller Manager Administration][ccmadmin] and [Migrate Replicated Control Plane To Use Cloud Controller Manager][ccmha]. See below for links to specific cloud provider implementations. | ||
|
|
||
| ## Where to find a cloud controller manager? | ||
|
|
||
| At its core, this post is about the cloud controllers that were previously included in Kubernetes. As these components move out of the core Kubernetes code and into their own repositories it is important to note a few things. | ||
|
|
||
| First, SIG Cloud Provider maintains a reference framework for developers who wish to create cloud controller managers for any provider. See the [cloud-provider repository][cloud-provider] for more information about how these controllers work and how to get started creating your own. | ||
|
|
||
| Second, there are many cloud controller managers available for Kubernetes. This post is addressing the providers that have been historically included with Kubernetes but are now in the process of being removed. If you need a cloud controller manager for your provider and do not see it listed here, please reach out to the SIG Cloud Provider community (links below) for help and advice. | ||
|
|
||
| ### Cloud Controller Managers | ||
|
|
||
| * AWS - https://github.com/kubernetes/cloud-provider-aws | ||
| * Azure - https://github.com/kubernetes-sigs/cloud-provider-azure | ||
| * GCE - https://github.com/kubernetes/cloud-provider-gcp | ||
| * OpenStack - https://github.com/kubernetes/cloud-provider-openstack | ||
| * vSphere - https://github.com/kubernetes/cloud-provider-vsphere | ||
|
|
||
| ## Want to learn more? | ||
|
|
||
| Cloud providers and cloud controller managers serve a core function in Kubernetes. Cloud providers are often the substrate upon which Kubernetes is operated, and the cloud controller managers supply the essential lifeline between Kubernetes clusters and their physical infrastructure. | ||
|
|
||
| This post exposes one aspect of how the Kubernetes community interacts with the world of cloud infrastructure providers. If you are curious about this topic and want to learn more the Cloud Provider Special Interest Group (SIG) is the place to go. SIG Cloud Provider hosts bi-weekly meetings to discuss all manner of topics related to cloud providers and cloud controller managers in Kubernetes. | ||
|
|
||
| ### SIG Cloud Provider | ||
|
|
||
| * Regular SIG Meeting: [Wednesdays at 9:00 PT (Pacific Time)](https://zoom.us/j/508079177?pwd=ZmEvMksxdTFTc0N1eXFLRm91QUlyUT09) (biweekly). [Convert to your timezone](http://www.thetimezoneconverter.com/?t=9:00&tz=PT%20%28Pacific%20Time%29). | ||
| * [Kubernetes slack][kslack] channel `#sig-cloud-provider` | ||
| * [SIG Community page][sig] | ||
|
|
||
| [kep2395]: https://github.com/kubernetes/enhancements/tree/master/keps/sig-cloud-provider/2395-removing-in-tree-cloud-providers | ||
| [fg]: https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/ | ||
| [kubelet]: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ | ||
| [kcm]: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/ | ||
| [kapi]: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/ | ||
| [ccmadmin]: https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/ | ||
| [ccmha]: https://kubernetes.io/docs/tasks/administer-cluster/controller-manager-leader-migration/ | ||
| [kslack]: https://kubernetes.slack.com | ||
| [sig]: https://github.com/kubernetes/community/tree/master/sig-cloud-provider | ||
| [cloud-provider]: https://github.com/kubernetes/cloud-provider |