Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove in-tree support for ControlPlane and rely on extension controllers #1076

Merged
merged 1 commit into from Jun 25, 2019

Conversation

Projects
None yet
3 participants
@stoyanr
Copy link
Contributor

commented Jun 10, 2019

What this PR does / why we need it:
Modifies the Gardener shoot reconciliation flow to write a ControlPlane resource to the shoot namespace and rely on controlplane controllers and webhooks to customize the control plane.

Which issue(s) fixes this PR?:
Fixes #881

Special notes for your reviewer:

  • All provider-specific charts and logic have been removed from seed-controlplane and seed-operatingsystemconfig.
  • The corresponding logic in HybridBotanist has been removed or adapted. All CloudBotanist methods that are no longer needed have been removed as well.
  • Corresponding migration code has been added for all providers (currently except Packet).
  • Provider-specific charts and logic in shoot-core have not been removed due to gardener/gardener-extensions#111.
  • The shoot deletion flow has been modified as well.

The following gardener-extensions issues should be fixed before this PR could be merged with full feature parity:

Release note:

:warning: Gardener does no longer have in-tree supported for managing provider-specific control plane configuration and deployments of additional provider-specific components (like cloud-controller-manager or CSI controllers). Instead, it now relies on extension controllers to inject the provider-specific configuration via mutating webhooks and to deploy the respective cloud-controller-manager (and cloud-provider-config) for the specific provider they have been developed for, e.g. AWS, Azure, GCP, etc. (see [GEP-1 on extensibility](https://github.com/gardener/gardener/blob/master/docs/proposals/01-extensibility.md)). We have prepared the implementation of the [Alicloud provider](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-alicloud), [AWS provider](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-aws), [Azure provider](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-azure), [GCP provider](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-gcp), [OpenStack provider](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-openstack), and [Packet provider](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-packet). After updating Gardener you need to update your existing `ControllerRegistration` resources to make these extension controllers known to Gardener (otherwise none of your shoots will be able to get reconciled anymore). You should already have `ControllerRegistration` resources for various providers. Add `.spec.resources[].kind="ControlPlane"` and `.spec.resources[].type="<provider-name>"` to it. Please find example `ControllerRegistration` resources [here (Alicloud)](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-alicloud/example/controller-registration.yaml), [here (AWS)](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-aws/example/controller-registration.yaml), [here (Azure)](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-azure/example/controller-registration.yaml), [here (GCP)](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-gcp/example/controller-registration.yaml), [here (OpenStack)](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-openstack/example/controller-registration.yaml), and [here (Packet)](https://github.com/gardener/gardener-extensions/blob/0.7.0/controllers/provider-packet/example/controller-registration.yaml). It is recommended to use more than one replica as the control plane extensions use webhooks to inject configuration into the standard control plane, and you want the webhook to be highly available. To get information about more details please walk through [these documents](https://github.com/gardener/gardener/tree/master/docs/extensions).
Due to the introduction of the control plane extension controllers the `etcd`s, `kube-apiserver`s, `cloud-controller-manager`s, and `csi-{*}` controllers of existing shoots will be restarted. The reason for this is that the extension controllers use another mechanism to compute the checksum of mounted secrets and configmaps.

@stoyanr stoyanr requested a review from gardener/gardener-maintainers as a code owner Jun 10, 2019

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch from f578c62 to 5ec87e3 Jun 10, 2019

@rfranzke rfranzke changed the title Write ControlPlane resource and rely on controlplane controller and webhooks [WIP] Write ControlPlane resource and rely on controlplane controller and webhooks Jun 11, 2019

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch 5 times, most recently from 7e1f663 to b5e78da Jun 12, 2019

@rfranzke
Copy link
Member

left a comment

Great work! I found a few things here and there, PTAL.

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch 3 times, most recently from fff3ec8 to d74cf43 Jun 14, 2019

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch 2 times, most recently from b3c01f0 to d2785cc Jun 14, 2019

@rfranzke rfranzke changed the title [WIP] Write ControlPlane resource and rely on controlplane controller and webhooks [WIP] Remove in-tree support for ControlPlane and rely on extension controllers Jun 15, 2019

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch from d2785cc to b16d881 Jun 18, 2019

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch 2 times, most recently from 6fa3795 to fb39abc Jun 19, 2019

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch from fb39abc to 4f06ce5 Jun 20, 2019

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch from 4f06ce5 to 894038e Jun 21, 2019

@rfranzke rfranzke changed the title [WIP] Remove in-tree support for ControlPlane and rely on extension controllers Remove in-tree support for ControlPlane and rely on extension controllers Jun 24, 2019

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch from 894038e to 917791e Jun 24, 2019

Show resolved Hide resolved charts/images.yaml
Show resolved Hide resolved pkg/operation/botanist/botanist_suite_test.go Outdated
Show resolved Hide resolved pkg/operation/botanist/cleanup.go Outdated
Show resolved Hide resolved pkg/operation/botanist/controlplane.go Outdated
Show resolved Hide resolved pkg/operation/hybridbotanist/controlplane.go

@stoyanr stoyanr force-pushed the stoyanr:feature/extension-controlplane branch from 917791e to cbb86ab Jun 24, 2019

@rfranzke

This comment has been minimized.

Copy link
Member

commented Jun 24, 2019

Can you add a cp alias and a new printer column for .status.lastOperation.state to https://github.com/gardener/gardener/blob/master/charts/seed-bootstrap/templates/extensions/crd-controlplane.yaml (like in all other extension CRDs)?

@rfranzke rfranzke referenced this pull request Jun 24, 2019

Closed

Implement `ControlPlane` extension resource #881

17 of 17 tasks complete

@rfranzke rfranzke force-pushed the stoyanr:feature/extension-controlplane branch from cbb86ab to 8066956 Jun 25, 2019

@rfranzke
Copy link
Member

left a comment

/lgtm

@rfranzke rfranzke merged commit a213166 into gardener:master Jun 25, 2019

4 checks passed

concourse-ci/check Concourse CI build success
Details
concourse-ci/publish Concourse CI build success
Details
concourse-ci/test Concourse CI build success
Details
license/cla Contributor License Agreement is signed.
Details

@stoyanr stoyanr deleted the stoyanr:feature/extension-controlplane branch Jun 25, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.