From d12bdfb0e7da56458e84a2b4ef9f01fe40904619 Mon Sep 17 00:00:00 2001 From: abrennan Date: Tue, 12 May 2020 15:58:56 -0500 Subject: [PATCH 1/6] [WIP] Added HA details --- docs/README.md | 1 + docs/config-ha.md | 55 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+) create mode 100644 docs/config-ha.md diff --git a/docs/README.md b/docs/README.md index 4791a9658f2..8ad1dad32bb 100644 --- a/docs/README.md +++ b/docs/README.md @@ -85,6 +85,7 @@ Follow the links below to learn more about Knative. - [Using a custom domain](./serving/using-a-custom-domain.md) - [Assigning a static IP address for Knative on Google Kubernetes Engine](./serving/gke-assigning-static-ip-address.md) - [Configuring HTTPS with a custom certificate](./serving/using-a-tls-cert.md) +- [Configuring high availability](./config-ha.md) ### Samples and demos diff --git a/docs/config-ha.md b/docs/config-ha.md new file mode 100644 index 00000000000..031014087bd --- /dev/null +++ b/docs/config-ha.md @@ -0,0 +1,55 @@ +--- +title: "Configuring high-availability components" +linkTitle: "Configuring high-availability components" +weight: 20 +type: "docs" +--- + +Active/passive high availability (HA) is a standard feature of Kubernetes APIs that helps to ensure that APIs stay operational if a disruption occurs. In an HA deployment, if an active controller crashes or is deleted, another controller is available to take over processing of the APIs that were being serviced by the controller that is now unavailable. + +Active/passive HA in Knative is available through leader election, which can be enabled after Knative Serving control plane is installed. + +When using a leader election HA pattern, instances of controllers are already scheduled and running inside the cluster before they are required. These controller instances compete to use a shared resource, known as the leader election lock. The instance of the controller that has access to the leader election lock resource at any given time is referred to as the leader. + +HA functionality is available on Knative for the `autoscaler-hpa`, `controller` and `activator` components. + +## Enabling leader election + +1. Enable leader election for the control plane controllers: +``` +$ kubectl patch configmap/config-leader-election \ + --namespace knative-serving \ + --type merge \ + --patch '{"data":{"enabledComponents": "controller,hpaautoscaler,certcontroller,istiocontroller,nscontroller"}}' +``` + +1. Restart the controllers: +``` + $ kubectl rollout restart deployment -n knative-serving +``` + + **NOTE:** You will experience temporary control plane downtime during this step. + + When your controllers come back up, they should be running as leader-elected. + At this point, we've configured the controllers to use leader election and we + can scale the control plane up! + +1. After the controllers have been configured to use leader election, the control plane can be scaled up: +``` +$ kubectl rollouts restart deployment -n knative-serving controller +``` + +## Scaling the control plane + +The following serving controller deployments can be scaled up once leader election is enabled: + +- `controller` +- `autoscaler-hpa` +- `networking-certmanager` +- `networking-istio` (if Istio is installed) +- `networking-ns-cert` + +Scale up the deployment(s): +``` +$ kubectl scale --replicas=2 +``` From 29c4fa8a7db5737b58cbe050d17744b2b263c462 Mon Sep 17 00:00:00 2001 From: abrennan Date: Fri, 5 Jun 2020 11:49:10 -0500 Subject: [PATCH 2/6] updates from review feedback --- docs/{ => serving}/config-ha.md | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-) rename docs/{ => serving}/config-ha.md (75%) diff --git a/docs/config-ha.md b/docs/serving/config-ha.md similarity index 75% rename from docs/config-ha.md rename to docs/serving/config-ha.md index 031014087bd..9677e6e73a5 100644 --- a/docs/config-ha.md +++ b/docs/serving/config-ha.md @@ -11,10 +11,23 @@ Active/passive HA in Knative is available through leader election, which can be When using a leader election HA pattern, instances of controllers are already scheduled and running inside the cluster before they are required. These controller instances compete to use a shared resource, known as the leader election lock. The instance of the controller that has access to the leader election lock resource at any given time is referred to as the leader. -HA functionality is available on Knative for the `autoscaler-hpa`, `controller` and `activator` components. +HA functionality is available on Knative for the following components: + +- `autoscaler-hpa` +- `controller` +- `activator` + +HA functionality is not currently available for the following components: + +- `autoscaler` +- `webhook` +- `queueproxy` +- `net-kourier` ## Enabling leader election +**NOTE:** Leader election functionality is still an alpha phase feature currently in development. + 1. Enable leader election for the control plane controllers: ``` $ kubectl patch configmap/config-leader-election \ @@ -41,15 +54,24 @@ $ kubectl rollouts restart deployment -n knative-serving controller ## Scaling the control plane -The following serving controller deployments can be scaled up once leader election is enabled: +The following serving controller deployments can be scaled up once leader election is enabled. + +Standard deployments: - `controller` -- `autoscaler-hpa` -- `networking-certmanager` - `networking-istio` (if Istio is installed) + +Optionally installed deployments: + +- `autoscaler-hpa` - `networking-ns-cert` +- `networking-certmanager` Scale up the deployment(s): ``` $ kubectl scale --replicas=2 ``` + +- Setting `--replicas` to a value of `2` enables HA. +- You can use a higher value if you have a use case that requires more replicas of a deployment. For example, if you require a minimum of 3 `controller` deployments, set `--replicas=3`. +- Setting `--replicas=1` disables HA. From 435c0c6930db0dc5ca157c24386c9d9ae00dd86f Mon Sep 17 00:00:00 2001 From: abrennan Date: Fri, 5 Jun 2020 11:51:20 -0500 Subject: [PATCH 3/6] fix link --- docs/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index 8ad1dad32bb..6dc0b225261 100644 --- a/docs/README.md +++ b/docs/README.md @@ -85,7 +85,7 @@ Follow the links below to learn more about Knative. - [Using a custom domain](./serving/using-a-custom-domain.md) - [Assigning a static IP address for Knative on Google Kubernetes Engine](./serving/gke-assigning-static-ip-address.md) - [Configuring HTTPS with a custom certificate](./serving/using-a-tls-cert.md) -- [Configuring high availability](./config-ha.md) +- [Configuring high availability](./serving/config-ha.md) ### Samples and demos From 387ae8246b412c75ec840b3933fe9bd2a13c68dc Mon Sep 17 00:00:00 2001 From: abrennan Date: Fri, 5 Jun 2020 12:38:53 -0500 Subject: [PATCH 4/6] removed linktitle --- docs/serving/config-ha.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/serving/config-ha.md b/docs/serving/config-ha.md index 9677e6e73a5..ada110879f0 100644 --- a/docs/serving/config-ha.md +++ b/docs/serving/config-ha.md @@ -1,7 +1,6 @@ --- title: "Configuring high-availability components" -linkTitle: "Configuring high-availability components" -weight: 20 +weight: 50 type: "docs" --- From 40cfa16831a37ea65fa48b620adcf61d07be4f46 Mon Sep 17 00:00:00 2001 From: abrennan Date: Fri, 5 Jun 2020 12:52:57 -0500 Subject: [PATCH 5/6] fixed scale command --- docs/serving/config-ha.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/serving/config-ha.md b/docs/serving/config-ha.md index ada110879f0..cfc2624d295 100644 --- a/docs/serving/config-ha.md +++ b/docs/serving/config-ha.md @@ -37,7 +37,7 @@ $ kubectl patch configmap/config-leader-election \ 1. Restart the controllers: ``` - $ kubectl rollout restart deployment -n knative-serving +$ kubectl rollout restart deployment -n knative-serving ``` **NOTE:** You will experience temporary control plane downtime during this step. @@ -48,7 +48,7 @@ $ kubectl patch configmap/config-leader-election \ 1. After the controllers have been configured to use leader election, the control plane can be scaled up: ``` -$ kubectl rollouts restart deployment -n knative-serving controller +$ kubectl scale --replicas=N ``` ## Scaling the control plane From 5a88f9cfa3beb3d6cb809a9b473592e1dc11c2ff Mon Sep 17 00:00:00 2001 From: abrennan Date: Fri, 5 Jun 2020 13:00:52 -0500 Subject: [PATCH 6/6] updates to commands --- docs/serving/config-ha.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/serving/config-ha.md b/docs/serving/config-ha.md index cfc2624d295..4d1d251170e 100644 --- a/docs/serving/config-ha.md +++ b/docs/serving/config-ha.md @@ -37,7 +37,7 @@ $ kubectl patch configmap/config-leader-election \ 1. Restart the controllers: ``` -$ kubectl rollout restart deployment -n knative-serving +$ kubectl rollout restart deployment -n knative-serving ``` **NOTE:** You will experience temporary control plane downtime during this step. @@ -48,7 +48,7 @@ $ kubectl rollout restart deployment -n knative-serving 1. After the controllers have been configured to use leader election, the control plane can be scaled up: ``` -$ kubectl scale --replicas=N +$ kubectl -n knative-serving scale deployment --replicas=2 ``` ## Scaling the control plane @@ -68,7 +68,7 @@ Optionally installed deployments: Scale up the deployment(s): ``` -$ kubectl scale --replicas=2 +$ kubectl -n knative-serving scale deployment --replicas=2 ``` - Setting `--replicas` to a value of `2` enables HA.