feat(kuma-cp) add horizontal pod autoscaler #1271

austince · 2020-12-02T17:50:23Z

Summary

Lifts the HPA configuration from the Kong chart to enable Control Plane horizontal autoscaling.

~~Will rebase and resync chart readme after we merge #1209.~~ done!

Full changelog

Add HorizontalPodAutoscaler with default target CPU of 80%, with min 2 and max 5 pods
Add kind/deploy/metrics-server make task for installing the k8s metrics-server, which is necessary for HPA
- Should this go in another makefile?

Issues resolved

Fix #1207

Documentation

N/A

austince · 2020-12-03T19:35:39Z

@tharun208 would you mind taking a look at this too?

Signed-off-by: austin ce <austin.cawley@gmail.com>

nickolaev · 2020-12-04T09:13:02Z

mk/kind.mk

@@ -116,6 +116,13 @@ kind/deploy/metrics: build/kumactl
 	@${BUILD_ARTIFACTS_DIR}/kumactl/kumactl install metrics $(KUMACTL_INSTALL_METRICS_IMAGES) | kubectl apply -f -
 	@KUBECONFIG=$(KIND_KUBECONFIG) kubectl wait --timeout=60s --for=condition=Ready -n kuma-metrics pods -l app=prometheus

+.PHONY: kind/deploy/metrics-server
+kind/deploy/metrics-server:
+	@KUBECONFIG=$(KIND_KUBECONFIG) kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.4.1/components.yaml


Please parametrize the version so we can bump it easier.

nickolaev · 2020-12-04T09:15:18Z

mk/kind.mk

+	@KUBECONFIG=$(KIND_KUBECONFIG) kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.4.1/components.yaml
+	@KUBECONFIG=$(KIND_KUBECONFIG) kubectl patch -n kube-system deployment/metrics-server \
+		--patch='{"spec":{"template":{"spec":{"containers":[{"name":"metrics-server","args":["--cert-dir=/tmp", "--secure-port=4443", "--kubelet-insecure-tls", "--kubelet-preferred-address-types=InternalIP"]}]}}}}'
+	@KUBECONFIG=$(KIND_KUBECONFIG) kubectl wait --timeout=60s --for=condition=Available -n kube-system deployment/metrics-server


So, how kind specific is this? You say in the readme or on a cluster with a similarly self-signed certz, what are these clusters? Is this a public cloud, or on-prem?
My point is, is, does this belong to kind.mk or can go a bit more generic?

Definitely more generic, just wasn't sure which makefile to put it in -- maybe dev.mk? It should be up to the user to configure the metrics-server separately from kuma, this make target is just meant to make the dev setup easier.

This might also be the case for things like minikube and microk8s. Do you generally use kind for local dev?

nickolaev · 2020-12-04T09:16:20Z

deployments/charts/kuma/README.md

+| controlPlane.autoscaling.minReplicas | int | `2` | The minimum CP pods to allow |
+| controlPlane.autoscaling.maxReplicas | int | `5` | The max CP pods to scale to |
+| controlPlane.autoscaling.targetCPUUtilizationPercentage | int | `80` | For clusters that don't support autoscaling/v2beta, autoscaling/v1 is used |
+| controlPlane.autoscaling.metrics | list | `[{"resource":{"name":"cpu","target":{"averageUtilization":80,"type":"Utilization"}},"type":"Resource"}]` | For clusters that do support autoscaling/v2beta, use metrics |


OK, how are we setting these on the command line? is it even possible?

This is set dynamically by helm in the Capabilities.APIVersions builtin object based on what is available in the target cluster. I think it is similar to the kubectl api-versions command.

nickolaev · 2020-12-04T12:06:48Z

@Mergifyio update

mergify · 2020-12-04T12:07:22Z

Command update: success

Branch has been successfully updated

CLAassistant · 2020-12-04T12:07:29Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ nickolaev
✅ austince
❌ mergify[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

Signed-off-by: austin ce <austin.cawley@gmail.com>

…arts/hpa Signed-off-by: austin ce <austin.cawley@gmail.com>

* feat(deployments) add horizontal pod autoscaler * test(deployments) add e2e test for helm autoscaling * docs(kuma-cp) add annotations to chart values * docs(kuma-cp) fix tag value doc * chore(*) parameterize metrics-server make target Signed-off-by: austin ce <austin.cawley@gmail.com> (cherry picked from commit 7d703bc) # Conflicts: # app/kumactl/pkg/install/k8s/control-plane/helmtemplates_vfsdata.go

* feat(kuma-cp) add horizontal pod autoscaler (#1271) * feat(deployments) add horizontal pod autoscaler * test(deployments) add e2e test for helm autoscaling * docs(kuma-cp) add annotations to chart values * docs(kuma-cp) fix tag value doc * chore(*) parameterize metrics-server make target Signed-off-by: austin ce <austin.cawley@gmail.com> (cherry picked from commit 7d703bc) # Conflicts: # app/kumactl/pkg/install/k8s/control-plane/helmtemplates_vfsdata.go * fix(*) conflicts Signed-off-by: Nikolay Nikolaev <nikolay.nikolaev@konghq.com> Co-authored-by: Austin Cawley-Edwards <austin.cawley@gmail.com> Co-authored-by: Nikolay Nikolaev <nikolay.nikolaev@konghq.com>

austince requested a review from a team as a code owner December 2, 2020 17:50

austince force-pushed the feat/charts/hpa branch from eca3f6a to 75f5862 Compare December 2, 2020 18:57

austince changed the title ~~feat(deployments) add horizontal pod autoscaler~~ feat(kuma-cp) add horizontal pod autoscaler Dec 3, 2020

nickolaev added the backport-to-stable label Dec 3, 2020

austince force-pushed the feat/charts/hpa branch from dc12036 to 2f578be Compare December 3, 2020 19:32

austince added 8 commits December 3, 2020 18:17

feat(deployments) add horizontal pod autoscaler

c496c2e

Signed-off-by: austin ce <austin.cawley@gmail.com>

test(deployments) add e2e test for helm autoscaling

916c5b1

Signed-off-by: austin ce <austin.cawley@gmail.com>

docs(kuma-cp) add annotations to chart values

2ed2d1f

Signed-off-by: austin ce <austin.cawley@gmail.com>

docs(kuma-cp) fix tag value doc

97046c6

Signed-off-by: austin ce <austin.cawley@gmail.com>

fix(*) make check

b820046

Signed-off-by: austin ce <austin.cawley@gmail.com>

fix(*) make check

9859507

Signed-off-by: austin ce <austin.cawley@gmail.com>

fix(*) make check

ac2d090

Signed-off-by: austin ce <austin.cawley@gmail.com>

fix(*) generate

1cf9ad0

Signed-off-by: austin ce <austin.cawley@gmail.com>

austince force-pushed the feat/charts/hpa branch from 1a94722 to 1cf9ad0 Compare December 3, 2020 23:17

nickolaev reviewed Dec 4, 2020

View reviewed changes

Merge branch 'master' into feat/charts/hpa

6d95aaa

austince and others added 3 commits December 4, 2020 10:07

chore(*) parameterize metrics-server make target

1fda8bc

Signed-off-by: austin ce <austin.cawley@gmail.com>

Merge branch 'feat/charts/hpa' of github.com:kumahq/kuma into feat/ch…

f05160f

…arts/hpa Signed-off-by: austin ce <austin.cawley@gmail.com>

Merge branch 'master' into feat/charts/hpa

1b635fd

jakubdyszkiewicz approved these changes Dec 14, 2020

View reviewed changes

nickolaev approved these changes Dec 14, 2020

View reviewed changes

nickolaev merged commit 7d703bc into master Dec 14, 2020

nickolaev deleted the feat/charts/hpa branch December 14, 2020 14:47

mergify bot mentioned this pull request Dec 14, 2020

feat(kuma-cp) add horizontal pod autoscaler (bp #1271) #1317

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(kuma-cp) add horizontal pod autoscaler #1271

feat(kuma-cp) add horizontal pod autoscaler #1271

austince commented Dec 2, 2020 •

edited

austince commented Dec 3, 2020

nickolaev Dec 4, 2020 •

edited

nickolaev Dec 4, 2020

austince Dec 4, 2020 •

edited

austince Dec 7, 2020

nickolaev Dec 4, 2020

austince Dec 4, 2020

nickolaev commented Dec 4, 2020

mergify bot commented Dec 4, 2020

CLAassistant commented Dec 4, 2020 •

edited

feat(kuma-cp) add horizontal pod autoscaler #1271

feat(kuma-cp) add horizontal pod autoscaler #1271

Conversation

austince commented Dec 2, 2020 • edited

Summary

Full changelog

Issues resolved

Documentation

austince commented Dec 3, 2020

nickolaev Dec 4, 2020 • edited

Choose a reason for hiding this comment

nickolaev Dec 4, 2020

Choose a reason for hiding this comment

austince Dec 4, 2020 • edited

Choose a reason for hiding this comment

austince Dec 7, 2020

Choose a reason for hiding this comment

nickolaev Dec 4, 2020

Choose a reason for hiding this comment

austince Dec 4, 2020

Choose a reason for hiding this comment

nickolaev commented Dec 4, 2020

mergify bot commented Dec 4, 2020

CLAassistant commented Dec 4, 2020 • edited

austince commented Dec 2, 2020 •

edited

nickolaev Dec 4, 2020 •

edited

austince Dec 4, 2020 •

edited

CLAassistant commented Dec 4, 2020 •

edited