Skip to content

Commit

Permalink
Use CronJob for deployment scaler
Browse files Browse the repository at this point in the history
This is mostly to get coverage of using CronJobs with Istio, something
we don't have a lot of testing of but that our customers often use.

Fixes istio/istio#17353
  • Loading branch information
howardjohn committed Sep 27, 2019
1 parent e47b140 commit ae370b9
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 17 deletions.
46 changes: 46 additions & 0 deletions perf/load/templates/config-changer.yaml
Expand Up @@ -52,6 +52,52 @@ subjects:
name: config-change-service-account
namespace: {{ .Values.Namespace }}
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: deployment-scaler
namespace: {{ .Values.Namespace }}
spec:
# The job takes on average 5 mins to complete, scaling 20 deployments with a 10-20s sleep in between
# We schedule the job for every 10min, giving us 5min on/off
schedule: "*/10 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: kube
image: gcr.io/mixologist-142215/hyperkube:v1.12.1
args:
- /bin/sh
- -c
- |-
set -x
n=0
until [ $n -ge 15 ]; do
kubectl version && break # wait for envoy
n=$[$n+1]
sleep 1
done
set -e
deployments=$(kubectl get deployments -n {{ .Values.Namespace }} -l app=service-graph -o jsonpath="{.items[*].metadata.name}")
while [[ true ]];do
for deployment in ${deployments}; do
kubectl scale deployments -n {{ .Values.Namespace }} ${deployment} --replicas $((1 + RANDOM % 3))
sleep {{ .Values.replicasSleep }}
# add jitter
sleep $[ ( $RANDOM % {{ .Values.replicasSleep }} ) + 1 ]s
done
done
# Kill envoy
curl localhost:15020/quitquitquit
restartPolicy: Never
serviceAccountName: config-change-service-account
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
Expand Down
17 changes: 0 additions & 17 deletions perf/load/templates/config-map.yaml
Expand Up @@ -46,7 +46,6 @@ data:
export INGRESS_IP={{ .Values.ingress }}
${WD}/rollout.sh &
${WD}/scaling.sh &
${WD}/rotate.sh {{ .Values.serviceNamePrefix }}0 {{ .Values.domain }} 60m &
wait
rollout.sh: |
Expand All @@ -61,22 +60,6 @@ data:
sleep $[ ( $RANDOM % {{ .Values.configSleep }} ) + 1 ]s
done
done
scaling.sh: |
#!/bin/bash
set -ex
WD=$(dirname $0)
WD=$(cd $WD;pwd)
deployments=$(kubectl get deployments -n {{ .Values.Namespace }} -l app=service-graph -o jsonpath="{.items[*].metadata.name}")
while [[ true ]];do
for deployment in ${deployments}; do
kubectl scale deployments -n {{ .Values.Namespace }} ${deployment} --replicas $((1 + RANDOM % 3))
sleep {{ .Values.replicasSleep }}
# add jitter
sleep $[ ( $RANDOM % {{ .Values.replicasSleep }} ) + 1 ]s
done
done
rotate.sh: |
set -x
errs=0
Expand Down

0 comments on commit ae370b9

Please sign in to comment.