Configure workers to autoscale down to 0

[chaws]

I was reading thru this issue (kubernetes/kubernetes#69687) I noticed our workers sit idle when there aren't tasks in queue. We could save costs by setting minimum number of pods to 0 (zero) when no task is present in queue and fire up workers depending on the amount of tasks.

The PR (kubernetes/kubernetes#74526) seems to be in alpha but not yet available in stable K8s nor in EKS, but it'll be nice to keep an eye on this feature when it becomes available to save costs running qa-reports

[chaws]

To attempt next week https://itnext.io/horizontal-pod-autoscale-with-custom-metrics-8cb13e9d475