Add Autoscaler to CSE

[adambarreiro]

Overview

This PR automates the steps to deploy an Autoscaler into a Kubernetes cluster stated in this document:
https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/docs/vmw-whitepaper-cluster-auto-scaler.pdf

This procedure affects cluster creation and cluster update operations.

Built on top of #674

Description

A new autoscaler.tmpl file has been added to the collection. A new configuration block Autoscaler has been added to both the CseWorkerPoolSettings (settings to create a worker pool in a new cluster) and CseWorkerPoolUpdateInput (input to update a worker pool in an existing cluster).

When any of the Worker Pools settings has an Autoscaler struct set, we add the autoscaler.tmpl rendered YAML to the final bunch of YAML documents that describe the whole cluster. Also, each of the worker pool deployments will have special metadata entries, and won't render the replicas field (as it conflicts with the autoscaler).

When none of the Worker Pools settings have the Autoscaler struct set, we do nothing, we just work with replicas as usual and no extra steps are required. This case would define an existing use case for regression tests.

With these two scenarios, the cluster is then created with or without autoscaling elements present in the cluster, and also the new metadata entries in each worker pool, if needed.

What about updating a cluster?

In this case, a user can enable or disable the Autoscaler in the same way, using CseWorkerPoolUpdateInput to update existing worker pools or CseWorkerPoolSettings to create new worker pools (this mechanism doesn't change). Both have the new Autoscaler struct (this is the new feature).

When updating a cluster, we search for the Autoscaler YAML document, to check whether it is present in the cluster or not.

• If we need autoscaling capabilities and the YAML is not there (that would mean that the cluster was created without autoscaling), we update/add the YAML and the required metadata to the existing/new worker pools.
• If we need autoscaling capabilities and the YAML is there, we just update/add the YAML and the required metadata to the existing/new worker pools.
• If we don't need autoscaling, then we need to check whether it is enabled in the cluster, and disable the Autoscaler if so (this doesn't remove the autoscaler YAML, just deactivates it).

Tests

Updated Test_Cse (the modifications check Autoscaler updates) and created Test_CseWithAutoscaler (this one checks cluster creation with Autoscaler). Both check that the autoscaler is created and the information is retrieved (set in the structure) correctly.

GOVCD_SKIP_VAPP_CREATION=1 TEST_VCD_CSE=1 go test -tags cse -check.f '^Test_Cse' -check.vv -timeout=0

[Didainius]

Tests passed