🐛 Cluster Autoscaler: provider is not customisable

[dalees]

The Cluster Autoscaler deployed by this Helm Chart uses the clusterapi provider.

This is fine when Magnum is not driving the Charts, but when using the magnum-capi-helm driver any scaling actions do not get reflected into Magnum. This leads to incorrect values for Node Groups showing in the Magnum API.

We should consider using --provider=magnum when Magnum is in use, so the Cluster Autoscaler talks to Magnum and not CAPI directly.

[dalees]

This does not appear to be immediately solvable simply using provider=magnum and giving it args as it appears this reaches into Heat Stacks[1] to determine Server UUIDs.

The Magnum API should really provide these via the nodegroup information, so the driver can implement this however it likes - for exactly this reason.

[1] https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/magnum/magnum_manager_impl.go#L232