update cluster-autoscaler addon to match upstream example

[c4tz]

What this PR does / why we need it:
Most importantly: Adds permissions to watch csistoragecapacities, csidrivers and namespaces objects, reducing errors such as:

Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:cluster-autoscaler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

Also, it adds labels to the ClusterRole, ClusterRoleBinding, etc. for better filtering as done in the offiicial cluster-autoscaler example for Hetzner.

Which issue(s) this PR fixes:
None, created this PR directly instead of opening an issue first.

Special notes for your reviewer:
I'm not quite sure whether it makes sense to also add the affinity bit to the deployment aswell:

      # Node affinity is used to force cluster-autoscaler to stick
      # to the master node. This allows the cluster to reliably downscale
      # to zero worker nodes when needed.
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: node-role.kubernetes.io/master
                    operator: Exists

Does this PR introduce a user-facing change?:
No.

NONE

[kubermatic-bot]

Hi @c4tz. Thanks for your PR.

I'm waiting for a kubermatic member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kron4eg]

@c4tz thank you!

/lgtm
/approve

[kubermatic-bot]

LGTM label has been added.

Git tree hash: 328aa82413642fa45c8344739c909e97b1eb2abd

[kubermatic-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: c4tz, kron4eg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [kron4eg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kron4eg]

/retest

[c4tz]

Thank you for the fast responses and merging @kron4eg ! :)

What about the affinity bit? Should I make a new PR for it (maybe with a release note)? As far as I currently understand the documentation, the toleration allows the autoscaler to also run on master nodes, while the affinity binds the pod to them, so that it doesn't schedule on worker nodes at all.

Is this correct? If yes, wouldn't that be behaviour we want cluster-autoscaler to have?

[kron4eg]

@c4tz given that CA has tolaration, it will be rescheduled to the control-plane node automatically in the event if it will kill the Machine where it is running and that Machine is the last worker. Should be OK. But having affinity in place wouldn't harm anyway.