Unable to disable kube-proxy with RKE2

[rancher-max]

Need a way to disable the new kube-proxy static pod during rke2 provisioning. The current option presented in the UI calls: disable: rke2-kube-proxy, whereas the latest release of rke2 introduces kube-proxy as a static pod and has a new way to disable it. See rancher/rke2#1444 (comment) for details.

The UI doesn't need to change, but should instead call disable-kube-proxy: true in the config when deselecting the option (see UI screenshot below):

[deniseschannon]

KDM has been updated to disable kube-proxy with this parameter.

      disable-kube-proxy:
        type: boolean

And the option to delete kube-proxy helm chart has been removed from this section:

      disable:
        type: array
        options:
        - rke2-coredns
        - rke2-ingress-nginx
        - rke2-metrics-server

UI needs to be reflected to use the new parameter of how to enable/disable kube-proxy.

The kube-proxy helm chart is being phased out and since it was never released yet, we should only enable/disable static pod of kube-proxy.

[deniseschannon]

Available as part of RKE2 v1.21.3-rc1+rke2e2

[deniseschannon]

After discussion with @ibuildthecloud, we only want to allow disabling through the YAML, but we don't want to expose this to the end user through the UI as it will break clusters unless you know what you're doing.

[deniseschannon]

KDM/yaml supports being able to disable the kube-proxy node, but we no longer have UI support to disable it.

[deniseschannon]

@rancher-max we should validate that we can disable it through the yaml and that it gets set correctly in RKE2.

[rancher-max]

The remaining testing on this is blocked by: #33961

[davidnuzik]

Reproduction:
Not required.

Validation Passed:
Rancher version: v2.6-head 7b9a232ac 8/6/21 2:12pm Pacific
Rancher cluster type: single-node rancher in docker container
Docker version: 19.03

Downstream cluster type: rke2
Downstream K8s version: v1.21.3-rc4+rke2r2

Validation steps:

1. Create a simple 1-node rke2 cluster, in my case with the digital ocean provider.
   Confirmed that the "Kube Proxy" system service is no longer visible in the UI, as expected:
   

2. Switch to the YAML view so we may disable the kube-proxy. Specify the relevant flag in the machineGlobalConfig of rkeConfig

    machineGlobalConfig:
      cni: calico
      disable-kube-proxy:
        - true

3. Once the node/cluster are ready, ssh into the node and confirm the setting took:
   (It did)

root@dave-rke2-try-diff-disable-pool1-q26jh:/etc/rancher/rke2/config.yaml.d# cat 50-rancher.yaml 
{
  "agent-token": "REDACTED",
  "cni": "calico",
  "disable-kube-proxy": true,
  "etcd-snapshot-retention": 5,
  "etcd-snapshot-schedule-cron": "0 */5 * * *",
  "node-label": [
    "rke.cattle.io/machine=440821ac-fc5f-42e9-aeac-a4bc291eb7ee"
  ],
  "protect-kernel-defaults": false,
  "token": "REDACTED"

And checked unit file and yes, I do not see the kube-proxy process running nor with ps.