[release-1.23] Add support for configuring the EgressSelector mode

[brandond]

Proposed Changes

• Missed backport from [release-1.23] Backport bugfixes and enhancements from master #5594

Add support for server configuration flag --egress-selector-mode to configure the apiserver egress selector configuration:

• disabled: The apiserver does not use agent tunnels to communicate with nodes. This mode requires that servers run agents, and have direct connectivity to the kubelet on agents, or the apiserver will not be able to access service endpoints or perform kubectl exec and kubectl logs.
  This is the historical default for RKE2
• agent: The apiserver uses agent tunnels to communicate with nodes. Nodes allow the tunnel connection if it is to a loopback address. This is mode requires that servers also run agents, or the apiserver will not be able to access service endpoints.
  This is the historical default for k3s.
• pod (default): The apiserver uses agent tunnels to communicate with nodes and service endpoints, routing endpoint connections to the correct agent by watching Nodes. Nodes allow the tunnel connection if it is to a loopback address, or a CIDR assigned to their node.
  NOTE: This will not work when using a CNI that uses its own IPAM and does not respect the node's PodCIDR allocation. cluster should be used with these CNIs instead.
• cluster: The apiserver uses agent tunnels to communicate with nodes and service endpoints, routing endpoint connections to the correct agent by watching Endpoints. Nodes allow the tunnel connection if it is to a loopback address, or the configured cluster CIDR range.
  This is marginally less secure and less efficient than pod mode.

Types of Changes

bugfix

Verification

Normal QA checks
Validate in RKE2 with Calico

Linked Issues

• [Release-1.23] - Allow configuring the EgressSelector mode #5602

User-Facing Change

The integrated apiserver network proxy's operational mode can now be set with `--egress-selector-mode`.

Further Comments

Will need a docs update.

[brandond]

Tests flaked; merging

[PARALLEL-mysql] + local 'failures=Summarizing 2 Failures:
[PARALLEL-mysql] 
[PARALLEL-mysql] [Fail] [sig-node] Variable Expansion [It] should fail substituting values in a volume subpath with backticks [Slow] [Conformance] 
[PARALLEL-mysql] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/node/expansion.go:379
[PARALLEL-mysql] 
[PARALLEL-mysql] [Fail] [sig-node] Probing container [It] should have monotonically increasing restart count [NodeConformance] [Conformance] 
[PARALLEL-mysql] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/node/container_probe.go:195
[PARALLEL-mysql] 
[PARALLEL-mysql] Ran 325 of 7044 Specs in 514.466 seconds
[PARALLEL-mysql] FAIL! -- 323 Passed | 2 Failed | 0 Pending | 6719 Skipped