[CIS] Check for PSP compatibility in target cluster

[MKlimuszka]

Parent ticket: #39366

Kubernetes 1.25 drops support for PSP, so additional checks need to be added.

Rancher team 3 is following the proposal outlined in "Kubernetes v1.25 PSP to PSA migration proposal document"

Every chart that ships with a PSP must be changed to add a new condition checking for the PSP capability in the target cluster:

Capabilities check for Helm

{{- if .Capabilities.APIVersions.Has "policy/v1beta1/PodSecurityPolicy" }}

This change allows the charts to still work with PSPs in Kubernetes versions prior to v1.25, and work in Kubernetes v1.25 or higher by skipping installation of PSPs.

Charts that need updating:

• rancher-cis-benchmark 

If a cluster role in a chart contains PSP's along with other permissions extra work may be needed to achieve the intended behavior of that role.

[prachidamle]

We dont have a CIS scan profile supported for k8s 1.25 yet in the upstream kube-bench tool https://github.com/aquasecurity/kube-bench/blob/main/docs/platforms.md#cis-kubernetes-benchmark-support

So we need to hold off supporting PSP removal and tests for k8s 1.25.

[MKlimuszka]

We do, however, need to make sure the behavior is appropriate.

[doflamingo721]

@prachidamle @MKlimuszka Following is the behaviour seen on 1.25 clusters:

1. RKE:

• rke-profile-permissive-1.23: The scans are working as expected. The scan is passing with the expected behaviour.
  

• rke-profile-hardened-1.23: The scans are working as expected. There are some tests which fail mentioned in the screenshot below. The reason for their failures is because these tests run PSP related audit commands and in k8s 1.25 there isn't a PSP resource found.

2. K3s:

• k3s-profile-permissive-1.23: The scans are working as expected. The scan is passing with the expected behaviour.
  

• k3s-profile-hardened-1.23: The scans are working as expected. The scan is passing with the expected behaviour.
  

3. RKE2:

• rke2-profile-permissive-1.23: The scans are working as expected. The scan is passing with the expected behaviour.
  

• rke2-profile-hardened-1.23: We were not able to verify the scan results for hardened rke2 cluster because the cluster provisioning for rke2 hardened cluster was failing consistently. For k8s 1.25, the 1.6 cis-profile was showing incorrect while cluster provisioning. Also after changing the profile to 1.23, the cluster provisioning was stuck on updating state. There is already an issue logged for this here.

cc: @mitulshah-suse

[vardhaman22]

tested running scan for profile rke2-profile-hardened-1.23 on rke2-hardened cluster k8s: v1.25.6+rke2r1 using cis-benchmark 4.0.0-rc1 chart.
The scan failed but all the failure cases have audit commands which try to do operation on psp so that is expected.

@prachidamle @MKlimuszka
cc: @mitulshah-suse

[martyav]

@MKlimuszka I'm on the docs team, helping with the 2.7.2 release notes. If the linked PRs are merged, why is this still open? Is there more to be done?

[mitulshah-suse]

@martyav The comments above need to be documented in the release notes as expected failures for CIS scans on 1.25 clusters.
#39851 (comment)
#39851 (comment)

[prachidamle]

@ronhorton can we close this as CIS chart has been tested with k8s 1.25?

[ronhorton]

Pass We've tested this thoroughly ;)
Going out with rancher 2.7.2

[martyav]

@ronhorton Working on release notes. Are the expected failures described above for RKE and RKE2 still valid, or can this be moved to bug fixes?