Remove PodSecurityPolicy from requiredPlugins and add a user-facing warning to consider migration

[shafeeqes]

How to categorize this PR?

/area usability control-plane testing
/kind enhancement

What this PR does / why we need it:
This PR adds a user-faced warning for shoots >= v1.23 and < v1.25, to consider migrating from PodSecurityPolicy to PodSecurity admission controller.
This PR also removes the PodSecurityPolicy from the required plugins.
The e2e tests for create-update-delete shoot is also enhanced.

Which issue(s) this PR fixes:
Part of #5250

Special notes for your reviewer:
/hold until gardener/gardener-extension-provider-openstack#485 is released.

Release note:

It is now possible to disable `PodSecurityPolicy` admission plugin, please make sure you have updated the extensions to a version which supports this change.

[gardener-prow]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[gardener-prow]

@shafeeqes: The label(s) area/test cannot be applied, because the repository doesn't have them.

In response to this:

How to categorize this PR?

/area usability control-plane test
/kind enhancement

What this PR does / why we need it:
This PR adds a user-faced warning for shoots >= v1.23 and < v1.25, to consider migrating from PodSecurityPolicy to PodSecurity admission controller.
This PR also removes the PodSecurityPolicy from the required plugins.

Which issue(s) this PR fixes:
Part of #5250

Special notes for your reviewer:
/hold until gardener/gardener-extension-provider-openstack#485 is released.

Release note:

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[shafeeqes]

/test all

[shafeeqes]

@shafeeqes: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-gardener-check-vulnerabilities 70c6179 link false /test pull-gardener-check-vulnerabilities
pull-gardener-e2e-kind 70c6179 link true /test pull-gardener-e2e-kind
Full PR test history. Your PR dashboard. Command help for this repository. Please help us cut down on flakes by linking this test failure to an open flake report or filing a new flake report if you can't find an existing one. Also see our testing guideline for how to avoid and hunt flakes.

 Expected success, but got an error:
      <*retry.Error | 0xc000512a40>: {
          ctxError: <context.deadlineExceededError>{},
          err: <*errors.StatusError | 0xc000827180>{
              ErrStatus: {
                  TypeMeta: {Kind: "", APIVersion: ""},
                  ListMeta: {
                      SelfLink: "",
                      ResourceVersion: "",
                      Continue: "",
                      RemainingItemCount: nil,
                  },
                  Status: "Failure",
                  Message: "Shoot.core.gardener.cloud \"e2e-default\" is invalid: [spec.kubernetes.version: Forbidden: admission plugin \"PodSecurityPolicy\" should be disabled for Kubernetes versions >=1.25, please check https://github.com/gardener/gardener/blob/master/docs/usage/pod-security.md#migrating-from-podsecuritypolicys-to-podsecurity-admission-controller, spec.kubernetes.allowPrivilegedContainers: Forbidden: for Kubernetes versions >= 1.25, allowPrivilegedContainers field should not be set, please see https://github.com/gardener/gardener/blob/master/docs/usage/pod-security.md#speckubernetesallowprivilegedcontainers-in-the-shoot-spec]",
                  Reason: "Invalid",
                  Details: {
                      Name: "e2e-default",
                      Group: "core.gardener.cloud",
                      Kind: "Shoot",
                      UID: "",
                      Causes: [
                          {
                              Type: "FieldValueForbidden",
                              Message: "Forbidden: admission plugin \"PodSecurityPolicy\" should be disabled for Kubernetes versions >=1.25, please check https://github.com/gardener/gardener/blob/master/docs/usage/pod-security.md#migrating-from-podsecuritypolicys-to-podsecurity-admission-controller",
                              Field: "spec.kubernetes.version",
                          },
                          {
                              Type: "FieldValueForbidden",
                              Message: "Forbidden: for Kubernetes versions >= 1.25, allowPrivilegedContainers field should not be set, please see https://github.com/gardener/gardener/blob/master/docs/usage/pod-security.md#speckubernetesallowprivilegedcontainers-in-the-shoot-spec",
                              Field: "spec.kubernetes.allowPrivilegedContainers",
                          },
                      ],
                      RetryAfterSeconds: 0,
                  },
                  Code: 422,
              },
          },
      }

This failure is expected. Will now fix this in another commit.

[rfranzke]

/assign

[shafeeqes]

@shafeeqes: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-gardener-e2e-kind 10216f8 link true /test pull-gardener-e2e-kind
pull-gardener-check-vulnerabilities c10e8fb link false /test pull-gardener-check-vulnerabilities
pull-gardener-integration c10e8fb link true /test pull-gardener-integration
Full PR test history. Your PR dashboard. Command help for this repository. Please help us cut down on flakes by linking this test failure to an open flake report or filing a new flake report if you can't find an existing one. Also see our testing guideline for how to avoid and hunt flakes.

Related to #6671
/test pull-gardener-integration

[shafeeqes]

/retest

[shafeeqes]

Is it safe to remove the check if PodSecurityPolicy is required for k8s versions before 1.22.0? IIUC this could cause some problems for older versions of the extensions that were not adapted to handle missing PSPs (@shafeeqes)?

This extension issue can still happen for versions > 1.22 also, right?

[shafeeqes]

I added a release note asking to update the extensions to versions which support this change. But we don't have a compatibility matrix as of now.

[rfranzke]

PSP is not really required for a functional cluster, is it? If it was disabled then there would simply be no effect (it's like "unprivileged" mode).

[shafeeqes]

PSP is not really required for a functional cluster, is it? If it was disabled then there would simply be no effect (it's like "unprivileged" mode).

Did you mean to say "privileged"? Without the PSP admission plugin, there won't be any "restrictions" for the pods.

[gardener-prow]

@shafeeqes: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-dependency-watchdog-verify-image-build	3ed9cf0	link	true	/test pull-dependency-watchdog-verify-image-build
pull-gardener-check-vulnerabilities	e6f1a78	link	false	/test pull-gardener-check-vulnerabilities

Full PR test history. Your PR dashboard. Command help for this repository.
Please help us cut down on flakes by linking this test failure to an open flake report or filing a new flake report if you can't find an existing one. Also see our testing guideline for how to avoid and hunt flakes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[acumino]

New provider-openstack version is released, We can now unhold this PR /cc @shafeeqes .

[rfranzke]

/lgtm

[rfranzke]

/milestone v1.58

[shafeeqes]

/unhold
All the extensions are adapted with necessary change and released.