Integrate with Pod security #8436

xpivarc · 2022-09-08T15:27:13Z

What this PR does / why we need it:
Pod Security Standards shows Kubevirt still can't run as a restricted workload. Clusters enforcing restricted policy are not able to run VMs without manual adjustments of namespaces. There is also an issue with our upgrade path to the cluster that starts to enforce the policy.

This PR enables opt-in to an automatic escalation of namespaces where VMs run. This should be used only with additional security in place or after auditing of RBAC for Kubevirt access(ability to create VM/VMI or higher abstractions).

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Release note:

Kubevirt is able to run with restricted Pod Security Standard enabled with an automatic escalation of namespace privileges.

kubevirt-bot · 2022-09-08T15:27:21Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

xpivarc · 2022-09-08T15:27:51Z

/test pull-kubevirt-e2e-k8s-1.24-psa-sig-compute
/test pull-kubevirt-e2e-k8s-1.24-psa-sig-storage

xpivarc · 2022-09-08T18:59:36Z

/test pull-kubevirt-e2e-k8s-1.24-psa-sig-compute
/test pull-kubevirt-e2e-k8s-1.24-psa-sig-storage

xpivarc · 2022-09-08T19:47:06Z

/test pull-kubevirt-e2e-k8s-1.24-psa-sig-compute
/test pull-kubevirt-e2e-k8s-1.24-psa-sig-storage

xpivarc · 2022-09-09T14:36:50Z

/test pull-kubevirt-e2e-k8s-1.24-psa-sig-compute
/test pull-kubevirt-e2e-k8s-1.24-psa-sig-storage

jean-edouard · 2022-09-09T14:43:08Z

/approve

kubevirt-bot · 2022-09-09T14:43:14Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jean-edouard

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jean-edouard]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

xpivarc · 2022-09-09T21:28:33Z

/retest
/test pull-kubevirt-e2e-k8s-1.24-psa-sig-compute
/test pull-kubevirt-e2e-k8s-1.24-psa-sig-storage

enp0s3 · 2022-09-11T12:30:55Z

tests/framework/storage/nfs.go

 )

-func RenderNFSServerWithPVC(generateName string, pvcName string) *k8sv1.Pod {


enp0s3 · 2022-09-11T15:05:56Z

automation/test.sh

@@ -70,6 +70,15 @@ else
  export KUBEVIRT_PROVIDER=${TARGET}
 fi

+if [[ $TARGET =~ psa ]]; then


@xpivarc I am not sure that it's going to work, since we use -apply-default-e2e-configuration flag in the functests.sh and it overrides the feature-gate list.

@xpivarc Sorry my bad, we append. Also checked in the failed test artifacts.

BTW. We have also test that verify this holds true in the future.

enp0s3 · 2022-09-11T15:06:34Z

/cc

kubevirt-commenter-bot · 2022-09-14T11:08:23Z

/retest-required
This bot automatically retries required jobs that failed/flaked on approved PRs.
Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

VMs are unfortunatly still privileged workload(in Kubevirt). We have to integrate with new Pod Security Standards in order to allow seamless integration, upgrades. This means we now make sure that target namespace allows privileged workloads if PSA feature gate is enabled. This unfortunatly means users escalate their privileges, in terms of Pod security, by having ability to create VMs. Signed-off-by: L. Pivarc <lpivarc@redhat.com>

Make sure we are not racing with cluster sync mechanism on Openshift. Signed-off-by: L. Pivarc <lpivarc@redhat.com>

Signed-off-by: L. Pivarc <lpivarc@redhat.com>

CDI is yet no integrated Signed-off-by: L. Pivarc <lpivarc@redhat.com>

xpivarc · 2022-09-14T13:19:34Z

@jean-edouard @acardace There was cycle dependency. It should be good now

acardace · 2022-09-14T14:24:26Z

/lgtm

We need privileged namespace in order to create some of our pods that we need for tests. This commit also aligns pods that can run as restricted PSS. Signed-off-by: L. Pivarc <lpivarc@redhat.com>

xpivarc · 2022-09-14T15:41:28Z

@acardace I just made Bazel happy

xpivarc · 2022-09-15T07:04:43Z

/retest

acardace · 2022-09-15T09:32:07Z

/lgtm

acardace · 2022-09-15T09:32:22Z

/test pull-kubevirt-e2e-kind-1.22-sriov-nonroot

brianmcarey · 2022-09-15T15:58:22Z

/test pull-kubevirt-verify-go-mod

kubevirt-bot · 2022-09-15T17:14:17Z

@xpivarc: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kubevirt-e2e-k8s-1.24-psa-sig-compute	`13aabe2`	link	false	`/test pull-kubevirt-e2e-k8s-1.24-psa-sig-compute`
pull-kubevirt-e2e-k8s-1.24-psa-sig-storage	`13aabe2`	link	false	`/test pull-kubevirt-e2e-k8s-1.24-psa-sig-storage`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

xpivarc · 2022-09-21T15:42:02Z

/cherry-pick release-0.49
/cherry-pick release-0.44

kubevirt-bot · 2022-09-21T15:42:52Z

@xpivarc: #8436 failed to apply on top of branch "release-0.49":

Applying: Enforce AllowPrivilegeEscalation
Using index info to reconstruct a base tree...
M	manifests/generated/operator-csv.yaml.in
M	pkg/virt-operator/resource/generate/components/deployments.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/virt-operator/resource/generate/components/deployments.go
Auto-merging manifests/generated/operator-csv.yaml.in
CONFLICT (content): Merge conflict in manifests/generated/operator-csv.yaml.in
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 Enforce AllowPrivilegeEscalation
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-0.49
/cherry-pick release-0.44

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

xpivarc · 2022-09-22T09:01:04Z

/cherry-pick release-0.53

kubevirt-bot · 2022-09-22T09:01:46Z

@xpivarc: #8436 failed to apply on top of branch "release-0.53":

Applying: Enforce AllowPrivilegeEscalation
Applying: Drop ALL capabilities
Applying: Set SeccompProfile
Applying: Pod security label for Kubevirt
Applying: Integrate with Pod security
Using index info to reconstruct a base tree...
M	manifests/generated/operator-csv.yaml.in
M	manifests/generated/rbac-operator.authorization.k8s.yaml.in
M	pkg/virt-config/feature-gates.go
M	pkg/virt-controller/watch/BUILD.bazel
M	pkg/virt-controller/watch/application.go
M	pkg/virt-controller/watch/application_test.go
M	pkg/virt-controller/watch/migration.go
M	pkg/virt-controller/watch/migration_test.go
M	pkg/virt-controller/watch/vmi.go
M	pkg/virt-controller/watch/vmi_test.go
M	pkg/virt-operator/resource/generate/rbac/controller.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/virt-operator/resource/generate/rbac/controller.go
Auto-merging pkg/virt-controller/watch/vmi_test.go
Auto-merging pkg/virt-controller/watch/vmi.go
Auto-merging pkg/virt-controller/watch/migration_test.go
Auto-merging pkg/virt-controller/watch/migration.go
Auto-merging pkg/virt-controller/watch/application_test.go
Auto-merging pkg/virt-controller/watch/application.go
Auto-merging pkg/virt-controller/watch/BUILD.bazel
Auto-merging pkg/virt-config/feature-gates.go
CONFLICT (content): Merge conflict in pkg/virt-config/feature-gates.go
Auto-merging manifests/generated/rbac-operator.authorization.k8s.yaml.in
Auto-merging manifests/generated/operator-csv.yaml.in
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0005 Integrate with Pod security
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-0.53

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Barakmor1 · 2022-10-26T10:54:48Z

pkg/virt-controller/watch/vmi.go

+	namespaceStore     cache.Store
+	onOpenshift        bool


Hey @xpivarc ,
Do we really want boolean field called "onOpenshift" in here(or in any other place in our repo) that looks like code smell...

As of now, we have some additional integration with OKD/Openshift and this just extends the integration. We can look at how we could remove this altogether.

kubevirt-bot requested review from marceloamaral and rthallisey September 8, 2022 15:27

xpivarc force-pushed the psa-enable branch from 8b04f90 to ed033ea Compare September 8, 2022 18:59

kubevirt-bot added size/XL and removed size/L labels Sep 9, 2022

xpivarc marked this pull request as ready for review September 9, 2022 14:36

kubevirt-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 9, 2022

kubevirt-bot requested review from jean-edouard and kbidarkar September 9, 2022 14:36

kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 9, 2022

xpivarc force-pushed the psa-enable branch 3 times, most recently from a37f81a to 13aabe2 Compare September 9, 2022 18:53

enp0s3 reviewed Sep 11, 2022

View reviewed changes

kubevirt-bot requested a review from enp0s3 September 11, 2022 15:06

mhenriks mentioned this pull request Sep 11, 2022

Support restricted PSA for worker pods kubevirt/containerized-data-importer#2410

Merged

4 tasks

acardace approved these changes Sep 12, 2022

View reviewed changes

xpivarc added 4 commits September 14, 2022 14:42

Pod secuirty on Openshift

230676f

Make sure we are not racing with cluster sync mechanism on Openshift. Signed-off-by: L. Pivarc <lpivarc@redhat.com>

Enable PSA feature gate with psa target

6c6746e

Signed-off-by: L. Pivarc <lpivarc@redhat.com>

Disable CDI with PSA lane

cd579e4

CDI is yet no integrated Signed-off-by: L. Pivarc <lpivarc@redhat.com>

xpivarc force-pushed the psa-enable branch from 13aabe2 to e5f97f9 Compare September 14, 2022 13:14

kubevirt-bot removed the lgtm Indicates that a PR is ready to be merged. label Sep 14, 2022

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Sep 14, 2022

Tests: Privileged namespace

0cb928e

We need privileged namespace in order to create some of our pods that we need for tests. This commit also aligns pods that can run as restricted PSS. Signed-off-by: L. Pivarc <lpivarc@redhat.com>

xpivarc force-pushed the psa-enable branch from e5f97f9 to 0cb928e Compare September 14, 2022 15:40

kubevirt-bot removed the lgtm Indicates that a PR is ready to be merged. label Sep 14, 2022

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Sep 15, 2022

kubevirt-bot merged commit ddef49d into kubevirt:main Sep 15, 2022
42 checks passed

xpivarc mentioned this pull request Sep 19, 2022

Pod Security Admission/Standards #8275

Closed

xpivarc mentioned this pull request Sep 26, 2022

[release-0.53] Integrate with Pod security #8518

Merged

kubevirt-bot mentioned this pull request Sep 26, 2022

Patching node set kubevirt.io/schedulable to false upon exiting heartbeat #8238

Merged

xpivarc mentioned this pull request Sep 27, 2022

[release-0.49] Integrate with Pod security #8529

Merged

Barakmor1 reviewed Oct 26, 2022

View reviewed changes

Integrate with Pod security #8436

Integrate with Pod security #8436

xpivarc commented Sep 8, 2022

kubevirt-bot commented Sep 8, 2022

xpivarc commented Sep 8, 2022

xpivarc commented Sep 8, 2022

xpivarc commented Sep 8, 2022

xpivarc commented Sep 9, 2022

jean-edouard commented Sep 9, 2022

kubevirt-bot commented Sep 9, 2022

xpivarc commented Sep 9, 2022

enp0s3 Sep 11, 2022

enp0s3 Sep 11, 2022

enp0s3 Sep 11, 2022

xpivarc Sep 11, 2022

enp0s3 commented Sep 11, 2022

kubevirt-commenter-bot commented Sep 14, 2022

xpivarc commented Sep 14, 2022

acardace commented Sep 14, 2022

xpivarc commented Sep 14, 2022

xpivarc commented Sep 15, 2022

acardace commented Sep 15, 2022

acardace commented Sep 15, 2022

brianmcarey commented Sep 15, 2022

kubevirt-bot commented Sep 15, 2022 •

edited

xpivarc commented Sep 21, 2022

kubevirt-bot commented Sep 21, 2022

xpivarc commented Sep 22, 2022

kubevirt-bot commented Sep 22, 2022

Barakmor1 Oct 26, 2022

xpivarc Oct 26, 2022

		)

		func RenderNFSServerWithPVC(generateName string, pvcName string) *k8sv1.Pod {

Integrate with Pod security #8436

Integrate with Pod security #8436

Conversation

xpivarc commented Sep 8, 2022

kubevirt-bot commented Sep 8, 2022

xpivarc commented Sep 8, 2022

xpivarc commented Sep 8, 2022

xpivarc commented Sep 8, 2022

xpivarc commented Sep 9, 2022

jean-edouard commented Sep 9, 2022

kubevirt-bot commented Sep 9, 2022

xpivarc commented Sep 9, 2022

enp0s3 Sep 11, 2022

Choose a reason for hiding this comment

enp0s3 Sep 11, 2022

Choose a reason for hiding this comment

enp0s3 Sep 11, 2022

Choose a reason for hiding this comment

xpivarc Sep 11, 2022

Choose a reason for hiding this comment

enp0s3 commented Sep 11, 2022

kubevirt-commenter-bot commented Sep 14, 2022

xpivarc commented Sep 14, 2022

acardace commented Sep 14, 2022

xpivarc commented Sep 14, 2022

xpivarc commented Sep 15, 2022

acardace commented Sep 15, 2022

acardace commented Sep 15, 2022

brianmcarey commented Sep 15, 2022

kubevirt-bot commented Sep 15, 2022 • edited

xpivarc commented Sep 21, 2022

kubevirt-bot commented Sep 21, 2022

xpivarc commented Sep 22, 2022

kubevirt-bot commented Sep 22, 2022

Barakmor1 Oct 26, 2022

Choose a reason for hiding this comment

xpivarc Oct 26, 2022

Choose a reason for hiding this comment

kubevirt-bot commented Sep 15, 2022 •

edited