OPCT-6 - fix/rbac: manage custom ServiceAccount and roles #34

mtulio · 2022-11-22T04:16:59Z

https://issues.redhat.com/browse/OPCT-6

This PR creates custom permissions to be used on Sonobuoy's service accounts.

The default sonobuoy's permissions are crashing the cluster upgrade feature (implemented on #33), described on OPCT-6. The goal is to avoid using system groups to not fall into pod admissions security admissions introduced on kube 1.24 (4.11).

The groups system:authenticated and system:serviceaccounts previously used were crashing the upgrade. Ref KCS 5875621

$ oc adm policy who-can use scc anyuid | grep serviceaccounts
        system:serviceaccounts

Tests checklist:

Run a full test on regular mode on baseline platform (AWS) and check the results. Expected: no failures increased when comparing with latest baseline results payload
Run the full test on upgrade feature (SPLAT-651/OPCT-5: support upgrade execution mode #33) to validate full functional in both modes (regular and upgrade).

mtulio · 2022-11-22T14:49:32Z

Converted to draft as the results with this RBAC seems not to be good: https://issues.redhat.com/browse/SPLAT-874?focusedCommentId=21301414&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-21301414

bostrt · 2022-11-30T17:13:37Z

/hold

bostrt · 2023-01-06T17:20:28Z

/unhold
/lgtm

This lgtm so far but I'll run a quick test before approving. @mtulio can you remove draft state from this PR?

pkg/types.go

pkg/run/run.go

mtulio · 2023-01-09T20:08:50Z

@bostrt this PR is ready for review and validation. It is the same RBAC changes used on #33 .

bostrt · 2023-01-09T21:19:28Z

/lgtm

I have a test running in AWS right now and if it looks good, I'll approve.

pkg/run/run.go

…tem#33

mtulio · 2023-01-10T03:41:21Z

@bostrt I ran tests onto 4.11 (regular mode) and got good results, similar failures ratio as v0.2.1+ (less than 1%). I am ok to a final review, then backport to v0.2.

$ echo $CLUSTER
opct-411pr34

$ ./.$CLUSTER-openshift-provider-cert version
OpenShift Provider Certification Tool: 0.0.0+aed8313
Sonobuoy Version: v0.56.10

$ ./.$CLUSTER-openshift-provider-cert results .$CLUSTER/clusters/$CLUSTER/opct/*.tar.gz  |egrep ^'(Plugin|Run)' -A 5
Plugin: 99-openshift-artifacts-collector
Status: passed
Total: 3
Passed: 3
Failed: 0
Skipped: 0
--
Plugin: 10-openshift-kube-conformance
Status: failed
Total: 662
Passed: 657
Failed: 5
Skipped: 0
--
Plugin: 20-openshift-conformance-validated
Status: failed
Total: 3805
Passed: 1717
Failed: 38
Skipped: 2050
--
Run Details:
API Server version: v1.24.6+5658434
Node health: 7/7 (100%)
Pods health: 262/262 (100%)

$ bc <<< "scale=4; (5/662)*100"
.7500
$ bc <<< "scale=4; (38/3805)*100"
.9900

/unhold

mtulio · 2023-01-10T03:43:43Z

/unhold

mtulio · 2023-01-10T04:53:23Z

@bostrt I am thinking that we don't need to backport this PR to v0.2 for the following reasons:

4.12 is not supported on v0.2
upgrade feature is not supported on v0.2

Let me know your thoughts.

bostrt · 2023-01-10T16:51:17Z

@bostrt I am thinking that we don't need to backport this PR to v0.2 for the following reasons:
* 4.12 is not supported on v0.2

* upgrade feature is not supported on v0.2
Let me know your thoughts.

@mtulio I was originally thinking we could separate the RBAC work from the upgrade work in 0.3 but if you want to work on stabilizing 0.2 that works for me.

mtulio · 2023-01-10T17:27:01Z

@mtulio I was originally thinking we could separate the RBAC work from the upgrade work in 0.3 but if you want to work on stabilizing 0.2 that works for me.

@bostrt yes, we are working to stabilize v0.3/upgrades, but I was thinking in scenarios if we need it on the older releases. But nothing came to my mind.

I am ok to merge here and not back port to v0.2.

bostrt · 2023-01-10T17:30:05Z

I am ok to merge here and not back port to v0.2.

This works for me too. If anything comes up, we can always backport.

bostrt · 2023-01-11T18:07:59Z

/lgtm
/approve

openshift-ci · 2023-01-11T18:08:06Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bostrt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [bostrt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

https://issues.redhat.com/browse/SPLAT-651 Support upgrade conformance. - Introduces new flags to control whether the execution needs to run the upgrade cluster or not: - `--mode=upgrade` - `--upgrade-to-image=<release_digest>` (`$(oc adm release info 4.Y+1.Z -o jsonpath={.image}`) - Create config map with plugin variables (the sonobuoy native feature wipes all existing from `podSpec` which is undesired) - add a new plugin instance of openshift-tests to run upgrades: `05-openshift-cluster-upgrade` Blocked by: - [x] #31 - [x] #34 Blocked by Plugin release: - [x] redhat-openshift-ecosystem/provider-certification-plugins#24 Checklist: - [x] CLI changes to run in upgrade mode - [x] CLI changes to get the release image digest - [x] Plugin implementation: redhat-openshift-ecosystem/provider-certification-plugins#24 - [x] Validate y-stream upgrades - [x] Fix RBAC #34 for Cluster upgrade - [x] Fix SecurityContextMode for Sonobuoy aggregator stuck on 4.10->4.11 #39 - [x] MachineConfigPool validation: 'opct' object is validated if present when running `mode=upgrade` on the runtime (plugin execution). Failures will be raised by the plugin when the MCP is not present (the User Docs should keep it very explicit): Tests described here: redhat-openshift-ecosystem/provider-certification-plugins#24 (comment) - [x] User Documentation Tests checklist: - [x] upgrade 4.12-> 4.13

openshift-ci bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Nov 22, 2022

mtulio marked this pull request as ready for review November 22, 2022 04:22

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 22, 2022

openshift-ci bot requested review from bostrt and rvanderp3 November 22, 2022 04:22

mtulio assigned mtulio and bostrt and unassigned mtulio Nov 22, 2022

mtulio changed the title ~~SPLAT-874 - fix/scc-upgrade: SPLAT874 - creating custom RBAC resources~~ SPLAT-874 - fix/rbac-upgrade: SPLAT874 - create and bind serviceaccount to SCC cluster roles Nov 22, 2022

mtulio force-pushed the fix-rbac-upgrade branch from 95fd8cd to 469f2cb Compare November 22, 2022 04:29

mtulio marked this pull request as draft November 22, 2022 14:19

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 22, 2022

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 30, 2022

mtulio changed the title ~~SPLAT-874 - fix/rbac-upgrade: SPLAT874 - create and bind serviceaccount to SCC cluster roles~~ SPLAT-874 - fix/rbac-upgrade: SPLAT874 - create serviceaccount to SCC cluster roles Dec 1, 2022

mtulio marked this pull request as ready for review December 21, 2022 17:37

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 21, 2022

mtulio force-pushed the fix-rbac-upgrade branch from 469f2cb to 8ff982f Compare December 22, 2022 18:16

openshift-ci bot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 22, 2022

mtulio changed the title ~~SPLAT-874 - fix/rbac-upgrade: SPLAT874 - create serviceaccount to SCC cluster roles~~ SPLAT-874 - fix/rbac-upgrade: create serviceaccount to SCC cluster roles Jan 5, 2023

mtulio marked this pull request as draft January 6, 2023 13:38

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 6, 2023

mtulio mentioned this pull request Jan 6, 2023

OPCT-5: support upgrade execution mode #33

Merged

12 tasks

openshift-ci bot added lgtm Indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Jan 6, 2023

bostrt reviewed Jan 6, 2023

View reviewed changes

pkg/types.go Outdated Show resolved Hide resolved

bostrt reviewed Jan 6, 2023

View reviewed changes

pkg/run/run.go Outdated Show resolved Hide resolved

openshift-ci bot requested a review from bostrt January 9, 2023 20:03

mtulio force-pushed the fix-rbac-upgrade branch 2 times, most recently from 543dd23 to a0b0488 Compare January 9, 2023 20:07

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 9, 2023

mtulio commented Jan 9, 2023

View reviewed changes

pkg/run/run.go Outdated Show resolved Hide resolved

mtulio commented Jan 9, 2023

View reviewed changes

pkg/run/run.go Outdated Show resolved Hide resolved

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Jan 9, 2023

mtulio force-pushed the fix-rbac-upgrade branch from 034ee81 to ff191d9 Compare January 9, 2023 22:43

mtulio added 3 commits January 9, 2023 19:47

fix/scc-upgrade: SPLAT874 - creating custom RBAC resources

155b266

fixes/rbac: create custom SCC according tests redhat-openshift-ecosys…

cf24a6f

…tem#33

chore/nit: expand GroupVersion, and improve err handler

aed8313

mtulio force-pushed the fix-rbac-upgrade branch from ff191d9 to aed8313 Compare January 9, 2023 22:47

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 10, 2023

mtulio changed the title ~~SPLAT-874 - fix/rbac: manage custom ServiceAccount and roles~~ OPCT-6 - fix/rbac: manage custom ServiceAccount and roles Jan 10, 2023

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 11, 2023

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 11, 2023

openshift-merge-robot merged commit d31aa37 into redhat-openshift-ecosystem:main Jan 11, 2023

mtulio deleted the fix-rbac-upgrade branch January 11, 2023 19:44

mtulio mentioned this pull request Jan 18, 2023

OPCT-7 - fix/securityContext: disable default SecurityContext created by Sonobuoy #39

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OPCT-6 - fix/rbac: manage custom ServiceAccount and roles #34

OPCT-6 - fix/rbac: manage custom ServiceAccount and roles #34

mtulio commented Nov 22, 2022 •

edited

mtulio commented Nov 22, 2022

bostrt commented Nov 30, 2022

bostrt commented Jan 6, 2023

mtulio commented Jan 9, 2023

bostrt commented Jan 9, 2023

mtulio commented Jan 10, 2023 •

edited

mtulio commented Jan 10, 2023

mtulio commented Jan 10, 2023

bostrt commented Jan 10, 2023 •

edited

mtulio commented Jan 10, 2023

bostrt commented Jan 10, 2023

bostrt commented Jan 11, 2023

openshift-ci bot commented Jan 11, 2023

OPCT-6 - fix/rbac: manage custom ServiceAccount and roles #34

OPCT-6 - fix/rbac: manage custom ServiceAccount and roles #34

Conversation

mtulio commented Nov 22, 2022 • edited

mtulio commented Nov 22, 2022

bostrt commented Nov 30, 2022

bostrt commented Jan 6, 2023

mtulio commented Jan 9, 2023

bostrt commented Jan 9, 2023

mtulio commented Jan 10, 2023 • edited

mtulio commented Jan 10, 2023

mtulio commented Jan 10, 2023

bostrt commented Jan 10, 2023 • edited

mtulio commented Jan 10, 2023

bostrt commented Jan 10, 2023

bostrt commented Jan 11, 2023

openshift-ci bot commented Jan 11, 2023

mtulio commented Nov 22, 2022 •

edited

mtulio commented Jan 10, 2023 •

edited

bostrt commented Jan 10, 2023 •

edited