Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OPCT-7 - fix/securityContext: disable default SecurityContext created by Sonobuoy #39

Merged

Conversation

mtulio
Copy link
Contributor

@mtulio mtulio commented Jan 11, 2023

The goal of this PR is to change the default mode of the SecurityContext of jobs, encapsulated by "modes". The SecurityContext created by Sonobuoy is not working as expected on the pod admission security pods introduced on the OCP 4.11/Kube 1.24.

We get the issue when running the upgrade from 4.10 to 4.11(using conformance tests / #33 ), the sonobuoy aggregator stuck trying to annotate the resources managed by it right after the cluster finished the upgrade. The issue is detailed on OPCT-7.

@mtulio
Copy link
Contributor Author

mtulio commented Jan 11, 2023

PR under validation
/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 11, 2023
@mtulio mtulio marked this pull request as draft January 17, 2023 14:58
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 17, 2023
@mtulio
Copy link
Contributor Author

mtulio commented Jan 18, 2023

This change, and RBAC #34, are mandatory to the 'upgrade' (#33) feature work correctly.

I just tested the upgrade without this change, leaving the mode==default (nonroot), and it raised those errors:

$ KUBECONFIG=$PWD/.opct-410t411/clusters/opct-410t411/auth/kubeconfig oc  logs -n openshift-provider-certification sonobuoy |grep error= |head -n1
time="2023-01-18T14:27:32Z" level=info msg="couldn't annotate sonobuoy pod" 
error="couldn't patch pod annotation: 
pods \"sonobuoy\" is forbidden: 
unable to validate against any security context constraint: 
[provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{2000}: 2000 is not an allowed group, 
spec.containers[0].securityContext.runAsUser: Invalid value: 1000: must be in the ranges: [1000650000, 1000659999], 
provider restricted: .spec.securityContext.fsGroup: Invalid value: []int64{2000}: 2000 is not an allowed group, 
provider machine-api-termination-handler: .spec.securityContext.fsGroup: Invalid value: []int64{2000}: 2000 is not an allowed group, 
spec.volumes[0]: Invalid value: \"configMap\": configMap volumes are not allowed to be used, 
spec.volumes[1]: Invalid value: \"configMap\": configMap volumes are not allowed to be used, 
spec.volumes[2]: Invalid value: \"emptyDir\": emptyDir volumes are not allowed to be used, 
spec.volumes[3]: Invalid value: \"projected\": projected volumes are not allowed to be used, 
provider hostnetwork-v2: .spec.securityContext.fsGroup: Invalid value: []int64{2000}: 2000 is not an allowed group, 
provider hostnetwork: .spec.securityContext.fsGroup: Invalid value: []int64{2000}: 2000 is not an allowed group, 
provider hostaccess: .spec.securityContext.fsGroup: Invalid value: []int64{2000}: 2000 is not an allowed group]"

$ KUBECONFIG=$PWD/.opct-410t411/clusters/opct-410t411/auth/kubeconfig oc  logs -n openshift-provider-certification sonobuoy |grep error= |wc -l
405

@mtulio mtulio marked this pull request as ready for review January 18, 2023 19:06
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 18, 2023
@mtulio
Copy link
Contributor Author

mtulio commented Jan 18, 2023

I've created two cards to address upstream issues found while testing this PR:

This card will resolve OPCT-7 for PR #33 (OPCT-5).

@bostrt
Copy link
Contributor

bostrt commented Jan 18, 2023

/lgtm
/approve

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 18, 2023
@openshift-ci
Copy link

openshift-ci bot commented Jan 18, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bostrt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 18, 2023
@bostrt
Copy link
Contributor

bostrt commented Jan 18, 2023

/unhold

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 18, 2023
@openshift-merge-robot openshift-merge-robot merged commit d6552ec into redhat-openshift-ecosystem:main Jan 18, 2023
@mtulio mtulio mentioned this pull request Jan 18, 2023
12 tasks
@mtulio mtulio changed the title OPCT-7 - fix/scc: disable SCC creation by sonobuoy OPCT-7 - fix/securityContext: disable default SecurityContex creation by Sonobuoy Jan 19, 2023
@mtulio mtulio changed the title OPCT-7 - fix/securityContext: disable default SecurityContex creation by Sonobuoy OPCT-7 - fix/securityContext: disable default SecurityContext created by Sonobuoy Jan 19, 2023
openshift-merge-robot pushed a commit that referenced this pull request Jan 26, 2023
https://issues.redhat.com/browse/SPLAT-651

Support upgrade conformance.
- Introduces new flags to control whether the execution needs to run the
upgrade cluster or not:
  - `--mode=upgrade`
- `--upgrade-to-image=<release_digest>` (`$(oc adm release info 4.Y+1.Z
-o jsonpath={.image}`)
- Create config map with plugin variables (the sonobuoy native feature
wipes all existing from `podSpec` which is undesired)
- add a new plugin instance of openshift-tests to run upgrades:
`05-openshift-cluster-upgrade`

Blocked by:
- [x]
#31
- [x]
#34

Blocked by Plugin release:

- [x]
redhat-openshift-ecosystem/provider-certification-plugins#24

Checklist:
- [x] CLI changes to run in upgrade mode
- [x] CLI changes to get the release image digest
- [x] Plugin implementation:
redhat-openshift-ecosystem/provider-certification-plugins#24
- [x] Validate y-stream upgrades
- [x] Fix RBAC #34 for Cluster upgrade
- [x] Fix SecurityContextMode for Sonobuoy aggregator stuck on
4.10->4.11 #39
- [x] MachineConfigPool validation: 'opct' object is validated if
present when running `mode=upgrade` on the runtime (plugin execution).
Failures will be raised by the plugin when the MCP is not present (the
User Docs should keep it very explicit): Tests described here:
redhat-openshift-ecosystem/provider-certification-plugins#24 (comment)
- [x] User Documentation

Tests checklist:
- [x] upgrade 4.12-> 4.13
@mtulio mtulio added the kind/bug Categorizes issue or PR as related to a bug. label Apr 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants