OCP4: Add rule to check ACS sensor deployed #11675

Vincent056 · 2024-03-12T04:22:08Z

Description:

Add rule acs_sensor_exists, so we can check that ACS sensor is being deployed on the cluster

github-actions · 2024-03-12T04:23:24Z

Start a new ephemeral environment with changes proposed in this pull request:

ocp4 (from CTF) Environment (using Fedora as testing environment)

Fedora Testing Environment

Oracle Linux 8 Environment

github-actions · 2024-03-12T04:33:57Z

🤖 A k8s content image for this PR is available at:
ghcr.io/complianceascode/k8scontent:11675
This image was built from commit: 397a3e6

Click here to see how to deploy it

If you alread have Compliance Operator deployed:
utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:11675

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and:
CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:11675 make deploy-local

controls/pcidss_4.yml

applications/openshift/general/acs_sensor_exists/tests/ocp4/e2e.yml

applications/openshift/general/acs_sensor_exists/rule.yml

Vincent056 · 2024-03-18T14:56:57Z

/test

openshift-ci · 2024-03-18T14:57:01Z

@Vincent056: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

/test 4.13-e2e-aws-ocp4-cis
/test 4.13-e2e-aws-ocp4-cis-node
/test 4.13-e2e-aws-ocp4-e8
/test 4.13-e2e-aws-ocp4-high
/test 4.13-e2e-aws-ocp4-high-node
/test 4.13-e2e-aws-ocp4-moderate
/test 4.13-e2e-aws-ocp4-moderate-node
/test 4.13-e2e-aws-ocp4-pci-dss
/test 4.13-e2e-aws-ocp4-pci-dss-node
/test 4.13-e2e-aws-ocp4-stig
/test 4.13-e2e-aws-ocp4-stig-node
/test 4.13-e2e-aws-rhcos4-e8
/test 4.13-e2e-aws-rhcos4-high
/test 4.13-e2e-aws-rhcos4-moderate
/test 4.13-e2e-aws-rhcos4-stig
/test 4.13-images
/test 4.14-images
/test 4.15-e2e-aws-ocp4-cis
/test 4.15-e2e-aws-ocp4-cis-node
/test 4.15-e2e-aws-ocp4-e8
/test 4.15-e2e-aws-ocp4-high
/test 4.15-e2e-aws-ocp4-high-node
/test 4.15-e2e-aws-ocp4-moderate
/test 4.15-e2e-aws-ocp4-moderate-node
/test 4.15-e2e-aws-ocp4-pci-dss
/test 4.15-e2e-aws-ocp4-pci-dss-node
/test 4.15-e2e-aws-ocp4-stig
/test 4.15-e2e-aws-ocp4-stig-node
/test 4.15-e2e-aws-rhcos4-e8
/test 4.15-e2e-aws-rhcos4-high
/test 4.15-e2e-aws-rhcos4-moderate
/test 4.15-e2e-aws-rhcos4-stig
/test 4.15-images
/test 4.16-e2e-aws-ocp4-cis
/test 4.16-e2e-aws-ocp4-cis-node
/test 4.16-e2e-aws-ocp4-e8
/test 4.16-e2e-aws-ocp4-high
/test 4.16-e2e-aws-ocp4-high-node
/test 4.16-e2e-aws-ocp4-moderate
/test 4.16-e2e-aws-ocp4-moderate-node
/test 4.16-e2e-aws-ocp4-pci-dss
/test 4.16-e2e-aws-ocp4-pci-dss-node
/test 4.16-e2e-aws-ocp4-stig
/test 4.16-e2e-aws-ocp4-stig-node
/test 4.16-e2e-aws-rhcos4-e8
/test 4.16-e2e-aws-rhcos4-high
/test 4.16-e2e-aws-rhcos4-moderate
/test 4.16-e2e-aws-rhcos4-stig
/test 4.16-images
/test e2e-aws-ocp4-cis
/test e2e-aws-ocp4-cis-node
/test e2e-aws-ocp4-e8
/test e2e-aws-ocp4-high
/test e2e-aws-ocp4-high-node
/test e2e-aws-ocp4-moderate
/test e2e-aws-ocp4-moderate-node
/test e2e-aws-ocp4-pci-dss
/test e2e-aws-ocp4-pci-dss-node
/test e2e-aws-ocp4-stig
/test e2e-aws-ocp4-stig-node
/test e2e-aws-rhcos4-e8
/test e2e-aws-rhcos4-high
/test e2e-aws-rhcos4-moderate
/test e2e-aws-rhcos4-stig
/test images

Use /test all to run the following jobs that were automatically triggered:

pull-ci-ComplianceAsCode-content-master-4.13-images
pull-ci-ComplianceAsCode-content-master-4.14-images
pull-ci-ComplianceAsCode-content-master-4.15-images
pull-ci-ComplianceAsCode-content-master-4.16-images
pull-ci-ComplianceAsCode-content-master-images

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Vincent056 · 2024-03-18T15:00:57Z

/test e2e-aws-ocp4-pci-dss

rhmdnd · 2024-03-18T21:06:58Z

applications/openshift/general/acs_sensor_exists/tests/ocp4/e2e.yml

@@ -0,0 +1,2 @@
+---
+default_result: FAIL


This goes back to @yuumasato's comment, but now that we have a manual remediation, we should be testing that it works in the second scan, right?

rhmdnd · 2024-03-18T21:10:37Z

applications/openshift/general/acs_sensor_exists/rule.yml

+
+ocil_clause: 'ACS Sensor is not deployed'
+
+{{% set jqfilter = '[ .items[] | select(.metadata.name == "sensor" and .metadata.labels["app.kubernetes.io/name"] == "stackrox") | .status.availableReplicas]' %}}


Looks reasonable so long as it's safe to rely on sensor as the deployment name and stackrox as the label.

@dashrews78 does this look ok to you?

on the surface looks OK. There may be another caveat though. Just because sensor is deployed doesn't necessarily mean it is connected to a central. But that may be a more advanced workload type check.

do you know where I can check so we know it is being connected to the central?

Luis could answer this, I've sent him this conversation.
~~My gut says it is sufficient to look into the Deployment status. A non-connecting sensor eventually will go to CrashLoopBackoff.~~

If ROX_PREVENT_SENSOR_RESTART_ON_DISCONNECT is enabled (which it should be by default), sensor should not go to CrashLoopBackoff. The only way to know for sure if sensor is connected with central is by checking logs, metrics, or central's UI.

I'm lacking some context here but I would say knowing if sensor is connected is not necessary because even if sensor is not connected to central, if sensor is deployed, it should flush all the resources to central upon reconnection.

thanks for the input, I wonder if there is a way we can find it out through any of the kube api resources? metrics might not be easily accessed by CO at the moment.

applications/openshift/general/acs_sensor_exists/rule.yml

rhmdnd · 2024-03-18T21:13:54Z

/test e2e-aws-ocp4-pci-dss

I don't think this tested anything new from the patch since we're not including the new rule in a profile, yet.

Add rule acs_sensor_exists, so we can check that ACS sensor is being deployed on the cluster

openshift-ci · 2024-04-01T15:04:10Z

@Vincent056: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/4.13-images	`397a3e6`	link	true	`/test 4.13-images`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

codeclimate · 2024-04-01T15:16:36Z

Code Climate has analyzed commit 397a3e6 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.3% (0.0% change).

View more on Code Climate.

BhargaviGudi · 2024-04-01T16:30:26Z

Verification failed with 4.16.0-0.nightly-2024-03-31-180021 + compliance-operator.v1.4.0 + rhacs-operator.v4.4.0

Install Co
Install rhacs-operator

$ oc get pods -n rhacs-operator 
NAME                                                 READY   STATUS    RESTARTS   AGE
rhacs-operator-controller-manager-6f647b746b-jsh4d   1/1     Running   0          6h27m

Create ssb

$ oc compliance bind -N test profile/upstream-ocp4-pci-dss-4-0 profile/upstream-ocp4-pci-dss-node-4-0
Creating ScanSettingBinding test
$ oc get scan
NAME                                    PHASE   RESULT
upstream-ocp4-pci-dss-4-0               DONE    NON-COMPLIANT
upstream-ocp4-pci-dss-node-4-0-master   DONE    NON-COMPLIANT
upstream-ocp4-pci-dss-node-4-0-worker   DONE    NON-COMPLIANT
$ oc get rules | grep acs
[bgudi@bgudi-thinkpadt14sgen2i openshift]$ oc get cr
NAME                                                              STATE
upstream-ocp4-pci-dss-4-0-api-server-encryption-provider-cipher   NotApplied
upstream-ocp4-pci-dss-4-0-audit-profile-set                       NotApplied
[bgudi@bgudi-thinkpadt14sgen2i openshift]$ oc get ccr -l compliance.openshift.io/automated-remediation=,compliance.openshift.io/check-status=FAIL
NAME                                                              STATUS   SEVERITY
upstream-ocp4-pci-dss-4-0-api-server-encryption-provider-cipher   FAIL     medium
upstream-ocp4-pci-dss-4-0-audit-profile-set                       FAIL     medium

Observation 1: upstream-ocp4-pci-dss-4-0-acs-sensor-exists is failed even though acs was installed on the cluster

$ oc get ccr | grep acs
upstream-ocp4-pci-dss-4-0-acs-sensor-exists                                                    FAIL     medium
$ oc describe ccr upstream-ocp4-pci-dss-4-0-acs-sensor-exists
Name:         upstream-ocp4-pci-dss-4-0-acs-sensor-exists
Namespace:    openshift-compliance
Labels:       compliance.openshift.io/check-severity=medium
              compliance.openshift.io/check-status=FAIL
              compliance.openshift.io/scan-name=upstream-ocp4-pci-dss-4-0
              compliance.openshift.io/suite=test
Annotations:  compliance.openshift.io/rule: acs-sensor-exists
API Version:  compliance.openshift.io/v1alpha1
Description:  Ensure that Advanced Cluster Security (ACS) Sensor is deployed
Red Hat Advanced Cluster Security (ACS) for Kubernetes provides comprehensive security for containerized environments. It offers deep visibility into deployed resources across Kubernetes clusters, enabling teams to detect vulnerabilities in all images, manage compliance, and enforce security policies. By integrating ACS into the Kubernetes environment, organizations can automate security checks and configurations, ensuring that every deployed application is scanned and secured according to best practices and organizational policies. Sensor is the service responsible for analyzing and monitoring the cluster. Sensor listens to the OpenShift Container Platform or Kubernetes API and Collector events to report the current state of the cluster. Sensor also triggers deploy-time and runtime violations based on RHACS Cloud Service policies. In addition, Sensor is responsible for all cluster interactions, such as applying network policies, initiating reprocessing of RHACS Cloud Service policies, and interacting with the Admission controller.
Id:    xccdf_org.ssgproject.content_rule_acs_sensor_exists
Kind:  ComplianceCheckResult
Metadata:
  Creation Timestamp:  2024-04-01T16:07:23Z
  Generation:          1
  Managed Fields:
    API Version:  compliance.openshift.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:description:
      f:id:
      f:metadata:
        f:annotations:
          .:
          f:compliance.openshift.io/rule:
        f:labels:
          .:
          f:compliance.openshift.io/check-severity:
          f:compliance.openshift.io/check-status:
          f:compliance.openshift.io/scan-name:
          f:compliance.openshift.io/suite:
        f:ownerReferences:
          .:
          k:{"uid":"3ac6fa8a-461e-41d7-96ce-4f13aef69f27"}:
      f:rationale:
      f:severity:
      f:status:
    Manager:    compliance-operator
    Operation:  Update
    Time:       2024-04-01T16:07:23Z
  Owner References:
    API Version:           compliance.openshift.io/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  ComplianceScan
    Name:                  upstream-ocp4-pci-dss-4-0
    UID:                   3ac6fa8a-461e-41d7-96ce-4f13aef69f27
  Resource Version:        208234
  UID:                     841d3b17-a19b-4789-ad5b-adb3cfb61601
Rationale:                 ACS provides a method to continuously monitor and protect the Kubernetes environment against vulnerabilities and misconfigurations. This ensures that the infrastructure and applications are compliant with security standards and regulations, reducing the risk of security breaches.
Severity:                  medium
Status:                    FAIL
Events:                    <none>

Observation 2: upstream-ocp4-pci-dss-4-0-acs-sensor-exists rule is not listed in oc get rules
@Vincent056 Could you please help me check this issue? Thanks

BhargaviGudi · 2024-04-02T08:58:34Z

Verification passed with 4.16.0-0.nightly-2024-04-01-213440 + compliance-operator.v1.4.0 + rhacs-operator.v4.4.0

Install CO and rhacs-operator
Create ssb

$ oc compliance bind -N test profile/upstream-ocp4-pci-dss-4-0 profile/upstream-ocp4-pci-dss-node-4-0
Creating ScanSettingBinding test
$ oc get suite 
NAME   PHASE   RESULT
test   DONE    NON-COMPLIANT
$ oc get scan
NAME                                    PHASE   RESULT
upstream-ocp4-pci-dss-4-0               DONE    NON-COMPLIANT
upstream-ocp4-pci-dss-node-4-0-master   DONE    NON-COMPLIANT
upstream-ocp4-pci-dss-node-4-0-worker   DONE    NON-COMPLIANT
$ oc get ccr | grep acs
upstream-ocp4-pci-dss-4-0-acs-sensor-exists                                                    FAIL     medium

Apply manual remediation

$ sh applications/openshift/general/acs_sensor_exists/tests/ocp4/e2e-remediation.sh
+ echo 'Mimicking the behavior of a deployed scanner'
Mimicking the behavior of a deployed scanner
+ oc apply -f ocp-resources/e2e/acs-sensor-install.yaml --server-side=true
namespace/stackrox serverside-applied
deployment.apps/sensor serverside-applied
+ sleep 30
+ echo 'waiting for gitops deployment to exist'
waiting for gitops deployment to exist
++ oc wait -n stackrox --for=condition=Available --timeout=300s deployment/sensor
+ '[' -z 'deployment.apps/sensor condition met' ']'
+ echo 'waiting for gitops deployment to be ready'
waiting for gitops deployment to be ready
+ oc wait -n stackrox --for=condition=Available --timeout=300s deployment/sensor
deployment.apps/sensor condition met

$ oc get ccr | grep acs
upstream-ocp4-pci-dss-4-0-acs-sensor-exists                                                    FAIL     medium
$ oc compliance rerun-now scansettingbinding test
Rerunning scans from 'test': upstream-ocp4-pci-dss-4-0, upstream-ocp4-pci-dss-node-4-0-master, upstream-ocp4-pci-dss-node-4-0-worker
Re-running scan 'openshift-compliance/upstream-ocp4-pci-dss-4-0'
Re-running scan 'openshift-compliance/upstream-ocp4-pci-dss-node-4-0-master'
Re-running scan 'openshift-compliance/upstream-ocp4-pci-dss-node-4-0-worker'
$ oc get suite
NAME   PHASE   RESULT
test   DONE    NON-COMPLIANT
$ oc get ccr | grep acs
upstream-ocp4-pci-dss-4-0-acs-sensor-exists                                                    PASS     medium

BhargaviGudi · 2024-04-02T08:58:42Z

/unhold

BhargaviGudi · 2024-04-02T08:58:47Z

/lgtm

rhmdnd · 2024-04-05T20:21:32Z

applications/openshift/general/acs_sensor_exists/rule.yml

+  runtime violations based on RHACS Cloud Service policies. In addition, Sensor is
+  responsible for all cluster interactions, such as applying network policies,
+  initiating reprocessing of RHACS Cloud Service policies, and interacting with
+  the Admission controller.


One minor note is that we can probably add a link to either stackrox or ACS documentation as a way for reader to find more information.

rhmdnd · 2024-04-05T20:23:06Z

applications/openshift/general/acs_sensor_exists/rule.yml

+  cce@ocp4: CCE-86171-6
+
+references:
+  pcidss: Req-6.3.2,Req-11.3.1.1,Req-11.5.1.1


We'll need to incorporate versions here before we release PCI-DSS v4.0.0.

rhmdnd

/lgtm

We can cleanup the documentation reference in a follow up. It also makes sense to do the versioning for the profile references separately from this change.

rhmdnd · 2024-04-05T20:26:36Z

I believe all @yuumasato's feedback was resolved.

We've incorporated Yuuma's feedback into the latest PR.

Vincent056 force-pushed the acs branch from 50705e6 to 862af69 Compare March 12, 2024 04:26

yuumasato self-assigned this Mar 13, 2024

yuumasato previously requested changes Mar 13, 2024

View reviewed changes

Vincent056 force-pushed the acs branch 3 times, most recently from 1a4b1b8 to 5105016 Compare March 18, 2024 14:56

rhmdnd reviewed Mar 18, 2024

View reviewed changes

applications/openshift/general/acs_sensor_exists/rule.yml Outdated Show resolved Hide resolved

Vincent056 force-pushed the acs branch from 5105016 to 1c60063 Compare March 21, 2024 01:28

OCP4: Add rule to check ACS sensor deployed

397a3e6

Add rule acs_sensor_exists, so we can check that ACS sensor is being deployed on the cluster

Vincent056 force-pushed the acs branch from 1c60063 to 397a3e6 Compare April 1, 2024 14:58

rhmdnd reviewed Apr 5, 2024

View reviewed changes

rhmdnd approved these changes Apr 5, 2024

View reviewed changes

rhmdnd requested a review from yuumasato April 5, 2024 20:26

rhmdnd merged commit 12735ee into ComplianceAsCode:master Apr 5, 2024
41 of 45 checks passed

yuumasato added this to the 0.1.73 milestone Apr 8, 2024

yuumasato mentioned this pull request Apr 23, 2024

OCP4: Add container_security_operator_exists to PCIDSS profile #11776

Merged

Mab879 added the Update Rule Issues or pull requests related to Rules updates. label May 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCP4: Add rule to check ACS sensor deployed #11675

OCP4: Add rule to check ACS sensor deployed #11675

Vincent056 commented Mar 12, 2024

github-actions bot commented Mar 12, 2024 •

edited

github-actions bot commented Mar 12, 2024 •

edited

Vincent056 commented Mar 18, 2024

openshift-ci bot commented Mar 18, 2024

Vincent056 commented Mar 18, 2024

rhmdnd Mar 18, 2024

rhmdnd Mar 18, 2024

dashrews78 Mar 18, 2024

Vincent056 Mar 19, 2024

SimonBaeumer Mar 21, 2024 •

edited

lvalerom Mar 21, 2024

Vincent056 Mar 26, 2024

rhmdnd commented Mar 18, 2024

openshift-ci bot commented Apr 1, 2024

codeclimate bot commented Apr 1, 2024

BhargaviGudi commented Apr 1, 2024

BhargaviGudi commented Apr 2, 2024

BhargaviGudi commented Apr 2, 2024

BhargaviGudi commented Apr 2, 2024

rhmdnd Apr 5, 2024

rhmdnd Apr 5, 2024

rhmdnd left a comment

rhmdnd commented Apr 5, 2024


		ocil_clause: 'ACS Sensor is not deployed'

		{{% set jqfilter = '[ .items[] \| select(.metadata.name == "sensor" and .metadata.labels["app.kubernetes.io/name"] == "stackrox") \| .status.availableReplicas]' %}}

OCP4: Add rule to check ACS sensor deployed #11675

OCP4: Add rule to check ACS sensor deployed #11675

Conversation

Vincent056 commented Mar 12, 2024

Description:

github-actions bot commented Mar 12, 2024 • edited

github-actions bot commented Mar 12, 2024 • edited

Vincent056 commented Mar 18, 2024

openshift-ci bot commented Mar 18, 2024

Vincent056 commented Mar 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SimonBaeumer Mar 21, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhmdnd commented Mar 18, 2024

openshift-ci bot commented Apr 1, 2024

codeclimate bot commented Apr 1, 2024

BhargaviGudi commented Apr 1, 2024

BhargaviGudi commented Apr 2, 2024

BhargaviGudi commented Apr 2, 2024

BhargaviGudi commented Apr 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhmdnd left a comment

Choose a reason for hiding this comment

rhmdnd commented Apr 5, 2024

github-actions bot commented Mar 12, 2024 •

edited

github-actions bot commented Mar 12, 2024 •

edited

SimonBaeumer Mar 21, 2024 •

edited