STOR-1277: Add Prometheus rules for SELinux telemetry

[jsafrane]

Create new SELinux metrics to send through telemetry to remove all labels except for volume_plugin. See https://issues.redhat.com/browse/STOR-1277.

cc @openshift/storage

[openshift-ci-robot]

@jsafrane: This pull request references STOR-1277 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but no target version was set.

In response to this:

Create new SELinux metrics to send through telemetry to remove all labels except for volume_plugin. See https://issues.redhat.com/browse/STOR-1277.

cc @openshift/storage

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jsafrane]

/hold
for the metrics to be approved

[jan--f]

This looks good to me. The only comment I'd have is to consider using a different prefix to indicate this comes from the volume_manager.
Perhaps cluster:volume_manager:kube_selinux_ or cluster:volume_manager_selinux.

[jsafrane]

This looks good to me. The only comment I'd have is to consider using a different prefix to indicate this comes from the volume_manager. Perhaps cluster:volume_manager:kube_selinux_ or cluster:volume_manager_selinux.

Updated to cluster:volume_manager_selinux_*

[jan--f]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jan--f, jsafrane

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jsafrane]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jsafrane]

/retest-required

[jsafrane]

/hold cancel
@jan--f from monitoring team lgtm-ed.

[jsafrane]

/label px-approved
/label docs-approved

[chao007]

curl -G -k -H "Authorization: Bearer $token" \
https://$(oc get route prometheus-k8s-federate -n \
openshift-monitoring -o jsonpath="{.spec.host}")/federate \
--data-urlencode 'match[]={__name__=~"cluster:volume_manager_selinux_volumes_admitted_total"}'

# TYPE cluster:volume_manager_selinux_volumes_admitted_total untyped
cluster:volume_manager_selinux_volumes_admitted_total{volume_plugin="kubernetes.io/csi/ebs.csi.aws.com",instance="",prometheus="openshift-monitoring/k8s",prometheus_replica="prometheus-k8s-0"} 1 1701331867402

curl -G -k -H "Authorization: Bearer $token" \
https://$(oc get route prometheus-k8s-federate -n \
openshift-monitoring -o jsonpath="{.spec.host}")/federate \
--data-urlencode 'match[]={__name__=~"cluster:volume_manager_selinux_pod_context_mismatch_total"}'
# TYPE cluster:volume_manager_selinux_pod_context_mismatch_total untyped
cluster:volume_manager_selinux_pod_context_mismatch_total{instance="",prometheus="openshift-monitoring/k8s",prometheus_replica="prometheus-k8s-0"} 49 1701332137402

[chao007]

/label qe-approved

[openshift-ci-robot]

@jsafrane: This pull request references STOR-1277 which is a valid jira issue.

In response to this:

Create new SELinux metrics to send through telemetry to remove all labels except for volume_plugin. See https://issues.redhat.com/browse/STOR-1277.

cc @openshift/storage

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 101f600 and 2 for PR HEAD 8c3435e in total

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 297bfca and 1 for PR HEAD 8c3435e in total

[Phaow]

/retest-required

[Phaow]

/retest-required

[Phaow]

/retest-required

[openshift-ci]

@jsafrane: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-ibmcloud-csi	8c3435e	link	false	/test e2e-ibmcloud-csi

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 06dc1e7 and 0 for PR HEAD 8c3435e in total

[openshift-ci-robot]

/hold

Revision 8c3435e was retested 3 times: holding

[jsafrane]

/retest-required

This PR seems to have really bad luck.

[jsafrane]

/hold cancel

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 06dc1e7 and 2 for PR HEAD 8c3435e in total

[openshift-bot]

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-cluster-storage-operator-container-v4.15.0-202312012031.p0.gcfaf9e3.assembly.stream for distgit cluster-storage-operator.
All builds following this will include this PR.