OCPBUGS-17216: Update rotate certificates check for OCP 4.14 #10973

rhmdnd · 2023-08-09T19:20:37Z

By default, the rotate certificates rules for CIS 1.4.0 (section 4.2)
fail on OpenShift 4.14.

This commit updates the rule to check for the proper configuration so
that it passes by default, since certificate rotation is enabled by
default. This patch also updates the instructions to use a valid command
for users looking to verify the configuration manually. The old command
didn't return anything because it was looking in the wrong configuration
section.

This is documented upstream in the following doc:

https://kubernetes.io/docs/reference/access-authn-authz/kubelet-tls-bootstrapping/#certificate-rotation

github-actions · 2023-08-09T19:22:11Z

Start a new ephemeral environment with changes proposed in this pull request:

ocp4 (from CTF) Environment (using Fedora as testing environment)

Fedora Testing Environment

Oracle Linux 8 Environment

github-actions · 2023-08-09T19:23:29Z

This datastream diff is auto generated by the check Compare DS/Generate Diff

Click here to see the full diff

New content has different text for rule 'xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation'.
--- xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation
+++ xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation
@@ -7,10 +7,7 @@
 file /etc/kubernetes/kubelet.conf
 on the kubelet node(s) and set the below parameter:
 
-featureGates:
-...
-  RotateKubeletServerCertificate: true
-...
+serverTLSBootstrap: true
 
 [warning]:
 This rule's check operates on the cluster configuration dump. This will be a Platform rule, var_role_worker and var_role_master needed to be set if scan is not expected to run on master, and worker nodes.

OCIL for rule 'xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation' differs.
--- ocil:ssg-kubelet_enable_server_cert_rotation_ocil:questionnaire:1
+++ ocil:ssg-kubelet_enable_server_cert_rotation_ocil:questionnaire:1
@@ -1,5 +1,5 @@
 Run the following command on the kubelet node(s):
-$ for NODE_NAME in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$NODE_NAME/proxy/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubelet.config.k8s.io/v1beta1"' | grep RotateKubeletServerCertificate; done
+$ for node in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$node/proxy/configz | jq '.kubeletconfig.serverTLSBootstrap'; done
 The output should return true.
       Is it the case that the kubelet cannot rotate server certificate?
       
New content has different text for rule 'xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation_master'.
--- xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation_master
+++ xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation_master
@@ -7,10 +7,7 @@
 file /etc/kubernetes/kubelet.conf
 on the kubelet node(s) and set the below parameter:
 
-featureGates:
-...
-  RotateKubeletServerCertificate: true
-...
+serverTLSBootstrap: true
 
 [reference]:
 CIP-003-8 R6

OCIL for rule 'xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation_master' differs.
--- ocil:ssg-kubelet_enable_server_cert_rotation_master_ocil:questionnaire:1
+++ ocil:ssg-kubelet_enable_server_cert_rotation_master_ocil:questionnaire:1
@@ -1,5 +1,5 @@
 Run the following command on the kubelet node(s):
-$ for NODE_NAME in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$NODE_NAME/proxy/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubelet.config.k8s.io/v1beta1"' | grep RotateKubeletServerCertificate; done
+$ for node in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$node/proxy/configz | jq '.kubeletconfig.serverTLSBootstrap' done
 The output should return true.
       Is it the case that the kubelet cannot rotate server certificate?
       
New content has different text for rule 'xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation_worker'.
--- xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation_worker
+++ xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation_worker
@@ -7,10 +7,7 @@
 file /etc/kubernetes/kubelet.conf
 on the kubelet node(s) and set the below parameter:
 
-featureGates:
-...
-  RotateKubeletServerCertificate: true
-...
+serverTLSBootstrap: true
 
 [reference]:
 CIP-003-8 R6

OCIL for rule 'xccdf_org.ssgproject.content_rule_kubelet_enable_server_cert_rotation_worker' differs.
--- ocil:ssg-kubelet_enable_server_cert_rotation_worker_ocil:questionnaire:1
+++ ocil:ssg-kubelet_enable_server_cert_rotation_worker_ocil:questionnaire:1
@@ -1,5 +1,5 @@
 Run the following command on the kubelet node(s):
-$ for NODE_NAME in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$NODE_NAME/proxy/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubelet.config.k8s.io/v1beta1"' | grep RotateKubeletServerCertificate; done
+$ for node in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$node/proxy/configz | jq '.kubeletconfig.serverTLSBootstrap'; done
 The output should return true.
       Is it the case that the kubelet cannot rotate server certificate?

applications/openshift/kubelet/kubelet_enable_server_cert_rotation_master/rule.yml

applications/openshift/kubelet/kubelet_enable_server_cert_rotation_worker/rule.yml

applications/openshift/kubelet/kubelet_enable_server_cert_rotation/rule.yml

By default, the rotate certificates rules for CIS 1.4.0 (section 4.2) fail on OpenShift 4.14. This commit updates the rule to check for the proper configuration so that it passes by default, since certificate rotation is enabled by default. This patch also updates the instructions to use a valid command for users looking to verify the configuration manually. The old command didn't return anything because it was looking in the wrong configuration section. This is documented upstream in the following doc: https://kubernetes.io/docs/reference/access-authn-authz/kubelet-tls-bootstrapping/#certificate-rotation

rhmdnd · 2023-08-10T17:55:49Z

/test help

openshift-ci · 2023-08-10T17:57:45Z

@rhmdnd: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test e2e-aws-ocp4-cis
/test e2e-aws-ocp4-cis-node
/test e2e-aws-ocp4-e8
/test e2e-aws-ocp4-high
/test e2e-aws-ocp4-high-node
/test e2e-aws-ocp4-moderate
/test e2e-aws-ocp4-moderate-node
/test e2e-aws-ocp4-pci-dss
/test e2e-aws-ocp4-pci-dss-node
/test e2e-aws-ocp4-stig
/test e2e-aws-ocp4-stig-node
/test e2e-aws-rhcos4-e8
/test e2e-aws-rhcos4-high
/test e2e-aws-rhcos4-moderate
/test e2e-aws-rhcos4-stig
/test images

Use /test all to run the following jobs that were automatically triggered:

pull-ci-ComplianceAsCode-content-master-images

In response to this:

/test help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

rhmdnd · 2023-08-10T17:58:04Z

/test e2e-aws-ocp4-cis
/test e2e-aws-ocp4-cis-node

codeclimate · 2023-08-10T18:27:55Z

Code Climate has analyzed commit 325aaa5 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 53.3% (0.0% change).

View more on Code Climate.

Vincent056

do we need to put the CPE in here now?

xiaojiey · 2023-08-11T05:41:53Z

Verification pass with 4.14.0-0.nightly-2023-08-10-072041 + content in the PR:
$ oc compliance bind -N test-u profile/upstream-ocp4-cis
Creating ScanSettingBinding test-u
$ oc get suite
NAME PHASE RESULT
test-u DONE NON-COMPLIANT
$ oc get ccr upstream-ocp4-cis-kubelet-enable-server-cert-rotation
NAME STATUS SEVERITY
upstream-ocp4-cis-kubelet-enable-server-cert-rotation PASS medium
$ oc get ccr upstream-ocp4-cis-kubelet-enable-server-cert-rotation -o=jsonpath={.instructions}
Run the following command on the kubelet node(s):
$ for node in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$node/proxy/configz | jq '.kubeletconfig.serverTLSBootstrap'; done
The output should return true.
$ oc get ccr upstream-ocp4-cis-kubelet-enable-server-cert-rotation -o=jsonpath={.description}
kubelet - Enable Server Certificate Rotation
To enable the kubelet to rotate server certificates, edit the kubelet configuration file /etc/kubernetes/kubelet.conf on the kubelet node(s) and set the below parameter:

serverTLSBootstrap: true[xiyuafor node in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$node/proxy/configz | jq '.kubeletconfig.serverTLSBootstrap'; doneerverTLSBootstrap'; done
true
true
true
true
true
true

yuumasato · 2023-08-11T10:19:56Z

do we need to put the CPE in here now?

@Vincent056 It is not clear to me why we would need CPE.

It seems that serverTLSBootstrap has been available for quite some time already.

rhmdnd · 2023-08-11T13:53:20Z

It seems that serverTLSBootstrap has been available for quite some time already.

I think so.

I was able to find serverTLSBootstrap in the Kubernetes 1.23 documentation reference, which is what we use for OpenShift 4.10.

This rule should work across all supported versions of OpenShift.

Vincent056 · 2023-08-11T14:53:46Z

/lgtm

rhmdnd requested review from Vincent056, yuumasato and xiaojiey August 9, 2023 19:21

xiaojiey reviewed Aug 10, 2023

View reviewed changes

applications/openshift/kubelet/kubelet_enable_server_cert_rotation_master/rule.yml Outdated Show resolved Hide resolved

xiaojiey reviewed Aug 10, 2023

View reviewed changes

applications/openshift/kubelet/kubelet_enable_server_cert_rotation_worker/rule.yml Outdated Show resolved Hide resolved

xiaojiey reviewed Aug 10, 2023

View reviewed changes

applications/openshift/kubelet/kubelet_enable_server_cert_rotation/rule.yml Outdated Show resolved Hide resolved

marcusburghardt added OpenShift OpenShift product related. CIS CIS Benchmark related. labels Aug 10, 2023

rhmdnd force-pushed the OCPBUGS-17216 branch from 9133713 to 325aaa5 Compare August 10, 2023 17:54

rhmdnd requested a review from xiaojiey August 10, 2023 17:55

Vincent056 reviewed Aug 10, 2023

View reviewed changes

yuumasato self-assigned this Aug 11, 2023

yuumasato approved these changes Aug 11, 2023

View reviewed changes

rhmdnd merged commit 2147636 into ComplianceAsCode:master Aug 11, 2023
32 of 36 checks passed

yuumasato added this to the 0.1.70 milestone Aug 14, 2023

Mab879 added the Update Rule Issues or pull requests related to Rules updates. label Oct 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-17216: Update rotate certificates check for OCP 4.14 #10973

OCPBUGS-17216: Update rotate certificates check for OCP 4.14 #10973

rhmdnd commented Aug 9, 2023 •

edited

github-actions bot commented Aug 9, 2023

github-actions bot commented Aug 9, 2023 •

edited

rhmdnd commented Aug 10, 2023

openshift-ci bot commented Aug 10, 2023

rhmdnd commented Aug 10, 2023

codeclimate bot commented Aug 10, 2023

Vincent056 left a comment

xiaojiey commented Aug 11, 2023

yuumasato commented Aug 11, 2023

rhmdnd commented Aug 11, 2023

Vincent056 commented Aug 11, 2023

OCPBUGS-17216: Update rotate certificates check for OCP 4.14 #10973

OCPBUGS-17216: Update rotate certificates check for OCP 4.14 #10973

Conversation

rhmdnd commented Aug 9, 2023 • edited

github-actions bot commented Aug 9, 2023

github-actions bot commented Aug 9, 2023 • edited

rhmdnd commented Aug 10, 2023

openshift-ci bot commented Aug 10, 2023

rhmdnd commented Aug 10, 2023

codeclimate bot commented Aug 10, 2023

Vincent056 left a comment

Choose a reason for hiding this comment

xiaojiey commented Aug 11, 2023

yuumasato commented Aug 11, 2023

rhmdnd commented Aug 11, 2023

Vincent056 commented Aug 11, 2023

rhmdnd commented Aug 9, 2023 •

edited

github-actions bot commented Aug 9, 2023 •

edited