OCP4 content cleanup #4970

evgenyz · 2019-11-04T18:49:33Z

Make OCP4 content usable

jhrozek · 2019-11-05T12:14:27Z

Unfortunately this still does not work for me. I pushed the content into this repo/branch:
https://github.com/jhrozek/content/tree/ek_ocp4
from there I built this container: quay.io/jhrozek/ocp4-openscap-content:ek_test
you can view all the builds here:
https://quay.io/repository/jhrozek/ocp4-openscap-content?tab=builds
and still I'm getting not applicable.

I used this pod to test:

apiVersion: "v1"
kind: Pod
metadata:
    name: scanner-test
spec:
    initContainers:
        - name: content-container
          image: quay.io/jhrozek/ocp4-openscap-content:ek_test
          command: ["sh", "-c", "cp /var/lib/content/*.xml /content"]
          volumeMounts:
              - mountPath: "/content"
                name: content-volume
    containers:
        - name: openscap-scanner
          image: quay.io/jhrozek/openscap-ocp
          command: ["sh", "-c", "sleep 3600"]
          env:
              - name: HOSTROOT
                value: "/host"
              - name: PROFILE
                value: "xccdf_org.ssgproject.content_profile_coreos-fedramp"
              - name: CONTENT
                value: "ssg-ocp4-ds.xml"
              - name: REPORT_DIR
                value: "/reports"
              - name: RULE
                value: "xccdf_org.ssgproject.content_rule_selinux_state"
          volumeMounts:
              - mountPath: "/reports"
                name: report-volume
              - mountPath: "/content"
                name: content-volume
              - mountPath: "/host"
                name: host-root
    restartPolicy: Never
    volumes:
        - name: report-volume
          emptyDir: {}
        - name: content-volume
          emptyDir: {}
        - name: host-root
          hostPath:
              path: /

Then you can create the pod with:

oc create -f openscap-pod-with-init.yaml

This would first copy the content from the init container and then the scanner container would just sleep. So you can rsh into it:

oc rsh --container openscap-scanner pods/scanner-test sh

And then you can just do whatever you like, inspect the content:

oscap info /content/ssg-ocp4-ds.xml 
Document type: Source Data Stream
Imported: 2019-11-05T12:06:43

Stream: scap_org.open-scap_datastream_from_xccdf_ssg-ocp4-xccdf-1.2.xml
Generated: (null)
Version: 1.3
Checklists:
        Ref-Id: scap_org.open-scap_cref_ssg-ocp4-xccdf-1.2.xml
                Status: draft
                Generated: 2019-11-04
                Resolved: true
                Profiles:
                        Title: Open Computing Information Security Profile for OpenShift Node
                                Id: xccdf_org.ssgproject.content_profile_opencis-node
                        Title: Open Computing Information Security Profile for OpenShift Node
                                Id: xccdf_org.ssgproject.content_profile_coreos-fedramp
                Referenced check files:
                        ssg-ocp4-oval.xml
                                system: http://oval.mitre.org/XMLSchema/oval-definitions-5
                        ssg-ocp4-ocil.xml
                                system: http://scap.nist.gov/schema/ocil/2
Checks:
        Ref-Id: scap_org.open-scap_cref_ssg-ocp4-oval.xml
        Ref-Id: scap_org.open-scap_cref_ssg-ocp4-ocil.xml
        Ref-Id: scap_org.open-scap_cref_ssg-ocp4-cpe-oval.xml
Dictionaries:
        Ref-Id: scap_org.open-scap_cref_ssg-ocp4-cpe-dictionary.xml

Or run the scan:

oscap-chroot /host xccdf eval --fetch-remote-resources --profile xccdf_org.ssgproject.content_profile_coreos-fedramp --report /tmp/report.xml --rule xccdf_org.ssgproject.content_rule_selinux_state /content/ssg-ocp4-ds.xml
Title   Ensure SELinux State is Enforcing                                 
Rule    xccdf_org.ssgproject.content_rule_selinux_state
E: oscap:       RPM: cannot open Packages database in    
                                                                               
error: cannot open Packages database in                      
E: oscap:       RPM: cannot open Packages database in 
                                                                               
error: cannot open Packages database in 
Result  notapplicable

jhrozek · 2019-11-05T12:29:43Z

feel free to ping me if you need access to an openshift cluster.
btw if I can ask for one more modification: we should name the file fedramp-moderate not just fedramp. This is my fault, I named the file originally, but it's better to include the level in the name from the start.

jan-cerny · 2019-11-05T13:53:26Z

If there isn't any RPM database in the scanned container we will have to invent a different way of determining platform applicability, eg. based on reading /etc/os-release.

jhrozek · 2019-11-05T14:22:42Z

@ashcrow had some idea how could a RHCOS node be detected, but sadly, I didn't take notes. Do you still remember what you proposed @ashcrow ?

ashcrow · 2019-11-05T15:17:38Z

@jhrozek on the node itself /etc/os-release does denote it's RHCOS. Here is an example from a local build:

NAME="Red Hat Enterprise Linux CoreOS"
VERSION="42.80.20191022.0"
VERSION_ID="4.2"
PRETTY_NAME="Red Hat Enterprise Linux CoreOS 42.80.20191022.0 (Ootpa)"
ID="rhcos"
ID_LIKE="rhel fedora"
ANSI_COLOR="0;31"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="OpenShift Container Platform"
REDHAT_BUGZILLA_PRODUCT_VERSION="4.2"
REDHAT_SUPPORT_PRODUCT="OpenShift Container Platform"
REDHAT_SUPPORT_PRODUCT_VERSION="4.2"
OSTREE_VERSION=42.80.20191022.0

You could also look at rpm-ostree status, but the parsing may be not as easy.

jhrozek · 2019-11-05T20:40:36Z

Yeah, I think this is good, we're not trying to write content for RHCOS in any other context than OCP node (edited: earlier I said worker which is ambiguous), so at least for now we can assume that RHCOS == OCP. If the REDHAT_SUPPORT_PRODUCT and/or REDHAT_BUGZILLA_PRODUCT fields are at least sort of stable we could also look at them to distinguish between 'bare' RHCOS and OCP.

That said, we're likely need to go a step further (not needed /right now/, but still..) because AFAIK it's possible to use RHEL for the OCP worker nodes. So we'll also need to allow the OCP content to run on RHEL nodes. But, that's not needed for this first iteration, we should just keep that in mind to not shoot ourselves in the foot now.

ashcrow · 2019-11-05T20:45:38Z

@jhrozek 👍. The fields should be stable enough. You could also look at NAME which should be "Red Hat Enterprise Linux CoreOS"

shared/checks/oval/installed_app_is_ocp4.xml

redhatrises · 2019-11-05T22:04:27Z

ocp4/profiles/coreos-fedramp.profile

@@ -0,0 +1,17 @@
+documentation_complete: true


This profile should not be added at this point in time. While Fedramp is the starting target, there are other profiles that are named differently that are essentially the same profile.

shared/checks/oval/installed_env_is_ocp4.xml

redhatrises · 2019-11-05T22:08:52Z

The rpm database definitely exists on CoreOS and the redhat-release-coreos rpm can be used to identify CoreOS. If the scanning container doesn't have access to scan all of CoreOS, this is a problem.

JAORMX · 2019-11-06T14:30:51Z

shared/checks/oval/installed_OS_is_rhel8.xml

+  </ind:textfilecontent54_test>
+  <ind:textfilecontent54_object id="obj_rhel8_coreos" version="1">
+    <ind:filepath>/etc/os-release</ind:filepath>
+    <ind:pattern operation="pattern match">^PRETTY_NAME=&quot;Red Hat Enterprise Linux CoreOS \d+\.(\d+)\.\d+\.\d+ \([\w\s]+\)&quot;$</ind:pattern>


@ashcrow what do you think of this, to check if a node is RHCOS?

NAME would be a simpler check if all that matters is if it's RHCOS or not ... but I think this would work. NOTE: I just eyeballed the regex 🙂

I hope that it did not put too much strain to your eyes. I'm open to suggestions for improvements of the regex used in the check :)

shared/checks/oval/installed_OS_is_rhel8.xml

evgenyz · 2019-11-07T08:22:07Z

So, first of all, the RPM-based check is no good at this moment because, while there is an RPM database in the CoreOS edition of RHEL (as a backward compatibility) it is at least located in a different place and current OVAL probe of the oscap scanner is unable to work with it. We will most likely be fixing the probe as there are a lot of RPM checks in the rules and they might be needed in the future. But, to continue with the OCP4 content evolution, I suggest to stick with the os-release file check for now.

Next, the version. Original check, submited by @redhatrises, was checking for package version of redhat-release-coreos to be 8. But the CoreOS edition actually bears two versions mixed together, the CoreOS itself and RHEL (e.g. 4.3 and 8.1) forming a very peculiar string VERSION="42.80.20191022.0". I used the (\d)\d+ part out of it, in coherence with previous RPM-based check, but I'm not sure if it's a good idea. This check is a complimentary check to the base RHEL's, and maybe we'd like to check here for CoreOS version instead.

jhrozek · 2019-11-07T08:31:27Z

On 11/7/19 9:22 AM, Evgeny Kolesnikov wrote: Next, the version. Original check, submited by @redhatrises <https://github.com/redhatrises>, was checking for package version of |redhat-release-coreos| to be 8. But the CoreOS edition actually bears two versions mixed together, the CoreOS itself and RHEL (e.g. 4.3 and 8.1) forming a very peculiar string |VERSION="42.80.20191022.0"|.

It sounds to me like|the 42 in||VERSION="42.80.20191022.0" |is actually|the OCP version and 8.0 is the base RHEL version. My OCP 4.3 cluster has 43.81.... |

I used the |(\d)\d+| part out of it, in coherence with previous RPM-based check, but I'm not sure if it's a good idea. This check is a complimentary check to the base RHEL's, and maybe we'd like to check here for CoreOS version instead.

I'm not sure how easy it is, but checking if the OCP version is 4[digit] and the RHEL version is 8[digit] sounds the safest to me in the sense that it checks both version substrings. But maybe we're overdoing this and as was said already earlier in the PR, just checking the RHCOS is OK. Let me test your latest PR and if it works, then I guess it could be good enough.

Add initial fedramp profile for CoreOS; simplified OCP4 environment check. The "test_rhel8_coreos" check is for now based on /etc/os-release file contents.

jan-cerny · 2019-11-07T10:22:45Z

Looks good to me, I agree with you that we need to check the version == 8.*, because the check is a part of a check that checks if the operating system is RHEL 8. I think a check that would check only CoreOS without version might match future versions that will not be based on RHEL8.

jhrozek · 2019-11-07T11:37:12Z

I tested the latest version and can confirm that it works:

oscap-chroot /host xccdf eval --fetch-remote-resources --profile xccdf_org.ssgproject.content_profile_coreos-fedramp --report /tmp/report.xml /content/ssg-ocp4-ds.xml
Title   Ensure SELinux State is Enforcing
Rule    xccdf_org.ssgproject.content_rule_selinux_state
E: oscap:       RPM: cannot open Packages database in 
                                                                               
error: cannot open Packages database in 
E: oscap:       RPM: cannot open Packages database in 
                                                                               
error: cannot open Packages database in 
WARNING: Skipping rule that requires an unregistered check system or incorrect content reference to evaluate. Please consider providing a valid SCAP/OVAL instead of http://scap.nist.gov/schema/ocil/2
Result  notchecked                
                                                                               
Title   Enable auditd Service                                     
Rule    xccdf_org.ssgproject.content_rule_service_auditd_enabled
E: oscap:     RPM: cannot open Packages database in 
                                                                               
error: cannot open Packages database in 
E: oscap:     RPM: cannot open Packages database in                                                                                                           
                                                                               
error: cannot open Packages database in 
Result  fail             
                                                                               
Title   Verify All Account Password Hashes are Shadowed
Rule    xccdf_org.ssgproject.content_rule_accounts_password_all_shadowed
Result  pass

redhatrises · 2019-11-07T17:51:02Z

shared/checks/oval/installed_app_is_ocp4.xml

@@ -10,19 +10,6 @@
    </metadata>
    <criteria>
      <extend_definition comment="RHEL8 OS installed" definition_ref="installed_OS_is_rhel8" />
-      <criterion comment="OpenShift Node is installed" test_ref="test_ocp4_node" />


You still need to check if OCP4 is installed. Checking just the OS install is not enough.

@redhatrises we don't support running RHCOS standalone. So OCP4 has to be there.

@JAORMX agreed. That is my original statement in this part of the review.

If you tell me how exactly this node can be detected, I'd happily add this check.

I mean beside the obvious way of copying the checks from ocp3 and replacing the number, because RPM-based checks would not work.

In fact while we are having this discussion, there is going to have to be a way to validate with a gpg check not only the software installed in CoreOS, but also a container that is installed on coreos.

In fact while we are having this discussion, there is going to have to be a way to validate with a gpg check not only the software installed in CoreOS, but also a container that is installed on coreos.

Nothing above or beyond RHEL itself (same packages) for installed software, or OCP itself for containers.

Still have to validate regardless of source against a key to meet compliance requirements.

I say we merge this for now so we can start using this to build content already. And we start iterating on this as we go. I'm fine with merging this as it is.

I understand where you're coming from and especially the points about RHEL nodes are valid. But we need to start working on the content and even just RHCOS-based is good enough.

On the RHCOS nodes: just knowing that "a node is RHCOS" is good enough at least for now because 1) RHCOS only ever exists as an OCP node 2) The content is supposed to be used from an operator anyway which can only run in OCP.

Again, I agree with your points. But if possible, I would prefer to merge the stub of the content ASAP and then iterate on it to provide a technically more sound solution.

JAORMX

I'm fine with merging this. We can iterate on this later. Let's unblock folks so we can all continue going forward. Great work here!

jan-cerny · 2019-11-08T11:04:05Z

The support for RHEL nodes can be implemented later in separate PR. This is good enough for this moment.

evgenyz force-pushed the ocp4_ironing branch from bbd3615 to 1d2c530 Compare November 5, 2019 08:55

redhatrises reviewed Nov 5, 2019

View reviewed changes

shared/checks/oval/installed_app_is_ocp4.xml Outdated Show resolved Hide resolved

redhatrises reviewed Nov 5, 2019

View reviewed changes

shared/checks/oval/installed_env_is_ocp4.xml Outdated Show resolved Hide resolved

evgenyz force-pushed the ocp4_ironing branch from 1d2c530 to 644f314 Compare November 6, 2019 13:40

JAORMX reviewed Nov 6, 2019

View reviewed changes

jhrozek reviewed Nov 7, 2019

View reviewed changes

shared/checks/oval/installed_OS_is_rhel8.xml Outdated Show resolved Hide resolved

evgenyz force-pushed the ocp4_ironing branch from 644f314 to 09bf293 Compare November 7, 2019 08:23

OCP4 content cleanup

0188a81

Add initial fedramp profile for CoreOS; simplified OCP4 environment check. The "test_rhel8_coreos" check is for now based on /etc/os-release file contents.

evgenyz force-pushed the ocp4_ironing branch from 09bf293 to 0188a81 Compare November 7, 2019 08:40

evgenyz marked this pull request as ready for review November 7, 2019 08:43

redhatrises suggested changes Nov 7, 2019

View reviewed changes

JAORMX approved these changes Nov 8, 2019

View reviewed changes

jhrozek approved these changes Nov 8, 2019

View reviewed changes

jan-cerny self-assigned this Nov 8, 2019

jan-cerny added this to the 0.1.48 milestone Nov 8, 2019

jan-cerny merged commit dbf0b36 into ComplianceAsCode:master Nov 8, 2019

evgenyz deleted the ocp4_ironing branch November 13, 2019 06:22

OCP4 content cleanup #4970

OCP4 content cleanup #4970

Uh oh!

Conversation

evgenyz commented Nov 4, 2019

Uh oh!

jhrozek commented Nov 5, 2019

Uh oh!

jhrozek commented Nov 5, 2019

Uh oh!

jan-cerny commented Nov 5, 2019

Uh oh!

jhrozek commented Nov 5, 2019

Uh oh!

ashcrow commented Nov 5, 2019

Uh oh!

jhrozek commented Nov 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ashcrow commented Nov 5, 2019

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

redhatrises commented Nov 5, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

evgenyz commented Nov 7, 2019

Uh oh!

jhrozek commented Nov 7, 2019 via email

Uh oh!

jan-cerny commented Nov 7, 2019

Uh oh!

jhrozek commented Nov 7, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JAORMX Nov 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashcrow Nov 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JAORMX left a comment

Choose a reason for hiding this comment

Uh oh!

jan-cerny commented Nov 8, 2019

Uh oh!

Uh oh!

jhrozek commented Nov 5, 2019 •

edited

Loading

JAORMX Nov 7, 2019 •

edited

Loading

ashcrow Nov 7, 2019 •

edited

Loading