Add documentation of the steps that OVAL content goes through during the build #11336
Conversation
openshift-ci
bot
added
the
do-not-merge/work-in-progress
|
Skipping CI for Draft Pull Request. |
Honny1
added
Documentation
|
Start a new ephemeral environment with changes proposed in this pull request: Fedora Environment Oracle Linux 8 Environment |
openshift-ci
bot
removed
the
do-not-merge/work-in-progress
| @@ -86,6 +86,13 @@ of occurrence: | |||
| - Generate content for derived products (such as CentOS and Scientific Linux). | |||
| - Generate HTML tables, Bash scripts, Ansible Playbooks and other secondary artifacts. | |||
|
|
|||
| ## How OVAL is build | |||
There was a problem hiding this comment.
| ## How OVAL is build | |
| ## How OVAL is Built |
Title case per the style guide and consider change the tense for clarity.
| @@ -86,6 +86,13 @@ of occurrence: | |||
| - Generate content for derived products (such as CentOS and Scientific Linux). | |||
| - Generate HTML tables, Bash scripts, Ansible Playbooks and other secondary artifacts. | |||
|
|
|||
| ## How OVAL is build | |||
|
|
|||
| Creating OVAL checks is done in two steps. First, all available OVAL checks are combined into a one unlinked OVAL document. The OVAL shorthands are loaded into the OVAL Document object and, in the case of template shorthand, extended using jinja macros before loading. If the shorthand is already loaded into the OVAL Document object, it is skipped. Shorthands are loaded in the order that benchmark checks are loaded first, followed by shared directory checks. | |||
There was a problem hiding this comment.
Per the style guide please have one sentence per line.
There was a problem hiding this comment.
I expect this to be more verbose and detailed.
You have recognized two major steps of the process. But, I would like to see to split them into more granular substeps.
As you describe a sequence of steps, I think a form of numbered list can be used to describe the process of processing OVALs. You can use a list with nested lists.
I have add some helpful questions below that will help you with extending the text.
| @@ -86,6 +86,13 @@ of occurrence: | |||
| - Generate content for derived products (such as CentOS and Scientific Linux). | |||
| - Generate HTML tables, Bash scripts, Ansible Playbooks and other secondary artifacts. | |||
|
|
|||
| ## How OVAL is build | |||
There was a problem hiding this comment.
| ## How OVAL is build | |
| ## How OVAL is built |
| @@ -86,6 +86,13 @@ of occurrence: | |||
| - Generate content for derived products (such as CentOS and Scientific Linux). | |||
| - Generate HTML tables, Bash scripts, Ansible Playbooks and other secondary artifacts. | |||
|
|
|||
| ## How OVAL is build | |||
|
|
|||
| Creating OVAL checks is done in two steps. First, all available OVAL checks are combined into a one unlinked OVAL document. The OVAL shorthands are loaded into the OVAL Document object and, in the case of template shorthand, extended using jinja macros before loading. If the shorthand is already loaded into the OVAL Document object, it is skipped. Shorthands are loaded in the order that benchmark checks are loaded first, followed by shared directory checks. | |||
There was a problem hiding this comment.
This is against the style guide. There should be one sentence per line.
| @@ -86,6 +86,13 @@ of occurrence: | |||
| - Generate content for derived products (such as CentOS and Scientific Linux). | |||
| - Generate HTML tables, Bash scripts, Ansible Playbooks and other secondary artifacts. | |||
|
|
|||
| ## How OVAL is build | |||
|
|
|||
| Creating OVAL checks is done in two steps. First, all available OVAL checks are combined into a one unlinked OVAL document. The OVAL shorthands are loaded into the OVAL Document object and, in the case of template shorthand, extended using jinja macros before loading. If the shorthand is already loaded into the OVAL Document object, it is skipped. Shorthands are loaded in the order that benchmark checks are loaded first, followed by shared directory checks. | |||
There was a problem hiding this comment.
I want information about which scripts perform each of these actions.
| Creating OVAL checks is done in two steps. First, all available OVAL checks are combined into a one unlinked OVAL document. The OVAL shorthands are loaded into the OVAL Document object and, in the case of template shorthand, extended using jinja macros before loading. If the shorthand is already loaded into the OVAL Document object, it is skipped. Shorthands are loaded in the order that benchmark checks are loaded first, followed by shared directory checks. | ||
|
|
||
| The second step is to link the generated document from the previous step with the XCCDF document. | ||
| The unlinked OVAL document is loaded into the OVAL Document object while the XML file is being loaded into the object instance. Validation of the OVAL document is performed during and after loading (for example, whether all the checks listed in the XCCDF are present in the OVAL document). After validation of the OVAL document, the IDs are converted to valid OVAL IDs. The OVAL document is then saved as an XML file. During saving, a minimal OVAL document is generated for each rule as an artifact. The last step is to link the file to the XCCDF document. |
There was a problem hiding this comment.
Which XML file is being loaded into the object instance?
There was a problem hiding this comment.
Can you elaborate more on the validation step?
There was a problem hiding this comment.
Where is the "minimal OVAL document" stored?
There was a problem hiding this comment.
What does it take "to link the file to the XCCDF"? What is difference between linked and unlinked document?
There was a problem hiding this comment.
Which script performs this step?
|
I fixed the style and rewrote the description of the steps the OVAL content goes through during build. |
|
/packit retest failed |
|
|
||
| ### 1. Combination of OVALs | ||
|
|
||
| In the first step, all available and applicable for platform OVAL checks are builded into a single unlinked OVAL document stored in the `build/${PRODUCT}/oval-unlinked.xml` directory. |
There was a problem hiding this comment.
| In the first step, all available and applicable for platform OVAL checks are builded into a single unlinked OVAL document stored in the `build/${PRODUCT}/oval-unlinked.xml` directory. | |
| In the first step, all available and applicable OVAL checks are built into a single unlinked OVAL document stored in the `build/${PRODUCT}/oval-unlinked.xml` directory. |
|
|
||
| In the first step, all available and applicable for platform OVAL checks are builded into a single unlinked OVAL document stored in the `build/${PRODUCT}/oval-unlinked.xml` directory. | ||
| The `oval-unlinked.xml` document is generated using the `combine_ovals.py` script. | ||
| The OVAL shortcuts are loaded into the OVAL Document object in the order that the benchmark checks are loaded first, followed by the shared directory checks. |
There was a problem hiding this comment.
What are shortcuts? It's a new term here
|
|
||
| Steps of loading the OVAL shorthand: | ||
|
|
||
| 1. The OVAL Shorthand file is loaded as a string, and in the case of templated Shorthand, it is expanded using Jinja macros before loading. |
There was a problem hiding this comment.
Shouldn't it be the opposite?
| 1. The OVAL Shorthand file is loaded as a string, and in the case of templated Shorthand, it is expanded using Jinja macros before loading. | |
| 1. The OVAL Shorthand file is loaded as a string, and in the case of not templated Shorthand, it is expanded using Jinja macros before loading. |
| 2. The OVAL Shorthand string is processed by the OVAL Document object. | ||
| 1. The OVAL Shorthand string is loaded into the OVAL Shorthand object. | ||
| 2. the OVAL Shorthand object is validated | ||
| It is checked: |
There was a problem hiding this comment.
| It is checked: | |
| The following properties are checked: |
| 4. The OVAL definition referenced by the XCCDF is checked to be defined in the OVAL document. | ||
| 5. Verify if `<xccdf:Value>` `type` to corresponding OVAL variable `datatype` export matching [constraint](http://csrc.nist.gov/publications/nistpubs/800-126-rev2/SP800-126r2.pdf#page=30&zoom=auto,69,313) is met. | ||
| Also correct the `type` attribute of those `<xccdf:Value>` elements where necessary in order the produced content to meet this constraint. | ||
| 6. verify that the referenced CCE identifiers are correct |
There was a problem hiding this comment.
| 6. verify that the referenced CCE identifiers are correct | |
| 6. Verify that the referenced CCE identifiers are correct. |
| 5. Verify if `<xccdf:Value>` `type` to corresponding OVAL variable `datatype` export matching [constraint](http://csrc.nist.gov/publications/nistpubs/800-126-rev2/SP800-126r2.pdf#page=30&zoom=auto,69,313) is met. | ||
| Also correct the `type` attribute of those `<xccdf:Value>` elements where necessary in order the produced content to meet this constraint. | ||
| 6. verify that the referenced CCE identifiers are correct | ||
| 7. translate the identifiers in the OVAL Document object using `IDTranslator`. |
There was a problem hiding this comment.
| 7. translate the identifiers in the OVAL Document object using `IDTranslator`. | |
| 7. Translate the identifiers in the OVAL Document object using `IDTranslator`. |
| 6. verify that the referenced CCE identifiers are correct | ||
| 7. translate the identifiers in the OVAL Document object using `IDTranslator`. | ||
| 8. The OVAL Document object is stored as an XML file `build/ssg-${PRODUCT}-oval.xml`. | ||
| 9. for each XCCDF rule, a minimal OVAL Documents document is generated as an artifact |
There was a problem hiding this comment.
| 9. for each XCCDF rule, a minimal OVAL Documents document is generated as an artifact | |
| 9. For each XCCDF rule, a minimal OVAL Documents document is generated as an artifact. |
| 10. The last step is to link the XCCDF document | ||
| - For each reference of OVAL check in XCCDF, a link to the check content and a check export element is added. |
There was a problem hiding this comment.
| 10. The last step is to link the XCCDF document | |
| - For each reference of OVAL check in XCCDF, a link to the check content and a check export element is added. | |
| 10. For each reference of OVAL check in XCCDF, a link to the `check-content` and a `check-export` element is added. |
| ### 2. Linking OVAL Document | ||
|
|
||
| The second step is performed when building an XCCDF document using the `build_xccdf.py` script. | ||
| In this step, the `oval-unlinked.xml` document from the previous step is linked to the XCCDF document being built. |
There was a problem hiding this comment.
It would be great to explain that in this context the term "linking" means alignment of IDs between rules and checks.
| 1. The OVAL Shorthand file is loaded as a string, and in the case of templated Shorthand, it is expanded using Jinja macros before loading. | ||
| 2. The OVAL Shorthand string is processed by the OVAL Document object. | ||
| 1. The OVAL Shorthand string is loaded into the OVAL Shorthand object. | ||
| 2. the OVAL Shorthand object is validated |
There was a problem hiding this comment.
| 2. the OVAL Shorthand object is validated | |
| 2. The OVAL Shorthand object is validated. |
Honny1
force-pushed
the
docs-build-of-content
branch
from
f91b141
to
6efad1c
Compare
|
@jan-cerny I've incorporated your comments. |
|
Code Climate has analyzed commit 6efad1c and detected 0 issues on this pull request. The test coverage on the diff in this pull request is 100.0% (50% is the threshold). This pull request will bring the total coverage in the repository to 58.5%. View more on Code Climate. |
|
/packit retest-failed |
|
The testing farm problem on Fedora 39 is an infrastructure problem. This PR changes only the documentation, so I think the PR is harmless. |
This PR documents the steps the OVAL content goes through during the build.