Add utils/controlrefcheck.py

[Mab879]

Description:

Add a script to test if the control id equals the reference of a rule.

Rationale:

To ensure that the control file and the rules are in sync.

Review Hints:

$ ./utils/controlrefcheck.py rhel8 cis_rhel8 cis

Example output:

Skipping reload_dconf_db as it does not match a CIS id.
Skipping enable_authselect as it does not match a CIS id.
selinux_state cis@rhel8 1.7.1.4 does not match the control id 1.6.1.5
configure_crypto_policy cis@rhel8 1.10,1.11 does not match the control id 1.1
package_xinetd_removed cis@rhel8 2.1.1 does not match the control id 2.2.1
rsyslog_files_permissions cis@rhel8 4.2.3 does not match the control id 4.2.1.4
sudo_require_reauthentication cis@rhel8 5.3.6 does not match the control id 5.3.5
accounts_password_pam_retry cis@rhel8 5.4.1 does not match the control id 5.5.1
accounts_passwords_pam_faillock_unlock_time cis@rhel8 5.4.2 does not match the control id 5.5.2
set_password_hashing_algorithm_systemauth cis@rhel8 5.4.4 does not match the control id 5.5.4
set_password_hashing_algorithm_passwordauth cis@rhel8 5.4.4 does not match the control id 5.5.4

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[github-actions]

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment

Oracle Linux 8 Environment

[jhrozek]

This is great! I already used the script to find bugs in OCP4 PCI-DSS (PR #10098).
One suggestion: In get_rule_object, could we handle None returned from all_rules.get() to handle the situation I ran into?

[jhrozek]

And a question: This seems like something that should be ran in CI after we fix the initial issues. Is that what the hunk that modifies ctest does?

[Mab879]

This is great! I already used the script to find bugs in OCP4 PCI-DSS (PR #10098). One suggestion: In get_rule_object, could we handle None returned from all_rules.get() to handle the situation I ran into?

Yes, I can work on that.

And a question: This seems like something that should be ran in CI after we fix the initial issues. Is that what the hunk that modifies ctest does?

That is correct.

[jhrozek]

LGTM
I tried running the tool against the RHEL profiles that you fixed and verified that they are indeed fixed. Running against OCP4 profiles that have issues does produce meaningful messages.

ship it :-)

[jan-cerny]

the function is too complex

[Mab879]

Do we want to lower the threshold in Code Climate, as this function passed the complexity before I changed it?

[jan-cerny]

probably not because in other situations I find Codeclimate strict enough

[jan-cerny]

/packit test

[jan-cerny]

I have tried to run it and I also tried to break reference consistency in some rule.ymls and the script has shown the expected output. I have also executed the CTest locally and I have seen that it runs in the upstream GitHub Actions in the Fedora latest and other jobs.

[codeclimate]

Code Climate has analyzed commit ba3056f and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 49.5% (0.0% change).

View more on Code Climate.