-
Notifications
You must be signed in to change notification settings - Fork 671
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix Automatus in CI #11494
Fix Automatus in CI #11494
Conversation
Skipping CI for Draft Pull Request. |
2b1a253
to
13eb69d
Compare
Some rules depend on rule `installed_OS_is_FIPS_certified` which makes it difficult to run test scenarios of these rules on different systems that aren't certified by FIPS. For example, this happens when you run test scenarios on Fedora container or CentOS container. This situation is currently handled by a feature of Automatus using the `--add-product-to-fips-certified` option which extends the OVAL in rule `installed_OS_is_FIPS_certified` to make it pass on a selected platform. Unfortunately, this depends on assumption that there exist an OVAL definition `installed_OS_is_${product}`. After recent changes in build system, it doesn't have to be true, because if this definition isn't used by any rule, it gets removed by code that filters out unused definitions. We will add a new Automatus option '--remove-fips-certified' which will remove all `<oval-def:extend_definition>` elements that reference OVAL definition for rule `installed_OS_is_FIPS_certified` from all other rules. As a result, no rule will depend on `installed_OS_is_FIPS_ceritifed` when this option will be used.
Use `--remove-fips-certified` in CI instead of using `--add-product-to-fips-certified` which is unreliable and broken at this moment.
This option depends on assumption that the given SCAP source data stream contains definitions of all possible products which might not be the case if the definiton isn't used by any rule in that data stream. The feature is fragile. We have replaced it by the `--remove-fips-certified` option.
40ae62e
to
4a362c9
Compare
This is a link to the output of the Automatus Fedora / Run Tests job: https://github.com/ComplianceAsCode/content/actions/runs/7707656693/job/21005286727?pr=11494. Now, I will remove the test commit so that I can make it ready for review. |
4a362c9
to
3ca2eb0
Compare
Code Climate has analyzed commit 3ca2eb0 and detected 0 issues on this pull request. The test coverage on the diff in this pull request is 100.0% (50% is the threshold). This pull request will bring the total coverage in the repository to 58.3% (0.0% change). View more on Code Climate. |
/packit retest-failed |
1 similar comment
/packit retest-failed |
@mildas are aware anything else that might need adjusting with this change? |
I don't think. Other places where AutoMatus is used doesn't use this option. |
Description:
Some rules depend on rule
installed_OS_is_FIPS_certified
which makesit difficult to run test scenarios of these rules on different systems
that aren't certified by FIPS. For example, this happens when you run
test scenarios on Fedora container or CentOS container.
This situation is currently handled by a feature of Automatus using the
--add-product-to-fips-certified
option which extends the OVAL in ruleinstalled_OS_is_FIPS_certified
to make it pass on a selected platform.Unfortunately, this depends on assumption that there exist an OVAL
definition
installed_OS_is_${product}
. After recent changes in buildsystem, it doesn't have to be true, because if this definition isn't
used by any rule, it gets removed by code that filters out unused
definitions.
This option depends on assumption that the given SCAP source data
stream contains definitions of all possible products which might
not be the case if the definiton isn't used by any rule in that
data stream. The feature is fragile
We will add a new Automatus option '--remove-fips-certified' which will
remove all
<oval-def:extend_definition>
elements that reference OVALdefinition for rule
installed_OS_is_FIPS_certified
from all otherrules. As a result, no rule will depend on
installed_OS_is_FIPS_certifed
when this option will be used.Review Hints:
Check the Automatus CI jobs.