Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove machine pruning from gating #10453

Merged
merged 3 commits into from
Apr 13, 2023
Merged

Remove machine pruning from gating #10453

merged 3 commits into from
Apr 13, 2023

Conversation

ggbecker
Copy link
Member

Description:

  • Add option to keep machine platform in Automatus gating.

Rationale

  • It's time to remove the machine platform pruning and let tests return notapplicable and do not get confused by bogus test results.

There is a commit test to trigger Automatus gating test which should not be part of the final PR.

@ggbecker ggbecker added this to the 0.1.68 milestone Apr 11, 2023
@ggbecker
Copy link
Member Author

A different way of mitigate what this is proposing: #10387

@github-actions
Copy link

Start a new ephemeral environment with changes proposed in this pull request:

rhel8 (from CTF) Environment (using Fedora as testing environment)
Open in Gitpod

Fedora Testing Environment
Open in Gitpod

Oracle Linux 8 Environment
Open in Gitpod

@ggbecker ggbecker force-pushed the remove-machine-pruning-from-gating branch from 1dbe470 to 398ea18 Compare April 11, 2023 16:00
@jan-cerny jan-cerny self-assigned this Apr 12, 2023
jan-cerny
jan-cerny reviewed Apr 12, 2023
View reviewed changes
Copy link
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The testing farm job on CS8 and 9 fails due to #10450 and isn't related to this PR.

tests/test_rule_in_container.sh Outdated
printf '\t%s\n' "<rule>: The short rule ID. Wildcards are supported."
printf '\t%s\n' "-n, --name: Name of the test image (default: 'ssg_test_suite')"
printf '\t%s\n' "-s, --scenarios: Regex to reduce selection of tested scenarios (no default)"
printf '\t%s\n' "-d, --datastream: Path to the datastream to use in tests. Autodetected by default. (no default)"
printf '\t%s\n' "-r, --remediate-using: What to remediate with. Can be one of: 'oscap', 'bash' and 'ansible' (default: 'oscap')"
printf '\t%s\n' "-l, --logdir: Directory where logs will be stored (no default)"
printf '\t%s\n' "--dontclean, --no-dontclean: Dont remove HTML reports from the log directory. (off by default)"
printf '\t%s\n' "--dontclean, --no-dontclean: Don't remove HTML reports from the log directory. (off by default)"
printf '\t%s\n' "--keep-machine-platform, --no-keep-machine-platform: Don't remove machine platforms. (off by default)"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can confuse someone. Does "off by default" mean that if people don't use this option the "machine" platforms will be kept or does it mean that they will be removed?

I suggest renaming this to --remove-machine-platform and the default behavior should be to not remove the machine platform.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another naming proposal would be to rename it to --remove-machine-only which would be consistent with Automatus as this script is a wrapper over Automatus.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense, I'm going to rename to --remove-machine-only but make it on by default so it doesn't change the behavior of the script if there is already someone using it.

@ggbecker
Copy link
Member Author

It seems to work as expected:

Setting console output to log level INFO
INFO - The base image option has been specified, choosing Podman-based test environment.
INFO - Logging into logs_bash/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_service_rsyslog_enabled
WARNING - Script service_disabled.fail.sh using profile (all) notapplicable
WARNING - Rule xccdf_org.ssgproject.content_rule_service_rsyslog_enabled evaluation resulted in notapplicable
WARNING - Script service_enabled.pass.sh using profile (all) notapplicable
WARNING - Rule xccdf_org.ssgproject.content_rule_service_rsyslog_enabled evaluation resulted in notapplicable

@ggbecker ggbecker force-pushed the remove-machine-pruning-from-gating branch from 398ea18 to 4430928 Compare April 12, 2023 08:11
@ggbecker ggbecker requested a review from jan-cerny April 12, 2023 08:13
@ggbecker ggbecker force-pushed the remove-machine-pruning-from-gating branch 2 times, most recently from 7ae366e to aba93fc Compare April 12, 2023 08:15
@ggbecker ggbecker force-pushed the remove-machine-pruning-from-gating branch from aba93fc to 06d377c Compare April 12, 2023 10:17
@codeclimate
Copy link

codeclimate bot commented Apr 12, 2023

Code Climate has analyzed commit 06d377c and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 52.4% (0.0% change).

View more on Code Climate.

jan-cerny
jan-cerny approved these changes Apr 13, 2023
View reviewed changes
Copy link
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes!

The TF on CS 8 and 9 fails due to #10450 , this fail is unrelated to the contents of this PR.

@jan-cerny jan-cerny merged commit 01ee23c into ComplianceAsCode:master Apr 13, 2023
27 of 29 checks passed
@jan-cerny jan-cerny added the Test Suite Update in Test Suite. label Apr 13, 2023
@ggbecker
Copy link
Member Author

ggbecker commented Apr 13, 2023 •
edited

The commit 06d377c was not supposed to be merged :(

It was just for testing purpose.

I'm going to open a PR to remove undo the change.

@ggbecker ggbecker mentioned this pull request Apr 13, 2023
Merged
@jan-cerny jan-cerny mentioned this pull request Jun 6, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jan-cerny jan-cerny jan-cerny approved these changes

Assignees

@jan-cerny jan-cerny

Labels
Test Suite Update in Test Suite.
Projects
None yet
Milestone
0.1.68
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@ggbecker @jan-cerny