Remove unreliable test scenarios#14703
Merged
Merged
Conversation
This commit removes 3 test scenarios. These scenarios create a violating file in `/tmp`. However, these rules should skip remote and special file system. That creates a problem that the evaluation result depends on whether `/tmp` on the target system is a partition on the disk or a `tmpfs` file system. If the `/tmp` is a partition on the disk, OpenSCAP examines the files in the `/tmp` directory, the violating file in `/tmp` is detected and the rule fails as expected by the test scenarios. If the `/tmp` is a `tmpfs` file system, OpenSCAP considers it a special file system, it skips it, doesn't examine the `/tmp` directory, therefore it doesn't detect the violating file in `/tmp` and rule result is pass, which isn't expected by test scenarios. In general, we don't want test scenarios with variable behavior. Also, we don't have a way to control the expected test scenario result based on the target system properties. Therefore, we will remove the test scenarios. Fixes: ComplianceAsCode#13778
jan-cerny
requested review from
Mab879,
matusmarhefka and
vojtapolasek
as code owners
jan-cerny
added
Test Suite
Mab879
approved these changes
May 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit removes 3 test scenarios. These scenarios create a violating file in
/tmp. However, these rules should skip remote and special file system. That creates a problem that the evaluation result depends on whether/tmpon the target system is a partition on the disk or atmpfsfile system.If the
/tmpis a partition on the disk, OpenSCAP examines the files in the/tmpdirectory, the violating file in/tmpis detected and the rule fails as expected by the test scenarios.If the
/tmpis atmpfsfile system, OpenSCAP considers it a special file system, it skips it, doesn't examine the/tmpdirectory, therefore it doesn't detect the violating file in/tmpand rule result is pass, which isn't expected by test scenarios.In general, we don't want test scenarios with variable behavior. Also, we don't have a way to control the expected test scenario result based on the target system properties. Therefore, we will remove the test scenarios.
Fixes: #13778
Review Hints:
Run
/per-rule/oscap/from-envfrom Contest, add totest_varsthis item:"RULE=file_permissions_ungroupowned no_files_unowned_by_user file_permissions_unauthorized_world_writable"