Add Smoke Tests for Some Scripts #9787
Conversation
Mab879
added
enhancement
|
Start a new ephemeral environment with changes proposed in this pull request: Fedora Environment Oracle Linux 8 Environment |
| COMMAND env "PYTHONPATH=$ENV{PYTHONPATH}" "${PYTHON_EXECUTABLE}" "${CMAKE_SOURCE_DIR}/utils/create_srg_export.py" --root "${CMAKE_SOURCE_DIR}" --json "${CMAKE_SOURCE_DIR}/build/rule_dirs.json" --control "${CMAKE_SOURCE_DIR}/controls/srg_gpos.yml" --product rhel9 --out-format xlsx --output "${CMAKE_BINARY_DIR}/cac_stig_output.xlsx" --build-config-yaml "${CMAKE_BINARY_DIR}/build_config.yml" | ||
| ) | ||
| set_tests_properties("srg-export-rhel9" PROPERTIES LABELS quick) | ||
| set_tests_properties("srg-export-rhel9" PROPERTIES DEPENDS "test-rule-dir-json") |
There was a problem hiding this comment.
I don't like the DEPENDS on other tests, because I can't run the test standalone using the -R option. I was looking into fixtures https://cmake.org/cmake/help/latest/prop_test/FIXTURES_REQUIRED.html but they are new in CMake 3.7. So I would propose to create the rule_dir.json as a part of this test.
There was a problem hiding this comment.
I will need to do some research but my initial reaction to that is that we might not be be able to use fixtures as RHEL 7 has CMake 2.7.
There was a problem hiding this comment.
It appears we can use fixures even with the older CMake that RHEL 7 uses. I left the DEPENDS so that we are backward compatible in the full runs.
This allow for running one test with `ctest -R` with CMake 3.7+.
Make sure that only Python 3 is used and all python deps are installed.
Mab879
force-pushed
the
smoke_test_srg_diff
branch
from
f4605d4
to
9abae2e
Compare
|
Code Climate has analyzed commit 9abae2e and detected 0 issues on this pull request. The test coverage on the diff in this pull request is 100.0% (50% is the threshold). This pull request will bring the total coverage in the repository to 46.8% (0.0% change). View more on Code Climate. |
|
/packit retest-failed |
|
Thanks, I like those new smoke tests. |
|
It should be possible; we would need to write XML files that follow how DISA does it. |
|
I was thinking in the opposite direction - take the current test content and throw about 99% of it away, so only matching skeletons remain. Skeleton in, skeleton out. |
Description:
utils/compare_ds.pyutils/create_srg_export.pyutils/srg_diff.pyRationale:
Improve CI coverage.