Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SCAPVal to CTest #10802

Merged
merged 5 commits into from
Jul 12, 2023
Merged

Add SCAPVal to CTest #10802

merged 5 commits into from
Jul 12, 2023

Conversation

Mab879
Copy link
Member

@Mab879 Mab879 commented Jul 6, 2023

Description:

This PR adds the ability to

  • Run SCAPVal using ctest if the ENABLE_SCAPVAL13 option is enabled
  • use tests/run_scalval.py to test one file

Rationale:

  • Parallelism on the SCAPVal test

Review Hints:

export ADDITIONAL_CMAKE_OPTIONS="-DENABLE_SCAPVAL13:BOOL=ON"
./build_product -j8 fedora      
ctest -R 'scapval' --output-on-failure -j8       

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Used by openshift-ci bot. label Jul 6, 2023
@openshift-ci
Copy link

openshift-ci bot commented Jul 6, 2023

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@Mab879 Mab879 changed the title Add SCAPVal to Test Add SCAPVal to CTest Jul 6, 2023
@github-actions
Copy link

github-actions bot commented Jul 6, 2023

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment
Open in Gitpod

Oracle Linux 8 Environment
Open in Gitpod

@Mab879
Copy link
Member Author

Mab879 commented Jul 6, 2023

/packit rebuild-failed

1 similar comment
@Mab879
Copy link
Member Author

Mab879 commented Jul 7, 2023

/packit rebuild-failed

@Mab879 Mab879 marked this pull request as ready for review July 7, 2023 18:06
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Used by openshift-ci bot. label Jul 7, 2023
@jan-cerny jan-cerny added this to the 0.1.69 milestone Jul 10, 2023
@jan-cerny jan-cerny self-assigned this Jul 10, 2023
CMakeLists.txt Outdated
# SCAPVal Options
option(ENABLE_SCAPVAL13 "Set to ON to enable scapval tests" OFF)
set(SCAPVAL_VERSION "1.3.5" CACHE STRING "What version of SCAPVal to use")
set(SCAPVAL_PATH "/opt/scapval/SCAP-Content-Validation-Tool-${SCAPVAL_VERSION}/scapval-${SCAPVAL_VERSION}.jar" CACHE PATH "Which minor version of OVAL are we targetting. Only 11 is supported at the moment.")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In CMake the paths to tools can't be hardcoded but they need to be found by the find_program or by a custom find module or similar construction.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just removed the default value, you need to provide -DSCAPVAL_PATH='/opt/scapval/SCAP-Content-Validation-Tool-1.3.5/scapval-1.3.5.jar' to give the path to the SCAPVal jar.

tests/run_scapval.py Outdated Show resolved Hide resolved
@codeclimate
Copy link

codeclimate bot commented Jul 10, 2023

Code Climate has analyzed commit 2dc4f96 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 53.5% (0.0% change).

View more on Code Climate.

Copy link
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have built RHEL 9 content using this command:

[jcerny@fedora scap-security-guide{pr/10802}]$ ADDITIONAL_CMAKE_OPTIONS="-DENABLE_SCAPVAL13:BOOL=ON -DSCAPVAL_PATH=/opt/scapval/scapval-1.3.5.jar" ./build_product -d rhel9 

Then, I have executed the new CTest test introduced by your PR:

[jcerny@fedora build{pr/10802}]$ ctest --verbose -R scapval
UpdateCTestConfiguration  from :/home/jcerny/work/git/scap-security-guide/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/jcerny/work/git/scap-security-guide/build/DartConfiguration.tcl
Test project /home/jcerny/work/git/scap-security-guide/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 52
    Start 52: scapval-rhel9

52: Test command: /usr/bin/env "PYTHONPATH=/home/jcerny/work/git/scap-security-guide" "/usr/bin/python3" "/home/jcerny/work/git/scap-security-guide/tests/run_scapval.py" "--scap-version" "1.3" "--scapval-path" "/opt/scapval/scapval-1.3.5.jar" "--datastream" "/home/jcerny/work/git/scap-security-guide/build/ssg-rhel9-ds.xml"
52: Working Directory: /home/jcerny/work/git/scap-security-guide/build/rhel9
52: Test timeout computed to be: 10000000
52: Build dir: /home/jcerny/work/git/scap-security-guide/build
52: Testing ssg-rhel9-ds.xml ...
52: ssg-rhel9-ds.xml: PASS
1/1 Test #52: scapval-rhel9 ....................   Passed   91.73 sec

The following tests passed:
	scapval-rhel9

100% tests passed, 0 tests failed out of 1

Label Time Summary:
scapval    =  91.73 sec*proc (1 test)

Total Test time (real) =  91.73 sec
[jcerny@fedora build{pr/10802}]$ 

@jan-cerny
Copy link
Collaborator

The CI fails on Rawhide aren't related to the content of this PR.

@jan-cerny jan-cerny merged commit f7dc5dd into ComplianceAsCode:master Jul 12, 2023
28 of 30 checks passed
@jan-cerny jan-cerny added the Test Suite Update in Test Suite. label Jul 12, 2023
@Mab879 Mab879 deleted the ctest_scapval branch July 12, 2023 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Test Suite Update in Test Suite.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants