You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Profile selections in test case metadata are ignored when running SSGTS in combined mode for certain profile. This causes that the test scenarios which are not targeted for the tested profile are also run and will fail.
Example:
The test policy_default_set.pass.sh for the configure_crypto_policy rule fails when running SSGTS for the ospp profile because it is implemented only for the standard profile and it is N/A for the ospp. This is caused by the SSGTS (combined mode) overloading profile metadata in test scenarios.
The text was updated successfully, but these errors were encountered:
I think that this behavior has been created because the test scenarios aren't labeled properly - eg. some rule is part of multiple profiles on multiple products but the test scenario is labeled only that it belongs to 1 profile. Labeling them properly is very costly, though. I think you would have to add a lot of profiles to the headers and then test it. Also maintaining it is be difficult - each time anybody changes a profile he should change all the test scenarios that test rules added/removed from the profile. I think we can't ask the our contributors to do this. Instead, we should rethink the profile metadata.
Description of problem:
Profile selections in test case metadata are ignored when running SSGTS in combined mode for certain profile. This causes that the test scenarios which are not targeted for the tested profile are also run and will fail.
Example:
The test
policy_default_set.pass.sh
for theconfigure_crypto_policy
rule fails when running SSGTS for theospp
profile because it is implemented only for thestandard
profile and it is N/A for theospp
. This is caused by the SSGTS (combined mode) overloading profile metadata in test scenarios.The text was updated successfully, but these errors were encountered: