-
Notifications
You must be signed in to change notification settings - Fork 117
Description
When writing a test about a feature which is only available from a certain version of a given application onwards, it would be a good thing to skip the test on old versions which do not support the feature, in order to avoid noise due to failures.
Currently the only documented ways to skip tests all take action from the ReFrame command line, using flags/options to filter tests based on name/PrgEnv/tags.
However, that requires to modify the CLI depending on the version of a tested software or on the environment variables currently loaded/mapped.
Would it be possible to have a way to skip a test from the ReFrame's Python API?
Having something like pytest's skipping functions/decorators would be great. Integrating packages like semver could also be an idea.
This way, tests could be self-sufficient in determining if they are suitable for a given application version, gracefully integrating into automated ReFrame runs.