Skip to content

Add possibility to skip tests from Python code #1474

@Madeeks

Description

@Madeeks

When writing a test about a feature which is only available from a certain version of a given application onwards, it would be a good thing to skip the test on old versions which do not support the feature, in order to avoid noise due to failures.
Currently the only documented ways to skip tests all take action from the ReFrame command line, using flags/options to filter tests based on name/PrgEnv/tags.
However, that requires to modify the CLI depending on the version of a tested software or on the environment variables currently loaded/mapped.

Would it be possible to have a way to skip a test from the ReFrame's Python API?
Having something like pytest's skipping functions/decorators would be great. Integrating packages like semver could also be an idea.
This way, tests could be self-sufficient in determining if they are suitable for a given application version, gracefully integrating into automated ReFrame runs.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions