Many of these words are used pretty much interchangeably what reduces the clarity of the code and the documentation. So, we should define a clear meaning to each of these words within ReFrame and its documentation and use only as few different words as minimally required (BTW in the documentation, this kind of definitions could go into a glossary if at some point we have a considerable amount of such).
Here are some notes from Vasilis on the subject:
-
"test" and "check" are indeed used interchangeably. The origin of the term "check" has though some historic background. When I started implementing the framework and describing/explaining it to people, I wanted to differentiate the unit "tests" of the framework itself from the regression "checks" written with the framework. That's why I came up with the word "check", which, I admit, I didn't find very attractive from the beginning.
-
The "job" is a separate concept from the the test case. A "job" refers to the job instance that is associated with the test. The "test case", on the other hand, is essentially the tuple "regression test, current partition, current environment".
Please note here your thoughts on this subject.
Internal issue: https://madra.cscs.ch/scs/reframe/issues/422