RP testing: proposal on structure reorganization

Reorganization of test modules and test data [LM]

All tests are located at radical.pilot/tests. This reorganization is focused on tests for agent's Launch Methods at radical.pilot/tests/test_lm.

Current structure

/test_cases                 # - includes input and output for all/some LMs,
|                           #   1 input and many outputs (1 output per LM) per test case
|-- unit.00000.json
/test_lm

Proposed changes

/test_lm
|-- /test_cases             # - every test case is for a particular LM
    |-- srun.general.json   # - "<lm>.general.json" would have outputs for general/common inputs
    |-- srun.00000.json     # - regular test cases would have 1..n inputs/outputs per random configuration
    |-- srun.00001.json     #   or test case for a certain project

General / common inputs

If we would like to have test cases (collection of inputs) that would be used for all subcomponents (e.g., LM, RM, Schedulers, etc.), thus the following information might be used (and located inside radical.pilot/tests):

• test includes compute unit (task) and resource descriptions - all in one file

/test_inputs                # - 1 input per test case
|                           # - going from test to test - increasing the number of active parameters per test
|                           # - use real configurations along with random settings
|-- unit.summit.00001.json
|-- unit.00000.json

• unit and resource descriptions are separated - each description per file

/test_inputs
|-- unit.00000.json
|-- unit.00001.json
|-- resource.00000.json
|-- resource.summit.00000.json

• use real configs (have a reference to them) for resource descriptions

NOTE: General / common inputs (if it is ok to have them) could be considered as additional test cases, but not as "core" test cases that would be per component/class.