[feat] Parameterise test from command line

[vkarak]

This PR introduces parameterisation from the command line on existing variables. Passing -P num_tasks=1,2,4,8 will create a parameterised version of a test with different num_tasks values. This feature cannot parameterise fixtures.

Closes #2665.

Todos

• Update the docs

[codecov]

Codecov Report

Patch coverage: 94.44% and project coverage change: +0.33 🎉

Comparison is base (eaad7e1) 87.01% compared to head (9ae80f2) 87.35%.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2913      +/-   ##
===========================================
+ Coverage    87.01%   87.35%   +0.33%     
===========================================
  Files           60       60              
  Lines        11649    11676      +27     
===========================================
+ Hits         10136    10199      +63     
+ Misses        1513     1477      -36     

Impacted Files	Coverage Δ
reframe/frontend/testgenerators.py	95.65% <93.10%> (-0.35%)	⬇️
reframe/core/meta.py	99.12% <100.00%> (-0.01%)	⬇️
reframe/frontend/cli.py	82.19% <100.00%> (+5.87%)	⬆️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[ekouts]

Looks good overall. Should we mention somewhere in the docs that this parameterisation doesn't work for fixtures?

[vkarak]

Looks good overall. Should we mention somewhere in the docs that this parameterisation doesn't work for fixtures?

It's not exactly that it does not work with fixtures; it parameterises the leaf test, which itself can have fixtures. It's that you can't parameterise a nested fixture, such as -P TEST.FIXT.var=1,2,3,4. I will add a small note in the docs.