Fix pattern matching produced by --ci-generate

[cedricjourdain]

While upgrading Reframe from version 3.11.2 to 4.1.1, I have issues with the command generated with the --ci-generate option and the use of subtest = parameter(['xxxx']) in the reframe python script. The command reframe --purge-env -R -c cines-checks/apps -r --ci-generate=tmp_apps.yml generates the command reframe --prefix=rfm-stage/${CI_COMMIT_SHORT_SHA} -C /lus/home/BCINES/dci/jourdain/gaia/reframe/settings. py -c cines-checks/libraries/magma -R -n '^MagmaCheck_2$' -r with in particular this last part (^MagmaCheck_2$) which is then misinterpreted by reframe (i.e. it doesn't find any test [ PASSED ] Ran 0/0 test case(s) from 0 check(s) (0 failure(s), 0 skipped), due to the bad matching pattern framed by ^$). Replacing 2$ with * makes the tests work.

@vkarak told me that "The way -n option’s argument is interpreted has changed and the --ci-generate command was not updated accordingly."

$ reframe --version
3.11.2
$ reframe --purge-env -R -c reframe/cines-checks/libraries --keep-stage-files -r --ci-generate=tmp_libs_3.11.2.yml
[ReFrame Setup]
  version:           3.11.2
  command:           '/lus/home/BCINES/dci/jourdain/reframe-3.11.2/reframe-repo/bin/reframe --purge-env -R -c reframe/cines-checks/libraries --keep-stage-files -r --ci-generate=tmp_libs_3.11.2.yml'
  launched by:       jourdain@login0
  working directory: '/lus/home/BCINES/dci/jourdain/git/gaia'
  settings file:     '/lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py'
  check search path: (R) '/lus/home/BCINES/dci/jourdain/git/gaia/reframe/cines-checks/libraries'
  stage directory:   '/lus/home/BCINES/dci/jourdain/git/gaia/stage'
  output directory:  '/lus/home/BCINES/dci/jourdain/git/gaia/output'

[List of matched checks]
- BoostPythonBindingsTest
- MagmaCheck %subtest=zunmbr
- MagmaCheck %subtest=ztranspose
- MagmaCheck %subtest=zsymmetrize
- MagmaCheck %subtest=zgemm
- TestLibsciAccDgemm %subtest=F %linkage=dynamic
- TestLibsciAccDgemm %subtest=C %linkage=dynamic
Found 7 check(s)

[Generate CI]
  Gitlab pipeline generated successfully in 'tmp_libs_3.11.2.yml'.

Log file(s) saved in '/lus/home/BCINES/dci/jourdain/git/gaia/rfm.log'

$ grep -i MagmaCheck_zgemm tmp_libs_3.11.2.yml | grep reframe
  - reframe --prefix=rfm-stage/${CI_COMMIT_SHORT_SHA} -C /lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py -c reframe/cines-checks/libraries -R --report-file=MagmaCheck_zgemm-report.json  --report-junit=MagmaCheck_zgemm-report.xml  -n '^MagmaCheck_zgemm$' -r
 
$ reframe --prefix=rfm-stage/${CI_COMMIT_SHORT_SHA} -C /lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py -c reframe/cines-checks/libraries -R --report-file=MagmaCheck_zgemm-report.json  --report-junit=MagmaCheck_zgemm-report.xml  -n '^MagmaCheck_zgemm$' -r
[ReFrame Setup]
  version:           3.11.2
  command:           '/lus/home/BCINES/dci/jourdain/reframe-3.11.2/reframe-repo/bin/reframe --prefix=rfm-stage/ -C /lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py -c reframe/cines-checks/libraries -R --report-file=MagmaCheck_zgemm-report.json --report-junit=MagmaCheck_zgemm-report.xml -n ^MagmaCheck_zgemm$ -r'
  launched by:       jourdain@login0
  working directory: '/lus/home/BCINES/dci/jourdain/git/gaia'
  settings file:     '/lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py'
  check search path: (R) '/lus/home/BCINES/dci/jourdain/git/gaia/reframe/cines-checks/libraries'
  stage directory:   '/lus/home/BCINES/dci/jourdain/git/gaia/rfm-stage/stage'
  output directory:  '/lus/home/BCINES/dci/jourdain/git/gaia/rfm-stage/output'

[==========] Running 1 check(s)
[==========] Started on Wed Mar 22 16:18:00 2023

[----------] start processing checks
[ RUN      ] MagmaCheck %subtest=zgemm @adastra:gpu+CPE-22.11-gcc-11.2.0
[ RUN      ] MagmaCheck %subtest=zgemm @adastra:gpu+CPE-22.11-cce-15.0.0
[ RUN      ] MagmaCheck %subtest=zgemm @adastra:gpu+CPE-22.11-rocmcc-5.2.3
[...]

$ export PATH=/lus/home/BCINES/dci/jourdain/reframe-4.1.1/reframe-repo/bin:$PATH
$ reframe --version
4.1.1
$ reframe --purge-env -R -c reframe/cines-checks/libraries --keep-stage-files -r --ci-generate=tmp_libs_4.1.1.yml
WARNING: RFM_CONFIG_FILE is deprecated; please use RFM_CONFIG_FILES instead
[ReFrame Setup]
  version:           4.1.1
  command:           '/lus/home/BCINES/dci/jourdain/reframe-4.1.1/reframe-repo/bin/reframe --purge-env -R -c reframe/cines-checks/libraries --keep-stage-files -r --ci-generate=tmp_libs_4.1.1.yml'
  launched by:       jourdain@login0
  working directory: '/lus/home/BCINES/dci/jourdain/git/gaia'
  settings files:    '<builtin>', '/lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py'
  check search path: (R) '/lus/home/BCINES/dci/jourdain/git/gaia/reframe/cines-checks/libraries'
  stage directory:   '/lus/home/BCINES/dci/jourdain/git/gaia/stage'
  output directory:  '/lus/home/BCINES/dci/jourdain/git/gaia/output'
  log files:         '/lus/home/BCINES/dci/jourdain/git/gaia/rfm.log'

[List of matched checks]
- BoostPythonBindingsTest /171f5d2e
- MagmaCheck %subtest=zunmbr /4483be18
- MagmaCheck %subtest=ztranspose /95473fbe
- MagmaCheck %subtest=zsymmetrize /40c93851
- MagmaCheck %subtest=zgemm /90de14a6
- TestLibsciAccDgemm %subtest=F %linkage=dynamic /121f557e
- TestLibsciAccDgemm %subtest=C %linkage=dynamic /1122df39
Found 7 check(s)

[Generate CI]
  Gitlab pipeline generated successfully in 'tmp_libs_4.1.1.yml'.

Log file(s) saved in '/lus/home/BCINES/dci/jourdain/git/gaia/rfm.log'

$ grep -i MagmaCheck_0tmp_libs_4.1.1.yml | grep reframe
  - reframe --prefix=rfm-stage/${CI_COMMIT_SHORT_SHA} -C /lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py -c reframe/cines-checks/libraries -R --report-file=MagmaCheck_0-report.json  --report-junit=MagmaCheck_0-report.xml  -n '^MagmaCheck_0$' -r

$ reframe --prefix=rfm-stage/${CI_COMMIT_SHORT_SHA} -C /lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py -c reframe/cines-checks/libraries -R --report-file=MagmaCheck_0-report.json  --report-junit=MagmaCheck_0-report.xml  -n '^MagmaCheck_0$' -r
WARNING: RFM_CONFIG_FILE is deprecated; please use RFM_CONFIG_FILES instead
[ReFrame Setup]
  version:           4.1.1
  command:           '/lus/home/BCINES/dci/jourdain/reframe-4.1.1/reframe-repo/bin/reframe --prefix=rfm-stage/ -C /lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py -c reframe/cines-checks/libraries -R --report-file=MagmaCheck_0-report.json --report-junit=MagmaCheck_0-report.xml -n ^MagmaCheck_0$ -r'
  launched by:       jourdain@login0
  working directory: '/lus/home/BCINES/dci/jourdain/git/gaia'
  settings files:    '<builtin>', '/lus/home/BCINES/dci/jourdain/git/gaia/reframe/settings.py'
  check search path: (R) '/lus/home/BCINES/dci/jourdain/git/gaia/reframe/cines-checks/libraries'
  stage directory:   '/lus/home/BCINES/dci/jourdain/git/gaia/rfm-stage/stage'
  output directory:  '/lus/home/BCINES/dci/jourdain/git/gaia/rfm-stage/output'
  log files:         '/lus/home/BCINES/dci/jourdain/git/gaia/rfm.log'

[==========] Running 0 check(s)
[==========] Started on Wed Mar 22 16:18:56 2023

[----------] start processing checks
[----------] all spawned checks have finished

[  PASSED  ] Ran 0/0 test case(s) from 0 check(s) (0 failure(s), 0 skipped)
[==========] Finished on Wed Mar 22 16:18:56 2023
Run report saved in 'MagmaCheck_0-report.json'
Log file(s) saved in '/lus/home/BCINES/dci/jourdain/git/gaia/rfm.log'

Looking to issues and PR, this one PR 2160 seems to indicate where to do changes