[testlib] Add OSU benchmarks in the test library and adapt CSCS checks accordingly

[rsarm]

No description provided.

[codecov-commenter]

Codecov Report

Merging #2421 (0a330ac) into master (bc20ca7) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2421   +/-   ##
=======================================
  Coverage   85.76%   85.76%           
=======================================
  Files          57       57           
  Lines       10580    10580           
=======================================
  Hits         9074     9074           
  Misses       1506     1506           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b71da59...0a330ac. Read the comment docs.

[rsarm]

I'm having troubles passing the version of the benchmark from the cscs test to the fetch_osu_benchmark fixture on the hpclib. I'm creating variables version = variable(str, value='5.6.2') and passing them to the fixtures with {'version': version}. The problem is that in the body of the class, where the fixture is defined, the value of version is the default one (from the definition of the variable) and not the one passed through variables, which only will take effect after init right?

[pep8speaks]

Hello @rsarm, Thank you for updating!

Cheers! There are no PEP8 issues in this Pull Request!Do see the ReFrame Coding Style Guide

Comment last updated at 2022-04-01 15:03:15 UTC

[vkarak]

I think that the benchmarks, especially on the CSCS side are more complex than they should. Also, follow the Amber and Gromacs examples for writing the library tests. That means, you should not name the base file as __init__.py and the library tests must also be proper tests that can be run from the command line.

[vkarak]

@rsarm I think you haven't add the new file, but the placement need to change as well. You should be able to simply import the tests from hpctestlib.microbenchmarks.mpi.osu and not hpctestlib.microbenchmarks.mpi.osu.mpi.

[rsarm]

The lib tests can be run like this:

RFM_COMPACT_TEST_NAMES=true ./bin/reframe -C config/cscs.py -r -c hpctestlib/microbenchmarks/mpi/osu.py -n 'osu_cpu_latency_pt2pt.osu_binaries.build_type=cuda' -S valid_systems='dom:gpu' -S valid_prog_environs=PrgEnv-gnu -S osu_cpu_latency_pt2pt.device_buffers=cuda

[vkarak]

Actually, I managed to run the test on Dom as follows:

./bin/reframe -C config/cscs.py -r -c hpctestlib/microbenchmarks/mpi/osu.py -n 'osu_cpu_latency_pt2pt.osu_binaries.build_type=cuda' -S valid_systems='dom:gpu' -S valid_prog_environs=PrgEnv-gnu -S osu_cpu_latency_pt2pt.device_buffers=cuda -S osu_cpu_latency_pt2pt.variables=MPICH_RDMA_ENABLED_CUDA:1 -M 'PrgEnv-gnu: PrgEnv-gnu craype-accel-nvidia60 gcc/9.3.0' -r

I don't like that we have to use the -M option, but that's due to #2477. Ideally, we would replace that with -S build_osu_benchmarks.modules=craype-accel-nvidia60,gcc/9.3.0.

[vkarak]

I will also do further fine tuning pass.

[vkarak]

We still need to update the docs for the test library version and then it'll be ready for merging.

[rsarm]

@vkarak I gave it a try, it looks good to me. The ci failure was only a performance error.

[vkarak]

@jenkins-cscs retry daint dom