ReFrame fails to load modules before job submission

[vkarak]

Issue reported by @bcfriesen in the Slack channel:

I have a question about how ReFarme handles Tcl modules. A few tests I am writing for NERSC require certain modules to be loaded before the job is submitted (the Slurm submit filter checks that the modules are loaded and rejects the job submission if they are not). However, when I add the necessary modules to the setup() phase, the logs indicate that ReFrame is indeed attempting to load them via modulecmd python module load <module>, but that seems to fail because when it submits the job to Slurm, the job is rejected because the module is not loaded. Any ideas what is going on?
I notice that modulecmd python load <module> takes no action other than printing the environment changes that it would make to the terminal. I wonder if ReFrame is not actually evaluating these outputs for some reason?

This looks like a regression in ReFrame. Some time ago (when we introduced the build systems) we changed slightly the behaviour of the setup(), and its only purpose now is to record the necessary load/unload sequence of modules. In the past, we kept the environment (in Python) until the end of the test, that’s why this scenario worked then but not now. I will open an issue about that. For the moment, you can run ReFrame with -m MODULE, which will load MODULE before starting running any test.