[bugfix] Get performance info from within the changedir

[teojgo]

This allows performing more complicated performance queries such as:

def reference_meminfo(self):
        regex = 'memory from sysconf: total: \S+ \S+ avail: (?P<mem>\S+) GB'
        return sn.extractsingle(regex, self.stdout, 'mem', int)

If the above is executed outside the stagedir then self.stdout is not found.

[teojgo]

@jgphpc can you check this?

[codecov-commenter]

Codecov Report

Base: 86.26% // Head: 86.26% // Increases project coverage by +0.00% 🎉

Coverage data is based on head (3bee759) compared to base (db93224).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2625   +/-   ##
=======================================
  Coverage   86.26%   86.26%           
=======================================
  Files          60       60           
  Lines       10985    10987    +2     
=======================================
+ Hits         9476     9478    +2     
  Misses       1509     1509           

Impacted Files	Coverage Δ
reframe/frontend/executors/policies.py	92.54% <100.00%> (+0.04%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[ekouts]

@jenkins-cscs retry daint

[vkarak]

It looks like that this fix is a workaround for something different. At this point the test has finished and perfvalues is a "static" value; it does not compute anything on-the-fly, so I was wondering how this could fail?

[teojgo]

It looks like that this fix is a workaround for something different. At this point the test has finished and perfvalues is a "static" value; it does not compute anything on-the-fly, so I was wondering how this could fail?

The problem was coming when trying to run the test given here: eth-cscs/cscs-reframe-tests#30

It was because the performance info was querying self.stdout which is just the name of the output file and cannot be retrieved as is if outside the stage dir.

[vkarak]

The problem in this case is not how we call the _print_perf() in that part of the code. The problem is two-fold here. The test is ill-formed and ReFrame accepts it nonetheless. More specifically, the problem is that reference contains a deferred expression (the call to reference_meminfo()), which shouldn't be accepted. Then when we try to print the reference from the perfvalues after the test has finished, we (re)trigger it's evaluation and of course we fail at a strange location. The fix atm should be at the test's side, because it's ill formed and as far as the framework is concerned, I'll open issue to restrict the types accepted as references.