how to print on terminal the values found with reframe.utility.sanity.extractall

[smarocchi]

Hello,
I am starting to write regression tests with ReFrame, I am using the sanity functions for extracting values. In particular

   found_measures=reframe.utility.sanity.extractall(
        r'write\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+(\w+)\s+(\w+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+([\d\.]+)\s+(\w+)\s+(\d+)', self.stdout)

to match the lines in the output

Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
write        1949.03    1949.03    1949.03       0.00    7796.10    7796.10    7796.10       0.00    0.00051         NA            NA     0      1   1    1   0     0        1         0    0      1  1048576   262144       1.0 POSIX      0

the match seems to work, but when I try to print out some values

        print(found_measures)

I got the error

File "/home/simone/Documents/reframe-test/ior_reframe_run_bench_only.py", line 22, in __init__
    print(found_measures)
  File "/home/simone/EasyBuild_INSTALL/software/ReFrame/2.19/lib/python3.6/site-packages/ReFrame_HPC-2.19-py3.6.egg/reframe/core/deferrable.py", line 69, in __str__
    return str(self.evaluate())
  File "/home/simone/EasyBuild_INSTALL/software/ReFrame/2.19/lib/python3.6/site-packages/ReFrame_HPC-2.19-py3.6.egg/reframe/core/deferrable.py", line 46, in evaluate
    arg.evaluate() if isinstance(arg, type(self)) else arg)
  File "/home/simone/EasyBuild_INSTALL/software/ReFrame/2.19/lib/python3.6/site-packages/ReFrame_HPC-2.19-py3.6.egg/reframe/core/deferrable.py", line 53, in evaluate
    ret = self._fn(*fn_args, **fn_kwargs)
  File "/home/simone/EasyBuild_INSTALL/software/ReFrame/2.19/lib/python3.6/site-packages/ReFrame_HPC-2.19-py3.6.egg/reframe/core/pipeline.py", line 744, in stdout
    return self._job.stdout
AttributeError: 'NoneType' object has no attribute 'stdout'

So, I was wondering, how can I print the elements of my match ?
Is it possible to use something like

 print(found_measures.group(1))

to obtain the single strings of the regular expression match ?

Thanks for the help, maybe it is a simple question but I have not found something about that inside Reframe documentation.

Thanks

[vkarak]

Hi @alchemroz,

The sanity functions are lazily evaluated but there are some cases where they are immediately evaluated. One of such cases is when you (explicitly or implicitly) call str() on the result of a sanity function. This is what print() does. If you want to print the elements that extractall() gives you, you have to write your own sanity function as follows:

@sn.sanity_function
def my_print(x):
    print(x)
    return True

# inside your test

sn.sanity_patterns = my_print(sn.extractall(...))

We could possibly provide a deferrable version of print in the framework in the future, since I agree it is useful sometimes.

You can find more information about the sanity functions, how are implemented and how they are evaluated here: https://reframe-hpc.readthedocs.io/en/latest/deferrables.html

[smarocchi]

Dear @vkarak,
many thanks for the help, I just tried your suggestion and it works. I needed some time to understand execution of deferrable function was trigger by calling bool on it. Anyway your method solved my problems. I have just another quick and less important question:

when I run extract all

 write_re=r'write\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s+(\w+)\s+(\w+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+([\d\.]+)\s+(\w+)\s+(\d+)'

        write_max_speed = sn.extractall(write_re, self.stdout, tag=2 )

I can isolate the group 2 of a regular expression using the tag. Can I select more than 1 tag simultaneously ?

Because I also tried to use tag=0 (return all the matched regular expression in a list ) but then when I try to select the first element of the list, the whole string with a command like

        write_all = sn.extractall(write_re, self.stdout, tag=0)

        self.sanity_patterns = sn.all([
            print_rfm(write_all[0].split() )
        ])

I obtain the error

AttributeError: '_DeferredExpression' object has no attribute 'split'

So it means that if I need to get several variables from the same output line I have to run the extract all many times with different tags ?

Thank you for the help, anyway the issue for me can be already considered solved.

Bests.

[vkarak]

Hi @alchemroz, with extractall() you can only select one capturing group. If tag == 0 the the whole match (line) will be extracted as a string. This wouldn't work even if your example with split() was correct (I'll come back to that later), because [0] in a string gives you the first character. To work with with multiple capturing groups you can use either findall() or finditer(). These functions return the regular expression match objects, so you can retrieve any information from there:

@sn.sanity_function
def extract_multiple(outfile, tag1, tag2):
    val1 = []
    val2 = []
    for match in sn.finditer(write_re, outfile):
        # I am just assuming integers here
        val1.append(int(match.group(1)))
        val2.append(int(match.group(2)))

    return val1, val2

Here I just return the extracted values as two lists, but you could also process them inside your sanity function.

Coming back to the error you are getting. When you write the following:

        write_all = sn.extractall(write_re, self.stdout, tag=0)
        self.sanity_patterns = sn.all([
            print_rfm(write_all[0].split() )
        ])

the return value of write_all is not yet a list; it will become a list as soon as it is evaluated in the sanity checking phase. For the moment, it is a special object that has captured the call and its arguments; this is the _DeferredExpression object. You can use this object in pretty much any Python expression (except in for loops, if clauses and with the str() function) or function call and you will get another deferred expression. So write_all[0] above is valid; it will be evaluated to the expected value in the future. But trying to call split() on it, it will fail, because it's not a real string object; it is a "deferred expression". To achieve the result you want you have to write another sanity function that wraps the split:

@sn.sanity_function
def my_split(s):
    return s.split()

self.sanity_patterns = sn.all([print_rfm(my_split(write_all[0]))])

[vkarak]

Hi @alchemroz, this is fixed in #1062. A print() function is now available as a sanity functions. I will close this issue.

Cheers.