monkeypatching in e.g. run_before() methods

[sjpb]

I've got a test method decorated with a @run_before() method, like (for example):

@rfm.run_before('run')
def no_run(self):
        """ Turn the run phase into a no-op to help debugging. """

        with open(os.path.join(self.stagedir, 'noop.sh'), 'w') as noop:
            noop.write('#!/bin/bash\necho "noop $@"\n')
        self.executable = "./noop.sh"

which really I want to pull out into a separate library so I can use it on various classes (it's more complex than the method here so it makes more sense!).

I can't figure out how to monkey-patch this in though - the run_before decorator seems to make a difference. Ideally I'd do something like:

@set_no_run
@rfm.simple_test
class MyTest(rfm.RegressionTest):
    <snip>

I tried this:

def set_no_run(cls):

    def no_run(self):
        """ Turn the run phase into a no-op. """

        with open(os.path.join(self.stagedir, 'noop.sh'), 'w') as noop:
            noop.write('#!/bin/bash\necho "noop $@"\n')
        self.executable = "./noop.sh"

    cls.no_run = no_run
    cls.no_run = rfm.run_before('run')(cls.no_run)
    return cls

but (unsurprisingly?) that doesn't work (as in, doesn't change the executable).

Maybe I'm missing something about the way binding works in Py3 here but I'd appreciate any suggestions for how to do something like this, even if it's via another route.

[sjpb]

I guess one key motivation not shown in the above is that really the library-provided decorator (or however it's done) is going to add methods at multiple pipeline stages.

[vkarak]

The reason that this does not work is that hooks mechanism works essentially through a metaclass, which is responsible for the creation of the RegressionTest. All the decorator does is set a function attribute. This attribute will be used by the metaclass when creating the RegressionTest class in order to set up the list of pipeline hooks for each pipeline stage. That means that in order of this mechanism to work, the body of the test class should have been executed before. A class decorator, as in your case, acts too late, after the class has been created, so whichever method you decorate it will be simply ignored.

One thing that could potentially help you is that pipeline hooks are inherited, so you could possibly define a mixin class that your test could inherit from as well:

class DisableRun(rfm.RegressionTest):
    @rfm.run_before('run')
    def no_run(self):
        """ Turn the run phase into a no-op to help debugging. """

        with open(os.path.join(self.stagedir, 'noop.sh'), 'w') as noop:
            noop.write('#!/bin/bash\necho "noop $@"\n')

        self.executable = "./noop.sh"


@rfm.simple_test
class MyTest(rfm.RunOnlyRegressionTest):
    ...

I haven't tested this, but it could work in principle. The downside is that you might have to have different classes to wrap each of your utility decorators and if MyTest sets self.executable in another @rfm.run_before('run') decorator, it will essentially override yours.

If I come up with a better solution, I will share it here.

[vkarak]

A bit more "esoteric" solution if you don't want to wrap your library hook in a class every time is the following, which works nicely:

import functools
import os
import reframe as rfm
import reframe.utility.sanity as sn


def sticky_hook(func):
    class _T(rfm.RegressionTest):
        @functools.wraps(func)
        def _hook(self):
            func(self)

    # Copy any ReFrame attributes from func to the internal test method
    for k, v in func.__dict__.items():
        if k.startswith('_rfm'):
            _T._hook.__dict__[k] = v

    return _T


@sticky_hook
@rfm.run_before('run')
def no_run(test):
    """ Turn the run phase into a no-op to help debugging. """

    with open(os.path.join(test.stagedir, 'noop.sh'), 'w') as noop:
        noop.write('#!/bin/bash\necho "noop $@"\n')

    test.executable = "./noop.sh"


@rfm.simple_test
class MyTest(rfm.RunOnlyRegressionTest, no_run):
    def __init__(self):
        self.valid_systems = ['*']
        self.valid_prog_environs = ['*']
        self.executable = 'echo'
        self.executable_opts = ['hello']
        self.sanity_patterns = sn.assert_found(r'hello', self.stdout)

Ideally, you the wrapper class should not inherit from RegressionTest, but if you don't do that it crashes currently:

Traceback (most recent call last):
  File "/Users/karakasv/Repositories/reframe/reframe/frontend/cli.py", line 515, in main
    checks_found = loader.load_all()
  File "/Users/karakasv/Repositories/reframe/reframe/frontend/loader.py", line 165, in load_all
    checks.extend(self.load_from_file(d))
  File "/Users/karakasv/Repositories/reframe/reframe/frontend/loader.py", line 135, in load_from_file
    return self.load_from_module(util.import_module_from_file(filename))
  File "/Users/karakasv/Repositories/reframe/reframe/utility/__init__.py", line 72, in import_module_from_file
    return _do_import_module_from_file(filename, module_name)
  File "/Users/karakasv/Repositories/reframe/reframe/utility/__init__.py", line 54, in _do_import_module_from_file
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/karakasv/Repositories/misc/playground/hook_inheritance.py", line 44, in <module>
    class MyTest(rfm.RunOnlyRegressionTest, no_run):
  File "/Users/karakasv/Repositories/reframe/reframe/core/meta.py", line 51, in __init__
    if callable(v) and v.__name__ in b._final_methods:
AttributeError: type object '_T' has no attribute '_final_methods'

However, this is fixable from ReFrame's side. It's just that we haven't imagined such a use case. :-)

[sjpb]

Thank you, that's very helpful. I think the limitation with other run_before('run') methods setting self.executable overriding this is probably inherent to anything which hooks into the reframe pipeline. I'll give this a go.

That said (and maybe I'm misunderstanding your comment "A class decorator, as in your case, acts too late"), seeing as a class decorator can change the class definition, I guess it could alter the metaclass to a monkeypatched one too. Sounds like a terrible approach though.

[vkarak]

I guess it could alter the metaclass to a monkeypatched one too.

No, you can't do that with a class decorator. But you could define your own test creation metaclass that would derive from RegressionTestMeta. Then you could define your own hierarchy of basic tests, from which your actual test would derive. Something along the following lines:

from reframe.core.meta import RegressionTestMeta

class MyExtendedMeta(RegressionTestMeta):
    '''Do your magic here'''

class MyBaseRegressionTest(metaclass=RegressionTestMeta):
    '''Basic test for all of your tests'''

class MyTest(MyBaseRegressionTest):
    '''You actual test'''

But is this actually something that you want to do??

In my solution above, you can just decorate any function and make it usable as a mixin (the fact that you have to derive this from RegressionTest is something that we can fix in the framework). You could also make the _T class callable so that you can call it as a normal function, making completely transparent the fact that decorated function is essentially converted to a class :-)