Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 37 additions & 6 deletions docs/regression_test_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ Built-in types
.. py:function:: RegressionMixin.parameter(values=None, inherit_params=False, filter_params=None)

Inserts or modifies a regression test parameter.
At the class level, these parameters are stored in a separate namespace referred to as the *parameter space*.
If a parameter with a matching name is already present in the parameter space of a parent class, the existing parameter values will be combined with those provided by this method following the inheritance behavior set by the arguments ``inherit_params`` and ``filter_params``.
Instead, if no parameter with a matching name exists in any of the parent parameter spaces, a new regression test parameter is created.
A regression test can be parameterized as follows:
Expand Down Expand Up @@ -79,14 +80,44 @@ Built-in types
else:
override_other()

Moreover, a derived class may extend, partially extend and/or modify the parameter values provided in the base class as shown below.

:param values: A list containing the parameter values.
If no values are passed when creating a new parameter, the parameter is considered as *declared* but not *defined* (i.e. an abstract parameter).
Instead, for an existing parameter, this depends on the parameter's inheritance behaviour and on whether any values where provided in any of the parent parameter spaces.
:param inherit_params: If :obj:`False`, no parameter values that may have been defined in any of the parent parameter spaces will be inherited.
.. code:: python

class ExtendVariant(Bar):
# Extend the full set of inherited variant parameter values to ['A', 'B', 'C']
variant = parameter(['C'], inherit_params=True)

class PartiallyExtendVariant(Bar):
# Extend a subset of the inherited variant parameter values to ['A', 'D']
variant = parameter(['D'], inherit_params=True,
filter_params=lambda x: x[:1])

class ModifyVariant(Bar):
# Modify the variant parameter values to ['AA', 'BA']
variant = parameter(inherit_params=True,
filter_params=lambda x: map(lambda y: y+'A', x))

A parameter with no values is referred to as an *abstract parameter* (i.e. a parameter that is declared but not defined).
Therefore, classes with at least one abstract parameter are considered abstract classes.

.. code:: python

class AbstractA(Bar):
variant = parameter()

class AbstractB(Bar):
variant = parameter(inherit_params=True, filter_params=lambda x: [])


:param values: An iterable containing the parameter values.
:param inherit_params: If :obj:`True`, the parameter values defined in any base class will be inherited.
In this case, the parameter values provided in the current class will extend the set of inherited parameter values.
If the parameter does not exist in any of the parent parameter spaces, this option has no effect.
:param filter_params: Function to filter/modify the inherited parameter values that may have been provided in any of the parent parameter spaces.
This function must accept a single argument, which will be passed as an iterable containing the inherited parameter values.
This only has an effect if used with ``inherit_params=True``.
This function must accept a single iterable argument and return an iterable.
It will be called with the inherited parameter values and it must return the filtered set of parameter values.
This function will only have an effect if used with ``inherit_params=True``.


.. py:function:: RegressionMixin.variable(*types, value=None)
Expand Down
15 changes: 12 additions & 3 deletions reframe/core/parameters.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,9 +124,18 @@ def extend(self, cls):

local_param_space = getattr(cls, self.local_namespace_name)
for name, p in local_param_space.items():
self.params[name] = (
p.filter_params(self.params.get(name, ())) + p.values
)
try:
filt_vals = p.filter_params(self.params.get(name, ()))
except Exception:
raise
else:
try:
self.params[name] = (tuple(filt_vals) + p.values)
except TypeError:
raise ReframeSyntaxError(
f"'filter_param' must return an iterable "
f"(parameter {name!r})"
) from None

# If any previously declared parameter was defined in the class body
# by directly assigning it a value, raise an error. Parameters must be
Expand Down
18 changes: 17 additions & 1 deletion unittests/test_parameters.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,29 @@ class MyTest(TwoParams):

def test_filter_params():
class MyTest(ExtendParams):
P1 = parameter(inherit_params=True, filter_params=lambda x: x[2:])
# We make the return type of the filtering function a list to ensure
# that other iterables different to tuples are also valid.
P1 = parameter(inherit_params=True,
filter_params=lambda x: list(x[2:]))

assert MyTest.param_space['P0'] == ('a',)
assert MyTest.param_space['P1'] == ('d', 'e',)
assert MyTest.param_space['P2'] == ('f', 'g',)


def test_wrong_filter():
def bad_filter(x):
raise RuntimeError('bad filter')

with pytest.raises(RuntimeError):
class Foo(ExtendParams):
P1 = parameter(inherit_params=True, filter_params=bad_filter)

with pytest.raises(ReframeSyntaxError):
class Foo(ExtendParams):
P1 = parameter(inherit_params=True, filter_params=lambda x: 1)


def test_is_abstract_test():
class MyTest(Abstract):
pass
Expand Down