DM-36649: Move ConfigurableActions from pipe_tasks

.github/workflows/build.yaml

@@ -13,7 +13,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.10", "3.11"]
 
     steps:
       - uses: actions/checkout@v3
@@ -66,7 +66,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v4
         with:
-          python-version: 3.8
+          python-version: 3.10
           cache: "pip"
           cache-dependency-path: "setup.cfg"
 

doc/changes/DM-36649.feature.rst

@@ -0,0 +1,2 @@
+Introduce ConfigurableActions. These are pex_config fields and config types which function as a functor, state
+is set at config time, but the ConfigurableActions are callable at runtime to produce an action.

doc/lsst.pex.config/field-types.rst

@@ -7,7 +7,7 @@ Types of configuration fields
 .. TODO: improve this page to summarize the purpose of each field, and then have a dedicated section for each field. https://jira.lsstcorp.org/browse/DM-17196
 
 Attributes of the configuration object must be subclasses of `Field`.
-A number of these are predefined: `Field`, `RangeField`, `ChoiceField`, `ListField`, `ConfigField`, `ConfigChoiceField`, `RegistryField` and `ConfigurableField`.
+A number of these are predefined: `Field`, `RangeField`, `ChoiceField`, `ListField`, `ConfigField`, `ConfigChoiceField`, `RegistryField`, `ConfigurableField`, `ConfigurableActionField`, and `ConfigurableActionStructField`.
 
 Example of `RangeField`:
 
@@ -78,3 +78,43 @@ Examples of `ChoiceField` and `ConfigField` and the use of the `Config` object's
             if self.doComputeApCorr and not self.doPsf:
                 raise ValueError("Cannot compute aperture correction "
                                  "without doing PSF determination.")
+
+Examples of `ConfigurableActionField` and `ConfigurableActionStructField` making use of `ConfigurableAction`\ s in a `Config` object.
+
+.. code-block:: python
+
+    class ExampleAction(pexConfig.configurableActions.ConfigurableAction):
+        """A ConfigurableAction that performs a simple calculation"""
+
+        numerator = pexConfig.Field[float](doc="Numerator for division operation")
+        divisor = pexConfig.Field[float](doc="Divisor for division operation")
+
+        def __call__(self, **kwargs):
+            return self.numerator / self.divisor
+
+
+    class ExampleConfig(pexConfig.Config):
+        """An example Config class which contains multiple `ConfigurableAction`\ s."""
+
+        divideAction = pexConfig.configurableActions.ConfigurableActionField(
+            doc="A field which points to a single action"
+            default=ExampleAction
+        )
+
+        multipleDivisionActions = pexConfig.configurableActions.ConfigurableActionStructField(
+            doc="A field which acts as a struct, referring to multiple ConfigurableActions"
+        )
+
+        def setDefaults(self):
+            """Example of setting multiple default configurations with `ConfigurableAction`\ s.
+            """
+            self.divideAction.numerator = 1
+            self.divideAction.divisor = 2
+
+            self.multipleDivisionActions.subDivide1 = ExampleAction()
+            self.multipleDivisionActions.subDivide1.numerator = 5
+            self.multipleDivisionActions.subDivide1.divisor = 10
+
+            self.multipleDivisionActions.subDivide2 = ExampleAction()
+            self.multipleDivisionActions.subDivide2.numerator = 7
+            self.multipleDivisionActions.subDivide2.divisor = 8

doc/lsst.pex.config/overview.rst

@@ -167,3 +167,15 @@ Alternatively, if you wish to locate another configuration file using LSST infra
    from lsst.utils import getPackageDir
 
    config.load(os.path.join(getPackageDir("product_x"), "config", "otherconfig.py"))
+
+Specialized Config subclasses
+=============================
+
+There exists a subclass of `Config` which is designed to be configurable like a standard config, but have a runtime call interface.
+These specialized subclasses are named `~lsst.pex.config.configurableActions.ConfigurableAction`\ s, or actions for short.
+These actions are not intended to replace other runtime components, but compliment them.
+They provide configuration time mechanics to a simple runtime function.
+This interface allows for both configuration of an action as well as making which action to run configurable.
+These configurations are serialized out in a standard way, and thus allow complete functional states to be completely restored.
+The selection (thus configuration) of which `~lsst.pex.config.configurableActions.ConfigurableAction`\ s to run is made possible through the use of special `Field`\ s named `~lsst.pex.config.configurableActions.ConfigurableActionField` and `~lsst.pex.config.configurableActions.ConfigurableActionStructField`.
+See :doc:`field-types` for more details and examples of both `ConfigurableAction`\ s and the corresponding fields.

pyproject.toml

@@ -16,9 +16,8 @@ classifiers = [
     "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
     "Operating System :: OS Independent",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.8",
-    "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
     "Topic :: Scientific/Engineering :: Astronomy",
 ]
 keywords = ["lsst"]

python/lsst/pex/config/configurableActions/__init__.py

@@ -0,0 +1,23 @@
+# This file is part of pex_config.
+#
+# Developed for the LSST Data Management System.
+# This product includes software developed by the LSST Project
+# (https://www.lsst.org).
+# See the COPYRIGHT file at the top-level directory of this distribution
+# for details of code ownership.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+from ._configurableAction import *
+from ._configurableActionField import *
+from ._configurableActionStructField import *

python/lsst/pex/config/configurableActions/_configurableAction.py

@@ -0,0 +1,63 @@
+# This file is part of pex_config.
+#
+# Developed for the LSST Data Management System.
+# This product includes software developed by the LSST Project
+# (https://www.lsst.org).
+# See the COPYRIGHT file at the top-level directory of this distribution
+# for details of code ownership.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+from __future__ import annotations
+
+__all__ = ["ConfigurableAction", "ActionTypeVar"]
+
+from typing import Any, TypeVar
+
+from lsst.pex.config.config import Config
+
+ActionTypeVar = TypeVar("ActionTypeVar", bound="ConfigurableAction")
+
+
+class ConfigurableAction(Config):
+    """A `ConfigurableAction` is an interface that extends a
+    `lsst.pex.config.Config` class to include a `__call__` method.
+
+    This interface is designed to create an action that can be used at
+    runtime with a state that is determined during the configuration stage. A
+    single action thus may be assigned multiple times, each with different
+    configurations.
+
+    This allows state to be set and recorded at configuration time,
+    making future reproduction of results easy.
+
+    This class is intended to be an interface only, but because of various
+    inheritance conflicts this class can not be implemented as an Abstract
+    Base Class. Instead, the `__call__` method is purely virtual, meaning that
+    it will raise a `NotImplementedError` when called. Subclasses that
+    represent concrete actions must provide an override.
+    """
+
+    identity: str | None = None
+    """If a configurable action is assigned to a `ConfigurableActionField`, or
+    a `ConfigurableActionStructField` the name of the field will be bound to
+    this variable when it is retrieved.
+    """
+
+    def __setattr__(self, attr, value, at=None, label="assignment"):
+        if attr == "identity":
+            return object.__setattr__(self, attr, value)
+        return super().__setattr__(attr, value, at, label)
+
+    def __call__(self, *args: Any, **kwargs: Any) -> Any:
+        raise NotImplementedError("This method should be overloaded in subclasses")

python/lsst/pex/config/configurableActions/_configurableActionField.py

@@ -0,0 +1,104 @@
+# This file is part of pex_config.
+#
+# Developed for the LSST Data Management System.
+# This product includes software developed by the LSST Project
+# (https://www.lsst.org).
+# See the COPYRIGHT file at the top-level directory of this distribution
+# for details of code ownership.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+from __future__ import annotations
+
+__all__ = ("ConfigurableActionField",)
+
+from typing import Any, overload
+
+from lsst.pex.config import Config, ConfigField, FieldValidationError
+from lsst.pex.config.callStack import getCallStack
+from lsst.pex.config.config import _joinNamePath, _typeStr
+
+from . import ActionTypeVar, ConfigurableAction
+
+
+class ConfigurableActionField(ConfigField[ActionTypeVar]):
+    """`ConfigurableActionField` is a subclass of `~lsst.pex.config.Field` that
+    allows a single `ConfigurableAction` (or a subclass) to be assigned to it.
+    The `ConfigurableAction` is then accessed through this field for further
+    configuration.
+
+    Any configuration of this field that is done prior to having a new
+    `ConfigurableAction` assigned to it is forgotten.
+    """
+
+    # These attributes are dynamically assigned when constructing the base
+    # classes
+    name: str
+
+    def __set__(
+        self,
+        instance: Config,
+        value: ActionTypeVar | type[ActionTypeVar],
+        at: Any = None,
+        label: str = "assignment",
+    ) -> None:
+        if instance._frozen:
+            raise FieldValidationError(self, instance, "Cannot modify a frozen Config")
+        name = _joinNamePath(prefix=instance._name, name=self.name)
+
+        if not isinstance(value, self.dtype) and not issubclass(value, self.dtype):
+            msg = f"Value {value} is of incorrect type {_typeStr(value)}. Expected {_typeStr(self.dtype)}"
+            raise FieldValidationError(self, instance, msg)
+
+        if at is None:
+            at = getCallStack()
+
+        if isinstance(value, self.dtype):
+            instance._storage[self.name] = type(value)(__name=name, __at=at, __label=label, **value._storage)
+        else:
+            instance._storage[self.name] = value(__name=name, __at=at, __label=label)
+        history = instance._history.setdefault(self.name, [])
+        history.append(("config value set", at, label))
+
+    @overload
+    def __get__(
+        self, instance: None, owner: Any = None, at: Any = None, label: str = "default"
+    ) -> "ConfigurableActionField[ActionTypeVar]":
+        ...
+
+    @overload
+    def __get__(self, instance: "Config", owner: Any = None, at: Any = None, label: str = "default") -> Any:
+        ...
+
+    def __get__(self, instance, owner=None, at=None, label="default"):
+        result = super().__get__(instance, owner)
+        if instance is not None:
+            # ignore is due to typing resolved in overloads not translating to
+            # type checker not knowing this is not a Field
+            result.identity = self.name  # type: ignore
+        return result
+
+    def save(self, outfile, instance):
+        # docstring inherited from parent
+        # This is different that the parent class in that this field must
+        # serialize which config class is assigned to this field prior to
+        # serializing any assignments to that config class's fields.
+        value = self.__get__(instance)
+        fullname = _joinNamePath(instance._name, self.name)
+        outfile.write(f"{fullname}={_typeStr(value)}\n")
+        super().save(outfile, instance)
+
+    def __init__(self, doc, dtype=ConfigurableAction, default=None, check=None, deprecated=None):
+        if not issubclass(dtype, ConfigurableAction):
+            raise ValueError("dtype must be a subclass of ConfigurableAction")
+        super().__init__(doc=doc, dtype=dtype, default=default, check=check, deprecated=deprecated)