Merge pull request #105 from khaeru/enh/2023-W43

Miscellaneous improvements for 2023-W43

.github/workflows/pytest.yaml

@@ -29,8 +29,9 @@ jobs:
         - {version: "3.8", extras: ",sparse"}
         - {version: "3.9", extras: ",sparse"}
         - {version: "3.10", extras: ",sparse"}
+        - {version: "3.11", extras: ",sparse"}
         # Latest release / latest supported by genno / testable on GHA
-        - {version: "3.11", extras: ""}
+        - {version: "3.12", extras: ""}
 
         # For fresh releases and development versions of Python, compiled binary
         # wheels are not available for some dependencies, e.g. numpy, pandas.

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
 - repo: https://github.com/pre-commit/mirrors-mypy
-  rev: v1.4.1
+  rev: v1.6.1
   hooks:
   - id: mypy
     additional_dependencies:
@@ -16,11 +16,9 @@ repos:
     - types-setuptools
     - xarray
     args: []
-- repo: https://github.com/psf/black
-  rev: 23.7.0
-  hooks:
-  - id: black
 - repo: https://github.com/astral-sh/ruff-pre-commit
-  rev: v0.0.287
+  rev: v0.1.2
   hooks:
   - id: ruff
+  - id: ruff-format
+    args: [ --check ]

doc/api.rst

@@ -21,7 +21,7 @@ Top-level classes and functions
    :exclude-members: add, add_queue, apply, eval, graph
 
    A Computer is used to prepare (:meth:`add` and related methods) and then execute (:meth:`get` and related methods) **computations** stored in a :attr:`graph`.
-   Advanced users may manipulate the graph directly; but most computations can be prepared can be handled by using Computer methods.
+   Advanced users may manipulate the graph directly; but most computations can be prepared using the methods of Computer.
 
    Instance attributes:
 
@@ -57,7 +57,7 @@ Top-level classes and functions
       check_keys
       configure
       full_key
-      get_comp
+      get_operator
       infer_keys
       require_compat
 
@@ -77,16 +77,15 @@ Top-level classes and functions
 
       1. Any other, existing key in the Computer. This functions as an alias.
       2. Any other literal value or constant, to be returned directly.
-      3. A *task* :class:`tuple`: a callable (e.g. function), followed by zero
-         or more computations, e.g. keys for other tasks.
+      3. A *task* :class:`tuple`: a callable (such as a function or any object with a :meth:`~object.__call__` method), followed by zero or more keys (referring to the output of other computations), or computations directly.
       4. A :class:`list` containing zero or more of (1), (2), and/or (3).
 
       :mod:`genno` reserves some keys for special usage:
 
       ``"config"``
          A :class:`dict` storing configuration settings.
          See :doc:`config`.
-         Because this information is stored *in* the :attr:`graph`, it can be used as one input to other computations.
+         Because this information is stored *in* the :attr:`graph`, it can be used as one input to other operators.
 
       Some inputs to tasks may be confused for (1) or (4), above.
       The recommended way to protect these is:
@@ -103,7 +102,7 @@ Top-level classes and functions
          A list of computations, like :py:`[(list(args1), dict(kwargs1)), (list(args2), dict(kwargs2)), ...]` → passed to :meth:`add_queue`.
 
       :class:`str` naming an operator
-         e.g. "select", retrievable with :meth:`get_comp`.
+         e.g. "select", retrievable with :meth:`get_operator`.
          :meth:`add_single` is called with :py:`(key=args[0], data, *args[1], **kwargs)`, that is, applying the named operator to the other parameters.
 
       :class:`.Key` or other :class:`str`:
@@ -162,7 +161,7 @@ Top-level classes and functions
       .. code-block:: python
 
          def my_gen1(**kwargs):
-             op = partial(computations.load_file, **kwargs)
+             op = partial(operator.load_file, **kwargs)
              yield from (f"file:{i}", (op, "file{i}.txt")) for i in range(2)
 
          rep.apply(my_gen1, units="kg")
@@ -236,7 +235,7 @@ Top-level classes and functions
 
    A Key has the same hash, and compares equal to its :class:`str` representation.
    A Key also compares equal to another key or :class:`str` with the same dimensions in any other order.
-   ``repr(key)`` prints the Key in angle brackets ('<>') to signify that it is a Key object.
+   :py:`repr(key)` prints the Key in angle brackets ('<>') to signify that it is a Key object.
 
    >>> str(k1)
    'foo:a-b-c'
@@ -256,6 +255,35 @@ Top-level classes and functions
    >>> foo('a b c')
    <foo:a-b-c>
 
+   .. _key-arithmethic:
+
+   Keys can also be manipulated using some of the Python arithmetic operators:
+
+   - :py:`+`: same as :meth:`.add_tag`:
+
+     >>> k1 = Key("foo", "abc")
+     >>> k1
+     <foo:a-b-c>
+     >>> k1 + "tag"
+     <foo:a-b-c:tag>
+
+   - :py:`*` with a single string, an iterable of strings, or another Key: similar to :meth:`.append` and :meth:`.product`:
+
+     >>> k1 * "d"
+     <foo:a-b-c-d>
+     >>> k1 * ("e", "f")
+     <foo:a-b-c-e-f>
+     >>> k1 * Key("bar", "ghi")
+     <foo:a-b-c-g-h-i>
+
+   - :py:`/` with a single string or iterable of strings: similar to :meth:`drop`:
+
+     >>> k1 / "a"
+     <foo:b-c>
+     >>> k1 / ("a", "c")
+     <foo:b>
+     >>> k1 / Key("baz", "cde")
+     <foo:a-b>
 
 .. autoclass:: genno.Quantity
    :members:
@@ -284,7 +312,7 @@ The goal is that all :mod:`genno`-based code, including built-in and user functi
 Operators
 =========
 
-.. automodule:: genno.computations
+.. automodule:: genno.operator
    :members:
 
    Unless otherwise specified, these functions accept and return :class:`.Quantity` objects for data arguments/return values.
@@ -304,7 +332,6 @@ Operators
       index_to
       interpolate
       mul
-      add_mul
       pow
       product
       ratio
@@ -333,9 +360,9 @@ Operators
 Helper functions for adding tasks to Computers
 ----------------------------------------------
 
+.. autofunction:: add_binop
 .. autofunction:: add_load_file
-.. autofunction:: add_mul
-
+.. autofunction:: add_sum
 
 Internal format for quantities
 ==============================

doc/compat-pyam.rst

@@ -38,7 +38,7 @@ Pyam (:mod:`.compat.pyam`)
       keys = c.add(ACT, "as_pyam", "ya", collapse=m_t, drop=["t", "m"])
 
 
-.. automodule:: genno.compat.pyam.computations
+.. automodule:: genno.compat.pyam.operator
    :members:
 
    .. autosummary::
@@ -72,7 +72,7 @@ Configuration
    ``base:`` (:class:`str`)
       Key for the quantity to convert.
    ``select:`` (:class:`dict`, optional)
-      Keyword arguments to :func:`.computations.select`.
+      Keyword arguments to :func:`.operator.select`.
       This selection is performed while data is in :class:`.Quantity` format, before it is passed to :func:`.as_pyam`.
    ``rename:`` (:class:`dict`, optional)
       Passed to :func:`.as_pyam`.

doc/conf.py

@@ -38,6 +38,8 @@
 rst_prolog = """
 .. role:: py(code)
    :language: python
+
+.. |KeyLike| replace:: :obj:`~genno.core.key.KeyLike`
 """
 
 
@@ -98,6 +100,7 @@ def can_document_member(cls, member, membername, isattr, parent) -> bool:
     "pyam": ("https://pyam-iamc.readthedocs.io/en/stable/", None),
     "python": ("https://docs.python.org/3/", None),
     "sdmx1": ("https://sdmx1.readthedocs.io/en/stable", None),
+    "sparse": ("https://sparse.pydata.org/en/stable", None),
     "xarray": ("https://docs.xarray.dev/en/stable/", None),
 }
 

doc/config.rst

@@ -9,6 +9,8 @@ As shown in :ref:`Concepts and usage <describe-tasks>`, a :class:`.Computer` can
 Overview
 ========
 
+.. automodule:: genno.config
+
 Ways to configure
 -----------------
 
@@ -136,7 +138,7 @@ Specific sections
 
 Computer-specific configuration.
 
-Invokes :meth:`.Computer.aggregate` add tasks with :func:`.computations.aggregate` or :func:`.computations.sum`, computing sums across labels within one dimension of a quantity.
+Invokes :meth:`.Computer.aggregate` add tasks with :func:`.operator.aggregate` or :func:`.operator.sum`, computing sums across labels within one dimension of a quantity.
 Each entry contains:
 
 ``_quantities:`` list of 0 or more keys
@@ -267,7 +269,7 @@ This sets :attr:`.Computer.default_key`, used when :meth:`.get` is called withou
 
 Computer-specific configuration.
 
-Invokes :meth:`.Computer.add_file` to add :func:`.computations.load_file`.
+Invokes :meth:`.Computer.add_file` to add :func:`.operator.load_file`.
 If the ``path:`` key is a relative path, it is resolved relative to the directory that contains the configuration file, else the current working directory.
 
 .. code-block:: yaml
@@ -297,8 +299,8 @@ This is, as the name implies, the most generalized section.
 Each item contains:
 
 ``comp:``
-  Refers to the name of a operator that is available in the namespace of :mod:`genno.computations`, or a custom operator registered by compatibility modules or third-party packages.
-  See :meth:`.Computer.add` and :meth:`.Computer.get_comp`.
+  Refers to the name of a operator that is available in the namespace of :mod:`genno.operator`, or a custom operator registered by compatibility modules or third-party packages.
+  See :meth:`.Computer.add` and :meth:`.Computer.get_operator`.
   For instance, if "product", then :meth:`.Computer.add_product` is called, which also automatically infers the correct dimensions for each input.
 
   If omitted, :data:`None`, or YAML ``null``, no specific callable is used, but instead ``key:`` is configured to retrieve a simple :class:`list` of the ``inputs:``.

doc/usage.rst

@@ -262,7 +262,7 @@ Operators
 =========
 
 A operator is any Python function or callable that operates on Quantities or other data.
-:mod:`genno.computations` includes many common operators; see the API documentation for descriptions of each.
+:mod:`genno.operator` includes many common operators; see the API documentation for descriptions of each.
 
 The power of :mod:`genno` is the ability to link *any* code, no matter how complex, into the graph, and have it operate on the results of other code.
 Tasks can perform complex tasks such as: