DM-28598: minor doc improvements

doc/lsst.daf.butler/index.rst

@@ -24,27 +24,6 @@ Using the Butler
 This module provides an abstracted data access interface, known as the Butler.
 It can be used to read and write data without having to know the details of file formats or locations.
 
-.. _lsst.daf.butler-dev:
-
-Design and Development
-======================
-
-``lsst.daf.butler`` is developed at https://github.com/lsst/daf_butler.
-You can find Jira issues for this module under the `daf_butler <https://jira.lsstcorp.org/issues/?jql=project%20%3D%20DM%20AND%20component%20%3D%20daf_butler>`_ component.
-
-.. toctree::
-   :maxdepth: 1
-
-   dev/dataCoordinate.rst
-
-Butler Command Line Interface Development
------------------------------------------
-
-.. toctree::
-   :maxdepth: 1
-
-   writing-subcommands.rst
-
 .. _lsst.daf.butler-dimensions:
 
 The Dimensions System
@@ -63,16 +42,45 @@ Concrete Storage Classes
 
   concreteStorageClasses.rst
 
+..
+   The sphinx+click tooling generates docs using sphinx's built-in
+   "program" and "option" directive, but linking to those is broken (in
+   sphinx itself): https://github.com/sphinx-doc/sphinx/issues/880
+
+   It seems the best we can do is link to the anchor below (even putting a
+   manual anchor in scripts/butler.py.rst does not seem to work).
+
 .. _lsst.daf.butler-scripts:
 
-Butler Command Reference
-========================
+Butler Command-Line Reference
+=============================
 
 .. toctree::
    :maxdepth: 1
 
    scripts/options-file.rst
-   scripts/butler.py
+   scripts/butler
+
+.. _lsst.daf.butler-dev:
+
+Design and Development
+======================
+
+``lsst.daf.butler`` is developed at https://github.com/lsst/daf_butler.
+You can find Jira issues for this module under the `daf_butler <https://jira.lsstcorp.org/issues/?jql=project%20%3D%20DM%20AND%20component%20%3D%20daf_butler>`_ component.
+
+.. toctree::
+   :maxdepth: 1
+
+   dev/dataCoordinate.rst
+
+Butler Command Line Interface Development
+-----------------------------------------
+
+.. toctree::
+   :maxdepth: 1
+
+   writing-subcommands.rst
 
 .. _lsst.daf.butler-pyapi:
 

doc/lsst.daf.butler/queries.rst

@@ -8,7 +8,7 @@ Querying datasets
 =================
 
 Datasets in a butler-managed data repository are identified by the combination of their *dataset type* and *data ID* within a *collection*.
-The `Registry` class's query methods (`~Registry.queryDatasetTypes`, `~Registry.queryCollections`, `~Registry.queryDataIds`, and `~Registry.queryDatasets`) allow these to be specified either fully or partially in various ways.
+The `Registry` class's query methods (`~Registry.queryDatasetTypes`, `~Registry.queryCollections`, `~Registry.queryDataIds`, `~Registry.queryDatasets`, and `~Registry.queryDimensionRecords`) allow these to be specified either fully or partially in various ways.
 
 .. _daf_butler_dataset_type_expressions:
 

doc/lsst.daf.butler/scripts/butler.rst

@@ -0,0 +1,8 @@
+.. note::
+
+   The ``butler`` subcommands documented here are only those defined in the ``daf_butler`` package itself; downstream packages can implement additional subcommands via the plugin system described at :ref:`daf_butler_cli`.
+   The best way to get a complete list of subcommands is to use ``butler --help``.
+
+.. click:: lsst.daf.butler.cli.butler:cli
+   :prog: butler
+   :show-nested:

doc/lsst.daf.butler/writing-subcommands.rst

@@ -1,8 +1,14 @@
+
 .. _daf_butler_cli:
 
 The Butler Command
 ==================
 
+.. note::
+
+    This page describes how to extend the ``butler`` subcommand; see :ref:`lsst.daf.butler-scripts` for documentation on the subcommands defined in `lsst.daf.butler`.
+    Additional subcommands are defined in other packages, especially `lsst.obs.base`; use ``butler --help`` to get a complete list.
+
 ``daf_butler`` provides a command line interface command called ``butler``. It supports subcommands, some of
 which are implemented in ``daf_butler``.
 

python/lsst/daf/butler/registry/_registry.py

@@ -1316,11 +1316,11 @@ def queryCollections(self, expression: Any = ...,
         Parameters
         ----------
         expression : `Any`, optional
-            An expression that fully or partially identifies the collections
-            to return, such as a `str`, `re.Pattern`, or iterable thereof.
-            `...` can be used to return all collections, and is the default.
-            See :ref:`daf_butler_collection_expressions` for more
-            information.
+            An expression that identifies the collections to return, such as a
+            `str` (for full matches), `re.Pattern` (for partial matches), or
+            iterable thereof.  `...` can be used to return all collections,
+            and is the default.  See :ref:`daf_butler_collection_expressions`
+            for more information.
         datasetType : `DatasetType`, optional
             If provided, only yield collections that may contain datasets of
             this type.  This is a conservative approximation in general; it may
@@ -1400,13 +1400,13 @@ def queryDatasets(self, datasetType: Any, *,
             be used to query all dataset types.  See
             :ref:`daf_butler_dataset_type_expressions` for more information.
         collections: optional
-            An expression that fully or partially identifies the collections
-            to search for datasets, such as a `str`, `re.Pattern`, or iterable
-            thereof.  `...` can be used to find datasets from all
-            `~CollectionType.RUN` collections (no other collections are
-            necessary, because all datasets are in a ``RUN`` collection).  See
+            An expression that identifies the collections to search, such as a
+            `str` (for full matches), `re.Pattern` (for partial matches), or
+            iterable thereof.  `...` can be used to search all collections
+            (actually just all `~CollectionType.RUN` collections, because this
+            will still find all datasets).  If not provided,
+            ``self.default.collections`` is used.  See
             :ref:`daf_butler_collection_expressions` for more information.
-            If not provided, ``self.default.collections`` is used.
         dimensions : `~collections.abc.Iterable` of `Dimension` or `str`
             Dimensions to include in the query (in addition to those used
             to identify the queried dataset type(s)), either to constrain
@@ -1452,7 +1452,12 @@ def queryDatasets(self, datasetType: Any, *,
         Returns
         -------
         refs : `queries.DatasetQueryResults`
-            Dataset references matching the given query criteria.
+            Dataset references matching the given query criteria.  Nested data
+            IDs are guaranteed to include values for all implied dimensions
+            (i.e. `DataCoordinate.hasFull` will return `True`), but will not
+            include dimension records (`DataCoordinate.hasRecords` will be
+            `False`) unless `~queries.DatasetQueryResults.expanded` is called
+            on the result object (which returns a new one).
 
         Raises
         ------
@@ -1600,12 +1605,14 @@ def queryDataIds(self, dimensions: Union[Iterable[Union[Dimension, str]], Dimens
             See :ref:`daf_butler_dataset_type_expressions` for more
             information.
         collections: `Any`, optional
-            An expression that fully or partially identifies the collections
-            to search for datasets, such as a `str`, `re.Pattern`, or iterable
-            thereof.  `...` can be used to return all collections.  Must be
-            provided if ``datasets`` is, and is ignored if it is not.  See
+            An expression that identifies the collections to search for
+            datasets, such as a `str` (for full matches), `re.Pattern` (for
+            partial matches), or iterable thereof.  `...` can be used to search
+            all collections (actually just all `~CollectionType.RUN`
+            collections, because this will still find all datasets).  If not
+            provided, ``self.default.collections`` is used.  Ignored unless
+            ``datasets`` is also passed.  See
             :ref:`daf_butler_collection_expressions` for more information.
-            If not provided, ``self.default.collections`` is used.
         where : `str`, optional
             A string expression similar to a SQL WHERE clause.  May involve
             any column of a dimension table or (as a shortcut for the primary
@@ -1714,8 +1721,13 @@ def queryDimensionRecords(self, element: Union[DimensionElement, str], *,
             that should constrain the yielded records.  See `queryDataIds` and
             :ref:`daf_butler_dataset_type_expressions` for more information.
         collections: `Any`, optional
-            An expression that fully or partially identifies the collections
-            to search for datasets.  See `queryDataIds` and
+            An expression that identifies the collections to search for
+            datasets, such as a `str` (for full matches), `re.Pattern` (for
+            partial matches), or iterable thereof.  `...` can be used to search
+            all collections (actually just all `~CollectionType.RUN`
+            collections, because this will still find all datasets).  If not
+            provided, ``self.default.collections`` is used.  Ignored unless
+            ``datasets`` is also passed.  See
             :ref:`daf_butler_collection_expressions` for more information.
         where : `str`, optional
             A string expression similar to a SQL WHERE clause.  See
@@ -1774,10 +1786,13 @@ def queryDatasetAssociations(
         datasetType : `DatasetType` or `str`
             A dataset type object or the name of one.
         collections: `Any`, optional
-            An expression that fully or partially identifies the collections
-            to search for datasets.  See `queryCollections` and
+            An expression that identifies the collections to search for
+            datasets, such as a `str` (for full matches), `re.Pattern` (for
+            partial matches), or iterable thereof.  `...` can be used to search
+            all collections (actually just all `~CollectionType.RUN`
+            collections, because this will still find all datasets).  If not
+            provided, ``self.default.collections`` is used.  See
             :ref:`daf_butler_collection_expressions` for more information.
-            If not provided, ``self.default.collections`` is used.
         collectionTypes : `AbstractSet` [ `CollectionType` ], optional
             If provided, only yield associations from collections of these
             types.