Skip to content

Commit

Permalink
Merge branch 'tickets/DM-26775' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
kfindeisen committed Oct 8, 2020
2 parents 7a7ca39 + 64a7405 commit 918db98
Show file tree
Hide file tree
Showing 4 changed files with 19 additions and 1 deletion.
8 changes: 8 additions & 0 deletions doc/lsst.ap.verify/command-line-reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,20 @@ Required arguments are :option:`--dataset` and :option:`--output`.
Specify data ID to process.
If using :option:`--gen2`, this should use :doc:`data ID syntax </modules/lsst.pipe.base/command-line-task-dataid-howto>`, such as ``--id "visit=12345 ccd=1..6 filter=g"``.
If using :option:`--gen3`, this should use :ref:`dimension expression syntax <daf_butler_dimension_expressions>`, such as ``--id "visit=12345 and detector in (1..6) and band='g'"``.
Consider using :option:`--data-query` instead of ``--id`` for forward-compatibility and consistency with Gen 3 pipelines.

Multiple copies of this argument are allowed.
For compatibility with the syntax used by command line tasks, ``--id`` with no argument processes all data IDs.

If this argument is omitted, then all data IDs in the dataset will be processed.

.. option:: -d, --data-query <dataId>

**Butler data ID.**

This option is identical to :option:`--id`, and will become the primary data ID argument as Gen 2 is retired.
It is recommended over :option:`--id` for :option:`--gen3` runs.

.. option:: --dataset <dataset_name>

**Input dataset designation.**
Expand Down
1 change: 1 addition & 0 deletions doc/lsst.ap.verify/running.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ The command for running the pipeline on Gen 3 data is almost identical to Gen 2:
ap_verify.py --dataset HiTS2015 --gen3 --id "visit in (412518, 412568) and band='g'" --output workspaces/hits/

The only differences are substituting :option:`--gen3` for :option:`--gen2`, and formatting the (optional) data ID in the :ref:`Gen 3 query syntax <daf_butler_dimension_expressions>`.
For further compatibility with Gen 3 pipelines, :option:`--id` may be replaced with :option:`--data-query`.

.. note::

Expand Down
3 changes: 2 additions & 1 deletion python/lsst/ap/verify/pipeline_driver.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,8 @@ def __init__(self):
argparse.ArgumentParser.__init__(self, add_help=False)
# namespace.dataIds will always be a list of 0 or more nonempty strings, regardless of inputs.
# TODO: in Python 3.8+, action='extend' handles nargs='?' more naturally than 'append'.
self.add_argument('--id', dest='dataIds', action=self.AppendOptional, nargs='?', default=[],
self.add_argument('--id', '-d', '--data-query', dest='dataIds',
action=self.AppendOptional, nargs='?', default=[],
help='An identifier for the data to process.')
self.add_argument("-p", "--pipeline", default=defaultPipeline,
help="A custom version of the ap_verify pipeline (e.g., with different metrics).")
Expand Down
8 changes: 8 additions & 0 deletions tests/test_args.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,14 @@ def testDataId(self):
parsed = self._parseString(args)
self.assertEqual(parsed.dataIds, ["visit=54123", "filter=x"])

def testMixedDataId(self):
"""Verify that a command line with both --id and --data-query parses correctly.
"""
args = '--dataset %s --output tests/output/foo --id "visit=54123" -d "filter=x"' \
% CommandLineTestSuite.datasetKey
parsed = self._parseString(args)
self.assertEqual(parsed.dataIds, ["visit=54123", "filter=x"])

def testEmptyDataId(self):
"""Test that an --id argument may be not followed by a data ID.
"""
Expand Down

0 comments on commit 918db98

Please sign in to comment.