Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-34445 Update ap_verify tutorial #163

Merged
merged 2 commits into from
Jun 28, 2022
Merged

DM-34445 Update ap_verify tutorial #163

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
755feaa
removed unneeded argument
AndyTza Jun 28, 2022
d8dc257
add instructions to setup dataset
AndyTza Jun 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 8 additions & 0 deletions doc/lsst.ap.verify/datasets-install.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ For example, to install the `Cosmos PDR2 <https://github.com/lsst/ap_verify_ci_c

rebuild -u ap_verify_ci_cosmos_pdr2

Alternatively, the dataset can be cloned directly and setup:

.. prompt:: bash

git clone https://github.com/lsst/ap_verify_ci_cosmos_pdr2/
setup -r ap_verify_ci_cosmos_pdr2

The dataset should be setup each time before use.
Once this is done, ``ap_verify`` will be able to find the Cosmos data when requested through :option:`--dataset`.

Further reading
Expand Down
17 changes: 13 additions & 4 deletions doc/lsst.ap.verify/running.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,27 @@ For more details, see the :doc:`command-line-reference` or run :option:`ap_verif
How to run ap_verify in a new workspace
=======================================

Using the `Cosmos PDR2`_ CI dataset as an example, one can run :command:`ap_verify.py` as follows:
Using the `Cosmos PDR2`_ CI dataset as an example, one can run :command:`ap_verify.py` as follows.

.. _Cosmos PDR2: https://github.com/lsst/ap_verify_ci_cosmos_pdr2/
.. _Cosmos PDR2: https://github.com/lsst/ap_verify_ci_cosmos_pdr2/

First download and setup the dataset.

.. prompt:: bash

git clone https://github.com/lsst/ap_verify_ci_cosmos_pdr2/
setup -r ap_verify_ci_cosmos_pdr2

Comment on lines +24 to +30
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect. ap_verify datasets should only be downloaded once, and not every time ap_verify is run; they can be very large!

If you believe it is necessary to mention installation, I would recommend linking to the installation docs from this page's introduction. This practice is unfortunately not in the developer guide, but see DMTN-030.

You will need to setup the dataset each time you want to use it.

.. prompt:: bash

ap_verify.py --dataset ap_verify_ci_cosmos_pdr2 --data-query "visit in (59150, 59160) and band='g'" -j4 --output workspaces/cosmos/
ap_verify.py --dataset ap_verify_ci_cosmos_pdr2 --data-query "visit in (59150, 59160)" -j4 --output workspaces/cosmos/

Here the inputs are:

* :command:`ap_verify_ci_cosmos_pdr2` is the ``ap_verify`` :ref:`dataset <ap-verify-datasets>` to process,
* :command:`visit in (59150, 59160) and band='g'` is the :ref:`data ID query <daf_butler_dimension_expressions>` to process,
* :command:`visit in (59150, 59160)` is the :ref:`data ID query <daf_butler_dimension_expressions>` to process,
* :option:`-j` causes the ingest and processing pipelines to use 4 processes: choose a value appropriate for your machine; the system does not automatically determine how many parallel processes to use.

while the output is:
Expand Down