Skip to content

Commit

Permalink
Add documentation for the --rerun option
Browse files Browse the repository at this point in the history
The --rerun option brought over from HSC was lacking good
documentation. This ticket provides said documentation by sumamrizing
the relevent contents of RFC-95: Filesystem Organization & Argument
Parser Features.
  • Loading branch information
AstroVPK committed Jun 14, 2016
1 parent eb040e6 commit 935bf7c
Showing 1 changed file with 38 additions and 0 deletions.
38 changes: 38 additions & 0 deletions doc/main.dox
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,44 @@ Thus the following example specifies all sensors for four combinations of visit
plus all sensors for one raft of two other visits for calibrated lsstSim data:
- `--id visit=54123^55523 raft=1,1^2,1 --id visit=623459^293423 raft=0,0`

\subsection pipeBase_argumentParser_rerun The --rerun Option

The `--rerun` option is a convenient shortcut for specifying the output directory of a command line task and
serves the same purpose as the `--output` argument. However, the value supplied to `--rerun` is relative to
the "rerun" directory of the root input data repository (i.e. follow the "_parent" chain all the way back,
then add "rerun" to that). `--rerun` saves the user from typing in the input repository twice. For example:

`processCcd.py $root/hsc --output $root/hsc/rerun/vpk24/DM-4443`

which will run processesCcd on the (HSC) data in "$root/hsc" and put the output in
"$root/hsc/rerun/vpk24/DM-4443" can be shortened by using `--rerun` to

`processCcd.py $root/hsc ----rerun vpk24/DM-4443`

This is useful because outputs from the pipeline usually go within the camera-level root repository into a
subdirectory of a "rerun" directory present in the camera-level root repository. These output sub-directories
should usually be named using JIRA issue numbers and dates, again with a distinction between public and
private reruns. For example:

`$root/hsc/rerun/DM-3456_20150607` # a public rerun useful for many people

`$root/hsc/rerun/jbosch/DM-7890/fixed-psf` # one of several private reruns used to debug a single issue

`$root/lsstSim/krughoff/DM-5678/rerun/jbosch/DM-5679` # simulated by Simon, processed by Jim

As with any other output repository, reruns can be chained, but keep in mind that the name of the parent rerun
is not generally recorded in the name of the child. Chaining is made easy by supporting a two-value
form separated by a colon, which specifies both a parent rerun and the new output directory. For example:

`processCcd.py $root/hsc --rerun price/DM-5680 ...` # Paul processes some CCDs

`makeCoaddTempExp.py $root/hsc --rerun price/DM-5680:jbosch/DM-5681 ...` # Jim uses Paul's CCD processing to
start a coadd.

The `--rerun` can also specify a path, part of which can be a symbolic link. For example, if
"$root/hsc/rerun/rhl" -> "/other/disk/rhl/rerun", then `--rerun rhl/tmp/foo` creates "/other/disk/rhl/tmp/foo"
where note that if the path is absolute, it is taken to be the path.

\subsection pipeBase_argumentParser_argumentFiles Argument Files

You may specify long or repetitive command-line arguments in text files and reference those files
Expand Down

0 comments on commit 935bf7c

Please sign in to comment.