A sample dataset for running the LSST DM stack
==============================================

Overview
--------

This package contains the input files and astrometric reference catalogs
needed to process several SDSS frames, using the processCcdSdss.py script
from the pipe_tasks package of the LSST Data Management Stack.

Usage
-----

To run the example, do::

	$ source $LSST_HOME/loadLSST.sh
	$ setup -r .
	$ ./bin/demo.sh  (or: "./bin/demo.sh --small"  if local memory is constrained)

where $LSST_HOME is the directory where your LSST DM stack resides.

Look at the contents of demo.sh to get an understanding of how to run
processCcd scripts.

All outputs (calibrated images, FITS tables of detected sources, etc.) will be
stored in the ./output subdirectory.  These are "raw" outputs, readable by
other parts of the LSST pipeline

In detected-sources.txt, you'll find an ASCII whitespace separated list of
detected sources and their measured attributes.  This may be more useful for
post-processing purposes.  This file may be compared to the expected results
found in the ./expected subdirectory.  Note that variations in the numerical
libraries found on different platforms can produce minor variations in the
output; choose a result set appropriate for your system. The ./bin/compare
script can then be used to check if your results are within the expected
numerical tolerances::

    $ ./bin/compare detected-sources.txt
    Ok.

If you use the "--small" run option then the corresponding output file is
named detected-sources_small.txt and the corresponding output directory is
called "output_small".

Check the astrometric relative RMS with::

    $ python bin/check_astrometry.py output

Included Data
-------------

The package comes with raw data for run 4192 camcol 4 field 300 and run 6377
camcol 4 field 399 from SDSS stripe 82.  This is 10 frames (one in each of five
filters for the two fields).

input
  This directory contains the fpC files with calibrated pixels, fpM files with
  masks, and asTrans/tsField/psField calibration data.

reference catalog
  Astrometric and photometric reference catalog indexed by HTM id, pared down
  to the area required for the above fields.  This catalog is used only for
  estimating photometric zeropoints; the astrometry is not re-determined.

  This catalog is pre-ingested in the input repository, but the original
  data along with the associated configuration file can be found in the
  ref_cat directory.

Updating the reference results
------------------------------

When making a change to the stack which change the numerical outputs,
developers *must* update *all* of the references files in the ``./expected``
directory to match. This likely means running the demo on multiple different
systems and collecting the results.