Skip to content

Commit

Permalink
Expand README-based documentation
Browse files Browse the repository at this point in the history
We don't have real documentation yet (a sphinx user guide), but the main
README and READMEs in the metrics/ and specs/ directories should help
orient developers for now.
  • Loading branch information
jonathansick committed Feb 28, 2017
1 parent c4b6973 commit 254e627
Show file tree
Hide file tree
Showing 4 changed files with 57 additions and 6 deletions.
24 changes: 24 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,28 @@ lsst.validate.metrics

**Centralized metric and specification definitions for the lsst.validate framework.**

For background on the ``lsst.validate`` framework, see SQR-017_.

Registering metrics
===================

**Metrics** are *measureable things*.
They can be astronomy concepts (an astrometric RMS), or computational (a task's timing or memory usage).

Metrics that are measured by packages are **registered** here in ``validate_metrics`` by adding or extending a metric definition YAML file in the ``metrics/`` directory.
See `metrics/README.rst <./metrics/README.rst>`_ for more information.

Registering specifications
==========================

**Specifications** are tests of metric measurements.
A specification might say that a metric measurement should be less than a certain value if that measurement was made under a certain set of conditions (*provenance*).
Whenever metrics are measured, DM's SQUASH harness tests those measurements against specifications.
Depending on the pass/fail outcome, subscribers can receive notifications (alerts).

New specifications are **registered** here in ``validate_metrics`` by adding or extending a specification definition YAML file in the ``specs/`` directory.
See `specs/README.rst <./specs/README.rst>`_ for more information.

Getting help and reporting bugs
===============================

Expand All @@ -27,3 +49,5 @@ This product includes software developed by the `LSST Project <http://www.lsst.o
See the `COPYRIGHT <./COPYRIGHT>`_ file.

This product's source code is licensed under the terms of GPLv3 (see `LICENSE <./LICENSE>`_), and all documentation content is licensed under the `Creative Commons Attribution 4.0 International (CC BY 4.0) <https://creativecommons.org/licenses/by/4.0/>`_ license.

.. _SQR-017: https://sqr-017.lsst.io
10 changes: 8 additions & 2 deletions metrics/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,11 @@
metrics/
########

This directory contains metric definition YAML files for each EUPS package that produces metric measurements.
Documentation to come.
This directory contains *metric definition* YAML files for each EUPS package that produces metric measurements.

Files are named after the EUPS package.
For example: ``validate_drp.yaml`` for the `validate_drp`_ package.

The format of these YAML files is *currently* documented in https://sqr-017.lsst.io/v/DM-9533/index.html#metric-yaml.

.. _validate_drp: https://github.com/lsst/validate_drp
10 changes: 10 additions & 0 deletions specs/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
######
specs/
######

This directory contains *specification definition* YAML files for metrics defined in ``../metrics``.
Specifications are organized into sub-directories, each named after the corresponding package in ``../metrics``.
For a single package, specifications can be distributed across multiple YAML files and even sub-directories.
The READMEs of each specification sub-directory should state how specifications are organized to suggest where new specifications can be added.

For more information about the format of specification definition YAML files, see https://sqr-017.lsst.io/v/DM-9533/index.html#spec-yaml.
19 changes: 15 additions & 4 deletions specs/validate_drp/README.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
##################
specs/validate_drp
##################
###################
specs/validate_drp/
###################

Specifications for ``validate_drp`` metrics are defined in this directory as YAML documents.
Specifications for `validate_drp`_ metrics are defined in this directory as YAML documents.

Specification files prefixed with ``LPM-17-`` reflect **official** requirements, baselined in `LPM-17`_, the *Science Requirements Document*.
One ``LPM-17-`` file is provided per metric family.
Developers should not modify these files, except to support `LPM-17`_.

Developers can add ad-hoc `validate_drp`_ specifications by committing additional YAML files to this directory, or by adding to these:

- ``cfht_gri.yaml``: specifications against the https://github.com/lsst/validation_data_cfht.git dataset in g, r, and i bands.

.. _validate_drp: https://github.com/lsst/validate_drp
.. _LPM-17: http://ls.st/lpm-17

0 comments on commit 254e627

Please sign in to comment.