Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-9530: Initial metric and specification layout design #1

Merged
merged 5 commits into from
Feb 28, 2017

Commits on Feb 27, 2017

  1. Set up lsst.validate.metrics EUPS package

    - Minimal Python package (lsst.validate.metrics) to permit version
      discovery via sconsUtils version.py. Otherwise, lsst.validate.metrics
      does not contain Python code. It's meant as a standalone data package
      that contains YAML definition files that are nominally consumed by
      validate_base.
    
    - Uses prototype Sphinx documentation framework (same as in
      validate_base).
    
    - Adopting RFC-45-inspired license management (COPYRIGHT file with
      references to that file from source files).
    
    - Includes GitHub issue/PR/contributing files in .github
    jonathansick committed Feb 27, 2017
    Configuration menu
    Copy the full SHA
    c696fd8 View commit details
    Browse the repository at this point in the history
  2. Add metric definitions for validate_drp

    These metric definitions are ported from github.com/lsst/validate_drp's
    metrics/, and distilled into the form being prototyped for
    validate_metrics.
    
    - Each eups package now has a YAML file, named after the package, in
      metrics/.
    - metrics/validate_drp.yaml, for example, specifically only defines
      metrics; it doesn't include specifications. This will allow metrics
      to be fairly static, while any LSST staff can separately add
      additional specifications.
    - We're adopting astropy.units definitions for percentages (%) and
      unitless unscaled numbers (---). No more empty string units.
    - We're adopting the docs convention of one-sentence-per-line for metric
      descriptions.
    jonathansick committed Feb 27, 2017
    Configuration menu
    Copy the full SHA
    3b4ecd9 View commit details
    Browse the repository at this point in the history
  3. Proof-of-concept specification model (PA1)

    This shows how specifications for PA1, from the SRD, can be written.
    
    In LPM-17-PA1.yaml, specifications for PA1 are written in their most
    basic and general form. A `#PA1-base` *partial specification* contains
    common data for all PA1 specifications, namely the units, comparison
    operator, and metric name.
    
    (Our suggested best practice is to have one YAML file per metric for
    these types of base specification definitions.)
    
    Then in LPM-17-PA1.yaml, specifications inherit from `#PA1-base`, adding
    a unique name and the specification value. The name **suggests**
    applicable bandpasses, but this provenance query is not established
    here. This design allows validate_drp to test measurements against the
    SRD specifications regardless of the provenance of the dataset.
    
    Then in cfht_gri.yaml we add specifications that specifically apply to
    measurements made against the validation_data_cfht data repository in
    specific bandpasses. These *additional* specifications permits selective
    monitoring of specification tests.
    jonathansick committed Feb 27, 2017
    Configuration menu
    Copy the full SHA
    26f62fa View commit details
    Browse the repository at this point in the history
  4. Add PA2 and PF1 metrics and specifications

    These metrics are interesting because, in the SRD, the metric measurement
    is written such that a specification of the other metric is a
    configuration. In the early validate_base framework these are called
    "dependent metrics."
    
    Here we've decided to unroll these dependencies by creating a metric for
    each metric measurement function configuration. This means there are
    dimensions of stretch/design/minimum, and also gri/uzy.
    
    Then the specifications are simply named 'srd' since they are the SRD
    specification of that metric.
    
    In cfht_gri.yaml, I've added specifications that inherit from the srd
    specifications and mix in the provenance query for CFHT gri validation
    data.
    jonathansick committed Feb 27, 2017
    Configuration menu
    Copy the full SHA
    c4b6973 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2017

  1. Expand README-based documentation

    We don't have real documentation yet (a sphinx user guide), but the main
    README and READMEs in the metrics/ and specs/ directories should help
    orient developers for now.
    jonathansick committed Feb 28, 2017
    Configuration menu
    Copy the full SHA
    254e627 View commit details
    Browse the repository at this point in the history