Fix ap_verify config.

The old configuration broke the Policy class, because Policies are
not allowed to have keys with dots yet perform no input validation.

README.md

@@ -10,7 +10,7 @@ This package manages end-to-end testing and metric generation for the LSST DM Al
 
 * a dictionary named `datasets`, which maps from user-visible dataset names to the eups package that implements them (see `Setting Up a Dataset`, below)
 * a dictionary named `measurements`, which contains dictionaries needed for different metrics:
-    * `timing`: maps from names of metrics in the `verify_metrics` package to the Tasks or subTasks they time. The names of subTasks must be those assigned by the parent Task, and may be prefixed by the parent Task name(s) followed by a colon, as in "imageDifference:detection".
+    * `timing`: maps from Tasks or subTasks to the names of metrics that time them. The names of subTasks must be those assigned by the parent Task, and may be prefixed by the parent Task name(s) followed by a colon, as in "imageDifference:detection". Metric names must exist in `verify_metrics` and include the package they're associated with, as in "meas_algorithms.SourceDetectionTime".
 
 Other configuration options may be added in the future.
 

config/dataset_config.yaml

@@ -3,16 +3,16 @@ datasets:
     HiTS2015: ap_verify_hits2015
 measurements:
     timing:
-        pipe_tasks.ProcessCcdTime: processCcd
-        ip_isr.IsrTime: processCcd:isr
-        pipe_tasks.CharacterizeImageTime: processCcd:charImage
-        pipe_tasks.CalibrateTime: processCcd:calibrate
-        pipe_tasks.ImageDifferenceTime: imageDifference
-        meas_astrom.AstrometryTime: imageDifference:astrometer
-        pipe_tasks.RegisterImageTime: imageDifference:register
-        ip_diffim.ImagePsfMatchTime: imageDifference:subtract
-        meas_algorithms.SourceDetectionTime: imageDifference:detection
-        ip_diffim.DipoleFitTime: imageDifference:measurement
-        ap_association.AssociationTime: association
+        processCcd: pipe_tasks.ProcessCcdTime
+        processCcd:isr: ip_isr.IsrTime
+        processCcd:charImage: pipe_tasks.CharacterizeImageTime
+        processCcd:calibrate: pipe_tasks.CalibrateTime
+        imageDifference: pipe_tasks.ImageDifferenceTime
+        imageDifference:astrometer: meas_astrom.AstrometryTime
+        imageDifference:register: pipe_tasks.RegisterImageTime
+        imageDifference:subtract: ip_diffim.ImagePsfMatchTime
+        imageDifference:detection: meas_algorithms.SourceDetectionTime
+        imageDifference:measurement: ip_diffim.DipoleFitTime
+        association: ap_association.AssociationTime
 ...
 

python/lsst/ap/verify/measurements/compute_metrics.py

@@ -57,8 +57,8 @@ def measure_from_metadata(metadata):
     result = []
 
     timing_map = Config.instance['measurements.timing']
-    for metric in timing_map.names():
-        measurement = measure_runtime(metadata, timing_map[metric], metric)
+    for task in timing_map.names():
+        measurement = measure_runtime(metadata, task, timing_map[task])
         if measurement is not None:
             result.append(measurement)