Skip to content

Commit

Permalink
Incorporate diffimDRP into the DRP step subsets
Browse files Browse the repository at this point in the history
  • Loading branch information
yalsayyad committed Aug 5, 2021
1 parent 6d38613 commit b147318
Showing 1 changed file with 41 additions and 1 deletion.
42 changes: 41 additions & 1 deletion pipelines/imsim/DRP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,12 +70,13 @@ subsets:
- deblend
- measure
- mergeMeasurements
- forcedPhotCcd
- forcedPhotCoadd
- transformObjectTable
- writeObjectTable
- consolidateObjectTable
- healSparsePropertyMaps
- selectGoodSeeingVisits
- templateGen
description: >
Tasks that can be run together, but only after the 'step1' and 'step2'
subsets.
Expand All @@ -92,3 +93,42 @@ subsets:
This subset is considered a workaround for missing middleware and task
functionality. It may be removed in the future.
step4:
subset:
- forcedPhotCcd
- forcedPhotDiffim
- imageDifference
- transformDiaSourceCat
description: >
Tasks that can be run together, but only after the 'step1', 'step2'
and 'step3' subsets
These detector-level tasks should not be run with
'tract' or 'patch' as part of the data ID expression if all
reference catalogs or diffIm templates that cover these
detector-level quanta are desired.
step5:
subset:
- consolidateDiaSourceTable
- drpAssociation
- drpDiaCalculation
description: >
Tasks that can be run together, but only after the 'step1', 'step2',
'step3', and 'step4' subsets
These DIA aggregation steps include both a visit-level aggregation
step consolidateDiaSourceTable and tract-level aggregation steps
drpAssociation and drpDiaCalculation.
While processing DC2 test-med-1, running without tract or patch in
the data query is recommended. If processing many contiguous tracts
running with tract in the dataId will be unavoidable.
If run with tract or patch in the data query, drpAssociation and
drpDiaCalculation will be complete, but the outputs of
consolidateDiaSourceTable will not contain complete visits.
This subset is separate from step4 to signal to operators to pause
to assess unexpected image differencing failures before these
aggregation steps. Otherwise, if run in the same quantum graph,
aggregated data products (e.g. diaObjects) would not be created if
one or more of the expected inputs is missing.

0 comments on commit b147318

Please sign in to comment.