Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-37784: Add analysis_tools tasks to HSC pipelines #56

Merged
merged 2 commits into from
Feb 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions ingredients/HSC/DRP+fakes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ imports:
- makeVisitTable
- healSparsePropertyMaps
- consolidateHealSparsePropertyMaps
- analyzeObjectTableCore
- analyzeObjectTableSurveyCore
- catalogMatchTract
- refCatObjectTract
- faro_all
tasks:
singleFrameWithFakes:
Expand Down
1 change: 1 addition & 0 deletions ingredients/HSC/DRP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ description: The DRP pipeline specialized for Subaru's HyperSuprime-Cam
instrument: lsst.obs.subaru.HyperSuprimeCam
imports:
- $DRP_PIPE_DIR/ingredients/DRP-full.yaml
- $ANALYSIS_TOOLS_DIR/pipelines/coaddQualityCore.yaml
- $FARO_DIR/pipelines/metrics_pipeline_jointcal_fgcm.yaml
- $ANALYSIS_DRP_DIR/pipelines/analysis_drp_plots.yaml
tasks:
Expand Down
17 changes: 16 additions & 1 deletion pipelines/HSC/DRP-Prod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ description: The DRP pipeline specialized for large data release productions
instrument: lsst.obs.subaru.HyperSuprimeCam
imports:
- location: "$DRP_PIPE_DIR/ingredients/HSC/DRP.yaml"
exclude: faro_all
exclude:
- faro_all
tasks:
fgcmFitCycle:
class: lsst.fgcmcal.fgcmFitCycle.FgcmFitCycleTask
Expand Down Expand Up @@ -135,3 +136,17 @@ subsets:
This step has global aggregation tasks to run over all visits, detectors,
tracts, etc. This step should be run only with the instrument constraint
in the data query.
step8:
subset:
- analyzeObjectTableCore
- analyzeObjectTableSurveyCore
- catalogMatchTract
- refCatObjectTract
description: |
Core (most important), coadd-level plots/metrics from analysis_tools.

Must be run after 'step3' (writeObjectTable). Cannot be run with any
dataquery constraints if analyzeObjectTableSurveyCore is included.

Can be run together with analysis_drp subsets, e.g. pipeline.yaml
(hash/pound sign)step8,analysis_coadd_plots,imsim_analysis_coadd_plots
14 changes: 14 additions & 0 deletions pipelines/HSC/DRP-RC2.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -234,3 +234,17 @@ subsets:
This step has global aggregation tasks to run over all visits, detectors,
tracts, etc. This step should be run only with the instrument constraint
in the data query.
step8:
subset:
- analyzeObjectTableCore
- analyzeObjectTableSurveyCore
- catalogMatchTract
- refCatObjectTract
description: |
Core (most important), coadd-level plots/metrics from analysis_tools.

Must be run after 'step3' (writeObjectTable). Cannot be run with any
dataquery constraints if analyzeObjectTableSurveyCore is included.

Can be run together with analysis_drp subsets, e.g. pipeline.yaml
(hash/pound sign)step8,analysis_coadd_plots,imsim_analysis_coadd_plots
15 changes: 9 additions & 6 deletions pipelines/HSC/DRP-ci_hsc.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
description: The DRP pipeline specialized for the ci_hsc test dataset.
instrument: lsst.obs.subaru.HyperSuprimeCam
imports:
location: "$DRP_PIPE_DIR/ingredients/HSC/DRP.yaml"
exclude:
# Don't run jointcal, fgcm, or (some of) faro, as there isn't enough data.
- fgcm
- jointcal
- faro_all
- location: $DRP_PIPE_DIR/ingredients/HSC/DRP.yaml
exclude:
- fgcm
- jointcal
- faro_all
# Some of these plots fail with ci_hsc data
- analyzeObjectTableCore
- refCatObjectTract
- location: $ANALYSIS_TOOLS_DIR/pipelines/visitQualityCore.yaml
tasks:
# Do expect jointcal's photoCalibs to be present even though we didn't
# run it, since that's what the testdata_ci_hsc repo is set up to have.
Expand Down