Skip to content

Commit

Permalink
Merge pull request #150 from lsst/tickets/DM-33453
Browse files Browse the repository at this point in the history
DM-33453: Update pipeline subdirectories in response to RFC-775.
  • Loading branch information
erinleighh committed Jan 30, 2024
2 parents f2ef8f8 + 146d1b8 commit db5c06c
Show file tree
Hide file tree
Showing 19 changed files with 25 additions and 25 deletions.
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
description: End to end AP pipeline specialized for DECam
# (1) Run $AP_PIPE_DIR/pipelines/DarkEnergyCamera/RunIsrForCrosstalkSources.yaml
# (1) Run $AP_PIPE_DIR/pipelines/DECam/RunIsrForCrosstalkSources.yaml
# (2) Execute `make_apdb.py`, e.g.,
# make_apdb.py -c db_url="sqlite:////project/user/association.db"
# (3) Run this pipeline, setting appropriate diaPipe configs
# (diaPipe configs should match the make_apdb.py configs)

instrument: lsst.obs.decam.DarkEnergyCamera
imports:
- location: $AP_PIPE_DIR/pipelines/DarkEnergyCamera/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/DECam/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/_ingredients/ApPipe.yaml
exclude: # These tasks come from DarkEnergyCamera/ProcessCcd.yaml instead
exclude: # These tasks come from DECam/ProcessCcd.yaml instead
- processCcd
subsets:
# apPipe must be redefined from $AP_PIPE_DIR/pipelines/_ingredients/ApPipe.yaml
Expand Down
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# TODO DM-40388: This file will replace ApPipe.yaml, once the new pipeline is vetted.
description: End to end AP pipeline specialized for DECam
# (1) Run $AP_PIPE_DIR/pipelines/DarkEnergyCamera/RunIsrForCrosstalkSources.yaml
# (1) Run $AP_PIPE_DIR/pipelines/DECam/RunIsrForCrosstalkSources.yaml
# (2) Execute `make_apdb.py`, e.g.,
# make_apdb.py -c db_url="sqlite:////project/user/association.db"
# (3) Run this pipeline, setting appropriate diaPipe configs
# (diaPipe configs should match the make_apdb.py configs)

instrument: lsst.obs.decam.DarkEnergyCamera
imports:
- location: $AP_PIPE_DIR/pipelines/DarkEnergyCamera/ProcessCcdCalibrateImage.yaml
- location: $AP_PIPE_DIR/pipelines/DECam/ProcessCcdCalibrateImage.yaml
- location: $AP_PIPE_DIR/pipelines/_ingredients/ApPipeCalibrateImage.yaml
exclude: # These tasks come from DarkEnergyCamera/ProcessCcdCalibrateImage.yaml instead
exclude: # These tasks come from DECam/ProcessCcdCalibrateImage.yaml instead
- processCcd
subsets:
# apPipe must be redefined from $AP_PIPE_DIR/pipelines/_ingredients/ApPipeCalibrateImage.yaml
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
description: DECam AP Pipeline with synthetic/fake sources. Templates are inputs.
# Remember:
# (0) Ensure median calibration products and template coadds exist for the data being processed
# (1) Run $AP_PIPE_DIR/pipelines/DarkEnergyCamera/RunIsrForCrosstalkSources.yaml
# (1) Run $AP_PIPE_DIR/pipelines/DECam/RunIsrForCrosstalkSources.yaml
# (2) Execute `make_apdb.py`, e.g.,
# make_apdb.py -c db_url="sqlite:////project/user/association.db"
# (3) Run this pipeline, setting appropriate diaPipe configs
Expand All @@ -12,7 +12,7 @@ imports:
- location: $AP_PIPE_DIR/pipelines/_ingredients/ApPipeWithFakes.yaml
exclude: # These tasks frome from DECam's ApPipe.yaml instead
- processCcd
- location: $AP_PIPE_DIR/pipelines/DarkEnergyCamera/ApPipe.yaml
- location: $AP_PIPE_DIR/pipelines/DECam/ApPipe.yaml
exclude: # These tasks come from ApPipeWithFakes.yaml instead
- retrieveTemplate
- subtractImages
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ description: The AP template building pipeline specialized for DECam

instrument: lsst.obs.decam.DarkEnergyCamera
imports:
- location: $AP_PIPE_DIR/pipelines/DarkEnergyCamera/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/DECam/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/_ingredients/ApTemplate.yaml
exclude: # These tasks come from DarkEnergyCamera/ProcessCcd.yaml instead
exclude: # These tasks come from DECam/ProcessCcd.yaml instead
- processCcd

subsets:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
description: A set of tasks to run when processing raw images, specialized for DECam
# You must run $AP_PIPE_DIR/pipelines/DarkEnergyCamera/RunIsrForCrosstalkSources.yaml first!
# You must run $AP_PIPE_DIR/pipelines/DECam/RunIsrForCrosstalkSources.yaml first!

instrument: lsst.obs.decam.DarkEnergyCamera
imports:
- location: $AP_PIPE_DIR/pipelines/DarkEnergyCamera/RunIsrWithCrosstalk.yaml
- location: $AP_PIPE_DIR/pipelines/DECam/RunIsrWithCrosstalk.yaml
- location: $AP_PIPE_DIR/pipelines/_ingredients/ProcessCcd.yaml
exclude:
- isr
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# TODO DM-40388: This file will replace ProcessCcd.yaml, once the new pipeline is vetted.
description: A set of tasks to run when processing raw images, specialized for DECam
# You must run $AP_PIPE_DIR/pipelines/DarkEnergyCamera/RunIsrForCrosstalkSources.yaml first!
# You must run $AP_PIPE_DIR/pipelines/DECam/RunIsrForCrosstalkSources.yaml first!

instrument: lsst.obs.decam.DarkEnergyCamera
imports:
- location: $AP_PIPE_DIR/pipelines/DarkEnergyCamera/RunIsrWithCrosstalk.yaml
- location: $AP_PIPE_DIR/pipelines/DECam/RunIsrWithCrosstalk.yaml
- location: $AP_PIPE_DIR/pipelines/_ingredients/ProcessCcdCalibrateImage.yaml
exclude:
- isr
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ description: End to end AP pipeline specialized for HSC

instrument: lsst.obs.subaru.HyperSuprimeCam
imports:
- location: $AP_PIPE_DIR/pipelines/HyperSuprimeCam/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/HSC/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/_ingredients/ApPipe.yaml
exclude: # These tasks come from HyperSuprimeCam/ProcessCcd.yaml instead
exclude: # These tasks come from HSC/ProcessCcd.yaml instead
- processCcd
subsets:
# apPipe must be redefined from $AP_PIPE_DIR/pipelines/_ingredients/ApPipe.yaml
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ imports:
- location: $AP_PIPE_DIR/pipelines/_ingredients/ApPipeWithFakes.yaml
exclude: # These tasks frome from HSC's ApPipe.yaml instead
- processCcd
- location: $AP_PIPE_DIR/pipelines/HyperSuprimeCam/ApPipe.yaml
- location: $AP_PIPE_DIR/pipelines/HSC/ApPipe.yaml
exclude: # These tasks come from _ingredients/ApPipeWithFakes.yaml instead
- retrieveTemplate
- subtractImages
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ description: The AP template building pipeline specialized for HSC

instrument: lsst.obs.subaru.HyperSuprimeCam
imports:
- location: $AP_PIPE_DIR/pipelines/HyperSuprimeCam/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/HSC/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/_ingredients/ApTemplate.yaml
exclude: # These tasks come from HyperSuprimeCam/ProcessCcd.yaml instead
exclude: # These tasks come from HSC/ProcessCcd.yaml instead
- processCcd

tasks:
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ description: End to end AP pipeline specialized for ImSim

instrument: lsst.obs.lsst.LsstCamImSim
imports:
- location: $AP_PIPE_DIR/pipelines/LsstCamImSim/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/LSSTCam-imSim/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/_ingredients/ApPipe.yaml
exclude: # These tasks come from LsstCamImSim/ProcessCcd.yaml instead
exclude: # These tasks come from LSSTCam-imSim/ProcessCcd.yaml instead
- processCcd
subsets:
# apPipe must be redefined from $AP_PIPE_DIR/pipelines/_ingredients/ApPipe.yaml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ description: The AP template building pipeline specialized for ImSim

instrument: lsst.obs.lsst.LsstCamImSim
imports:
- location: $AP_PIPE_DIR/pipelines/LsstCamImSim/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/LSSTCam-imSim/ProcessCcd.yaml
- location: $AP_PIPE_DIR/pipelines/_ingredients/ApTemplate.yaml
exclude: # These tasks come from LsstCamImSim/ProcessCcd.yaml instead
exclude: # These tasks come from LSSTCam-imSim/ProcessCcd.yaml instead
- processCcd

tasks:
Expand Down
File renamed without changes.
4 changes: 2 additions & 2 deletions pipelines/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ The pipelines defined here come in three flavors: camera-specific (within named
Pipelines within the ingredients directory are meant to be imported by other pipelines, and are not intended to be used directly by end-users.

The `pipetask build` command can be used to expand a pipeline YAML and resolve any imports for the purposes of visualizing it.
For example, to visualize the `apPipe` subset from the [LSSTCam-imSim ApPipe pipeline](https://github.com/lsst/ap_pipe/blob/main/pipelines/LsstCamImSim/ApPipe.yaml) pipeline, run:
For example, to visualize the `apPipe` subset from the [LSSTCam-imSim ApPipe pipeline](https://github.com/lsst/ap_pipe/blob/main/pipelines/LSSTCam-imSim/ApPipe.yaml) pipeline, run:

```bash
pipetask build \
-p $AP_PIPE_DIR/pipelines/LsstCamImSim/ApPipe.yaml#apPipe \
-p $AP_PIPE_DIR/pipelines/LSSTCam-imSim/ApPipe.yaml#apPipe \
--show pipeline
```

0 comments on commit db5c06c

Please sign in to comment.