DM-32245: Reprocess HiTS AP with fakes and an APDB #96
Conversation
mrawls
force-pushed
the
tickets/DM-32245
branch
from
b0797ab
to
dac3a25
Compare
There was a problem hiding this comment.
Looks good! The main change I'd like to see is use of pipeline parameters that are cross-compatible with the existing ap_pipe and ap_verify pipelines. It's a bit of work now, but would make things easier if we start running the new pipelines in ap_verify, try to use MultiTractFakes as a drop-in replacement for the existing Fakes, need to change dataset names (which we've already done once for *Diff_diaSrcTable), etc.
| # make_apdb.py -c isolation_level=READ_UNCOMMITTED | ||
| # -c db_url="sqlite:////project/user/association.db" | ||
| # (3) Run this pipeline, setting appropriate diaPipe configs | ||
| # (diaPipe configs should match the make_apdb.py configs) |
| imports: | ||
| - location: $AP_PIPE_DIR/pipelines/ApPipeMultiTractFakes.yaml | ||
| exclude: # These tasks frome from DECam's ApPipe.yaml instead | ||
| - processCcd | ||
| - location: $AP_PIPE_DIR/pipelines/DarkEnergyCamera/ApPipe.yaml | ||
| exclude: # These tasks come from ApPipeMultiTractFakes.yaml instead | ||
| - imageDifference | ||
| - diaPipe | ||
| - transformDiaSrcCat |
There was a problem hiding this comment.
I'm a bit worried about the juggling here, since getting this right requires knowledge of which tasks ApPipeMultiTractFakes overrides from ApPipe. Might it be simpler to work directly in terms of ProcessCcd.yaml instead?
There was a problem hiding this comment.
I think this juggling will be resolved when we switch everything to multi-tract in the future. I don't immediately see another way to get all the right components and configs imported.
mrawls
force-pushed
the
tickets/DM-32245
branch
2 times, most recently
from
8b3cc28
to
7c60c63
Compare
mrawls
force-pushed
the
tickets/DM-32245
branch
from
7c60c63
to
83996a5
Compare
No description provided.