New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DM-21212: Update existing cp_pipe tasks to pipelineTasks #148
Conversation
91529ef
to
158bcd1
Compare
47c799f
to
782ad59
Compare
20b122d
to
ad16882
Compare
ad16882
to
ae5e5fc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would think that most of the contents of these files can be inherited (but I don't know for sure). You should trim as much as you possibly can, as having multiple layers doing the same thing is asking for trouble.
doCrosstalk: False | ||
doDefect: True | ||
doNanMasking: True | ||
doInterpolate: True |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can most of the above be inherited?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My understanding is that it cannot for complexity reasons (inheritance + config setting can be ambiguous/confusing if they both set config parameters). The real fix is in producing CT/Defects/etc for obs_lsst, which would allow the simply inherited version to be used (like obs_subaru).
pipelines/latiss/cpBias.yaml
Outdated
connections.inputExps: 'cpBiasProc' | ||
connections.outputData: 'biasProposal' | ||
calibrationType: 'bias' | ||
exposureScaling: "ExposureTime" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You look to be scaling by the exposure time, which should be zero...?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a mistake now fixed. It worked because there's a safe guard to prevent scaling by zero, but it's better to ask for the right thing, "None".
config: | ||
connections.inputExps: 'cpBiasProc' | ||
connections.outputData: 'biasProposal' | ||
calibrationType: 'bias' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't the above get inherited?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From the master cp_pipe pipeline definition? No.
From the camera? Yes, which was one of the issues making this change necessary.
pipelines/latiss/cpBias.yaml
Outdated
contracts: | ||
- isr.doBias == False | ||
- cpCombine.calibrationType == "bias" | ||
- cpCombine.exposureScaling == "ExposureTime" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Inherited?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Contracts are not inherited, and are there to ensure command line options don't break what the pipeline is supposed to do.
1096901
to
daec367
Compare
a9bf4ed
to
ae12075
Compare
I've filed a new ticket to handle this issue, https://jira.lsstcorp.org/browse/DM-25329 , in the interest of getting this ancient ticket merged. I'm planning on updating the ci_cpp_latiss code to fully test obs_lsst and LATISS on real data. |
Add LATISS cp_pipe pipelines.