DM-37984: Move rc2_subset DRP pipeline definitions to drp_pipe #54
Conversation
leeskelvin
force-pushed
the
tickets/DM-37984
branch
from
39189a5
to
bcf7001
Compare
leeskelvin
force-pushed
the
tickets/DM-37984
branch
2 times, most recently
from
20760be
to
f17fad2
Compare
pipelines/HSC/DRP-RC2_subset.yaml
Outdated
| time, because otherwise quanta will be created for jobs with only partial | ||
| visit coverage. | ||
| isolatedStarAssociation can be run with or without 'tract' constraints. | ||
| nightlyStep2cde: |
There was a problem hiding this comment.
Does it work to include FGCM in the same step with the other tasks? In rc2_subset, FGCM has been separate because it explicitly has to be run on a single processor. Is that true of the other tasks here, or would it improve performance to keep them separate?
There was a problem hiding this comment.
See the shell script that executes rc2_subset processing at https://github.com/lsst/rc2_subset/blob/bf14286d29ed909cd0b0949231dc2b19725cb8d1/bin/measureHscRC2Metrics.sh
There was a problem hiding this comment.
Thanks Jeff. Following a conversation with @erykoff, I've decided to revert this change and maintain the three FGCM tasks in their own nightlyStep3c subset, with the remainder continuing on as before in nightlyStep2d. To that end, the only actual pipeline changes should now be the addition of makeVisitTable and makeCcdVisitTable into nightlyStep2d. Other proposed changes, such as moving isolatedStarAssociation from nightlyStep2b into nightlyStep2a (as in DRP-RC2.yaml) have also been dropped.
| visit coverage. | ||
| isolatedStarAssociation can be run with or without 'tract' constraints. | ||
| nightlyStep2cde: | ||
| subset: |
There was a problem hiding this comment.
I think TE3 and TE4 should also be in this subset (per-visit tasks)?
There was a problem hiding this comment.
These have been moved back to their former home in nightlyStep2d.
pipelines/HSC/DRP-RC2_subset.yaml
Outdated
| - TE3 | ||
| - TE4 |
There was a problem hiding this comment.
See note above - TE3 and TE4 are per-visit tasks, so should be in Step2cde?
There was a problem hiding this comment.
These have been moved back to their former home in nightlyStep2d.
pipelines/HSC/DRP-RC2_subset.yaml
Outdated
| detector-level quanta are desired. | ||
| nightlyStep5: | ||
| subset: | ||
| - nsrcMeasVisit |
There was a problem hiding this comment.
This is a per-visit task, so should be in Step2cde?
There was a problem hiding this comment.
This has been moved back to its former home in nightlyStep2d.
leeskelvin
force-pushed
the
tickets/DM-37984
branch
from
f17fad2
to
772af6a
Compare
No description provided.