DM-26553: Create gen3 unittests and test CreateApFakes pipeline tasks on data. #409
Conversation
| @@ -681,8 +680,7 @@ def cleanCat(self, fakeCat, starCheckVal): | |||
| will be added. | |||
| """ | |||
|
|
|||
| rowsToKeep = (((fakeCat[self.config.bulgeHLR] != 0.0) & (fakeCat[self.config.diskHLR] != 0.0)) | |||
| | (fakeCat[self.config.sourceType] == starCheckVal)) | |||
| rowsToKeep = ((fakeCat[self.config.bulgeHLR] != 0.0) & (fakeCat[self.config.diskHLR] != 0.0)) | |||
There was a problem hiding this comment.
You've lost a line here from the check.
There was a problem hiding this comment.
Huh, don't know when that happened. Have put it back.
There was a problem hiding this comment.
Even after looking at CreateRandomApFakesTask, I'm a bit unsure about the use of tract as one of the dimensions for ProcessCcdWithFakesTask. It looks like the Gen 2 version of ProcessCcdWithFakesTask operated mostly on the detector level, with tracts only used for auxiliary data.
Is this a deliberate change between the Gen 2 and Gen 3 behavior? If not, it will cause problems with other tasks that assume fakes_calexp and fakes_src are detector-level.
I don't want to sign off on this until I understand what's going on.
|
|
||
| if not config.useUpdatedCalibs: | ||
| self.prerequisiteInputs.remove("wcs") | ||
| self.prerequisiteInputs.remove("photoCalib") |
There was a problem hiding this comment.
Should be inputs, not prerequisiteInputs.
| fakeCat = cT.Input( | ||
| doc="Catalog of fake sources to draw inputs from.", | ||
| name="{CoaddName}Coadd_fakeSourceCat", | ||
| storageClass="Parquet", | ||
| storageClass="DataFrame", | ||
| dimensions=("tract", "skymap") |
There was a problem hiding this comment.
@sr525, just to make sure, is it ok for the Gen 3 code to only take a tract-level *Coadd_fakeSourceCat, and not a detector-level fakeSourceCat? That seems to be the only substantial difference between the Gen 2 and Gen 3 versions of the task at the moment.
There was a problem hiding this comment.
Okay, so I get what you're asking now. The catalogs are not different in how they are stored really. Both will be at the tract level, they just have different fluxes for the visit data vs coadd data.
There was a problem hiding this comment.
Probably not? The visit level catalogues are needed for the variable sources, where the current way of using them is to make a catalogue specific to each calexp.
There was a problem hiding this comment.
Wait so are you saying we are going to produce specific catalogs for each ccdVisit vs cutting out the sources from a tract level catalog?
There was a problem hiding this comment.
Okay, this conversation doesn't seem to be continuing so I'm going to go ahead and merge the ticket.
morriscb
force-pushed
the
tickets/DM-26553
branch
from
8999cef
to
ae6adc7
Compare
Add dropped or statement from insertFakes. Remove tract from fakes output in procCcdWithFakes. Make jointcal connecdtions optional.
morriscb
force-pushed
the
tickets/DM-26553
branch
from
ae6adc7
to
f5dc18a
Compare
No description provided.