meas_base DM-1954: HeavyFootprints in forced coadd photometry #23
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # parent is broken and raises an exception if it occurs. | ||
| # | ||
| # NOTE: this culling should to be accounted for at the reference catalog creation | ||
| # stage (of which there are several) but is not currently implemented. |
There was a problem hiding this comment.
Trivial grammar error: "should to be accounted for".
There was a problem hiding this comment.
Also, instead of saying the culling is not implemented at the reference catalog stage, I think it's probably more accurate to say the code merely checks that this precondition is satisfied by whatever reference catalog provider is being paired with this code. We do in fact have reference catalog creation code that guarantees these conditions are satisfied under the conditions in which they are run.
When these Tasks are run in their own stand-alone scripts, the schema is loaded through the butler. However, when they are run as part of a common operation, the Tasks are instantiated before the schemas are written out, and so they need to be given the schemas at construction time. This change makes both possible: the schema is used if provided, otherwise loaded from the butler. Note: this is essentially a cherry-pick from HSC-1063 except that we still refer to Coadd_src_schema instead of Coadd_ref_schema. This will likely change once we have the multiband processing working on the LSST stack.
While we may want to load from deepCoadd_src, under the multiband processing scheme it's deepCoadd_ref, so that suffix needs to be customisable. Have NOT gone to deepCoadd_ref exclusively yet, so the old forced photometry should still work.
Now that we have consistent source IDs across all coadd measurement catalogs (because we're merging at the Footprint/Peak level before deblending), those catalogs can all have consistent IDs, instead of generating their own. That means we want to use the reference catalog ID as the source ID in forced photometry. This is essentially a cherry-pick from HSC-1062's commit: HyperSuprime-Cam/pipe_tasks@d85a382
Forced photometry previously relied on the MeasureSources class to transform reference Footprints and attach them to sources, but we've moved that responsibility here so we can override it at the Python level in the future. This is essentially a cherry-pick from HSC-1062's commit: HyperSuprime-Cam/pipe_tasks@0f29c18
This allows us to use deblend results from the single-band measurement phase to replace neighbors with noise in forced photometry. This is essentially a cherry-pick from HSC-1062's commit: HyperSuprime-Cam/pipe_tasks@2218b2e
MultiBandReferences can't delegate its config validation directly to CoaddSrcReferences, because that includes a check for chi-squared coadd that we don't want. This is essentially a cherry-pick from HSC-1062's commit: HyperSuprime-Cam/pipe_tasks@14efa55
This is essentially a cherry-pick from HSC-1062's commit: HyperSuprime-Cam/pipe_tasks@814036b
Printing out the number of sources isn't as useful as printing the data ID, especially when we already have other log messages that print the same number This is essentially a cherry-pick from HSC-1062's commit: HyperSuprime-Cam/pipe_tasks@da5a18d
Now that we're trying to use deblended HeavyFootprints here, we can't afford to have parent footprints rejected as being in the overlap region if their children might not be rejected, because the ReplaceWithNoiseTask requires all sources to have a parent. In order to change this back, we'd need more intelligent handling of the overlaps. But once this is loaded into the database, we should be able to join with the main measurement catalog and just use the "primary" flags there to handle overlaps. This is essentially a cherry-pick from HSC-1062's commit: HyperSuprime-Cam/pipe_tasks@681422d
Instead of having a separate ID for each forced source, we'll just have the foreign key to the coadd-based references. We'll do the same with the deblender fields (since there isn't any deblending in CCD forced photometry anyway). This is essentially a cherry-pick from HSC-1218's commit: HyperSuprime-Cam/pipe_tasks@9e72537
The ReplaceWithNoise procedure in measurement requires all child sources to have their parents present, so we can't do any kind of filtering that could cut a parent but not one or more of its children. This is essentially a cherry-pick of the changes to reference.py from HSC-1218's commit (i.e. the changes to forcedPhotImage.py were not made as they are not appropriate): HyperSuprime-Cam/pipe_tasks@c442bc9
This is essentially a cherry-pick of the changes to forcedPhotCcd.py from HSC-1256's commit (i.e. the changes to forcedPhotImage.py were not made as they are not appropriate): HyperSuprime-Cam/pipe_tasks@d369520
A broken, pre-review version of HSC-1256 was accidentally merged to master. This is essentially a cherry-pick of the changes to forcedPhotCcd.py from HSC's stand-alone commit (i.e. the changes to forcedPhotImage.py were not made as they are not appropriate): HyperSuprime-Cam/pipe_tasks@d6d652b
Check that the reference catalog does not contain any children for which any member of their parent chain is not within the list. This can occur at boundaries when the parent is outside and one of the children is within. Currently, the parent chain is always only one deep, but just in case, this code checks for any case where when the parent chain to a child's topmost parent is broken and raises an exception if it occurs.
Footprints are now attached to the sources using the class's implementation of attachFootprints().
Moving the source generation from ForcedMeasurementTask in forcedMeasurement.py to ProcessImageForcedTask in forcedPhotImage.py is required so that we can pass the sources to attachFootprints() before measurement of forced photometry.
Needed to access forcedPhotCcd.py and forcedPhotCoadd.py command line tasks.
In doing so we would lose the attributes of the SourceCatalog (e.g. getChildren()) which are required by the measurement.
This is required so that Footprints can be attached to the generated sources prior to forced measurement. ForcedPhotImageTask inherits this default method (which downgrades HeavyFootprints into regular Footprints). Subclasses for ForcedPhotImageTask must implement their own version of this method to define how those Footprints should be generated.
We need to do forced measurement using the exposure that has had it's background fields updated in the multi-band measurement stage. See lsst/pipe_tasks@7901922
laurenam
force-pushed
the
u/lauren/DM-1954
branch
from
9b810e8
to
a354455
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.