DM-44889: Investigate feasibility of resurrecting matchBackgrounds.py background fitting algorithms #956
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aemerywatkins
force-pushed
the
tickets/DM-44889
branch
from
d99bc60 to
8dfbe5e
Compare
leeskelvin
force-pushed
the
tickets/DM-44889
branch
2 times, most recently
from
a95c234 to
b0463a5
Compare
leeskelvin
force-pushed
the
tickets/DM-44889
branch
from
7c1554f to
fea12d4
Compare
aemerywatkins
force-pushed
the
tickets/DM-44889
branch
from
fea12d4 to
30ebfa0
Compare
aemerywatkins
force-pushed
the
tickets/DM-44889
branch
from
52df871 to
3210ec9
Compare
aemerywatkins
force-pushed
the
tickets/DM-44889
branch
from
a969a20 to
f379682
Compare
aemerywatkins
force-pushed
the
tickets/DM-44889
branch
from
ac42437 to
c652ce0
Compare
aemerywatkins
force-pushed
the
tickets/DM-44889
branch
from
6a6037f to
10cf7cc
Compare
leeskelvin
force-pushed
the
tickets/DM-44889
branch
from
10cf7cc to
072ece2
Compare
TallJimbo
approved these changes
Nov 12, 2025
Member
TallJimbo
left a comment
TallJimbo
left a comment
There was a problem hiding this comment.
Seems like a decent cleanup of the existing code. Only serious problem is the use of a single BackgroundList to represent the backgrounds for all visits, but that should be easy to fix (and I think using more visit-ID-keyed dicts instead of lists will help in other parts of the code, too).
aemerywatkins
force-pushed
the
tickets/DM-44889
branch
2 times, most recently
from
d957a3c to
b7318ec
Compare
TallJimbo
approved these changes
Nov 13, 2025
Initial version of this task is written using old architecture, and so needs updating. In MatchBackgroundsTask, and its method selectRefExposure, required parameters were equally outdated: DataId, DatasetType, and ImageScaler. All of these now seem consolidated under lsst.afw.image.Exposure, so separate calls to DataId and DatasetType are now single calls for Exposure objects. ImageScaler calls were replaced in-line with Exposure.getPhotoCalib() calls, to scale all image flux to the same zeropoint (nJy). Also, we want to process visit-level images using this, so a MatchBackgroundsConnections class was created, MatchBackgroundsConfig was updated to inherit from PipelineTaskConfig (and those connections), and a rudimentary runQuantum method was added to MatchBackgroundsTask.
Code now runs without complaint through self.matchBackgrounds. Also added a self._fluxScale method to replace repeat code blocks. Will decide later if scaling to nJy is the best way to do this.
Code is now functional, in that it accepts images and returns difference image background models as "psfMatchedWarpBackground_diff" (name likely to be altered later). Uses a fit to a blank image for that corresponding to the reference image.
Difference background models are now formatted properly, to allow for image creation from the spline parameters. Also did some adjustments to documentation for Flake8 formatting.
_defineWarps() now rejects any image with all NaNs along any image edge, and creates the cost function using a sky-subtracted image. This sky-subtraction fits a 1st order Chebyshev polynomial to the masked image background. Also fixed a bug from LSK refactor by inserting a blank sky model into the background model list at the chosen reference image index.
Otherwise, changes are clean-up from previous refactoring to restore functionality, plus a bug fix. Bug fix was the restoration of two lines of code in MatchBackgroundsTask.matchBackgrounds() which produced a difference image to work from.
All images and background models now returned in counts, not nJy.
`matchBackgrounds` in its original form matches by warps, i.e. single detectors. A more sensible thing to do is to match backgrounds across the whole focal plane. This functionality needed to be added to do this using warped exposures, in tract coordinates, so this is now added to `backgrounds.py`. This commit also includes a partial revision to `matchBackgrounds.py` using this new functionality, choosing a reference visit ID based on these background models instead of on individual warps. Full functionality has yet to be restored in light of these changes.
All methods, including run, updated to function properly with tract backgrounds rather than warps. Task completes without error when run on three full visits, and output appears roughly correct.
aemerywatkins
force-pushed
the
tickets/DM-44889
branch
from
74b8f32 to
14d3ca1
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The matchBackgrounds.py script was an initial attempt at matching multiple warped visit-level images to a given reference image. Taking the difference images between the reference and each subsequent image allows for each of these images to be modified to match the background of the reference image. These images may then be stacked (coadded), and a single background estimation made on the combined (and, importantly, deeper) coadd image.
The existing matchBackgroundsTask has not been utilized in some time. This ticket aims to explore the feasibility of resurrecting this code base, with a mind towards testing on HSC in the near-term.