DM-19852: Copy SkySubtractionTask from pipe_drivers #707
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erykoff
approved these changes
Jul 29, 2022
We combine a large-scale background model and a scaled sky frame and write a background model consisting of these (and un-doing the original) that can be used downstream to correct the calexp. This does a decent job over most of the field of view of HSC.
This provides a facility to produce a binned image of the entire focal plane (butler product visit_camera).
Makes the binning factor configurable, and reduces boilerplate.
We use a wide matched-filter and relatively low threshold to pick up faint extended wings of bright sources. This reduces the over-subtraction around bright sources.
Prepare Tasks for Gen3 PipelineTask conversion by: * Renaming CmdlineTask's entry method to `runDataRef`. The default CmdlineTask.TaskRunner now calls a Task's `runDataRef` method on the parsed command line inputs. `runDataRef` method can take any Gen2 Butler data products. * Renaming CmdlineTasks previous core methods (e.g. assemble, characterize) to `run` when they exist.
Tim Jenness points out that the 'maskedImage' variable isn't defined. Looks like it was renamed 'image' at some point, but not updated in this block.
This allows a better measurement of the sky level, less affected by the wings of bright objects.
This cleans up any remaining low-level sky after the original two stages (large-scale focalplane sky-subtraction and sky frame subtraction).
The new Gen3 implementation of the skyCorrection code is serialized, and duplicates some of the SkyCorrectionTask methods to provide this serialized form via a run() method (which matches the Gen2 runDataRef() method). Comments have been added to link the Gen2 and Gen3 methods, and both methods should be updated for any future code changes. An additional "rawLinker" input dataset is needed for Gen3, as the calExp (which has the dataset.visit column populated) cannot find a link to find the appropriate camera/sky calibration entries. Adding this rawLinker allows the calExp to connect to the raw data that was its source, and use that connection to find the calibration files. This dataset is used only for constructing the quantum graph, and is removed prior to processing in the task run() method.
If subtracting a background model fails, log the failure as an error and return immediately so any additional dataRefs can continue to run.
for changed gen3 interface: - storage class for raws - calibration_label not in use anymore - need lookup function as workaround for inability to query calibration collections
Order of inputs is not guaranteed, and the weighting per detector to the full focal plane sky model can depend (very weakly) on the order of inputs, so sort them by detector to guarantee reproducible results for the same set of inputs.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.