Skip to content

Commit

Permalink
Update comments and docstrings per flake8/PEP8
Browse files Browse the repository at this point in the history
  • Loading branch information
@laurenam
laurenam committed Sep 20, 2022
1 parent 791c1bb commit c71c93d
Show file tree
Hide file tree
Showing 3 changed files with 127 additions and 88 deletions.
13 changes: 8 additions & 5 deletions python/lsst/pipe/tasks/calibrate.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,8 @@ class CalibrateTask(pipeBase.PipelineTask):
RuntimeError
Raised if any of the following occur:
- isSourceCat is missing fields specified in icSourceFieldsToCopy.
- PipelineTask form of this task is initialized with reference object loaders.
- PipelineTask form of this task is initialized with reference object
loaders.
Notes
-----
Expand Down Expand Up @@ -522,12 +523,13 @@ def run(self, exposure, exposureIdInfo=None, background=None,
exposure : `lsst.afw.image.ExposureF`
Exposure to calibrate.
exposureIdInfo : `lsst.obs.baseExposureIdInfo`, optional
Exposure ID info. If not provided, returned SourceCatalog IDs will not
be globally unique.
Exposure ID info. If not provided, returned SourceCatalog IDs will
not be globally unique.
background : `lsst.afw.math.BackgroundList`, optional
Initial model of background already subtracted from exposure.
icSourceCat : `lsst.afw.image.SourceCatalog`, optional
SourceCatalog from CharacterizeImageTask from which we can copy some fields.
SourceCatalog from CharacterizeImageTask from which we can copy
some fields.
Returns
-------
Expand Down Expand Up @@ -725,7 +727,8 @@ def setMetadata(self, exposure, photoRes=None):
def copyIcSourceFields(self, icSourceCat, sourceCat):
"""Match sources in an icSourceCat and a sourceCat and copy fields.
The fields copied are those specified by ``config.icSourceFieldsToCopy``.
The fields copied are those specified by
``config.icSourceFieldsToCopy``.
Parameters
----------
Expand Down
133 changes: 80 additions & 53 deletions python/lsst/pipe/tasks/makeWarp.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,8 @@ class MakeWarpConnections(pipeBase.PipelineTaskConnections,
storageClass="ExposureF",
dimensions=("tract", "patch", "skymap", "visit", "instrument"),
)
# TODO DM-28769, have selectImages subtask indicate which connections they need:
# TODO DM-28769, have selectImages subtask indicate which connections they
# need:
wcsList = connectionTypes.Input(
doc="WCSs of calexps used by SelectImages subtask to determine if the calexp overlaps the patch",
name="{calexpType}calexp.wcs",
Expand Down Expand Up @@ -274,48 +275,69 @@ class MakeWarpTask(CoaddBaseTask):
To make `psfMatchedWarps`, select `config.makePsfMatched=True`. The subtask
`~lsst.ip.diffim.modelPsfMatch.ModelPsfMatchTask`
is responsible for the PSF-Matching, and its config is accessed via `config.warpAndPsfMatch.psfMatch`.
The optimal configuration depends on aspects of dataset: the pixel scale, average PSF FWHM and
dimensions of the PSF kernel. These configs include the requested model PSF, the matching kernel size,
padding of the science PSF thumbnail and spatial sampling frequency of the PSF.
*Config Guidelines*: The user must specify the size of the model PSF to which to match by setting
`config.modelPsf.defaultFwhm` in units of pixels. The appropriate values depends on science case.
In general, for a set of input images, this config should equal the FWHM of the visit
with the worst seeing. The smallest it should be set to is the median FWHM. The defaults
is responsible for the PSF-Matching, and its config is accessed via
`config.warpAndPsfMatch.psfMatch`.
The optimal configuration depends on aspects of dataset: the pixel scale,
average PSF FWHM and dimensions of the PSF kernel. These configs include
the requested model PSF, the matching kernel size, padding of the science
PSF thumbnail and spatial sampling frequency of the PSF.
*Config Guidelines*: The user must specify the size of the model PSF to
which to match by setting `config.modelPsf.defaultFwhm` in units of pixels.
The appropriate values depends on science case. In general, for a set of
input images, this config should equal the FWHM of the visit with the worst
seeing. The smallest it should be set to is the median FWHM. The defaults
of the other config options offer a reasonable starting point.
The following list presents the most common problems that arise from a misconfigured
@link ip::diffim::modelPsfMatch::ModelPsfMatchTask ModelPsfMatchTask @endlink
and corresponding solutions. All assume the default Alard-Lupton kernel, with configs accessed via
```config.warpAndPsfMatch.psfMatch.kernel['AL']```. Each item in the list is formatted as:
The following list presents the most common problems that arise from a
misconfigured
@link ip::diffim::modelPsfMatch::ModelPsfMatchTask ModelPsfMatchTask
@endlink
and corresponding solutions. All assume the default Alard-Lupton kernel,
with configs accessed via
```config.warpAndPsfMatch.psfMatch.kernel['AL']```. Each item in the list
is formatted as:
Problem: Explanation. *Solution*
*Troublshooting PSF-Matching Configuration:*
- Matched PSFs look boxy: The matching kernel is too small. _Increase the matching kernel size.
- Matched PSFs look boxy: The matching kernel is too small.
_Increase the matching kernel size.
For example:_
config.warpAndPsfMatch.psfMatch.kernel['AL'].kernelSize=27 # default 21
config.warpAndPsfMatch.psfMatch.kernel['AL'].kernelSize=27
# default 21
Note that increasing the kernel size also increases runtime.
- Matched PSFs look ugly (dipoles, quadropoles, donuts): unable to find good solution
for matching kernel. _Provide the matcher with more data by either increasing
- Matched PSFs look ugly (dipoles, quadropoles, donuts): unable to find
good solution for matching kernel.
_Provide the matcher with more data by either increasing
the spatial sampling by decreasing the spatial cell size,_
config.warpAndPsfMatch.psfMatch.kernel['AL'].sizeCellX = 64 # default 128
config.warpAndPsfMatch.psfMatch.kernel['AL'].sizeCellY = 64 # default 128
config.warpAndPsfMatch.psfMatch.kernel['AL'].sizeCellX = 64
# default 128
config.warpAndPsfMatch.psfMatch.kernel['AL'].sizeCellY = 64
# default 128
_or increasing the padding around the Science PSF, for example:_
config.warpAndPsfMatch.psfMatch.autoPadPsfTo=1.6 # default 1.4
Increasing `autoPadPsfTo` increases the minimum ratio of input PSF dimensions to the
matching kernel dimensions, thus increasing the number of pixels available to fit
after convolving the PSF with the matching kernel.
Optionally, for debugging the effects of padding, the level of padding may be manually
controlled by setting turning off the automatic padding and setting the number
of pixels by which to pad the PSF:
config.warpAndPsfMatch.psfMatch.doAutoPadPsf = False # default True
config.warpAndPsfMatch.psfMatch.padPsfBy = 6 # pixels. default 0
Increasing `autoPadPsfTo` increases the minimum ratio of input PSF
dimensions to the matching kernel dimensions, thus increasing the
number of pixels available to fit after convolving the PSF with the
matching kernel. Optionally, for debugging the effects of padding, the
level of padding may be manually controlled by setting turning off the
automatic padding and setting the number of pixels by which to pad the
PSF:
config.warpAndPsfMatch.psfMatch.doAutoPadPsf = False
# default True
config.warpAndPsfMatch.psfMatch.padPsfBy = 6
# pixels. default 0
- Deconvolution: Matching a large PSF to a smaller PSF produces
a telltale noise pattern which looks like ripples or a brain.
_Increase the size of the requested model PSF. For example:_
config.modelPsf.defaultFwhm = 11 # Gaussian sigma in units of pixels.
- High frequency (sometimes checkered) noise: The matching basis functions are too small.
config.modelPsf.defaultFwhm = 11 # Gaussian sigma in units of
pixels.
- High frequency (sometimes checkered) noise: The matching basis functions
are too small.
_Increase the width of the Gaussian basis functions. For example:_
config.warpAndPsfMatch.psfMatch.kernel['AL'].alardSigGauss=[1.5, 3.0, 6.0]
# from default [0.7, 1.5, 3.0]
config.warpAndPsfMatch.psfMatch.kernel['AL'].alardSigGauss=
[1.5, 3.0, 6.0] # from default [0.7, 1.5, 3.0]
"""
ConfigClass = MakeWarpConfig
_DefaultName = "makeWarp"
Expand Down Expand Up @@ -346,9 +368,9 @@ def runQuantum(self, butlerQC, inputRefs, outputRefs):
quantumDataId = butlerQC.quantum.dataId
skyInfo = makeSkyInfo(skyMap, tractId=quantumDataId['tract'], patchId=quantumDataId['patch'])

# Construct list of input DataIds expected by `run`
# Construct list of input DataIds expected by `run`.
dataIdList = [ref.datasetRef.dataId for ref in inputRefs.calExpList]
# Construct list of packed integer IDs expected by `run`
# Construct list of packed integer IDs expected by `run`.
ccdIdList = [dataId.pack("visit_detector") for dataId in dataIdList]

if self.config.doApplyExternalSkyWcs:
Expand Down Expand Up @@ -389,7 +411,7 @@ def runQuantum(self, butlerQC, inputRefs, outputRefs):
goodIndices = self.select.run(**inputs, coordList=coordList, dataIds=dataIdList)
inputs = self.filterInputs(indices=goodIndices, inputs=inputs)

# Extract integer visitId requested by `run`
# Extract integer visitId requested by `run`.
visitId = dataIdList[0]["visit"]

results = self.run(**inputs, visitId=visitId,
Expand Down Expand Up @@ -431,7 +453,8 @@ def run(self, calExpList, ccdIdList, skyInfo, visitId=0, dataIdList=None, **kwar
``exposures``
A dictionary containing the warps requested:
"direct": direct warp if ``config.makeDirect``
"psfMatched": PSF-matched warp if ``config.makePsfMatched`` (`dict`).
"psfMatched": PSF-matched warp if ``config.makePsfMatched``
(`dict`).
"""
warpTypeList = self.getWarpTypeList()

Expand Down Expand Up @@ -485,11 +508,13 @@ def run(self, calExpList, ccdIdList, skyInfo, visitId=0, dataIdList=None, **kwar
warp.setPhotoCalib(exposure.getPhotoCalib())
warp.setFilter(exposure.getFilter())
warp.getInfo().setVisitInfo(exposure.getInfo().getVisitInfo())
# PSF replaced with CoaddPsf after loop if and only if creating direct warp
# PSF replaced with CoaddPsf after loop if and only if
# creating direct warp.
warp.setPsf(exposure.getPsf())
didSetMetadata[warpType] = True

# Need inputRecorder for CoaddApCorrMap for both direct and PSF-matched
# Need inputRecorder for CoaddApCorrMap for both direct and
# PSF-matched.
inputRecorder[warpType].addCalExp(calExp, ccdId, numGoodPix[warpType])

except Exception as e:
Expand All @@ -508,7 +533,7 @@ def run(self, calExpList, ccdIdList, skyInfo, visitId=0, dataIdList=None, **kwar
self.config.coaddPsf.makeControl()))
else:
if not self.config.doWriteEmptyWarps:
# No good pixels. Exposure still empty
# No good pixels. Exposure still empty.
warps[warpType] = None
# NoWorkFound is unnecessary as the downstream tasks will
# adjust the quantum accordingly.
Expand All @@ -521,13 +546,13 @@ def filterInputs(self, indices, inputs):
Parameters
----------
indices : `list` of `int`
inputs : `dict` of `list`
indices : `list` [`int`]
inputs : `dict` [`list`]
A dictionary of input connections to be passed to run.
Returns
-------
inputs : `dict` of `list`
inputs : `dict` [`list`]
Task inputs with their lists filtered by indices.
"""
for key in inputs.keys():
Expand Down Expand Up @@ -568,18 +593,18 @@ def _prepareCalibratedExposures(self, calExpList=[], wcsList=None, backgroundLis
Parameters
----------
calExpList : `list` of `lsst.afw.image.Exposure` or
`lsst.daf.butler.DeferredDatasetHandle`
calExpList : `list` [`lsst.afw.image.Exposure` or
`lsst.daf.butler.DeferredDatasetHandle`]
Sequence of calexps to be modified in place.
wcsList : `list` of `lsst.afw.geom.SkyWcs`
wcsList : `list` [`lsst.afw.geom.SkyWcs`]
The WCSs of the calexps in ``calExpList``. When
``externalSkyCatalog`` is `None`, these are used to determine if
the calexp should be included in the warp, namely checking that it
is not `None`. If ``externalSkyCatalog`` is not `None`, this list
will be dynamically updated with the external sky WCS.
backgroundList : `list` of `lsst.afw.math.backgroundList`, optional
backgroundList : `list` [`lsst.afw.math.backgroundList`], optional
Sequence of backgrounds to be added back in if bgSubtracted=False.
skyCorrList : `list` of `lsst.afw.math.backgroundList`, optional
skyCorrList : `list` [`lsst.afw.math.backgroundList`], optional
Sequence of background corrections to be subtracted if
doApplySkyCorr=True.
externalSkyWcsCatalog : `lsst.afw.table.ExposureCatalog`, optional
Expand All @@ -601,7 +626,8 @@ def _prepareCalibratedExposures(self, calExpList=[], wcsList=None, backgroundLis
Returns
-------
indices : `list` [`int`]
Indices of calExpList and friends that have valid photoCalib/skyWcs.
Indices of ``calExpList`` and friends that have valid
photoCalib/skyWcs.
"""
wcsList = len(calExpList)*[None] if wcsList is None else wcsList
backgroundList = len(calExpList)*[None] if backgroundList is None else backgroundList
Expand All @@ -626,7 +652,7 @@ def _prepareCalibratedExposures(self, calExpList=[], wcsList=None, backgroundLis

detectorId = calexp.getInfo().getDetector().getId()

# Find the external photoCalib
# Find the external photoCalib.
if externalPhotoCalibCatalog is not None:
row = externalPhotoCalibCatalog.find(detectorId)
if row is None:
Expand All @@ -646,7 +672,7 @@ def _prepareCalibratedExposures(self, calExpList=[], wcsList=None, backgroundLis
"and will not be used in the warp.", detectorId)
continue

# Find and apply external skyWcs
# Find and apply external skyWcs.
if externalSkyWcsCatalog is not None:
row = externalSkyWcsCatalog.find(detectorId)
if row is None:
Expand All @@ -668,7 +694,7 @@ def _prepareCalibratedExposures(self, calExpList=[], wcsList=None, backgroundLis
"and will not be used in the warp.", detectorId)
continue

# Find and apply finalized psf and aperture correction
# Find and apply finalized psf and aperture correction.
if finalizedPsfApCorrCatalog is not None:
row = finalizedPsfApCorrCatalog.find(detectorId)
if row is None:
Expand All @@ -688,11 +714,12 @@ def _prepareCalibratedExposures(self, calExpList=[], wcsList=None, backgroundLis
continue
calexp.info.setApCorrMap(apCorrMap)

# Calibrate the image
# Calibrate the image.
calexp.maskedImage = photoCalib.calibrateImage(calexp.maskedImage,
includeScaleUncertainty=includeCalibVar)
calexp.maskedImage /= photoCalib.getCalibrationMean()
# TODO: The images will have a calibration of 1.0 everywhere once RFC-545 is implemented.
# TODO: The images will have a calibration of 1.0 everywhere once
# RFC-545 is implemented.
# exposure.setCalib(afwImage.Calib(1.0))

# Apply skycorr
Expand Down Expand Up @@ -759,7 +786,7 @@ def reorderRefs(inputRefs, outputSortKeyOrder, dataIdKey):
Returns
-------
inputRefs: `lsst.pipe.base.connections.QuantizedConnection`
inputRefs : `lsst.pipe.base.connections.QuantizedConnection`
Quantized Connection with sorted DatasetRef values sorted if iterable.
"""
for connectionName, refs in inputRefs:
Expand Down

0 comments on commit c71c93d

Please sign in to comment.