Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-31710: optimize crosstalkSources lookupFunction + small cleanups/fixes #195

Merged
merged 3 commits into from
Sep 13, 2021
Merged

DM-31710: optimize crosstalkSources lookupFunction + small cleanups/fixes #195

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b1dd5d7
Optimize lookupFunction for crosstalkSources.
TallJimbo Sep 9, 2021
9a8a4ba
Fix crosstalkSources connection discard when doCrosstalk=False.
TallJimbo Sep 9, 2021
53f3be4
Delete already-resolved TODO comments/docs.
TallJimbo Sep 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
21 changes: 9 additions & 12 deletions python/lsst/ip/isr/isrTask.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,6 @@ def crosstalkSourceLookup(datasetType, registry, quantumDataId, collections):
when inter-chip crosstalk has been identified should this be
populated.

This will be unused until DM-25348 resolves the quantum graph
generation issue.

Parameters
----------
datasetType : `str`
Expand All @@ -89,12 +86,14 @@ def crosstalkSourceLookup(datasetType, registry, quantumDataId, collections):
crosstalkSources.
"""
newDataId = quantumDataId.subset(DimensionGraph(registry.dimensions, names=["instrument", "exposure"]))
results = list(registry.queryDatasets(datasetType,
collections=collections,
dataId=newDataId,
findFirst=True,
).expanded())
return results
results = set(registry.queryDatasets(datasetType, collections=collections, dataId=newDataId,
findFirst=True))
# In some contexts, calling `.expanded()` to expand all data IDs in the
# query results can be a lot faster because it vectorizes lookups. But in
# this case, expandDataId shouldn't need to hit the database at all in the
# steady state, because only the detector record is unknown and those are
# cached in the registry.
return [ref.expanded(registry.expandDataId(ref.dataId, records=newDataId.records)) for ref in results]


class IsrTaskConnections(pipeBase.PipelineTaskConnections,
Expand Down Expand Up @@ -122,8 +121,6 @@ class IsrTaskConnections(pipeBase.PipelineTaskConnections,
isCalibration=True,
minimum=0, # can fall back to cameraGeom
)
# TODO: DM-25348. This does not work yet to correctly load
# possible crosstalk sources.
crosstalkSources = cT.PrerequisiteInput(
name="isrOverscanCorrected",
doc="Overscan corrected input images.",
Expand Down Expand Up @@ -279,7 +276,7 @@ def __init__(self, *, config=None):
if config.doLinearize is not True:
self.prerequisiteInputs.discard("linearizer")
if config.doCrosstalk is not True:
self.inputs.discard("crosstalkSources")
self.prerequisiteInputs.discard("crosstalkSources")
self.prerequisiteInputs.discard("crosstalk")
if config.doBrighterFatter is not True:
self.prerequisiteInputs.discard("bfKernel")
Expand Down