Skip to content

Commit

Permalink
Include tract in dimensions of intermediate forcedSource
Browse files Browse the repository at this point in the history
ForcedPhotCcdTask includes tract in its dimensions which means
that it produces an output catalog per-detector, per-tract.
WriteForcedSourceTableTask which merges catalogs of forced measurements from
calexps and diffims should do the same. TransformForcedSource temporally
aggregates the forcedSource catalogs per-patch anyway.

-rename forcedSourceTable to mergedForcedSourceTable because you cannot
change the dimensions  of an already registerd datasetType
  • Loading branch information
yalsayyad committed Oct 25, 2021
1 parent 0c33f26 commit 712bdd3
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 6 deletions.
4 changes: 2 additions & 2 deletions pipelines/DRP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -126,14 +126,14 @@ tasks:
key: diaObjectId
connections.inputCatalogDiff: forced_diff_diaObject
connections.inputCatalog: forced_src_diaObject
connections.outputCatalog: forcedSourceOnDiaObject
connections.outputCatalog: mergedForcedSourceOnDiaObject
transformForcedSourceOnDiaObjectTable:
class: lsst.pipe.tasks.postprocess.TransformForcedSourceTableTask
config:
referenceColumns: []
keyRef: diaObjectId
key: forcedSourceOnDiaObjectId
connections.inputCatalogs: forcedSourceOnDiaObject
connections.inputCatalogs: mergedForcedSourceOnDiaObject
connections.outputCatalog: forcedSourceOnDiaObjectTable
connections.referenceCatalog: goodSeeingDiff_fullDiaObjTable
consolidateForcedSourceOnDiaObjectTable:
Expand Down
16 changes: 12 additions & 4 deletions python/lsst/pipe/tasks/postprocess.py
Original file line number Diff line number Diff line change
Expand Up @@ -1578,9 +1578,9 @@ class WriteForcedSourceTableConnections(pipeBase.PipelineTaskConnections,
)
outputCatalog = connectionTypes.Output(
doc="InputCatalogs horizonatally joined on `objectId` in Parquet format",
name="forcedSource",
name="mergedForcedSource",
storageClass="DataFrame",
dimensions=("instrument", "visit", "detector")
dimensions=("instrument", "visit", "detector", "skymap", "tract")
)


Expand All @@ -1595,6 +1595,14 @@ class WriteForcedSourceTableConfig(WriteSourceTableConfig,

class WriteForcedSourceTableTask(pipeBase.PipelineTask):
"""Merge and convert per-detector forced source catalogs to parquet
Because the predecessor ForcedPhotCcdTask operates per-detector,
per-tract, (i.e., it has tract in its dimensions), detectors
on the tract boundary may have multiple forced source catalogs.
The successor task TransformForcedSourceTable runs per-patch
and temporally-aggregates overlapping mergedForcedSource catalogs from all
available multiple epochs.
"""
_DefaultName = "writeForcedSourceTable"
ConfigClass = WriteForcedSourceTableConfig
Expand Down Expand Up @@ -1628,9 +1636,9 @@ class TransformForcedSourceTableConnections(pipeBase.PipelineTaskConnections,

inputCatalogs = connectionTypes.Input(
doc="Parquet table of merged ForcedSources produced by WriteForcedSourceTableTask",
name="forcedSource",
name="mergedForcedSource",
storageClass="DataFrame",
dimensions=("instrument", "visit", "detector"),
dimensions=("instrument", "visit", "detector", "skymap", "tract"),
multiple=True,
deferLoad=True
)
Expand Down

0 comments on commit 712bdd3

Please sign in to comment.