Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tickets/DM-16844 #250

Merged
merged 1 commit into from Dec 24, 2018
Merged

tickets/DM-16844 #250

merged 1 commit into from Dec 24, 2018

Conversation

natelust
Copy link
Contributor

Introduce PipelineTasks for doing deblending

The stack has two different ways of deblending, one band at a time,
or all bands together. Because these have different units of work
in a quantum graph, the work that was once done by a single
command line task has now been split up into two different
PipelineTasks. Of note is that these tasks are only used when processing
in a PipelineTask sense, and the original DeblendCoaddSources task must
still be used when processing with command line tasks.

Copy link
Member

@TallJimbo TallJimbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Only minor comments.

storageClass="SourceCatalog"
)
outputSchema = InitOutputDatasetField(
doc='Output of the schema used in deblending task',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: seemingly random switch from double to single quotes here.

target=SourceDeblendTask,
doc="Task to deblend an image in one band"
)
exposure = InputDatasetField(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we call this "coadd" instead of "exposure", so the name reflects the concept rather than the (unfortunately-named) Python/C++ type?

name='',
nameTemplate='{}Coadd_deblendedFlux_schema',
storageClass='SourceCatalog'
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You already have a outputSchema (capital 'S') in the base class config; I suspect this (and the one in the multi variant) are not needed.


class DeblendCoaddSourcesMultiConfig(DeblendCoaddSourcesBaseConfig):
multiBandDeblend = ConfigurableField(
target=MultibandDeblendTask,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'B' in 'band' should either always be capitalized or never. Looks like the precedent is "always".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, everything needs to switch to "multiband" as the task name that is imported is "MultibandDeblendTask"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but we have multiBand.py here and multiBandDriver.py in pipe_drivers. I'm not actually opposed to switching conventions to Multiband, as I think that's actually more correct given that "multi" isn't a word in its own right, but in that case we should make sure we (eventually) go back and make those consistent with this.

return self.run(**inputData)

def _makeSourceCatalog(self, mergedDetections, idFactory):
# Need to do something more cleaver once we have a real Idfactory here
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, commenting on a typo in a TODO comment feels really nitpicky, but ... "cleaver".

return outputTypeDict

def adaptArgsAndRun(self, inputData, inputDataIds, outputDataIds, butler):
inputData['filters'] = [dId['abstract_filter'] for dId in inputDataIds['exposures']]
Copy link
Member

@TallJimbo TallJimbo Dec 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to add a code comment noting that adding a new entry to inputData results in that being passed as a kwarg to run. I suppose that's true for the base class, too, and that gets me wondering how we could improve the code clarity for this pattern without needing to comment it every time we use it, but I do think that for people who aren't you and mean it's not at all clear what's going on. Maybe we should rename the inputData parameter to something like runKwargs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a note on my to-do list to come back to this later, as discussed in person

The stack has two different ways of deblending, one band at a time,
or all bands together. Because these have different units of work
in a quantum graph, the work that was once done by a single
command line task has now been split up into two different
PipelineTasks. Of note is that these tasks are only used when processing
in a PipelineTask sense, and the original DeblendCoaddSources task must
still be used when processing with command line tasks.
@natelust natelust merged commit 6cf5cc0 into master Dec 24, 2018
@timj timj deleted the tickets/DM-16844 branch February 18, 2021 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants