Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-30444: Raise when template insufficiently covers the calexp #528

Merged
merged 2 commits into from Jun 10, 2021

Conversation

yalsayyad
Copy link
Contributor

No description provided.

@yalsayyad yalsayyad force-pushed the tickets/DM-30444 branch 2 times, most recently from b4d9fda to 03f30f6 Compare June 9, 2021 15:54
Copy link
Contributor

@isullivan isullivan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine for now, and I don't have a specific value in mind for requiredTemplateFraction I suggest changing it to 10% simply because 12% may look carefully chosen by a future developer.

More broadly, while I think this is an improvement and sufficient for now, I think we should change it substantially in the future. I suspect we will want to construct something like an nImage from coaddition and select images (or possibly even parts of images) based on the availability of images across the patch.

# In the meantime, raise to exit and register as a failure
message = "Insufficient Template Coverage. (%.1f%% < %.1f%%) Not attempting subtraction." % (
100*templateStruct.area/inputs['exposure'].getBBox().getArea(),
100*self.config.requiredTemplateFraction)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be too verbose, but you could include "Set requiredTemplateFraction=0 to force subtraction."

requiredTemplateFraction = pexConfig.Field(
dtype=float, default=0.12,
doc="Do not attempt to run task if template covers less than this fraction of pixels."
"Setting to 0 will always attempt image subtraction"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider setting the threshold to 10% and adding a comment that the specific value is somewhat arbitrary. I worry that future developers will give the specific value a lot of weight and cargo-cult it.

Copy link
Contributor Author

@yalsayyad yalsayyad Jun 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

17% is also on the table as the minimum needed to be able to drop this line excluding two visits that fail because they have 12 and 16% template coverage.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree 17% would also be a reasonable default. We tend to get more junk than good data from the edge regions, so it's reasonable to be more aggressive about throwing it out. Please do explain in the docstring where the number is coming from, though.

@yalsayyad
Copy link
Contributor Author

Yeah, thats what I was saying in the comment of the Jira ticket:
Of the 14 remaining failures due to bad templates with requiredFraction = 5% shown here:
https://jira.lsstcorp.org/secure/attachment/50020/50020_Untitled.html
Only half are due to insufficient numbers of pixels and the rest are insufficient numbers of pixels. Next version is going to use goodSeeing_nImage to filter out the latter.

@yalsayyad yalsayyad force-pushed the tickets/DM-30444 branch 2 times, most recently from 137ecbf to 0920d1d Compare June 10, 2021 00:40
A minimum of 3 visits is needed per pixel for a sufficient template.
Because of incomplete coverage of the visits (e.g. chip gaps, edges)
the nVisitMin should be higher than 3. No claims that 6 is optimal
What is considered sufficient template coverage is controlled by
the config parameter requiredTemplateFraction
@yalsayyad yalsayyad merged commit 362f0c0 into master Jun 10, 2021
@yalsayyad yalsayyad deleted the tickets/DM-30444 branch June 10, 2021 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants