-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ellipticity validation #113
Conversation
Thanks for the update @msimet - CC @dkorytov @evevkovacs - based on the updated DESCQA test Melanie posted above, it looks like the ellipticity model upgrade was applied to v4.4 and v4.5, but not v4.5_rescale. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @msimet! The change looks good. I wonder if it's better to specify validation_percentiles
in the yaml config file as oppose to hard writing them in the code as they currently are?
I'd be happy to make that change if you think it's useful! |
@msimet Yes, in particular because the config files will included in the web interface, it will make it easier for users to find out what validation criteria are used. Thank you for making that change! |
I think I have implemented these changes--you can view the new outputs here. |
descqa/EllipticityDistribution.py
Outdated
#check overall ellipticity distribution | ||
global_N = np.sum(np.sum(N_array, axis=1), axis=0) | ||
number_passed, percentiles = self.validate_percentiles(global_N) | ||
print("Percentiles for global distribution are:".format(morphology)+', '.join([" {:.3f} ({})".format(p, v) for p,v in zip(percentiles, self.validation_percentiles['percentiles'])])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the first .format
is not actually being used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I think this is fixed now...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, @msimet, for the quick fix! This looks good. I'll merge it on Tuesday.
This code contains the validation criteria for the ellipticity distribution test. A run of this test on the v4.* catalogs can be found here. Everything passes except for the late-type galaxies in proto-dc2_v4.5_rescale, where the 10th percentile is a little high (0.41 instead of <=0.4). I'm not sure that's actually important--I've left if for now so more knowledgeable people can advise me if that's expected or not. It would probably be okay to adjust that criterion if we needed to, however.
The actual validation criteria used here are:
This is tested for all morphologies separately and for the catalog as a whole; the returned float value is the total number of failed criteria. The criteria are currently hard-coded but the percentiles could be moved to config files if desired.