red sequence validation test

[j-dr]

I've implemented a red sequence validation test as discussed in #41. The latest run of this test can be found here: https://portal.nersc.gov/project/lsst/descqa/v2/?run=2018-04-08_16&test=RedSequence_DES_Y1&catalog=proto-dc2_v3.0.

I'm currently comparing to DES Y1, but don't have concrete validation criteria implemented yet. I'm using the actual redmapper catalog as run on protoDC2_v3.0 in order to calculate the red sequence. This is not totally ideal, since if the red-sequence training has issues as it does in this case, then this is not a totally accurate measure of the red sequence.

Pinging @erykoff, @aphearin, @yymao as interested parties.

[Edited by @yymao: This PR fixes #41 ]

[j-dr]

Hmm, looks travis has an issue with how I'm loading the redmapper catalog in the test that wasn't showing up in my runs. @yymao do you have a preferred way for me to do this.

[yymao]

Thanks, @j-dr!

Indeed, while the code you post here can actually be run without issue, our design is that the tests should agnostic about the catalogs and the tests should not need to know how to import/load catalogs, and that's why GCRCatalogs should not be imported. The "correct" way to fix this issue is either (1) to modify the add-on reader such that it also includes all the original quantities or (2) to modify the GCRCatalogs API to give users a new way to directly grab the add-on catalog (without calling GCRCatalogs.load_catalog again). I need to think about this a bit more.

(cc @EiffL because he's writing tests for add-on catalogs as well)

[yymao]

@j-dr is the only thing you need in this test the galaxy_id in redmapper?

[j-dr]

I just updated this PR to use the base extragalactic catalogs only. It now shows the full distribution of colors for centrals, and the mean and scatter of the color distributions in mass bins. This will be more useful for eyeballing and identifying issues like ones that have come up with discontinuities in g-r with mass and the like. See test output at https://portal.nersc.gov/project/lsst/descqa/v2/?run=2018-07-27_18&test=RedSequence_DES_Y1&catalog=protoDC2_test. Not very pretty yet, but you get the idea. @erykoff @yymao @evevkovacs @aphearin

[yymao]

Responding to this earlier comment #101 (comment) : we now finally have really redmapper add-on that we can use (e.g. proto-dc2_v4.3_addon_redmapper), and we no longer need to import GCRCatalogs within the test.

I can take a stab to make the necessary changes. @j-dr, is that ok with you?

[yymao]

Sorry, mis-hitted the close button..

[yymao]

@j-dr The new composite catalog allows you to access the main catalog and the redmapper add-on with a single catalog input, and hence remove the need of loading another catalog during the test.

In particular:

import GCRCatalogs
gc = GCRCatalogs.load_catalog('proto-dc2_v4.3_addon_redmapper')

# to obtain quantities from the main protoDC2 catalog, use `gc.master`
gc.master.get_quantities(['galaxy_id'])

# to obtain quantities from only entries that are redmapper galaxies
gc.get_quantities(['galaxy_id']) # you will obtain a dictionary of masked arrays

# use `.compressed()` to make a masked array regular array, with masked entries removed.
gc.get_quantities(['galaxy_id'])['galaxy_id'].compressed()

[yymao]

Here's the output of the code snippet above:

[yymao]

Comments to simplify some of the code

[yymao]

simplify Lines 55-56:

self.possible_mag_fields = [[f.format(band) for f in possible_mag_fields] for band in self.bands]

[yymao]

simplify Lines 78-81

        mag_fields = [gc.first_available(*fields) for fields in self.possible_mag_fields]
        quantities_needed.update(mag_fields)

[evevkovacs]

Sorry for the delay in reviewing this. The email got lost in my stack.