dN/dmag test

[yymao]

See LSSTDESC/DC2-production#21 for detail and validation data from HSC.

• code to reduce mock data
• code that works within DESCQA framework
• validation data
• validation criteria

[duncandc]

I am going to start working on this.

[yymao]

Here's a preliminary result from this test, made by @duncandc: (and the corresponding entry on DESCQA web interface)

Lines are protoDC2 and buzzard_test, points are HSC. Difference in normalization may be due to a bug (still be explored).

See also LSSTDESC/DC2-production#21 for related comments.

[rmandelb]

@yymao @duncandc - There has been further discussion of this test in #50 (see specifically from #50 (comment) onwards). A few outstanding questions:

• as discussed there, we'd like to have some way of disabling this test for catalogs that have redshift cuts such that the N(<mag) calculation is significantly affected. Can this be implemented based on the maximum redshift of the light cone?
• the last status of this issue was that there were bugs to work out. Is this still in progress?
• validation criterion: we had discussed this on constructing validation test for DC2 dN/dmag DC2-production#21 (see more specifically constructing validation test for DC2 dN/dmag DC2-production#21 (comment) ) and the proposal I would like to make is that at limiting magnitudes of 23, 24, and 25 in i-band, we should require 20% agreement in N(<mag). At limiting magnitudes of 26 and 27, we should require 40% agreement. Pinging a few people who may have comments on this: @janewman-pitt-edu , @msimet .

[yymao]

@rmandelb To answer your questions ---

• Yes, that can be easily implemented.
• We have not made much progress since the Sprint Week (but @duncandc can correct me if that's not the case). I am not sure if there was really a bug in getting dN/dmag from the mocks, or just that the way the validation data is constructed is not consistent with how we do it with the mock.
• Those criteria can be implemented. As mentioned above, we need to make sure the validation data is correctly/consistently constructed.

[janewman-pitt-edu]

The criteria Rachel suggests sounds reasonable to me.

[duncandc]

I am testing my calculation of dn/dmag. Here is a comparison between my calculation and the result in the HSC data release paper. This suggests that there is a problem with the test. But for the life of me I can not find the problem. Particularly confusing is the offset in the peak where incompleteness sets in. the magnitudes are offset by ~1.

[compare lime green line and points]

[yymao]

@duncandc is there any chance that there's a missing 5*log(h) ~= -0.77?

[duncandc]

apparent magnitudes don't scale with h, right?

[yymao]

no, i don't think they do

[duncandc]

everyone should feel free to take a look at the notebook in the desqagen branch of my fork. https://github.com/duncandc/descqa/tree/descqagen/descqagen/app_mag_func_test

[evevkovacs]

Apparent magnitudes have a contribution from the luminosity distance 5*np.log10(dl)+25 which assumes dl is in Mpc. But the catalog providers take care of this. What about magnitude definitions? Catalog fluxes are converted to magnitudes in the AB system. Does this match the definition in Hiroki et al?

[yymao]

@evevkovacs the most recent plot @duncandc showed is comparing the HSC data with HSC paper plot. So there seems to be something going on in Duncan's data reduction but we haven't figured out what it is.

It does seem that the dN/dmag in the HSC paper is closer to the catalogs'

[rmandelb]

I think I see the problem in the notebook. YOu have a variable representing the full area, but you seem to have assigned its value based on the area of just 1 region:

full_area_sq_arcmin = float(len(hsc_randoms[ran_field_mask_1]))/100.0
full_area_sq_deg = full_area_sq_arcmin/3600.0

So your "full area" is too small by a factor of ~5. You later use this area to normalize densities for the whole catalog.

[duncandc]

@rmandelb, that mostly does the trick.

[duncandc]

We are now about 20% lower than the data release paper. It would make me nervous if this is in the realm of details of magnitude types and flags used to mask the data...

[rmandelb]

Is this protoDC2 or Buzzard that is below the data release paper?

[rmandelb]

To be clear, the reason i ask is that we know about a few effects:

• in both sims, at low redshift, there are missing galaxies beyond some luminosity limit (and I don't remember what magnitude that would affect, but I believe it was @aphearin who showed some relevant plots on this)
• in protoDC2 (but not Buzzard) we are missing everything about z=1

Both of these effects have the right sign at least, but the severity and magnitude-dependence differs for the two effects.

[duncandc]

Although, it just occurred to me that I got these points from plot digitizer... so I don't know how much trust we put in my steady hand...

[duncandc]

This is a comparison between my measurement of dn/dmag and the data presented in figure 16 in Aihara+ 2017 (https://arxiv.org/abs/1702.08449).

[rmandelb]

Well, I would be comfortable at this point going back to your dn/dmag estimate from HSC vs. the sims...

[duncandc]

replace "spring week 2018" with "collaboration meeting 2018"!

better, but not exactly good...

[rmandelb]

Is the legend really correct? I believe the number counts in HSC should really flatten out beyond i=26, but the green points labeled HSC keep rising all the way to i=30.

[yymao]

@rmandelb I believe that is @duncandc's fit.

[duncandc]

That is correct @yymao and @rmandelb. I have fit a power law to to extrapolate HSC. That was part fo the original test proposed at the sprint week.

[rmandelb]

OK. Well, I think the suppression at protoDC2 beyond i=23 has to do with the z=1 limit, which is particularly important there. I had made a suggestion somewhere for how to account for this (basically taking the DEEP2 dN/dz for mag-limited samples, and using that to "correct" for missing galaxies), but perhaps for now we should do comparisons with Buzzard?

Also, can you by default have a separate panel for the ratio? Log plots covering many orders of magnitude can be hard to interpret.

[janewman-pitt-edu]

We could certainly look at dN(<m)/dmag and correct using the DEEP2 dN/dz estimates... then differentiate to get dN/dmag... it's better than nothing, at least.

[yymao]

@duncandc now that you have fixed the normalization issue, can you update PR #46 and #47 (together with Rachel's suggestion of adding a ratio panel in #47)? Once you update the PRs, I'll review and merge (if there's no further issues) them.

Also, do you have access to the DEEP2 validation data?

[yymao]

Done in #47.