Analysis of Battaglia Profile and sim results

[kbanker]

Continuing from issue #93 to discuss the results of the sims, both in isolation and compared to the sims from Sehgal 2010.

[bnord]

is this latest plot a difference plot?
#93 (comment)

[kbanker]

Yes, the last plot there is an average over 1000 loops of difference plots (Seghal sim - DeepSZSim).

[bnord]

Could you make a version where the z axis is a fractional error or a percentage? That will help us do some quick evaluations.

Could we also do a little looping over redshift ranges and mass ranges and make multiple of these plots?

e.g.,

for iz in zwindows:
    for im in mwindows:
        make_residual_tsz_map(iz, im)

[kbanker]

Here are the average difference maps for a couple different mass bins for 0<z<0.3 redshift

[kbanker]

And these are for 1.1e14 < M200 < 1.7e14; but varying redshift (The first map in the same as the one in the previous comment

[bnord]

The highest-mass bin having the smoothest map makes sense to me, because I expect the most massive structure to have smooth profiles or signals that are much higher than the noise.

[bnord]

I'm not sure how to interpret the high-redshift smoothness.

[kbanker]

After doing a percent difference graph (Seghal - DeepSZ) /DeepSZ, Im getting really high values around the edge, probably because our sim has really low values at those points. Here it is in log-scale:

[bnord]

@kbanker Could you add contours with labels so we can see more readily where the percent error?

[bnord]

@kbanker will you still make the rows or grids of these?

[kbanker]

The grid with uniform color bars is definitely something I am going to do. I am more unsure on how exactly I would add counters with labels but I can look into it (probably something in Matplotlib)

[bnord]

If there's something easy in matplotlib lets you add contours, I think it would be good to do, but if it's not easy/quick, it's probably not worth it.

[bnord]

you might have to use a 2D histogram function, I'm not sure.

[kbanker]

Here are the 4 mass bins in plots with absolute difference in uK being on the color bars. Here they are all on the same color scale so maybe a bit more comparable than before.

[bnord]

1. is it possible to still do percentages for the z-axis?
2. Could you make the title font smaller so it's a little easier to read? alternatively, could you put the mass and redshift range within each little map instead of the title?

[kbanker]

I still need to compute all the percentage maps, since I was adding contours, but here's the one that I have (just added contours). Will try to reduce the size of the title when I generate this for the percent maps.

[bnord]

interesting. These errors seem quite large. @evavagiakis What do you think about the errors here?

[evavagiakis]

I'd encourage talking about these as "differences" rather than errors, as we aren't expecting to reproduce the substructures and offcentering present in the independent hydrosim set. I do think the percentage differences are rather large and don't seem to jive with my understanding of the differences when presented in uK

[kbanker]

Here is the grid with the percentage maps, and clearer title size, along with contours.

[kbanker]

Yeah, sorry for not adding all the axes titles, but x and y are in arcmin and the values are still in percent error.

I agree that's they're really high, but I do think that is because I'm dividing by our profile, which is extremely small when away from the center (so small absolute errors lead to large percent errors)

[evavagiakis]

What's the entire formula you're using to determine the plotted percentage difference?

[kbanker]

Just while thinking about this, I tried running one of these the other way (so instead of (seghal -deepsz)/deepsz I did (signal -deepsz)/seghal), and this is the result I got, with far lower values around the edges.

[kbanker]

What's the entire formula you're using to determine the plotted percentage difference?

In the grid above I was using AVERAGE { (Seghal - DeepSZ)/DeepSZ } * 100%

[bnord]

In case you're curious, You can also do basic latex in github

$\langle \frac{sehgal - deepsz}{deepsz} \rangle * 100$
-->
$\langle \frac{sehgal - deepsz}{deepsz} \rangle * 100$

[kbanker]

Here are the (Seghal - Deepsz)/Seghal maps in a 3x3 grid; first one is percent difference, then next one is absolute difference.

[bnord]

these are pure tSZ with now noise or beam. is that correct?

[kbanker]

Yeah exactly, I'm now regenerating these zoomed in, then I'll do it with beam convolution

[kbanker]

Here are the zoomed in versions of the pervious maps (all from -4 to 4 arcmin) same bins as above. First map is percent difference, second one is absolute difference in uK

[kbanker]

Same as above but with a beam convolution added on both maps before the difference was taken. You can definitely see the edge effects from the convolution on the percent difference map. First map in percent difference, second is absolute difference in uK

[kbanker]

I will probably work on generating 1-d profiles next from these maps.

[evavagiakis]

That sounds great, it will be nice to see the various mass and redshift trends all in one 1-d profile plot. I think we need to be cropping the edges when we beam-convolve in general

[bnord]

cropping the edges when we beam-convolve in general
cropping the overall image? or within a contour?

[bnord]

It looks like the map differences are at the $\sim 5$% level between raw and convolved?

[evavagiakis]

After beam convolving there is a ~1 pixel edge effect that I believe we currently remove after beam convolution because it presents a "frame" around the image that is an artifact/not useful

[kbanker]

Here are the 1-d profiles varying over z (calculated using the zoomed in percent difference maps without beam convolution)

[kbanker]

I can definitely do these for the beam-convolved versions, but I worry that the edge effect "frame" will cause some weird effects

[bnord]

I think that is pretty close to doing annuli

[bnord]

I htink the edge effect only occurs along the edges of the image. Is that correct? If so, I think we can just pay attention to that at high radii.

[kbanker]

Yeah, the effect only occurs at the edge, but this is what we get with the convolution (Maybe it makes sense to chop it off before the edge effects start?)

[bnord]

we can keep it the edges for now. It's good information to have.

[kbanker]

Here's the 2x2 grid, varying over mass and redshift. Lower row is beam convolved. As in the single example above, the redshift for the mass-varying profiles is 0-0.3, and for the redshift varying profiles we have 1.1e14 SM < M200 < 1.7e14 SM

[kbanker]

Here are the same plots but all with the same scaling and with a cutoff at 3.5 arcmin

[bnord]

I'm wondering if we should put the convolved curves in the same plot as the non-convolved curves but with a different line shape (like dashed?)

[kbanker]

Here it is, beam convolved are dashed. Looks like there is not much difference at all between beam convolved vs not beam convolved (or all the differences were averaged out during the azimuthal average)

[bnord]

this is great.

@evavagiakis Do you see anything here that indicates a problem? Is this this time to share with Camille for her feedback?

[evavagiakis]

I'd add the dashed line definition to the legend and the fixed mass (for varying z) and redshift (for varying M) value to the subplot titles, otherwise looks great and we should ping Camille

[kbanker]

@evavagiakis @bnord Here is the final plot as we talked about

[bnord]

@kbanker Could you write out 50-150-word explanation of this plot in this discussion, and then send the link to Camille Avestruz via slack?

[evavagiakis]

I have written up an explanation of this plot and shared it to slack tagging Camille for discussion