Final Dust and Synchrotron models w/ Pol. Tens. Small scales

[giuspugl]

As agreed, i am posting two notebooks encoding a version of models for both dust and synchrotron that have been thoroughly validated. These models employ a unified approach in injecting small angular scales:

• construction of modulation maps benefitting of local informations by estimating the power spectrum in circular patches of ~11 degrees
• the procedure for building the modulation maps for both QU and I is the same, by means of respectively the EE and TT spectra
• the algorithm to inject small scales and to create the modulation maps is the same for both synchrotron and dust
• in regions including the Gal. midplane (since high SNR data ), e.g. GAL097 mask, we employ small scales coming from real data

employed Validations

• Spectra at different fraction of skies
• Polarization fraction pixel distributions
• EB ratios
• K. Lau & C. Pryke dust validation http://bicep.rc.fas.harvard.edu/CMB-S4/analysis_logbook/20230117_PySMsmallfield_Ad/

Modulation maps accessible @NERSC

• dust /global/cscratch1/sd/giuspugl/cmbs4/pysm_validation/dust_modulation_pysm3.fits
• synchro /global/cscratch1/sd/giuspugl/cmbs4/pysm_validation/synchro_modulation_pysm3.fits

Maps w/ small scales

• dust /global/cscratch1/sd/giuspugl/cmbs4/pysm_validation/dust_pysm3_small_scales.fits
• synchro /global/cscratch1/sd/giuspugl/cmbs4/pysm_validation/synchro_pysm3_small_scales.fits

Notebooks

• dust link to gist
• synchro link to gist

[giuspugl]

Also i've implemented the bug fix as recommended in #147. thanks @YJHelloWorld for catching that!

[zonca]

thanks, for testing purposes, can you please also provide the output templates at 353 GHz?
So I can compare the outputs I get with yours. They are going to be different because I need to run at 8192, but at least I can compare the spectra.

[giuspugl]

i've updated the first posting w/ map location as well

[zonca]

ok, I am executing the notebook at 2048 for documentation purposes, comparison of the modulation maps looks good:

https://gist.github.com/e9245bb5170362d01a8fde2f40f5a786

[zonca]

In the notebook the seed was fixed after synalm, so cannot reproduce the same realization, even at 2048.
Anyway, the difference in the maps seems compatible with a different realization, it has a noise-like feature, spectra are compatible.
See notebook https://gist.github.com/aba4a81fe93a8eb98e8a66e6b4f81b8c

[giuspugl]

In the notebook the seed was fixed after synalm, so cannot reproduce the same realization, even at 2048.

ah sorry about that! If you think is worth to implement this fix can rerun and reproduce the files to compare with.

in the notebook https://gist.github.com/aba4a81fe93a8eb98e8a66e6b4f81b8c, we notice some difference in the modulation maps ( as blobs ), that is totally expected as it might be due to different amplitude in the spectra wrt what i've gotten. Although this doesn't seem to affect neither the small scale maps nor the power spectra, could you check the relative difference between the two modulation maps ?

[zonca]

it is really strange, the first time I ran the notebook, I got the maps at #148 (comment), which are very similar to yours, without blobs.
Now instead I get the blobs, which while in temperature I could ignore, in polarization a single blob is at the 20% level.
Here is the relative difference:

[zonca]

the discrepancy in modulation is fixed.
I have not understood the source, possibly activating threading, it seems hardly plausible, I will investigate later.

Now relative difference is at 1e-8:

[zonca]

https://gist.github.com/c3695988a4c99fa185357e7a7a5ed05c

[giuspugl]

Thanks @zonca, i 'd say that a difference of 1e-8 is acceptable, but i leave the other speak. I can't wrap my head around the previous source of discrepancy, though.

[zonca]

@giuspugl I have completed the implementation of the templates.

Now for beta and Td, currently we are adding small scales only outside of the GAL080 mask. However, this technique, that was previously used for the templates, is not used anymore.

So I think the best way to be consistent with the templates is to also apply the "Galactic plane fix" the same way it is done for the templates and add the small scales everywhere else.
I will start to implement this, if you have any feedback please let me know asap.

[zonca]

completed the update for dust in #150

[yaojian95]

Hello, I think the latest implementation of the two pivot scales should be 100 and 2000, right? The numbers in the notebook (see following two places)(https://pysm3.readthedocs.io/en/latest/preprocess-templates/small_scale_dust_pysm3.html) seem to be the old ones.

[giuspugl]

Hello, thanks!
@yaojian95 yes the pivotal scales are ell=100 and 2000. Here the documentation didn't get updated! but ell=2000 is actually employed in the notebook.

[zonca]

finalized implementation in #150 and #152