Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PySM prerelease with GNILC models d9,d10,d11 and synchrotron models s4,s5,s6,s7 #112

Closed
zonca opened this issue Mar 4, 2022 · 14 comments
Closed

PySM prerelease with GNILC models d9,d10,d11 and synchrotron models s4,s5,s6,s7 #112

zonca opened this issue Mar 4, 2022 · 14 comments
Assignees
@zonca
Labels
enhancement New feature or request

Comments

@zonca
Copy link
Member

zonca commented Mar 4, 2022

I've published to PyPI PySM 3.4.0 beta 1, you can install it with:

pip install --pre pysm3

(--pre grabs pre-releases)

Alternatively you can test in your browser using Google Colab (or use the notebook code locally):

https://colab.research.google.com/drive/1LI7zGC2Exd8R0HiYMl6b3Bnw-0bRp0Ws?usp=sharing

Please report any bugs here or open dedicated issue.
Feedback on the docs is particualrly appreciated, see https://pysm3.readthedocs.io/en/latest/models.html#dust, you can also click on "Edit this file on Github" to create a pull request directly.
@zonca
Copy link
Member Author

zonca commented Mar 12, 2022

there is currently a bug in CO #113 , it is fixed in the main branch #114 , as soon as I have a first version of synchrotron in a few days I'll make another beta release with this fix.

@zonca zonca changed the title PySM prerelease with GNILC models d9,d10,d11 PySM prerelease with GNILC models d9,d10,d11 and synchrotron models s4,s5,s6 Mar 16, 2022
@zonca
Copy link
Member Author

zonca commented Mar 16, 2022

I just released 3.4.0b2, this release has:

You can install it with:

pip install --pre pysm3 --upgrade

Or checkout the Colab link above.

@zonca
Copy link
Member Author

zonca commented Mar 16, 2022

New PySM 3 models included:

  • s4, s5, s6 Synchrotron
  • co1, co2, co3 CO lines
  • d9, d10, d11 GNILC Dust
  • d12 MKD 3D Dust

@brandonshensley brandonshensley mentioned this issue Mar 17, 2022
Closed
@zonca zonca changed the title PySM prerelease with GNILC models d9,d10,d11 and synchrotron models s4,s5,s6 PySM prerelease with GNILC models d9,d10,d11 and synchrotron models s4,s5,s6,s7 Mar 28, 2022
@zonca
Copy link
Member Author

zonca commented Mar 28, 2022

Release 3.4.0b3 which also includes s7, synchrotron model with curvature.
I haven't implemented a stochastic model for synchrotron with curvature, please let me know if anyone is interested in that.

@zonca
Copy link
Member Author

zonca commented May 5, 2022

@brandonshensley @seclark @giuspugl @bthorne93 I prepared a fireslide for the CMB-S4 spring meeting with quick tour of the new models, please provide feedback: https://docs.google.com/presentation/d/1_JkiKpVvbCL4lMUPWonDszq4mR8S6XWnTmN6JHdh-v4/edit?usp=sharing

@brandonshensley
Copy link
Contributor

@brandonshensley @seclark @giuspugl @bthorne93 I prepared a fireslide for the CMB-S4 spring meeting with quick tour of the new models, please provide feedback: https://docs.google.com/presentation/d/1_JkiKpVvbCL4lMUPWonDszq4mR8S6XWnTmN6JHdh-v4/edit?usp=sharing

This looks great to me! Might be useful to give a reference (hyperlink/arXiv number/journal citation) for the MKD model. Susan and I have a talk on Friday where we can explain in a little more detail.

@zonca zonca added the enhancement New feature or request label Nov 15, 2022
@zonca zonca pinned this issue Nov 21, 2022
@zonca
Copy link
Member Author

zonca commented Nov 21, 2022

PySM 3.4.0b4

We have a new beta, this has been already used for production runs for CMB-S4, and will soon for Litebird and SO.

All the models have changed, due to changing how PySM does the spherical harmonics transform, on top of that GNILC and synchrotron have brand new small scales (thanks @giuspugl) and we have Websky extragalactic (thanks @xzackli)

List of the models:

  • s4, s5, s6, s7 Synchrotron
  • co1, co2, co3 CO lines
  • d9, d10, d11 GNILC Dust
  • d12 MKD 3D Dust
  • cib1, rg1, ksz1, tsz1 Websky 0.4 components (rg is Radio Galaxies)
  • c3, c4, Unlensed and Lensed Websky CIB components

As usual, update with:

pip install --pre pysm3 --upgrade

See https://github.com/galsci/pysm/releases/tag/3.4.0b4

@zonca zonca mentioned this issue Nov 21, 2022
Closed
@zonca
Copy link
Member Author

zonca commented Dec 5, 2022

PySM 3.4.0b5

This new beta release fixes the color correction in the MKD dust model (d12)
See #141

https://pypi.org/project/pysm3/3.4.0b5/

@zonca zonca self-assigned this Dec 13, 2022
@zonca
Copy link
Member Author

zonca commented Feb 18, 2023

PySM 3.4.0b6

Need to cleanup documentation, then hopefully this will become PySM 3.4.0

@zonca
Copy link
Member Author

zonca commented Feb 25, 2023

PySM 3.4.0b7

@zonca
Copy link
Member Author

zonca commented Jun 27, 2023 •
edited
Loading

PySM 3.4.0b9

Releasing 3.4.0b9, the most important improvement is a bugfix in the dust beta and Td
I moved the broken templates out of the way and marked the fixed templates with a datestamp. So old PySM version will stop working to avoid people running broken models by mistake.

https://pypi.org/project/pysm3/3.4.0b9/

@zonca
Copy link
Member Author

zonca commented Jun 27, 2023

Logs of unit tests of pysm 3.4.0b9 running at NERSC:

(pycmb) zonca@login32 pysm (main) $ pytest -v
======================================================= test session starts =======================================================
platform linux -- Python 3.10.0, pytest-7.2.0, pluggy-1.0.0 -- /global/common/software/cmb/zonca/conda/pycmb/bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/global/u2/z/zonca/p/software/pysm/.hypothesis/examples')
rootdir: /global/u2/z/zonca/p/software/pysm, configfile: pyproject.toml
plugins: arraydiff-0.5.0, filter-subpackage-0.1.1, openfiles-0.5.0, hypothesis-6.56.4, remotedata-0.3.3, cython-0.2.0, doctestplus-0.12.1, anyio-3.6.2, astropy-0.10.0, cov-4.0.0, astropy-header-0.2.2, nbval-0.9.6, mock-3.10.0
collected 104 items / 2 skipped                                                                                                   

pysm3/tests/test_ame.py::test_model[a1-30] PASSED                                                                           [  0%]
pysm3/tests/test_ame.py::test_model[a1-100] PASSED                                                                          [  1%]
pysm3/tests/test_ame2.py::test_model[a2-30] PASSED                                                                          [  2%]
pysm3/tests/test_ame2.py::test_model[a2-100] PASSED                                                                         [  3%]
pysm3/tests/test_bandpass_convert_units.py::test_Bandpass_Unit_Conversion::test_bandpass_unit_conversion_CMB2MJysr PASSED   [  4%]
pysm3/tests/test_bandpass_convert_units.py::test_Bandpass_Unit_Conversion::test_bandpass_unit_conversion_MJysr2KRJ PASSED   [  5%]
pysm3/tests/test_bandpass_convert_units.py::test_bandpass_convert_integration::test_bandpass_convert_units PASSED           [  6%]
pysm3/tests/test_cmb.py::test_cmb_map PASSED                                                                                [  7%]
pysm3/tests/test_cmb.py::test_cmb_map_bandpass PASSED                                                                       [  8%]
pysm3/tests/test_cmb.py::test_cmb_lensed[c1-30] PASSED                                                                      [  9%]
pysm3/tests/test_cmb.py::test_cmb_lensed[c1-100] PASSED                                                                     [ 10%]
pysm3/tests/test_cmb.py::test_cmb_lensed[c1-353] PASSED                                                                     [ 11%]
pysm3/tests/test_co.py::test_co[False] PASSED                                                                               [ 12%]
pysm3/tests/test_co.py::test_co[True] PASSED                                                                                [ 13%]
pysm3/tests/test_co.py::test_co_model[co2] PASSED                                                                           [ 14%]
pysm3/tests/test_co.py::test_co_model[co3] PASSED                                                                           [ 15%]
pysm3/tests/test_dust.py::test_blackbody_ratio PASSED                                                                       [ 16%]
pysm3/tests/test_dust.py::test_dust_model[d0-30] PASSED                                                                     [ 17%]
pysm3/tests/test_dust.py::test_dust_model[d0-100] PASSED                                                                    [ 18%]
pysm3/tests/test_dust.py::test_dust_model[d0-353] PASSED                                                                    [ 19%]
pysm3/tests/test_dust.py::test_dust_model[d1-30] PASSED                                                                     [ 20%]
pysm3/tests/test_dust.py::test_dust_model[d1-100] PASSED                                                                    [ 21%]
pysm3/tests/test_dust.py::test_dust_model[d1-353] PASSED                                                                    [ 22%]
pysm3/tests/test_dust.py::test_dust_model[d2-30] PASSED                                                                     [ 23%]
pysm3/tests/test_dust.py::test_dust_model[d2-100] PASSED                                                                    [ 24%]
pysm3/tests/test_dust.py::test_dust_model[d2-353] PASSED                                                                    [ 25%]
pysm3/tests/test_dust.py::test_dust_model[d3-30] PASSED                                                                     [ 25%]
pysm3/tests/test_dust.py::test_dust_model[d3-100] PASSED                                                                    [ 26%]
pysm3/tests/test_dust.py::test_dust_model[d3-353] PASSED                                                                    [ 27%]
pysm3/tests/test_dust.py::test_dust_model[d6-30] PASSED                                                                     [ 28%]
pysm3/tests/test_dust.py::test_dust_model[d6-100] PASSED                                                                    [ 29%]
pysm3/tests/test_dust.py::test_dust_model[d6-353] PASSED                                                                    [ 30%]
pysm3/tests/test_dust_layers.py::test_modified_black_body_class PASSED                                                      [ 31%]
pysm3/tests/test_dust_layers.py::test_model_d12[100] PASSED                                                                 [ 32%]
pysm3/tests/test_dust_layers.py::test_model_d12[353] PASSED                                                                 [ 33%]
pysm3/tests/test_dust_layers.py::test_model_d12[857] PASSED                                                                 [ 34%]
pysm3/tests/test_dust_pysm3.py::test_dust_model_353[d9] PASSED                                                              [ 35%]
pysm3/tests/test_dust_pysm3.py::test_dust_model_353[d10] PASSED                                                             [ 36%]
pysm3/tests/test_dust_pysm3.py::test_gnilc_857[d9] PASSED                                                                   [ 37%]
pysm3/tests/test_dust_pysm3.py::test_gnilc_857[d10] PASSED                                                                  [ 38%]
pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[353] PASSED                                                                 [ 39%]
pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[857] PASSED                                                                 [ 40%]
pysm3/tests/test_freefree.py::test_model[f1-30] PASSED                                                                      [ 41%]
pysm3/tests/test_freefree.py::test_model[f1-100] PASSED                                                                     [ 42%]
pysm3/tests/test_freefree.py::test_model[f1-353] PASSED                                                                     [ 43%]
pysm3/tests/test_hd17.py::test_highfreq_dust_model[d7-100] PASSED                                                           [ 44%]
pysm3/tests/test_hd17.py::test_highfreq_dust_model[d7-353] PASSED                                                           [ 45%]
pysm3/tests/test_hd17.py::test_highfreq_dust_model[d7-900] PASSED                                                           [ 46%]
pysm3/tests/test_hd17.py::test_highfreq_dust_model[d5-100] PASSED                                                           [ 47%]
pysm3/tests/test_hd17.py::test_highfreq_dust_model[d5-353] PASSED                                                           [ 48%]
pysm3/tests/test_hd17.py::test_highfreq_dust_model[d5-900] PASSED                                                           [ 49%]
pysm3/tests/test_hd17.py::test_highfreq_dust_model[d8-100] PASSED                                                           [ 50%]
pysm3/tests/test_hd17.py::test_highfreq_dust_model[d8-353] PASSED                                                           [ 50%]
pysm3/tests/test_hd17.py::test_highfreq_dust_model[d8-900] PASSED                                                           [ 51%]
pysm3/tests/test_interpolating.py::test_interpolating PASSED                                                                [ 52%]
pysm3/tests/test_interpolating.py::test_interpolating_bandpass_boundary_above PASSED                                        [ 53%]
pysm3/tests/test_interpolating.py::test_interpolating_bandpass_boundary_below PASSED                                        [ 54%]
pysm3/tests/test_max_nside.py::test_sky_max_nside PASSED                                                                    [ 55%]
pysm3/tests/test_max_nside.py::test_sky_max_nside_highres PASSED                                                            [ 56%]
pysm3/tests/test_read_map.py::test_read_map_unit PASSED                                                                     [ 57%]
pysm3/tests/test_read_map.py::test_read_map_unit_dimensionless PASSED                                                       [ 58%]
pysm3/tests/test_smoothing.py::test_smoothing_healpix PASSED                                                                [ 59%]
pysm3/tests/test_smoothing.py::test_car_nosmoothing PASSED                                                                  [ 60%]
pysm3/tests/test_smoothing.py::test_healpix_output_nside PASSED                                                             [ 61%]
pysm3/tests/test_synch_pysm3.py::test_synch_model_s7_noscaling[s7] PASSED                                                   [ 62%]
pysm3/tests/test_synch_pysm3.py::test_synch_model_s7_44[s7] PASSED                                                          [ 63%]
pysm3/tests/test_synch_pysm3.py::test_synch_model_noscaling[s4] PASSED                                                      [ 64%]
pysm3/tests/test_synch_pysm3.py::test_synch_model_noscaling[s5] PASSED                                                      [ 65%]
pysm3/tests/test_synch_pysm3.py::test_synch_44[s4] PASSED                                                                   [ 66%]
pysm3/tests/test_synch_pysm3.py::test_synch_44[s5] PASSED                                                                   [ 67%]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[23] PASSED                                                                   [ 68%]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[44] PASSED                                                                   [ 69%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s0-30] PASSED                                                       [ 70%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s0-100] PASSED                                                      [ 71%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s0-353] PASSED                                                      [ 72%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s1-30] PASSED                                                       [ 73%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s1-100] PASSED                                                      [ 74%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s1-353] PASSED                                                      [ 75%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s2-30] PASSED                                                       [ 75%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s2-100] PASSED                                                      [ 76%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s2-353] PASSED                                                      [ 77%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s3-30] PASSED                                                       [ 78%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s3-100] PASSED                                                      [ 79%]
pysm3/tests/test_synchrotron.py::test_synchrotron_model[s3-353] PASSED                                                      [ 80%]
pysm3/tests/test_unit_conversion_pysm2.py::TestConvertUnits::test_convert_units PASSED                                      [ 81%]
pysm3/tests/test_units.py::test_conversion PASSED                                                                           [ 82%]
pysm3/tests/test_units.py::test_fits_unit_functionality PASSED                                                              [ 83%]
pysm3/tests/test_utils.py::test_get_relevant_frequencies PASSED                                                             [ 84%]
pysm3/tests/test_utils.py::test_has_polarization PASSED                                                                     [ 85%]
pysm3/tests/test_utils.py::test_bandpass_unit_conversion PASSED                                                             [ 86%]
pysm3/tests/test_utils.py::test_bandpass_integration_tophat PASSED                                                          [ 87%]
pysm3/tests/test_utils.py::test_trapz[uniform] PASSED                                                                       [ 88%]
pysm3/tests/test_utils.py::test_trapz[non-uniform] PASSED                                                                   [ 89%]
pysm3/tests/test_utils.py::test_bandpass_integration_weights PASSED                                                         [ 90%]
pysm3/tests/test_utils.py::test_remotedata PASSED                                                                           [ 91%]
pysm3/tests/test_utils.py::test_remotedata_globalpath PASSED                                                                [ 92%]
pysm3/tests/test_utils.py::test_add_metadata PASSED                                                                         [ 93%]
pysm3/tests/test_utils.py::test_add_metadata_different_units PASSED                                                         [ 94%]
pysm3/tests/test_utils.py::test_data_raise PASSED                                                                           [ 95%]
pysm3/tests/test_websky.py::test_SPT_CIB_map_scaling PASSED                                                                 [ 96%]
pysm3/tests/test_websky.py::test_radiogalaxies PASSED                                                                       [ 97%]
pysm3/tests/test_websky.py::test_cib PASSED                                                                                 [ 98%]
pysm3/tests/test_websky.py::test_sz[thermal] PASSED                                                                         [ 99%]
pysm3/tests/test_websky.py::test_sz[kinetic] PASSED                                                                         [100%]

======================================================== warnings summary =========================================================
pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[353]
pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[857]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[23]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[44]
  /global/common/software/cmb/zonca/conda/pycmb/lib/python3.10/site-packages/healpy/fitsfunc.py:643: ComplexWarning: Casting complex values to real discards the imaginary part
    alm.real[i] = almr

pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[353]
pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[857]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[23]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[44]
  /global/common/software/cmb/zonca/conda/pycmb/lib/python3.10/site-packages/healpy/fitsfunc.py:644: ComplexWarning: Casting complex values to real discards the imaginary part
    alm.imag[i] = almi

pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[353]
pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[857]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[23]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[44]
  /global/common/software/cmb/zonca/conda/pycmb/lib/python3.10/site-packages/healpy/sphtfunc.py:607: ComplexWarning: Casting complex values to real discards the imaginary part
    (np.asarray(cl, dtype=np.float64) if cl is not None else None)

pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[353]
pysm3/tests/test_dust_pysm3.py::test_d10_vs_d11[857]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[23]
pysm3/tests/test_synch_pysm3.py::test_s6_vs_s5[44]
  /global/common/software/cmb/zonca/conda/pycmb/lib/python3.10/site-packages/healpy/sphtfunc.py:563: ComplexWarning: Casting complex values to real discards the imaginary part
    cls_list = [np.asarray(cls, dtype=np.float64)]

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================== 104 passed, 2 skipped, 16 warnings in 990.11s (0:16:30) =====================================

@zonca
Copy link
Member Author

zonca commented Oct 31, 2023

as discussed on the PanExperiment Galascic Science group call, the implementation of the new emission models in PySM 3.4.0b9 looks stable and has been extensively used in simulations by different experiments, for example:

The plan is to fix issue #155 about renaming templates to avoid caching issues for people that had previously installed a beta release and release 3.4.0.

Please write here if you have any other feedback about the release.

@zonca
Copy link
Member Author

zonca commented Dec 12, 2023

3.4.0 released now, see changelog for more details.

@zonca zonca closed this as completed Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zonca zonca

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@zonca @brandonshensley