Conversation
|
Not relevant for the PR, but just to keep a record of my tests and to keep in mind the amount of storage we can save with different compression schemes. It's also a testimony to Zstandard's awesomeness! I downloaded the PySM3 data archive and create a tarball: and then I compressed
Zstandard is the winner here. However, the most impressive result comes from compression speed:
Not only Zstandard has achieved the best compression ratio, but it has performed the compression in a fraction of the time required by the other two algorithms. |
|
There is a problem here, because the cache gets saved and restored as desired, but it is never accessed when running the After a few hours of debugging, I discovered that this happens because the environment variable @NicolettaK, if I understand correctly, you are using this feature so that we can pass our own CMB realizations to PySM3, is this correct? |
|
@NicolettaK , may you please have a look at mbs.py, line 568? I would like to remove the line where the code changes the value of the environment variable |
|
Testing the use of |


As it is often the case that PySM3 data files are not available due to network outage, this PR uses https://github.com/actions/cache to cache it. It should be quite efficient, as cache files are compressed using Zstandard, which in my tests showed very good compression ratios for PySM3 maps.
I use here the trick explained in the PySM3 User's Manual