Changes for running and testing the roman_rubin catalog #239
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@yymao Please review and comment. Thanks |
|
Oops forgot to add: the successful NumberDensityVersusRedshift Test and the successful Delta_Sigma Test |
yymao
requested changes
Nov 28, 2023
There was a problem hiding this comment.
Thanks, @evevkovacs. I have two minor suggestions but otherwise it looks good to me.
Co-authored-by: Yao-Yuan Mao <yymao.astro@gmail.com>
Co-authored-by: Yao-Yuan Mao <yymao.astro@gmail.com>
|
@yymao Thanks, I accepted both of your suggestions. |
|
Thanks @evevkovacs -- I'm going to merge this, but before we tag v2.8.0, do you mind do another run of |
yymao
approved these changes
Nov 28, 2023
|
Yes will do. |
|
Running now on an interactive node...no failures yet. |
|
OK, it ran... see here |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here are the changes to get the Delta_Sigma and NumberDensityVersusRedshift tests to run with the latest versions of numpy, sklearn etc. These are minor code changes to the actual tests.
There is also an addition required to the run_master script to set the number of OMP_NUM_THREADS. The issue is that the NumberDensityVersusRedshift test fails with an OPEN_BLAS error that complains about too many threads being used. (This is for a call to the kNN module in sklearn. ) The error did not occur when running on cori. I have experimented with trying to set the number of OPEN_BLAS_THREADS and OMP_NUM_THREADS by calling the python function to set these environmentals in the init function of NumberDensityVersusRedshift.py. None of these worked. The only attempt that worked was to set the environmental from the master script. I am of course open to better solutions.
Finally, there is a new configuration file for running the readiness test on the roman_rubin catalog.