-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Extend RNG options to three, addressing issue #348 #349
Conversation
…multi" corresponding to the centralized RNG, SingleRNG (separate streams per distribution), and MultiRNG (full support for common random numbers).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This works, but I'm not sure why we would want to give users one good option (multi-RNG) and two bad options (single
and centralized
)? I could imagine someone using single
instead of multi
if they only care about performance and their own code isn't RNG-safe, but I don't know why someone would want centralized
instead of single
.
Suggest we consider this together with #351 (once it's ready) and decide how to proceed.
From benchmark_full.py
, I got:
'centralized': 3451 ± 19 ms
'single': 3362 ± 13 ms
'multi': 4851 ± 37 ms
Unclear to me if most users will want 'single' or 'multi'. We currently only have one network (EmbeddingNet) and a few diseases that are CRN safe. It makes sense to run 'multi' for these, depending on the application, but otherwise users will use the default 'single' option. I do want to keep 'centralized' around, e.g. to create figures for the CRN paper. But it can be obscure (sorta already is?) and likely removed at a later time. |
What do you think about the names? I like 'centralized', that's clear as to what's going on. But 'single' vs 'multi' is a misnomer considering both establish one random number generator instance per distribution. Single could be 'per-distribution' or even 'multi'? And what's currently 'multi' could become 'CRN'? Thoughts? |
Of course, if we rename 'multi', I would also change the name of the 'MultiRNG' and we'd have to fix all the 'options.multirng' calls, but that's doable. And 'SingleRNG' could stand to be renamed as part of that as well. |
Ok, we can merge for now and refactor later. I agree the names aren't perfect, but would suggest holding off on a refactor until we decide on the long-term path forward. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
…e suggestion from @cliffckerr.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's merge this and then compare with refactor-rngs
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, will merge for now
Renames and extends the
multirng
option in settings, now called 'rng'.Set how random numbers are handled in Starsim with three options:
"single" is the default value for this option.
In comparing two simulations, the "single" option may be slightly better than the "centralized" option without any real disadvantages. Only "multi" can achieve full common random number (CRN) coherence, but is more computationally expensive and only produces CRN results when using CRN-safe code.
In practice, devtest_rng.py shows that multi > single > centralized, but that single and centralized have nearly identical performance. So this PR switches the default to single, which uses the SingleRNG under the hood to track seeds for each stream.
Closes #348