-
Notifications
You must be signed in to change notification settings - Fork 430
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Emcee and MPI with mpi4py #310
Comments
There isn't anything fundamentally wrong about this approach (it can actually be useful since you have many independent ensembles in this case). But one thing that you need to be careful about is making sure that the random number generator on each process gets properly initialized or you'll end up with chains that are correlated or even the same. |
I ran into a similar situation. Is it possible to random.seed() the random numbers used in emcee? Does emcee already adopt a particular way of seeding the random number generator (in this way I won't have a control)? |
The |
General information:
Problem description:
Hi all,
I am pretty new of MPI parallelization and this is indeed my first try to use it on a python code
I have tried a parallelization of emcee using mpi4py in a different way w.r.t. the minimal examples proposed in the documentation of the code (which actually uses multiprocessing).
The idea behind this workaround is that I want to have each process to write its own"backend" file when emcee is running which, to my knowledge so far, cannot be realized using the "pool" option of the sampler. Furthermore, w.r.t. the "pool" method I can draw much more model at
the same time with speed comparable to when I run the code without parallelization.
what I would like to know is if this is the right way to obtain what I wanted to achieve or if there's another way to do that using the "pool" method. The workaround works fine but I am concerned about the results I am drawing from this, are the results of each MCMC process really independent from one another in this way, or there's something I am missing ?
Best,
F.
Minimal example:
The text was updated successfully, but these errors were encountered: