Remove default multiprocessing when on Windows? #3403
Comments
|
There is a known problem with multiprocessing on windows that causes an exception to fail in this spectacular fashion. |
twiecki
changed the title
I guess that's one way of solving it, but if this only happens when exceptions are thrown, wouldn't it be simpler and better for final users to instead throw the exception after all processes have finished and their output collected? |
|
@david-cortes Unfortunately we never see the output in that case. |
|
@david-cortes, to address your particular problem, could you enclose the if __name__ == "__main__":
with model:
pm.sample()This could help you run the notebook on windows (but you would still reach the bad initial energy you saw on linux). The recurrent issue was discussed in many places (1, 2, 3, 4, and I imagine that other threads too). The problem is caused because The reason that we cannot catch these errors and handle them nicely is that the errors occur even before the new process is fully initialized. This means that the communication pipe between it and the root process is not entirely set up. So the child process fails, raises an exception which is printed to its
|
|
Unfortunately, it seems adding |
|
Could you share a minimum example notebook gist that produces your error? |
|
Found an easy way of making it get an error: passing some data with import numpy as np, pymc3 as pm
a = np.repeat(np.nan, 10)
with pm.Model() as model:
a_prior_mu = pm.Normal('a_prior_mu', mu = 0, sd = 1)
a_prior_sd = pm.Gamma('a_prior_sd', alpha = 1, beta = 1)
a_pm = pm.Normal('a', mu = a_prior_mu, sd = a_prior_sd, shape = 10, observed = a)
trace = pm.sample()Put under import numpy as np, pymc3 as pm
a = np.repeat(np.nan, 10)
with pm.Model() as model:
a_prior_mu = pm.Normal('a_prior_mu', mu = 0, sd = 1)
a_prior_sd = pm.Gamma('a_prior_sd', alpha = 1, beta = 1)
a_pm = pm.Normal('a', mu = a_prior_mu, sd = a_prior_sd, shape = 10, observed = a)
with model:
if __name__ == "__main__":
trace = pm.sample() |
|
Do you know if there are any plans to fix this issue? |
|
Since #4116 the situation has gotten much better. I'll close this issue since the reason and ways to debug it were explained above. |
I’m not sure how to reproduce it, but oftentimes, depending on the type of model and random seed, pymc3’s NUTS sampler would fail and crash the ipython notebook along with it in Windows 10. This is the stack trace that I get from running it in a regular python process (not IPython notebook):
Running it on jupyer, I would get the following message in the console:
If I run the same script on Linux, I would instead get
SamplingError: Bad initial energy, without crashing the notebook along.Setup: Python 3.6.5, running on Windows 10, theano 1.0.3, pymc3 3.6.
Can attach a script and data to reproduce it, but it's not a minimalistic example as I haven't been able to create that.
The text was updated successfully, but these errors were encountered: