Describe the bug
Every CustomSampler attached to one stochastic model is reset with the same seed. StochasticModel._validate_custom_sampler ends in
and seed is the model's seed, passed unchanged to each sampler in turn. A sampler that follows the documented pattern and builds np.random.default_rng(seed) therefore starts from the same state as every other sampler in that model. Two parameters that share a distribution family draw the same value, and two that do not still draw the same underlying deviate.
Quoting the NumPy documentation on parallel random number generation:
NumPy allows you to spawn new (with very high probability) independent BitGenerator and Generator instances via their spawn() method.
and, in the section on sequences of integer seeds, on deriving several seeds from one root:
It is quite likely that multiple invocations of the program with different seeds will get overlapping sets of worker seeds [...] subsets of the workers will return identical results, causing a bias in the overall ensemble of results.
Handing one integer to several independent generators is the case that guidance exists to steer away from. Here it is not merely likely to overlap, it is the same integer every time.
To Reproduce
Two samplers, different means and different standard deviations, on one rocket:
class Gaussian(CustomSampler):
def __init__(self, mean, sd):
self.mean, self.sd = mean, sd
def sample(self, n_samples=1):
return list(self.rng.normal(self.mean, self.sd, n_samples))
def reset_seed(self, seed=None):
self.rng = np.random.default_rng(seed)
rocket = StochasticRocket(
rocket=calisto, mass=Gaussian(14.426, 0.5), radius=Gaussian(0.0635, 0.001)
)
rocket._set_stochastic(4242)
drawn = next(rocket.dict_generator())
Recovering the standard normal deviate behind each draw:
mass 14.659110385288670 -> 0.466220770577340
radius 0.063966220770577 -> 0.466220770577341
The two agree to fifteen significant figures. The last digit is the scaling, not the draw. Every later pair agrees the same way, so the correlation is exactly 1 rather than merely high, and a study that varies both parameters is really varying one.
Expected behavior
Separate samplers are separate parameters and should be independent, with the model's seed deciding all of them together. Spawning one SeedSequence child per sampler would do that and stay reproducible.
The obvious fix has a case to answer first. docs/user/custom_sampler.rst documents a pair of wrappers that share one bivariate generator on purpose, so that wind X and wind Y stay correlated. Giving every sampler its own child would break that pattern, so the fix needs a way to say "these two share a stream deliberately" rather than simply spawning per attribute.
Additional context
Not a regression. This is the behaviour on develop and predates #1054, which changes neither _validate_custom_sampler nor its call sites. Found while reviewing that PR.
Related: #1053 (Monte Carlo reproducibility), #1042 (sensor noise outside the seed tree).
Verified on develop at 1691119, NumPy 2.5.1, Python 3.12.13.
Signed-off-by: thc1006 84045975+thc1006@users.noreply.github.com
Describe the bug
Every
CustomSamplerattached to one stochastic model is reset with the same seed.StochasticModel._validate_custom_samplerends inand
seedis the model's seed, passed unchanged to each sampler in turn. A sampler that follows the documented pattern and buildsnp.random.default_rng(seed)therefore starts from the same state as every other sampler in that model. Two parameters that share a distribution family draw the same value, and two that do not still draw the same underlying deviate.Quoting the NumPy documentation on parallel random number generation:
and, in the section on sequences of integer seeds, on deriving several seeds from one root:
Handing one integer to several independent generators is the case that guidance exists to steer away from. Here it is not merely likely to overlap, it is the same integer every time.
To Reproduce
Two samplers, different means and different standard deviations, on one rocket:
Recovering the standard normal deviate behind each draw:
The two agree to fifteen significant figures. The last digit is the scaling, not the draw. Every later pair agrees the same way, so the correlation is exactly 1 rather than merely high, and a study that varies both parameters is really varying one.
Expected behavior
Separate samplers are separate parameters and should be independent, with the model's seed deciding all of them together. Spawning one
SeedSequencechild per sampler would do that and stay reproducible.The obvious fix has a case to answer first.
docs/user/custom_sampler.rstdocuments a pair of wrappers that share one bivariate generator on purpose, so that wind X and wind Y stay correlated. Giving every sampler its own child would break that pattern, so the fix needs a way to say "these two share a stream deliberately" rather than simply spawning per attribute.Additional context
Not a regression. This is the behaviour on
developand predates #1054, which changes neither_validate_custom_samplernor its call sites. Found while reviewing that PR.Related: #1053 (Monte Carlo reproducibility), #1042 (sensor noise outside the seed tree).
Verified on
developat1691119, NumPy 2.5.1, Python 3.12.13.Signed-off-by: thc1006 84045975+thc1006@users.noreply.github.com