CustomSampler documentation teaches a reset_seed that does nothing

[thc1006]

Describe the bug

Both CustomSampler examples in docs/user/custom_sampler.rst build a generator and drop it on the floor:

def reset_seed(self, seed=None):
    np.random.default_rng(seed)

The return value is not assigned, so the call has no effect. sample then draws from the process-global RNG instead:

mixture_id_list = np.random.binomial(1, self.prob_tuple[0], n_samples)
samples_list[i] = np.random.normal(self.means_tuple[0], self.sd_tuple[0])

A user who copies the documented sampler gets one whose random_seed does nothing, and the failure is quiet: the study runs, the results look reasonable, and only a second run with the same seed shows they were never reproducible. The page is the reference for writing a sampler, so this is the shape most user samplers will have.

Lines 55, 91, 219 and 241 all have it.

To Reproduce

A sampler with the shape the page teaches, reset to the same seed twice:

class AsDocumented(CustomSampler):
    def sample(self, n_samples=1):
        return list(np.random.normal(14.426, 0.5, n_samples))

    def reset_seed(self, seed=None):
        np.random.default_rng(seed)


sampler = AsDocumented()
sampler.reset_seed(4242)
first = sampler.sample(3)
sampler.reset_seed(4242)
second = sampler.sample(3)

run 1  [14.901167, 14.230133, 14.249566]
run 2  [14.340185, 14.369796, 13.612586]

Same seed, different samples. Through MonteCarlo.simulate(random_seed=...) the effect is the same, since that is where reset_seed is called from.

Expected behavior

reset_seed keeps the generator and sample draws from it:

def reset_seed(self, seed=None):
    self.rng = np.random.default_rng(seed)

def sample(self, n_samples=1):
    mixture_id_list = self.rng.binomial(1, self.prob_tuple[0], n_samples)
    ...

The bivariate example needs one more thing. It caches a block of samples, so reset_seed has to discard the cache as well, otherwise the first draws after a reseed come from the generator that was replaced.

Additional context

The same mistake was in tests/fixtures/monte_carlo/custom_sampler_fixtures.py and is fixed there in #1054, which does not touch the documentation. Filing this so the two do not drift apart.

Happy to send the doc patch. It is small and I have the corrected form already.

Verified on develop at 1691119.

Signed-off-by: thc1006 84045975+thc1006@users.noreply.github.com

[thc1006]

Fixed on develop in 2cca437 (#1097). Open until develop reaches master, since the closing keyword only fires on the default branch.

One thing that PR left behind: the bivariate generator refills its 1000-sample cache inside reset_seed, and the two wind wrappers both call it, so each reseed builds the cache twice and throws the first one away. Measured at 0.20 ms per simulation, about 0.2% of a run, so it is tidiness rather than a problem. I will send it with the sampler seed-group work in #1093 rather than on its own.