Reuse up to max_examples examples when generation is disabled

[Zac-HD]

hypothesis/hypothesis-python/src/hypothesis/internal/conjecture/engine.py

Line 544 in caccabb

desired_size = max(2, ceil(0.1 * self.settings.max_examples))

No matter how many examples are in the database, we'll use at most 10% of our example budget replaying them - even if the generation phase is disabled. It would be nice if we could replay up to max_examples when Phase.generate not in self.settings.phases.

[oxfordhalfblood]

@Zac-HD I'm interested to take a look at this.

[Zac-HD]

Hi @oxfordhalfblood! https://hypothesis.readthedocs.io/en/latest/settings.html#phases explains how phases work in Hypothesis.

While a run with default settings will reuse (up to) 10 examples and then generate 90 examples, if we've disabled the generate phase Hypothesis should reuse max_examples=100 examples from the database instead. So the calculation for desired_size should multiply by (0.1 if Phase.generate in self.settings.phases else 1).