Fix flaky Hypothesis stateful test for unique name generation #10946
+35
−25
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Claude tried to fix the flakiness issue in CI. I probed its logic a bit, it looks reasonable but I don't fully understand. Our summary below...
Summary
DatasetStateMachinetest that was causing CI failures withFlakyStrategyDefinition: Inconsistent data generation!st.shared()strategy to state machine instance stateProblem
The nightly Slow Hypothesis CI has been failing:
https://github.com/pydata/xarray/actions/runs/19603404994
The test
properties/test_index_manipulation.py::DatasetTest::runTestfails intermittently with:hypothesis.errors.FlakyStrategyDefinition: Inconsistent data generation!
Data generation behaved differently between different runs.
Is your data generation depending on external state?
Root Cause
The original code used a pattern from [Stack Overflow](https://stackoverflow.com/questions/73737073/c
reate-hypothesis-strategy-that-returns-unique-values):
While this pattern works for regular
@giventests, it does not work for RuleBasedStateMachinebecause:
Per HypothesisWorks/hypothesis#2338:
"I think you'll need to set these up as instance attributes, rather than class attributes."
Solution
Move uniqueness tracking to state machine instance state:
This follows the Hypothesis-recommended pattern for stateful tests where cross-rule state belongs on
the instance.