increase trials in "We are elected" test #3093
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Soupstraw
approved these changes
Oct 19, 2022
The property test "We are elected as leader proportional to our stake" fails fairly infrequently, but often enough to be a disturbance. There does not seem to be a problem with the test, just that the variance with 500 trials is too high. I've increased the number of trials to 2,000. It still runs quite fast (under a second).
JaredCorduan
force-pushed
the
jc/moar-trials
branch
from
7013419 to
5d5ee85
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The property test "We are elected as leader proportional to our stake" fails fairly infrequently, but often enough to be a annoying. I cannot find a problem with the test, I think that the variance with 500 trials is just too high. I've increased the number of trials to 2,000. It still runs quite fast (under a second).
I can get this test to fail consistently by setting
withMaxSuccessto 500 when the number of trials is 500. With 2,000 trials, it succeeds often withwithMaxSuccessset to 10,000.If anyone else wants to check the math, that would be great too.