Skip to content

Conversation

HDembinski
Copy link
Member

Richard Samworth pointed out in a comment by email, that an iterative algorithm which computes the number permutation samples dynamically makes the number of permutations used by the test a function of the resampled data set, but it has to be independent of that set if the Type I error probabilities should hold theoretically.

One possibility was to use the worst case value 0.5 and compute the number of required samples from that, but this is very pessimistic when the pvalue is very small (orders of magnitude). As a compromise, this patch uses only two iterations. In the first iteration, a burn sample of 10 permutations is used to get a crude estimate of the pvalue. The required number of samples is then computed from the pvalue obtained in the burn sample. The second sample is statistically independent of the first, so this should avoid the problem.

@HDembinski HDembinski merged commit 8fdf1f2 into develop Feb 10, 2022
@HDembinski HDembinski deleted the fix_usp branch February 10, 2022 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant