Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Int overflow(?) in wilcoxonMatchedPairCriticalValue #18

Closed
Shimuuar opened this Issue · 4 comments

2 participants

@Shimuuar
Collaborator

There is a bug in calculation of critical value for Wilcoxon-T test.

λ wilcoxonMatchedPairCriticalValue 35 0.05
Just 213
λ  wilcoxonMatchedPairCriticalValue 36 0.05
Just 2147483647   --  Which equal to Just (2^31-1) 

It looks like at some point integer overflow occurs.

Also it cannot handle samples larger than 1023 at all since it uses 2**sampleSize in calculation of critical value which overflows to +∞ for sampleSize > 1023

Test probably should fall back to normal approximation at some point.

@Shimuuar
Collaborator

Indeed on 64-bit machine critical point is at N=67

@Shimuuar
Collaborator

Use of Integers solves problem with integer overflows (commit e04da30) but wilcoxonMatchedPairCriticalValue should revert to normal approximation at some point because of double overflow and since algorithm is really slow for large N.

@Shimuuar
Collaborator

fixed in 755a825

@Shimuuar Shimuuar closed this
@3noch

If this is fixed, has it not been released? The warning is still in haddocks: https://hackage.haskell.org/package/statistics-0.13.2.3/docs/Statistics-Test-WilcoxonT.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.