Int overflow(?) in wilcoxonMatchedPairCriticalValue #18

Closed
Shimuuar opened this Issue Dec 9, 2011 · 4 comments

Projects

None yet

2 participants

@Shimuuar
Collaborator

There is a bug in calculation of critical value for Wilcoxon-T test.

λ wilcoxonMatchedPairCriticalValue 35 0.05
Just 213
λ  wilcoxonMatchedPairCriticalValue 36 0.05
Just 2147483647   --  Which equal to Just (2^31-1) 

It looks like at some point integer overflow occurs.

Also it cannot handle samples larger than 1023 at all since it uses 2**sampleSize in calculation of critical value which overflows to +∞ for sampleSize > 1023

Test probably should fall back to normal approximation at some point.

@Shimuuar
Collaborator

Indeed on 64-bit machine critical point is at N=67

@Shimuuar
Collaborator

Use of Integers solves problem with integer overflows (commit e04da30) but wilcoxonMatchedPairCriticalValue should revert to normal approximation at some point because of double overflow and since algorithm is really slow for large N.

@Shimuuar Shimuuar added a commit that referenced this issue Jan 19, 2012
@Shimuuar Shimuuar Call error if critical value couldn't be calculated.
(related to bug #18)
3ef3132
@Shimuuar Shimuuar added a commit that referenced this issue Aug 31, 2012
@Shimuuar Shimuuar Call error if critical value couldn't be calculated.
(related to bug #18)
ee1ac13
@Shimuuar
Collaborator

fixed in 755a825

@Shimuuar Shimuuar closed this Sep 12, 2014
@3noch

If this is fixed, has it not been released? The warning is still in haddocks: https://hackage.haskell.org/package/statistics-0.13.2.3/docs/Statistics-Test-WilcoxonT.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment