Int overflow(?) in wilcoxonMatchedPairCriticalValue #18

Shimuuar opened this Issue Dec 9, 2011 · 4 comments


None yet

2 participants

Shimuuar commented Dec 9, 2011

There is a bug in calculation of critical value for Wilcoxon-T test.

λ wilcoxonMatchedPairCriticalValue 35 0.05
Just 213
λ  wilcoxonMatchedPairCriticalValue 36 0.05
Just 2147483647   --  Which equal to Just (2^31-1) 

It looks like at some point integer overflow occurs.

Also it cannot handle samples larger than 1023 at all since it uses 2**sampleSize in calculation of critical value which overflows to +∞ for sampleSize > 1023

Test probably should fall back to normal approximation at some point.


Indeed on 64-bit machine critical point is at N=67


Use of Integers solves problem with integer overflows (commit e04da30) but wilcoxonMatchedPairCriticalValue should revert to normal approximation at some point because of double overflow and since algorithm is really slow for large N.

@Shimuuar Shimuuar added a commit that referenced this issue Aug 31, 2012
@Shimuuar Shimuuar Call error if critical value couldn't be calculated.
(related to bug #18)

fixed in 755a825

@Shimuuar Shimuuar closed this Sep 12, 2014
3noch commented Aug 14, 2015

If this is fixed, has it not been released? The warning is still in haddocks:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment