REF: Multipletests reduce memory usage #1396

merged 7 commits into from Feb 19, 2014


None yet

1 participant


see #1394
reducing memory consumption and some duplicate, redundant calculations in multipletests

incomplete: not all methods work yet with 5300**2 pvalues, but I want to get TravisCI results for the refactoring


After the last commit, all methods except 'hommel' finish without memory error
I attached test script to #1394

I don't know what to do about hommel, but will always be slow since it has a len(pvals) loop.
'hommel' is the most complex method, even if it's just a few lines.


about finished with the last commit

I don't know if I should really still add overwrite_input=True
and I think returnsorted should be dropped. I don't really see a strong use case.


There is still a bit of slack left, memory consumption could be reduced a bit further by using out arguments in numpy functions, and more inplace operation.
However, since it reduces readability, I'm waiting until that is really necessary.

I'm going to merge when TravisCI comes back green.

@josef-pkt josef-pkt merged commit a167d72 into statsmodels:master Feb 19, 2014

1 check passed

default The Travis CI build passed
@josef-pkt josef-pkt deleted the josef-pkt:multitest_memory branch Feb 19, 2014
@josef-pkt josef-pkt added the PR label Mar 11, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment