Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REF: Multipletests reduce memory usage #1396

Merged
merged 7 commits into from Feb 19, 2014

Conversation

Projects
None yet
1 participant
@josef-pkt
Copy link
Member

commented Feb 16, 2014

see #1394
reducing memory consumption and some duplicate, redundant calculations in multipletests

incomplete: not all methods work yet with 5300**2 pvalues, but I want to get TravisCI results for the refactoring

@josef-pkt

This comment has been minimized.

Copy link
Member Author

commented Feb 16, 2014

After the last commit, all methods except 'hommel' finish without memory error
I attached test script to #1394

I don't know what to do about hommel, but will always be slow since it has a len(pvals) loop.
'hommel' is the most complex method, even if it's just a few lines.

@josef-pkt

This comment has been minimized.

Copy link
Member Author

commented Feb 17, 2014

about finished with the last commit

I don't know if I should really still add overwrite_input=True
and I think returnsorted should be dropped. I don't really see a strong use case.

@josef-pkt

This comment has been minimized.

Copy link
Member Author

commented Feb 19, 2014

There is still a bit of slack left, memory consumption could be reduced a bit further by using out arguments in numpy functions, and more inplace operation.
However, since it reduces readability, I'm waiting until that is really necessary.

I'm going to merge when TravisCI comes back green.

josef-pkt added a commit that referenced this pull request Feb 19, 2014

Merge pull request #1396 from josef-pkt/multitest_memory
REF: Multipletests reduce memory usage

@josef-pkt josef-pkt merged commit a167d72 into statsmodels:master Feb 19, 2014

1 check passed

default The Travis CI build passed
Details

@josef-pkt josef-pkt deleted the josef-pkt:multitest_memory branch Feb 19, 2014

@josef-pkt josef-pkt added the PR label Mar 11, 2014

PierreBdR pushed a commit to PierreBdR/statsmodels that referenced this pull request Sep 2, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.