Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[POC | not ready for merge] Speed up bootstrap using multiprocessing #185

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rebecca-burwei
Copy link
Contributor

@rebecca-burwei rebecca-burwei commented Jul 15, 2023

Trying out some things I learned at SciPy.

Locally, I'm seeing a 4-5x speed-up on the tests. I wonder what the speed-up would be in our production environments.

I understand we're introducing a bit more non-determinism here, but I bet it's possible to make the difference small enough.

If this works, we should implement it for the Bayesian bootstrap too.

Other tips:

  • This can probably be done with multithreading, which will use less memory and take less time.
  • Another way to speed things up is to use numba on stat_fn's that are not just simple compositions of numpy functions.

Tagging some folks for feedback on this proof of concept.

@scholtzan
Copy link
Contributor

how does #177 compare to this?

@mikewilli
Copy link
Contributor

@rebecca-burwei Should this be closed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants