Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel fitting #7

Merged
merged 8 commits into from
Jan 15, 2024
Merged

Parallel fitting #7

merged 8 commits into from
Jan 15, 2024

Conversation

loicdtx
Copy link
Collaborator

@loicdtx loicdtx commented Aug 14, 2023

I initially wanted to improve compatibility with dask (hence the branch name), and permit things like lazy/distributed fitting, but there were too many obstacles, starting with the fancy indexing used for masking and not supported by dask.

This instead adds parallel capabilities to most fitting methods thanks to numba's parallel accelerator.

For a couple of fitting functions I had to use np.linalg.solve in place of np.linalg.inv for the parallel version to work. More details here.

Using pinv instead of inv appears to work as well. I attempted to do that for _recresid, for which the inversion isn't easy to bypass, but parallel code ends up running much slower than the sequential, despite high cpu resources use (I can't explain why...). I have therefore reverted parallelism for _recresid and roc_stable_fit

One new gallery example on parallel fitting

Can you have a quick look @jonasViehweger ?

@loicdtx loicdtx merged commit 341f357 into master Jan 15, 2024
4 checks passed
@loicdtx loicdtx deleted the feature-dask branch January 15, 2024 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant