Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: ENH: improve _count_paths_outside_method via pythran #14314

Closed
wants to merge 1 commit into from

Conversation

charlotte12l
Copy link
Contributor

@charlotte12l charlotte12l commented Jun 28, 2021

Could speed up stats.ks_2samp(rvs1, rvs2, alternative='less', mode='exact') >20x times on my computer if went well with Pythran master

from scipy import stats
import numpy as np
from timeit import timeit

print(stats.__file__)
rng = np.random.default_rng(1234)
n1 = 200
n2 = 300

rvs1 = stats.norm.rvs(size=n1, loc=0., scale=1, random_state=rng)
rvs2 = stats.norm.rvs(size=n2, loc=0.5, scale=1.5, random_state=rng)
stats.ks_2samp(rvs1, rvs2, alternative='less', mode='exact')

@rgommers
Copy link
Member

CI failures can be ignore, they're due to Pythran master being needed (0.9.12 will be released soon, and it has support for scipy.special.binom that's needed here).

Right now this PR is also blocked by the macOS crash in gh-14315.

@mdhaber
Copy link
Contributor

mdhaber commented Jul 26, 2022

Wow, I'm seeing 90x speedup. Once gh-13265 merges, I'll merge main into this PR.
After that, if CI passes, is there anything else holding up this one @rgommers? Looks like the things you mentioned above have been resolved.

@rgommers
Copy link
Member

Wow, I'm seeing 90x speedup. Once gh-13265 merges, I'll merge main into this PR.

It'd be great to see this move forward indeed.

After that, if CI passes, is there anything else holding up this one @rgommers? Looks like the things you mentioned above have been resolved.

The crash report on macOS in gh-14315 was based on this PR, it may still be a problem - not sure, requires testing. The other thing is the Pythran annotation; it uses plain int's, which is likely not correct. The int/int32/int64/intp/intc issues in gh-16646 are probably relevant. Ideally this code uses fixed dtypes, perhaps only int64.

@mdhaber
Copy link
Contributor

mdhaber commented Jul 31, 2022

This is closely related to gh-13957 so let's continue this work there.

@charlotte12l
Copy link
Contributor Author

@rgommers @mdhaber Thank you so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Pythran Items related to internal Pythran code base scipy.stats
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants