ENH add n_jobs to mutual_info_regression and mutual_info_classif

[netomenoci]

Reference Issues/PRs

Feat: #27795 (comment)

What does this implement/fix? Explain your changes.

This implement the addition of parameter n_jobs to sklearn.feature_selection.mutual_info_regression and sklearn.feature_selection.mutual_info_classif

[github-actions]

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 5a54c2f. Link to the linter CI: here}

[glemaitre]

We will need an entry in doc/whats_new/v1.5.rst to acknowledge adding the parameter.
We also need to have a test that try different n_jobs. For instance we can test that n_jobs=None provide the same results than n_jobs=2.

[netomenoci]

@glemaitre thanks for the suggestions. I have implemented them.

[glemaitre]

It almost look good. Here are a couple of suggestions.

[netomenoci]

It almost look good. Here are a couple of suggestions.

@glemaitre i have made the suggestions, thanks. One of the circleci tests is failing, but i don't know why.

[glemaitre]

@netomenoci I push a piece of code that show how to make the parallelization if you are interested in.

[netomenoci]

awesome, thanks :)

[Saeid99Shadiw113]

Hi bary Wery good nice good luck bary

[glemaitre]

LGTM. We will need a second review.

[glemaitre]

One of the circleci tests is failing, but i don't know why.

It was because of a git conflict.

[lesteve]

Here is a quick benchmark, loosely adapted from #27795 (comment) (which unless I am mistaken was not really a benchmark of parallelizing mutual_info_regression).

I get 2.5 max speed improvement on my laptop, that has 8 logical cores (4 physical cores + hyper-threading).

from sklearn.datasets import make_sparse_uncorrelated
from sklearn.feature_selection import mutual_info_regression


n_features = 100
n_samples = int(1e4)
X, y = make_sparse_uncorrelated(random_state=0, n_features=n_features, n_samples=n_samples)

print('n_jobs=1')
%timeit mutual_info_regression(X, y, n_jobs=1)

print('n_jobs=4')
%timeit mutual_info_regression(X, y, n_jobs=4)

print('n_jobs=8')
%timeit mutual_info_regression(X, y, n_jobs=8)

I get:

n_jobs=1
3.56 s ± 127 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
n_jobs=4
1.41 s ± 179 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
n_jobs=8
1.34 s ± 28.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

[lesteve]

I used keyword arguments to make the code more readable in the _estimate_mi function call and also use keyword-only arguments in the _estimate_mi definition.

I guess that's fine since _estimate_mi is private. @glemaitre do you agree?

[glemaitre]

Yes I do agree.

[lesteve]

Let's merge this one, thanks @netomenoci!

[netomenoci]

Here is a quick benchmark, loosely adapted from #27795 (comment) (which unless I am mistaken was not really a benchmark of parallelizing mutual_info_regression).

I get 2.5 max speed improvement on my laptop, that has 8 logical cores (4 physical cores + hyper-threading).

from sklearn.datasets import make_sparse_uncorrelated

from sklearn.feature_selection import mutual_info_regression





n_features = 100

n_samples = int(1e4)

X, y = make_sparse_uncorrelated(random_state=0, n_features=n_features, n_samples=n_samples)



print('n_jobs=1')

%timeit mutual_info_regression(X, y, n_jobs=1)



print('n_jobs=4')

%timeit mutual_info_regression(X, y, n_jobs=4)



print('n_jobs=8')

%timeit mutual_info_regression(X, y, n_jobs=8)

I get:

n_jobs=1

3.56 s ± 127 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

n_jobs=4

1.41 s ± 179 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

n_jobs=8

1.34 s ± 28.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Using sklearn wrapper of parallel slows it down because there is some extra overhead on top of joblib (eg copies)

If u think it's useful , we can change (1 line change) the import of Parallel to joblib instead of sklearn utils and it should be faster and closer to the previous mentioned benchmark.

[glemaitre]

If u think it's useful , we can change (1 line change) the import of Parallel to joblib instead of sklearn utils and it should be faster and closer to the previous mentioned benchmark.

We cannot do that because we will have issue with the global config. This is the reason to have the wrapper.