Add support for scikit-learn>=1.0

[angela97lin]

Opening up a branch, enabling scikit-learn v1.0, and seeing what breaks :)

[codecov]

Codecov Report

Merging #3051 (a8cea5f) into main (24779e0) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #3051     +/-   ##
=======================================
+ Coverage   99.7%   99.8%   +0.1%     
=======================================
  Files        312     312             
  Lines      30344   30340      -4     
=======================================
- Hits       30252   30249      -3     
+ Misses        92      91      -1     

Impacted Files	Coverage Δ
evalml/model_understanding/graphs.py	100.0% <ø> (ø)
...lml/tests/objective_tests/test_standard_metrics.py	100.0% <ø> (ø)
...lml/tests/model_understanding_tests/test_graphs.py	100.0% <100.0%> (ø)
...derstanding/prediction_explanations/_algorithms.py	100.0% <0.0%> (+0.8%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 24779e0...a8cea5f. Read the comment docs.

[angela97lin]

Between 0.24.0 and 1.0.0+, the implementation was changed to just return a matthews_corrcoef score of 0.0 if the denominator was 0 (previously caused invalid divide by 0)

[angela97lin]

Note:
scikit-learn >1.0 will require scikit-optimize>=0.9.0 because modules / objects get shuffled around, causing the following:

evalml/pipelines/components/estimators/classifiers/logistic_regression_classifier.py:4: in <module>
    from skopt.space import Real
../evalml_venv/lib/python3.7/site-packages/skopt/__init__.py:55: in <module>
    from .searchcv import BayesSearchCV
../evalml_venv/lib/python3.7/site-packages/skopt/searchcv.py:16: in <module>
    from sklearn.utils.fixes import MaskedArray
E   ImportError: cannot import name 'MaskedArray' from 'sklearn.utils.fixes' (/Users/angela.lin/Desktop/evalml_venv/lib/python3.7/site-packages/sklearn/utils/fixes.py)```

Our tests, which automatically install the latest version of scikit-optimize (0.9.0) will thus not break, so maybe we don't need to set our requirement of scikit-optimize>=0.9.0 but open to discussion if it feels like we should.

[freddyaboulton]

How did you run into this?

[angela97lin]

My local environment uses scikit-optimize==0.8.1 so as soon as I tried using scikit-learn 1.0, things broke 🥲

[freddyaboulton]

I see. I'm wondering if this will affect users in some way/if we need to update the requirements file in light of this. Did you just install scikit-learn or did you do pip install -r requirements.txt ?

[angela97lin]

I just installed scikit-learn! Our tests didn't break because they call pip install -r requirements.txt and in a clean environment, 0.9.0 will be installed.

This is a known issue in the scikit-optimize repo where it seems like the community solution is to either use scikit-learn<=0.24.2 or use scikit-optimize>=0.9.0: scikit-optimize/scikit-optimize#569

[freddyaboulton]

Cool, makes sense to me!

[bchen1116]

Would it be simpler to make the core requirements use scikit-learn >= 1.0.0 versus having support for both but test coverage for the later version?

[angela97lin]

Hmm, I can be persuaded otherwise but I think this aligns with what we've currently been doing aka every time our dependencies come out with a new version, we pin until we can fix the tests to support the latest version--and thats the version that we run on!

[angela97lin]

Negative and zero values are no longer accepted: https://github.com/scikit-learn/scikit-learn/blob/0d378913be6d7e485b792ea36e9268be31ed52d0/sklearn/manifold/_t_sne.py#L816

[freddyaboulton]

@angela97lin This looks good to me!

[bchen1116]

LGTM! Is there a reason we chose to support >=0.24 versus >=1.0.0?

[bchen1116]

Would it be simpler to make the core requirements use scikit-learn >= 1.0.0 versus having support for both but test coverage for the later version?