Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG+1] MAINT drop SciPy < 0.13 #8854

Merged
merged 8 commits into from Jun 2, 2017
Merged

[MRG+1] MAINT drop SciPy < 0.13 #8854

merged 8 commits into from Jun 2, 2017

Conversation

@naoyak
Copy link
Contributor

@naoyak naoyak commented May 10, 2017

Ref: #8810

Removes or deprecates backport code covering scipy < 0.13.

  • utils.sparsetools.connected_components: deprecate and replace backport with wrapper (scipy 0.11+)
  • utils.fixes.expit: remove (scipy 0.10+)
  • utils.extmath.logsumexp: deprecate and replace backport (scipy 0.11+)
  • utils.fixes.rankdata: remove / utils.stats.rankdata: deprecate and replace backport (scipy 0.13+)
  • utils.arpack: deprecate and replace backport of sparse matrix functions (scipy 0.10+)
  • utils.extmath.norm: deprecate and replace backport (scipy 0.10+)
  • utils.exmath.pinvh: deprecate and replace backport (scipy 0.10+)
  • remove other version checks for scipy < 0.13 in examples and library code
@jnothman
Copy link
Member

@jnothman jnothman commented May 10, 2017

@jnothman
Copy link
Member

@jnothman jnothman commented May 10, 2017

@naoyak
Copy link
Contributor Author

@naoyak naoyak commented May 11, 2017

annoyingly i think we'll need to deprecate some of those things from utils.
Perhaps not from fixes, but if you're removing things from elsewhere...

Even plain backports from scipy (which the docs warn could disappear without notice)? I guess there is value in being strict about the deprecation contract...

@jnothman
Copy link
Member

@jnothman jnothman commented May 11, 2017

It hardly hurts to deprecate some of these things: delete any documentation that references them, but attach a deprecated decorator. The only annoyance is a potential drop in test coverage.

Where do we warn that something could disappear without notice?

@naoyak
Copy link
Contributor Author

@naoyak naoyak commented May 11, 2017

Where do we warn that something could disappear without notice?

My mistake, that's certainly not what the docs say:

Warning
These utilities are meant to be used internally within the scikit-learn package. They are not guaranteed to be stable between versions of scikit-learn. Backports, in particular, will be removed as the scikit-learn dependencies evolve.

I'll go back and add some deprecations. Something like:

@deprecated("sklearn.utils.extmath.logsumexp was deprecated in version 0.19 and will be "
            "removed in 0.21. Use scipy.misc.logsumexp instead.")
@jnothman
Copy link
Member

@jnothman jnothman commented May 11, 2017

@naoyak naoyak force-pushed the naoyak:scipy-deps branch 4 times, most recently from 941ad46 to fc982d8 May 11, 2017
@lesteve
Copy link
Member

@lesteve lesteve commented May 11, 2017

Update .travis.yml: SCIPY_VERSION="0.11" should become SCIPY_VERSION="0.13.1".

@lesteve
Copy link
Member

@lesteve lesteve commented May 11, 2017

Also please fix the flake8 errors in the Travis build. They looks like they are genuine problems.

@naoyak naoyak force-pushed the naoyak:scipy-deps branch from fc982d8 to 291073e May 11, 2017
@naoyak
Copy link
Contributor Author

@naoyak naoyak commented May 11, 2017

This is dependent on #8855.

Should we also replace calls to utils.extmath.norm with scipy.misc.norm at this point? @fabianp

@naoyak naoyak force-pushed the naoyak:scipy-deps branch from 291073e to 53c7589 May 11, 2017
@lesteve
Copy link
Member

@lesteve lesteve commented May 11, 2017

This is dependent on #8855.

I see I edited the issue to make this more clear.

@naoyak
Copy link
Contributor Author

@naoyak naoyak commented May 11, 2017

I wiped out as much old code as possible while preserving public API methods (except those in fixes). Once the CI dependencies on the other PR are worked out I'll rebase and see how it goes..

@naoyak naoyak force-pushed the naoyak:scipy-deps branch 4 times, most recently from 69d27c8 to 2fc77e3 May 12, 2017
@naoyak
Copy link
Contributor Author

@naoyak naoyak commented May 13, 2017

Travis is now passing on the builds with scipy 0.17.0/0.18.1. Rebasing once #8855 is merged.

@naoyak naoyak changed the title [WIP] MAINT drop SciPy < 0.13 MAINT drop SciPy < 0.13 May 13, 2017
@naoyak naoyak mentioned this pull request May 13, 2017
@naoyak naoyak force-pushed the naoyak:scipy-deps branch from 2fc77e3 to 8480ba9 May 14, 2017
@naoyak naoyak changed the title MAINT drop SciPy < 0.13 [MRG] MAINT drop SciPy < 0.13 May 14, 2017
@naoyak
Copy link
Contributor Author

@naoyak naoyak commented May 14, 2017

This is ready for review. I stuck an extra commit on to test that CI passes with the proposed changes from #8855 but will later rebase.

The same commits are also in #8874 but I figure the diff is big enough without the NumPy-related changes to examine at once.

logX = np.vstack([logx, logx])
assert_array_almost_equal(np.exp(logsumexp(logX, axis=0)), X.sum(axis=0))
assert_array_almost_equal(np.exp(logsumexp(logX, axis=1)), X.sum(axis=1))

This comment has been minimized.

@naoyak

naoyak May 15, 2017
Author Contributor

question: when deprecating a backport (and replacing the implementation with a wrapper of the upstream function), should the test be left in place until we remove it entirely?

@jnothman
Copy link
Member

@jnothman jnothman commented May 15, 2017

@naoyak naoyak force-pushed the naoyak:scipy-deps branch from 56388c9 to c09f67f May 15, 2017
@naoyak naoyak force-pushed the naoyak:scipy-deps branch 3 times, most recently from 6067501 to 05d6e26 Jun 2, 2017
@naoyak naoyak force-pushed the naoyak:scipy-deps branch 2 times, most recently from c1494f4 to de84176 Jun 2, 2017
@@ -1,22 +1,15 @@
import numpy
# Remove in version 0.21

This comment has been minimized.

@lesteve

lesteve Jun 2, 2017
Member

Given that there is no tests anymore, I think you can remove the tests folder and setup.py as well. You only need to keep __init__.py for the deprecated connected_components function.

This comment has been minimized.

@lesteve

lesteve Jun 2, 2017
Member

Or you could just add a test file with this content:

from sklearn.utils.sparsetools import connected_components

Your call

This comment has been minimized.

@naoyak

naoyak Jun 2, 2017
Author Contributor

Do we need this line in utils/setup.py?
#8854 (comment)
config.add_subpackage('sparsetools')

Apparently the lack of a setup.py and tests folder causes this test to fail..

I don't understand the implication of registering a subpackage with add_subpackage() - is this just for setting up the testrunner, or does it affect the ability to import submodules from sklearn?

This comment has been minimized.

@lesteve

lesteve Jun 2, 2017
Member

Apparently the lack of a setup.py and tests folder causes this test to fail..

OK I did not know that. I think this is fine like this then.

Do we need this line in utils/setup.py?
#8854 (comment)
config.add_subpackage('sparsetools')

I think so. One way to check would be to do python setup.py sdist and make sure that the folder sklearn.utils.sparsetools is in the sdist.

This comment has been minimized.

@naoyak

naoyak Jun 2, 2017
Author Contributor

got it. well, a bit annoying to require those placeholders but this seems to work.

@lesteve
Copy link
Member

@lesteve lesteve commented Jun 2, 2017

LGTM, I just pushed a small change to keep the signature of the original wrapper.

@@ -415,7 +415,7 @@ def logsumexp(arr, *args, **kwargs):
>>> logsumexp(a)
9.4586297444267107
"""
return scipy_logsumexp(arr, *args, **kwargs)
return scipy_logsumexp(arr, axis=0)

This comment has been minimized.

@naoyak

naoyak Jun 2, 2017
Author Contributor

oops ;)

@naoyak naoyak force-pushed the naoyak:scipy-deps branch from 1f3d055 to bfc351a Jun 2, 2017
@naoyak
Copy link
Contributor Author

@naoyak naoyak commented Jun 2, 2017

Thanks for the reviews - good to merge and move on to #8874?

@lesteve
Copy link
Member

@lesteve lesteve commented Jun 2, 2017

LGTM, merging, thanks a lot!

@lesteve lesteve merged commit bd0fc23 into scikit-learn:master Jun 2, 2017
4 of 5 checks passed
4 of 5 checks passed
@codecov
codecov/patch 94.17% of diff hit (target 94.88%)
Details
ci/circleci Your tests passed on CircleCI!
Details
@codecov
codecov/project 95.94% (+1.05%) compared to a150d93
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@naoyak naoyak deleted the naoyak:scipy-deps branch Jun 3, 2017
Sundrique added a commit to Sundrique/scikit-learn that referenced this pull request Jun 14, 2017
Remove sklearn.utils.fixes functions that are not needed for scipy >= 0.13 and keep deprecated wrappers in other modules.
dmohns added a commit to dmohns/scikit-learn that referenced this pull request Aug 7, 2017
Remove sklearn.utils.fixes functions that are not needed for scipy >= 0.13 and keep deprecated wrappers in other modules.
dmohns added a commit to dmohns/scikit-learn that referenced this pull request Aug 7, 2017
Remove sklearn.utils.fixes functions that are not needed for scipy >= 0.13 and keep deprecated wrappers in other modules.
NelleV added a commit to NelleV/scikit-learn that referenced this pull request Aug 11, 2017
Remove sklearn.utils.fixes functions that are not needed for scipy >= 0.13 and keep deprecated wrappers in other modules.
paulha added a commit to paulha/scikit-learn that referenced this pull request Aug 19, 2017
Remove sklearn.utils.fixes functions that are not needed for scipy >= 0.13 and keep deprecated wrappers in other modules.
AishwaryaRK added a commit to AishwaryaRK/scikit-learn that referenced this pull request Aug 29, 2017
Remove sklearn.utils.fixes functions that are not needed for scipy >= 0.13 and keep deprecated wrappers in other modules.
maskani-moh added a commit to maskani-moh/scikit-learn that referenced this pull request Nov 15, 2017
Remove sklearn.utils.fixes functions that are not needed for scipy >= 0.13 and keep deprecated wrappers in other modules.
jwjohnson314 pushed a commit to jwjohnson314/scikit-learn that referenced this pull request Dec 18, 2017
Remove sklearn.utils.fixes functions that are not needed for scipy >= 0.13 and keep deprecated wrappers in other modules.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants