New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG + 1] Remove np.isclose() from ROC curve calculation #7353

Merged
merged 1 commit into from Sep 11, 2016

Conversation

Projects
None yet
3 participants
@jblackburne
Contributor

jblackburne commented Sep 7, 2016

This PR removes the use of np.isclose() when finding indices where y_score changes during calculation of ROC curves. It is equivalent to changing the tolerance parameters of isclose to 0. It is strictly a reversion of #3268.

Fixes #3864, as well as dupes #4864 and #6688.

There is relevant discussion on #6693. To me it seemed that the consensus was to take this course of action. Further comments are welcome too.

If this is merged, #6693 can probably be closed as OBE.

@jblackburne jblackburne changed the title from Remove np.isclose() from ROC curve calculation to [MRG] Remove np.isclose() from ROC curve calculation Sep 7, 2016

@jnothman jnothman added this to the 0.18 milestone Sep 7, 2016

@jnothman

This comment has been minimized.

Member

jnothman commented Sep 7, 2016

I think at a minimum, this will require a what's new alerting users to the changed behaviour. Perhaps a non-regression test is also appropriate. Thanks for this. We'll squeeze it into an imminent release, perhaps?

@jnothman jnothman added the Bug label Sep 7, 2016

@amueller

This comment has been minimized.

Member

amueller commented Sep 7, 2016

+1 for non-regression test with small values. +1 for 0.18

@jblackburne

This comment has been minimized.

Contributor

jblackburne commented Sep 9, 2016

Modified an existing test (test_score_scale_invariance) rather than adding a new test. The modified test fails on master but succeeds with this PR.

@amueller

This comment has been minimized.

Member

amueller commented Sep 9, 2016

can you rebase please?

@@ -387,6 +387,11 @@ Bug fixes
- Fix :class:`linear_model.ElasticNet` sparse decision function to match
output with dense in the multioutput case.
- :func:`metrics.roc_curve` and :func:`metrics.precision_recall_curve` no
longer round `y_score` values when creating ROC curves; this was causing

This comment has been minimized.

@amueller

amueller Sep 9, 2016

Member

double backticks. methinks.

- :func:`metrics.roc_curve` and :func:`metrics.precision_recall_curve` no
longer round `y_score` values when creating ROC curves; this was causing
problems for users with very small differences in scores (`#3864
<https://github.com/scikit-learn/scikit-learn/issues/3864>`_).

This comment has been minimized.

@amueller

amueller Sep 9, 2016

Member

can you please reference the PR, not the issue? If you like, you can also attribute the fix to yourself and optionally add a link to your github account / website.

Removed isclose from _binary_clf_curve. Added a bugfix report to what…
…s_new.rst. Modified a unit test to check for the kind of problems caused by isclose.
@jblackburne

This comment has been minimized.

Contributor

jblackburne commented Sep 9, 2016

Applied @amueller's suggesed fixes to the whatsnew entry, then rebased and squashed.

@amueller

This comment has been minimized.

Member

amueller commented Sep 9, 2016

lgtm

@amueller amueller changed the title from [MRG] Remove np.isclose() from ROC curve calculation to [MRG + 1] Remove np.isclose() from ROC curve calculation Sep 9, 2016

@jnothman

This comment has been minimized.

Member

jnothman commented Sep 11, 2016

LGTM

@jnothman jnothman merged commit ba7cf38 into scikit-learn:master Sep 11, 2016

3 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@jnothman

This comment has been minimized.

Member

jnothman commented Sep 11, 2016

Thanks @jblackburne :)

@jblackburne jblackburne deleted the jblackburne:remove-isclose branch Sep 12, 2016

rsmith54 pushed a commit to rsmith54/scikit-learn that referenced this pull request Sep 14, 2016

Removed isclose from _binary_clf_curve. (scikit-learn#7353)
Added a bugfix report to whats_new.rst. Modified a unit test to check for the kind of problems caused by isclose.

TomDLT added a commit to TomDLT/scikit-learn that referenced this pull request Oct 3, 2016

Removed isclose from _binary_clf_curve. (scikit-learn#7353)
Added a bugfix report to whats_new.rst. Modified a unit test to check for the kind of problems caused by isclose.

@amueller amueller referenced this pull request Oct 11, 2016

Closed

Fix roc #6693

Sundrique added a commit to Sundrique/scikit-learn that referenced this pull request Jun 14, 2017

Removed isclose from _binary_clf_curve. (scikit-learn#7353)
Added a bugfix report to whats_new.rst. Modified a unit test to check for the kind of problems caused by isclose.

paulha added a commit to paulha/scikit-learn that referenced this pull request Aug 19, 2017

Removed isclose from _binary_clf_curve. (scikit-learn#7353)
Added a bugfix report to whats_new.rst. Modified a unit test to check for the kind of problems caused by isclose.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment