Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST add unit tests for current _get_response #21041

Merged
merged 9 commits into from Sep 18, 2021

Conversation

glemaitre
Copy link
Member

Working on #21038, I could catch an inconsistency regarding the shape of y_pred when pos_label is set to a value.
We were also raising a FutureWarning with a scalar/NumPy array in operation.

This PR adds minimum tests. They will be probably adapted later on in #20999 but this is safer to have them now.

@thomasjpfan thomasjpfan added this to the 1.0.1 milestone Sep 14, 2021
@glemaitre glemaitre added this to WAITING FOR REVIEW in Guillaume's pet Sep 14, 2021
Copy link
Member

@ogrisel ogrisel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the code could be simpler (see below) but otherwise, ok with the new tests.

sklearn/metrics/_plot/base.py Outdated Show resolved Hide resolved
Copy link
Member

@thomasjpfan thomasjpfan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this PR fixes a bug and could be considered for 1.0.1.

@@ -111,7 +113,7 @@ def _get_response(X, estimator, response_method, pos_label=None):
pos_label = estimator.classes_[1]
y_pred = y_pred[:, 1]
else:
class_idx = np.flatnonzero(estimator.classes_ == pos_label)
class_idx = np.flatnonzero(estimator.classes_ == pos_label)[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a bug fix. Should we include this in a whats_new?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is on an outdated line but the point is still valid.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a private function so I think that we are fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even tho we have different output shapes for y_pred, all the functions that call _get_response also ends up calling _binary_clf_curve. _binary_clf_curve will then force y_score to be 1d:

y_score = column_or_1d(y_score)

So in the end, not a bug fix (for a public API).

Comment on lines 73 to 74
except ValueError:
pass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May we add a comment on what version numpy will raise a ValueError?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add a comment. NumPy will not raise an error. Scikit-learn raise an error because pos_label="unknown". And the NumPy warning is only raised if pos_label is a string. I did not think about it but we could maybe generate the warning when pos_label is a valid string label.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I remove the test since we completely change the way to get the index.

glemaitre and others added 2 commits September 17, 2021 11:12
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
@thomasjpfan thomasjpfan removed this from the 1.0.1 milestone Sep 18, 2021
Copy link
Member

@thomasjpfan thomasjpfan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thomasjpfan thomasjpfan merged commit 722dd15 into scikit-learn:main Sep 18, 2021
@thomasjpfan
Copy link
Member

Failing test is on AppVeyor which should not be running.

@glemaitre glemaitre moved this from WAITING FOR REVIEW to REVIEWED AND WAITING FOR CHANGES in Guillaume's pet Sep 24, 2021
@glemaitre glemaitre moved this from REVIEWED AND WAITING FOR CHANGES to MERGED in Guillaume's pet Sep 24, 2021
@glemaitre glemaitre mentioned this pull request Oct 23, 2021
10 tasks
glemaitre added a commit to glemaitre/scikit-learn that referenced this pull request Oct 23, 2021
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
glemaitre added a commit that referenced this pull request Oct 25, 2021
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
samronsin pushed a commit to samronsin/scikit-learn that referenced this pull request Nov 30, 2021
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

3 participants