Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MAINT Clean up deprecations for 1.5: in log_loss #28851
MAINT Clean up deprecations for 1.5: in log_loss #28851
Changes from 5 commits
829fa65
12765b0
bb13aa2
bcb5f3a
fc4edaa
eb26d8e
dac73bc
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lorentzenchr I'm a bit confused. Does removing
eps
(deprecated in #25299) means that noweps
is always 0 oreps
is always computed based on the dtype ? The previous "auto" seems to indicate the latter but in that case testing edge cases is no longer possible.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See #24515 (comment).
My opinion is that eps=0 is the correct behavior (who are we to judge and MODIFY uncalibrated predicted probabilities!). The consensus was more in the direction of dtype dependent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At least, the clipping should not happen here to me. If y_true=0 and y_pred=0, the result should be exactly 0.
xlogy(0, 0) = 0
(no warning).The question is do we want to return
inf
when y_true != 0 and y_pred = 0, or a finite value. If the former, we should clip with eps=0, else we should clip the result ofxlogy
as suggested in #24515 (comment)I would go with returning
inf
, but I don't know if we rely on it being finite (maybe if *SearchCV and co), and the warning message said that eps will be non-zero in 1.5, so maybe we should better keep it as is.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have the same opinion as you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed, I find the following weird:
Shall to conditionally clip to
eps
only whenone_hot_encode(y_true) > 0
and clip to 0 otherwise?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually whatever the decision on this, there should be a test to cover the case where
log_loss
reaches its minimum (perfect predictions), both in binary and multiclass settings.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather leave this discussion for a separate issue/PR to (try to) keep the focus of this PR on the deprecations clean-up.
I added a test for perfect predictions that only checks that the result is close to 0 for now.