Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weighted average scores no nan #14595

Merged

Conversation

amueller
Copy link
Member

@amueller amueller commented Aug 7, 2019

Finishes #10891 by adding a regression test.
If there's no positives the code raises a division by zero warning, and there's a test that the warning is raised. I'm not sure if that's useful, but I'm keeping with the current behavior. For the case in #10891 I'm still raising the warning but providing a useful number.

kyu-sz and others added 5 commits Mar 29, 2018
Some metrics can produce NaN scores which are however 0-weighted (e.g. PR curve score calculated on class with no positive ground truth samples in the batch). When directly taking weighted average of them, the NaN scores can cause the average score to be NaN, even if they are 0-weighted.

To prevent this, before taking the average, we force the scores with 0 weight to be 0, so that they will not pose effect on the final average score.
@amueller
Copy link
Member Author

@amueller amueller commented Aug 12, 2019

These failures are somewhat disconcerning, in particular because they are version-dependent?!

@amueller
Copy link
Member Author

@amueller amueller commented Aug 12, 2019

I fix

@amueller
Copy link
Member Author

@amueller amueller commented Aug 13, 2019

actually good for reviews now.

Copy link
Member

@jnothman jnothman left a comment

A small change log entry?

@amueller
Copy link
Member Author

@amueller amueller commented Aug 13, 2019

What should it say? Allowing averaging metrics in the absence of true positives? Sure.

@jnothman
Copy link
Member

@jnothman jnothman commented Aug 13, 2019

@adrinjalali adrinjalali merged commit 1a14920 into scikit-learn:master Aug 14, 2019
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants