Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include confusion matrix threshold details in Performance and Prediction Section #8230

Closed
exalate-issue-sync bot opened this issue May 11, 2023 · 3 comments

Comments

@exalate-issue-sync
Copy link

+Details:+
Let's update the [Prediction and Performance section|http://docs.h2o.ai/h2o/latest-stable/h2o-docs/performance-and-prediction.html] to include details on how the confusion matrix threshold is applied. From the [code|https://github.com/h2oai/h2o-3/blob/2a6fdc715e8c4317df7320eaddf8bae2524889fa/h2o-genmodel/src/main/java/hex/genmodel/GenModel.java#L329] you can see that we predict the positive class (1, True, or categorical label that is second in the lexicographical ordering of the labels) if the predicted probability is *greater than or equal * to the threshold, which produces the best F1 score.

Documentation Update:
In the [confusion matrix section|http://docs.h2o.ai/h2o/latest-stable/h2o-docs/performance-and-prediction.html#confusion-matrix]

Below the paragraph and image in that section we should add:

Binary Classification
All predicted probabilities greater than or equal to the F1 Max threshold are labeled with the positive class (e.g., 1, True, or the second label in lexicographical order). The F1 Max threshold is selected to maximize the F1 score calculated from confusion matrix values (true positives, true negatives, false positives, and false negatives).

Multiclass Classification
Prediction class labels are based on the class with the highest predicted probability.

@exalate-issue-sync
Copy link
Author

Angela Bartz commented: Pull request submitted into rel-yule.

@exalate-issue-sync
Copy link
Author

Angela Bartz commented: Pull request merged into rel-yule.

@h2o-ops
Copy link
Collaborator

h2o-ops commented May 14, 2023

JIRA Issue Migration Info

Jira Issue: PUBDEV-7407
Assignee: Angela Bartz
Reporter: Lauren DiPerna
State: Resolved
Fix Version: 3.30.0.1
Attachments: N/A
Development PRs: Available

Linked PRs from JIRA

#4464

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant