Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add formula to KS Metric in Performance & Prediction #7978

Closed
exalate-issue-sync bot opened this issue May 11, 2023 · 3 comments
Closed

Add formula to KS Metric in Performance & Prediction #7978

exalate-issue-sync bot opened this issue May 11, 2023 · 3 comments
Assignees

Comments

@exalate-issue-sync
Copy link

Add the formula information for the KS Metric:

https://www.analyticsvidhya.com/blog/2019/08/11-important-model-evaluation-error-metrics/#:~:text=K-S%20or%20Kolmogorov-Smirnov%20chart,the%20positive%20and%20negative%20distributions <- part 4

"The Gains/Lift table contains n rows. Each row is a certain bin (the binning can now be controlled actually).
Per each bin, kolmogorov-smirnov metric is calculated as cumulative_event - cumulative_non_event. The value the function model.kolmogorov_smirnov() returns is maximal value from all the bins `max(cumulative_event_i - cumulative_non_event_i).
cumulative_event (1s) and cumulative_non_event (0s) are values from 0 to 1. With each bin, this number grows (or reimains the same, if there are no new events or non-event observed in the next bin).
cumulative_event is the sum of all events observed so far up to the bin i divided by total number of events.
cumulative_non_event is analogically the sum of all non-events observed so far up to the bin i divided by total number of non-events.
Subtract these two numbers from each other and you have kolmogorov-smirnov metric."

@exalate-issue-sync
Copy link
Author

hannah.tillman commented: Added KS equation to [https://0xdata.atlassian.net/browse/PUBDEV-7655|https://0xdata.atlassian.net/browse/PUBDEV-7655|smart-link]

@exalate-issue-sync
Copy link
Author

hannah.tillman commented: closed with [https://0xdata.atlassian.net/browse/PUBDEV-7655|https://0xdata.atlassian.net/browse/PUBDEV-7655|smart-link]

@h2o-ops
Copy link
Collaborator

h2o-ops commented May 14, 2023

JIRA Issue Migration Info

Jira Issue: PUBDEV-7661
Assignee: hannah.tillman
Reporter: hannah.tillman
State: Resolved
Fix Version: 3.30.1.1
Attachments: N/A
Development PRs: N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants