Model Governance in H2O-3

• AUC: Calculates metric AUC in R using library ROCR. h2o.auc only extracts the metric from model or model performance objects. For data scientists that run segmentation or stacks their models a standalone AUC function is more useful.
• Collinearity Analysis: Calculates condition indices which indicates when dependencies might affect regression estimates. As the value of the condition index increases it is an indication that there are some collinearity between features used in the regression. As mentioned in the script, we try to replicate SAS COLLIN/COLLINOINT.
• Information Value: Calculates information value or weight of evidence for variables in an H2OFrame. Information Value is used in credit scoring to compare predictive power among variables. Keep in mind this function doesn't use rpart but instead use manual quantiles. The IV or WOE would be different from using woe package but trends in the same direction in terms of how valuable a variable is. Example using iris data is included.
• Kolomogorov Smirnov Chart: K-S is a measure of the degree of separation between the positive and negative distributions. The K-S is 100, if the scores partition the population into two separate groups in which one group contains all the positives and the other all the negatives. Metric can be calculated off lift/gains chart.
• Logit Plot: Example code of how to plot the prediction probability on a logit scale.
• Variance Inflation Factors: Calculates the inflation in the variances of the parameter estimates due to collinearities that exist among the predictor variables. This is done in H2O by regression each predictor against all other predictor variables.
• Regression Performance Table: Creates a performance table for a regression model. The regression equivalent to lift/gains table.