probability calibration does not work in Sparkling Water Dataframe API

[exalate-issue-sync]

originally posted as a github issue [here|https://github.com//issues/1108]:

Although when calibration is enabled, EasyPredictModelWrapper returns a BinomialModelPrediction object which contains both raw probs and calibrated probs, the
implicit conversion defined here
{code}
sparkling-water/ml/src/main/scala/org/apache/spark/ml/h2o/models/H2OMOJOModel.scala

Lines 81 to 83 in 9968342

implicit def toBinomialPrediction(pred: AbstractPrediction) = BinomialPrediction(
pred.asInstanceOf[BinomialModelPrediction].classProbabilities(0),
pred.asInstanceOf[BinomialModelPrediction].classProbabilities(1))
{code}
transforms the BinomialModelPrediction object to a BinomialPrediction, which only contains raw probs (p0, and p1) and calibrated probabilities are NOT returned. Both raw and calibrated probabilities should be returned to the user.

[DinukaH2O]

JIRA Issue Migration Info

Jira Issue: SW-1249
Assignee: Jakub Hava
Reporter: Lauren DiPerna
State: Resolved
Fix Version: 2.1.54
Attachments: N/A
Development PRs: Available

Linked PRs from JIRA

#1174

[hasithjp]

JIRA Issue Migration Info Cont'd

Jira Issue Created Date: 2019-04-25T17:44:22.195-0700