Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SW-2559] Expose Scoring History and Variable Importances on H2OMOJOModel #2525

Merged
merged 13 commits into from May 24, 2021

Conversation

mn-mikke
Copy link
Collaborator

@mn-mikke mn-mikke commented May 6, 2021

@michalkurka
Copy link

When people are training CV models, how do they have access to scoring history of the CV models?

Also how do they access CV metrics, training metrics and validation metrics?

@mn-mikke
Copy link
Collaborator Author

mn-mikke commented May 7, 2021

When people are training CV models, how do they have access to scoring history of the CV models?

As discussed, it's not currently possible

Also how do they access CV metrics, training metrics and validation metrics?

scala> model.getTrainingMetrics()
res2: Map[String,Double] = Map(Logloss -> 0.5407563235669484, Gini -> 0.5738086864779322, RMSE -> 0.42418099556613403, MSE -> 0.17992951699947662, R2 -> 0.24477535552017637, AUC -> 0.7869043432389661)

scala> model.getValidationMetrics()
res3: Map[String,Double] = Map()

scala> model.getCrossValidationMetrics()
res4: Map[String,Double] = Map()

val rows = (0 until table.rows()).map { rowId =>
val rowData = (0 until table.columns()).map { colId =>
table.getCell(colId, rowId) match {
case str: String if table.getColTypes()(colId) == ColumnType.INT => Integer.parseInt(str)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, this should be handled on H2O-3 side. I'm not sure about the impact of the eventual change

@mn-mikke mn-mikke merged commit 3fab572 into master May 24, 2021
@mn-mikke mn-mikke deleted the mn/SW-2559 branch May 24, 2021 20:51
mn-mikke added a commit that referenced this pull request May 24, 2021
…odel (#2525)

* [SW-2559] Expose Scoring History and Variable Importances on H2OMOJOModel

* Deprecate variable importances on H2ODeepLearning

* Rename variable importances to feature importances

* Extend documentation

* Fix deserialization problem

* spotless apply

* Fix python tests and deprecation on DeepLearningMOJOModel

* fix python tests

* fix R tests

* typo

* fix R tests

* spotless apply

* null handling

(cherry picked from commit 3fab572)

# Conflicts:
#	api-generation/src/main/scala/ai/h2o/sparkling/api/generation/common/AlgorithmConfigurations.scala
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants