AutoML: improve handling of model builder params validation

[exalate-issue-sync]

Currently, when a model fails during validation (e.g. unsupported parameter), the error is simply logged and the entire model is just skipped.

This raises several concerns:

• if AutoML passes a global param incompatible with the training dataset (e.g. wrong default distribution for regression task), then all models will fail one after another but the training will keep going indefinitely, wasting time and resources.
• let's imagine that the user passes a param that is only incompatible with one specific algo, then AutoML could be more clever and apply some fallback logic:
  ** try to train model
  ** handle validation errors
  ** if validation errors are parameters errors, reset those params to defaults
  ** rerun model
  ** if validation fails again, skip model

[exalate-issue-sync]

Sebastien Poirier commented: Note: the fallback logic (retry after switching param back to default after validation error) hasn’t been implemented in this ticket. Need to reconsider if it’s a desired behaviour.

[h2o-ops-ro]

JIRA Issue Details

Jira Issue: PUBDEV-8420
Assignee: Sebastien Poirier
Reporter: Sebastien Poirier
State: Resolved
Fix Version: 3.36.0.1
Attachments: N/A
Development PRs: Available

[h2o-ops-ro]

Linked PRs from JIRA

#5938