Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make Custom Metrics an evaluation metric to find best grid model for gridsearch #15993

Closed
wendycwong opened this issue Dec 28, 2023 · 0 comments
Closed

Make Custom Metrics an evaluation metric to find best grid model for gridsearch #15993

wendycwong opened this issue Dec 28, 2023 · 0 comments
Assignees
@tomasfryda
Labels
bug
Milestone
3.46.0.1

Comments

@wendycwong
Copy link
Contributor

wendycwong commented Dec 28, 2023 •
edited

Support ticket: https://support.h2o.ai/a/tickets/107079

Currently, gridsearch chooses best model out of gridsearch on hyperparameters using some metric values. However, a customer will like to use custom metrics to be used as the evaluating metric for gridsearch to determine best model.
@wendycwong wendycwong added the bug label Dec 28, 2023
@wendycwong wendycwong changed the title Make Custom Metrics a hyperparameter for gridsearch Make Custom Metrics an evaluation metric to find best grid model for gridsearch Jan 2, 2024
@tomasfryda tomasfryda mentioned this issue Jan 3, 2024
Merged
@tomasfryda tomasfryda added this to the 3.44.0.4 milestone Jan 4, 2024
tomasfryda added a commit that referenced this issue Jan 18, 2024
@tomasfryda
GH-15993: Custom metric as a hyperparam for grid search (#15999)
1639968 
* Custom metric as a hyperparam for grid search

* Add test for custom_increasing

* Hide GAM's custom_metric_func until it's tested

* Revert changes in h2o-py/h2o/utils/shared_utils.py

* Fix python tests by hiding custom metrics when not calculated
@mn-mikke mn-mikke modified the milestones: 3.44.0.4, 3.46.0.2, 3.46.0.1 Mar 4, 2024
@mn-mikke mn-mikke closed this as completed Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tomasfryda tomasfryda

Labels
bug
Projects
None yet
Milestone
3.46.0.1
Development

No branches or pull requests
4 participants
@mn-mikke @wendycwong @sebhrusen @tomasfryda