You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We need the ability to get the best run in an experiement based on a specific metric.
For example, let's say we value the F1 score for a given machine learning model we are training (contained in an experiment). Each time we train the model (run), we want to know if this model is better than a previously recorded model or not.
The reason we want this is so we don't need to upload every model that we don't care about.
I'm thinking something like this
public void IsBestRun(Guid runId, string metricName)
{
// 1. Fetch the given metric value for that run
// 2. Given the experiment that run is apart of, fetch the best run for that metric
// 3. Compare if this is the best run or not
}
The text was updated successfully, but these errors were encountered:
aslotte
changed the title
Add ability to get best run in experiment based on metric
Add ability to get the best run in experiment based on a given metric
Jun 3, 2020
@aslotte , i feel we should save the model irrespective with each run and let the user decide which model to use. What if the best score was a result of a bug in the code or underlying data changed.
Or
We can pass in a boolean flag so that the user decides to save it not?
We need the ability to get the best run in an experiement based on a specific metric.
For example, let's say we value the F1 score for a given machine learning model we are training (contained in an experiment). Each time we train the model (run), we want to know if this model is better than a previously recorded model or not.
The reason we want this is so we don't need to upload every model that we don't care about.
I'm thinking something like this
The text was updated successfully, but these errors were encountered: