Use callback to report results to MLflow. #799
Conversation
|
@toshihikoyanase The script looks much simpler now. Thanks! |
examples/mlflow/keras_mlflow.py
Outdated
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| study = optuna.create_study(direction='minimize') | ||
| study.optimize(objective, n_trials=100, timeout=600) | ||
| study = optuna.create_study(study_name='keras-wine', direction='minimize') |
There was a problem hiding this comment.
I think 'study_name' argument can be removed since this example doesn't use an RDB storage.
There was a problem hiding this comment.
Thank you for your comment.
I set it to show somewhat meaningful names rather than no-name-xxx in MLflow UI. But we can distinguish studies if we remove the name. So I'll remove it.
There was a problem hiding this comment.
Thanks.
This is just a thought. Current default study names are not human-friendly, so it might worth to improve the name generation logic in the future (e.g., adopt the naming rule being used by Docker).
There was a problem hiding this comment.
I agree with you. I posted an issue about the name generation as #803. Thank you.
By the way, the following part of my comment #799 (comment) is not true because all studies created with InMemoryStorage have the same default name (i.e., no-name-00000000-0000-0000-0000-000000000000). I'm sorry to give you a wrong information.
But we can distinguish studies if we remove the name.
There was a problem hiding this comment.
I posted an issue about the name generation as #803.
Thanks!
all studies created with InMemoryStorage have the same default name (i.e., no-name-00000000-0000-0000-0000-000000000000).
Oops. That is, there is no actual merit to use the default name...?
There was a problem hiding this comment.
That's right. Maybe storage classes should share the name generator instead of adding name generation code for InMemoryStorage to this example.
Codecov Report
@@ Coverage Diff @@
## master #799 +/- ##
=======================================
Coverage 90.15% 90.15%
=======================================
Files 106 106
Lines 8769 8769
=======================================
Hits 7906 7906
Misses 863 863Continue to review full report at Codecov.
|
This PR uses a callback function to report optimization results to MLflow.
In this PR, I used
study_nameasrun_nameof MLflow, but I'm not fully confident if it is a relevant usage ofrun_name. Please let me know if you have any ideas about it.