Update OptunaSearch #1106
Update OptunaSearch #1106
Conversation
setup.py
Outdated
| @@ -15,6 +15,7 @@ | |||
|
|
|||
| install_requires = [ | |||
| "NumPy>=1.17.0rc1", | |||
| "packaging", | |||
There was a problem hiding this comment.
move to [blendsearch] option
test/tune/test_searcher.py
Outdated
| @@ -70,7 +70,7 @@ def test_searchers(): | |||
| searcher = OptunaSearch(["a", config["a"]], metric="m", mode="max") | |||
| try: | |||
| searcher.suggest("t0") | |||
| except ValueError: | |||
| except AttributeError: | |||
| # not enough values to unpack (expected 3, got 1) | |||
There was a problem hiding this comment.
Is this error message still correct?
flaml/tune/searcher/suggestion.py
Outdated
| @@ -33,6 +34,8 @@ | |||
| Uniform, | |||
| ) | |||
| from ..trial import flatten_dict, unflatten_dict | |||
| from packaging import version | |||
There was a problem hiding this comment.
use try...except. Or move to where it's needed, if performance is not an issue.
| [tutorial](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html). | ||
| Warning - No actual computation should take place in the define-by-run | ||
|
|
||
| `Optuna <https://optuna.org/>`_ is a hyperparameter optimization library. |
There was a problem hiding this comment.
This syntax of URL doesn't work in our doc compilation.
| metric: The training result objective value attribute. If | ||
| None but a mode was passed, the anonymous metric ``_metric`` | ||
| will be used per default. Can be a list of metrics for | ||
| multi-objective optimization. | ||
| mode: One of {min, max}. Determines whether objective is | ||
| minimizing or maximizing the metric attribute. Can be a list of | ||
| modes for multi-objective optimization (corresponding to | ||
| ``metric``). |
There was a problem hiding this comment.
I also think this is a natural way for lexicographic optimization and would like this kind of API in the beginning. Let's discuss the API after this PR is merged.
flaml/tune/searcher/suggestion.py
Outdated
|
|
||
| Multi-objective optimization is supported: | ||
|
|
||
| .. code-block:: python |
There was a problem hiding this comment.
syntax for doc compilation
flaml/tune/searcher/suggestion.py
Outdated
| You can pass configs that will be evaluated first using | ||
| ``points_to_evaluate``: | ||
|
|
||
| .. code-block:: python |
There was a problem hiding this comment.
syntax for doc compilation
flaml/tune/searcher/suggestion.py
Outdated
| Avoid re-running evaluated trials by passing the rewards together with | ||
| `points_to_evaluate`: | ||
|
|
||
| .. code-block:: python |
There was a problem hiding this comment.
syntax for doc compilation
flaml/tune/searcher/suggestion.py
Outdated
| define_search_space, | ||
| metric="loss", | ||
| mode="min") | ||
| tune.run(trainable, search_alg=optuna_search) | ||
| .. versionadded:: 0.8.8 |
There was a problem hiding this comment.
syntax for doc compilation
There was a problem hiding this comment.
add test to cover the changes.
flaml/tune/searcher/suggestion.py
Outdated
| define_search_space, | ||
| metric="loss", | ||
| mode="min") | ||
| tune.run(trainable, search_alg=optuna_search) | ||
| .. versionadded:: 0.8.8 |
There was a problem hiding this comment.
".. versionadded:: 0.8.8" Check if this version need to be changed?
skzhang1
requested review from
feiran-jia and
yiranwu0
and removed request for
feiran-jia
Why are these changes needed?
To support multiple objectives global search. Copy from ray (1.13.1) https://github.com/ray-project/ray/blob/ray-1.13.1/python/ray/tune/suggest/optuna.py.
Related issue number
Checks