Update katib-config document to describe suggestion service images #1907
Conversation
|
This change is |
There was a problem hiding this comment.
Thank you for this PR, I left few comments.
Also, can you update about algorithms implementation here: https://www.kubeflow.org/docs/components/hyperparameter-tuning/experiment/#random-search.
For example, for Random we say that Katib uses only hyperopt implementation.
| @@ -114,7 +114,13 @@ All of these settings except **`image`** can be omitted. If you don't specify an | |||
|
|
|||
There was a problem hiding this comment.
I think we should merge this PR after we update Katib Manifests images with the latest images.
/cc @johnugeorge
| **Must be specified**. | ||
| **Must be specified**. You can specify following images: | ||
|
|
||
| - `suggestion-chocolate`: [chocolate](https://github.com/AIworx-Labs/chocolate) based suggestion service which supports `grid`, `chocolate-random`, `chocolate-quasirandom`, `chocolate-bayesian-optimization` and `chocolate-mocmaes`. |
There was a problem hiding this comment.
Maybe it is better to say: "Which supports algorithm names: grid, ..." ?
And do we need to say about each algorithm name. For example:
chocolate-quasirandom: Chocolate implementation of quasirandom algorithm.
There was a problem hiding this comment.
I mind two things about this:
- In the discussion of Add Goptuna based suggestion service for CMA-ES. katib#1131 (comment), we decided to not introduce
goptuna-randomandgoptuna-tpe. So I don't know how should I describetpeandrandom. - I think it's better to describe
imagefield of suggestion settings in this section. How about describe algorithm name in other parts?
|
|
||
| - `suggestion-chocolate`: [chocolate](https://github.com/AIworx-Labs/chocolate) based suggestion service which supports `grid`, `chocolate-random`, `chocolate-quasirandom`, `chocolate-bayesian-optimization` and `chocolate-mocmaes`. | ||
| - `suggestion-goptuna`: [Goptuna](https://github.com/c-bata/goptuna) based suggestion service which supports `cmaes`, `tpe` and `random`. | ||
| - `suggestion-hyperband`: [HpBandSter](https://github.com/automl/HpBandSter) based suggestion service which supports `hyperband`. |
There was a problem hiding this comment.
I believe for hyperband we are using our own implementation. We can add link to Katib hyperband source code here.
There was a problem hiding this comment.
Thank you for your comment. Hm, we may change the file path of hyperband source code at kubeflow/katib#1144.
There was a problem hiding this comment.
Yes, we will change the file path for hyperband. In this PR maybe just add link to hyperband service: https://github.com/kubeflow/katib/blob/master/pkg/suggestion/v1alpha3/hyperband_service.py?
|
I believe the merge conflict on this PR is due to PR #1909, which inserted the |
c-bata
force-pushed
the
katib-supported-algorithms
branch
from
cb75738
to
cb9cd29
Compare
| @@ -181,7 +181,8 @@ sampling without replacement. Random search is therefore the best algorithm to | |||
| use when combinatorial exploration is not possible. If the number of continuous | |||
| variables is high, you should use quasi random sampling instead. | |||
|
|
|||
| Katib uses the [hyperopt](http://hyperopt.github.io/hyperopt/) optimization | |||
| Katib uses the [hyperopt](http://hyperopt.github.io/hyperopt/) or | |||
There was a problem hiding this comment.
I have no good idea how describe chocolate-random and chocolate-quasirandom. So I added Goptuna only.
It may be better to rename chocolate-random to random for consistency (related discussion: kubeflow/katib#1131 (comment)).
There was a problem hiding this comment.
I agree with @c-bata. What do you think @gaocegege @johnugeorge ?
|
/assign @andreyvelich |
There was a problem hiding this comment.
/lgtm
@sarahmaddox I think we can merge this PR. We will update manifest with the recent Katib version soon.
/cc @johnugeorge @gaocegege
|
/cc @jlewi Thanks @andreyvelich. I'm no longer working on Kubeflow, so I've cc-ed @jlewi to approve this PR. |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jlewi The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Update katib-config document to describe suggestion service images
Katib issue: kubeflow/katib#1147
cc: @gaocegege @andreyvelich @sarahmaddox