Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CMA-ES Sampler. #447

Merged
merged 23 commits into from Jul 26, 2019
Merged

Add CMA-ES Sampler. #447

merged 23 commits into from Jul 26, 2019

Conversation

@toshihikoyanase
Copy link
Collaborator

toshihikoyanase commented Jul 10, 2019

This PR adds a new sampler, which is based on the CMA-ES algorithm. This sampler is a thin wrapper of pycma.

Benchmark Results

Settings:

Solver Borda Firsts
(a) optuna#cma 29 36
(b) optuna#tpe 2 9

See this gist for further details.

Opinions Wanted

1. sigma0 argument of CmaEsSampler

CmaEsSampler.__init__ requires sigma0 argument, which is an initial standard deviation of CMA-ES (see here for further details). It is problem dependent, and pycma does not provide default values. I simply followed the pycma, but it may not be user-friendly. If you have any questions or comments, please let me know.

Current implementation
As a result of offline discussion with @sile, I decided to use range_of_variable / 6 as the default value for sigma0. This value is determined by the following comment on sigma0:

the optimum is expected to lie within about x0 +- 3*sigma0

2. Initial solution x0

CMAEvolutionStrategy in pycma also has a required parameter x0, which is a starting point of search. CmaEsSampler uses the best trial as x0 and does not provide the way to set the initial point externally. What do you think if I add a new argument to CmaEsSampler.__init__ to set x0.

Current implementation
The default value of x0 is determined by mean(low, high) to cover the range of variable with default sigma0 value.

@toshihikoyanase toshihikoyanase marked this pull request as ready for review Jul 17, 2019
optuna/integration/cma.py Outdated Show resolved Hide resolved
optuna/integration/cma.py Outdated Show resolved Hide resolved
optuna/integration/cma.py Outdated Show resolved Hide resolved
optuna/integration/cma.py Outdated Show resolved Hide resolved
optuna/integration/cma.py Show resolved Hide resolved
optuna/integration/cma.py Outdated Show resolved Hide resolved
@sile
sile approved these changes Jul 24, 2019
Copy link
Collaborator

sile left a comment

LGTM

Copy link
Collaborator

g-votte left a comment

LGTM. I added a few minor comments.

# `cma` cannot handle distributions that contain just a single value,
# so we skip this distribution.
#
# Note that `Trial` takes care of this distribution during suggestion.

This comment has been minimized.

Copy link
@g-votte

g-votte Jul 25, 2019

Collaborator

I think takes care of is too redundant to grasp what a Trial actually does.

This comment has been minimized.

Copy link
@toshihikoyanase

toshihikoyanase Jul 26, 2019

Author Collaborator

Thank you for pointing it out. I updated this comment as follows since the current Trial does not seem to have code to handle distributions with a single value. If I misunderstood the behavior or Trial, please let me know.

# `cma` cannot handle distributions that contain just a single value, so that
# the parameter values for such distributions are sampled by `sample_independent`.

This comment has been minimized.

Copy link
@toshihikoyanase

toshihikoyanase Jul 26, 2019

Author Collaborator

As a result of offline discussion with @sile, I fixed the comment as follows:

# `cma` cannot handle distributions that contain just a single value, so we skip
# them. Note that the parameter values for such distributions are sampled in
# `Trial`.

The Trial actually has code to handle distributions with a single value at
https://github.com/pfnet/optuna/blob/master/optuna/trial.py#L178, https://github.com/pfnet/optuna/blob/master/optuna/trial.py#L218 and so on.

class CmaEsSampler(BaseSampler):
"""A Sampler using cma library as the backend.
Note that parallel execution of trials may degrade the optimization performance of CMA-ES,

This comment has been minimized.

Copy link
@g-votte

g-votte Jul 25, 2019

Collaborator

This piece of information seems less important than the example. Would you mind putting this paragraph after the example?

This comment has been minimized.

Copy link
@toshihikoyanase

toshihikoyanase Jul 26, 2019

Author Collaborator

Thanks. I fixed it in 119372d.

@sile sile added this to the v0.14.0 milestone Jul 25, 2019
@sile sile added the feature label Jul 25, 2019
@g-votte g-votte merged commit 475efb1 into master Jul 26, 2019
7 checks passed
7 checks passed
ci/circleci: checks Your tests passed on CircleCI!
Details
ci/circleci: codecov Your tests passed on CircleCI!
Details
ci/circleci: document Your tests passed on CircleCI!
Details
ci/circleci: tests-python27 Your tests passed on CircleCI!
Details
ci/circleci: tests-python35 Your tests passed on CircleCI!
Details
ci/circleci: tests-python36 Your tests passed on CircleCI!
Details
ci/circleci: tests-python37 Your tests passed on CircleCI!
Details
@g-votte g-votte deleted the cma-sampler branch Jul 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.