Add IPOP-CMA-ES support in CmaEsSampler.

[c-bata]

Motivation

As the benchmark of two-dimensional Rastrigin function shows, IPOP-CMA-ES outperforms non-restarting CMA-ES on multimodal functions.

Auger, A., Hansen, N.: A restart CMA evolution strategy with increasing population size. In: Proceedings of the 2005 IEEE Congress on Evolutionary Computation (CEC’2005), pp. 1769–1776 (2005a)

Description of the changes

• Add restart_strategy argument. Currently accepts ipop or None only. I will implement BIPOP-CMA-ES in the following pull request.
• After each optimizer.tell(solutions) call, check whether optimizer.should_stop() method returns True or not.
  • If True, re-initialize CMA object with multiplied popsize by inc_popsize. inc_popsize is 2 by default. According to the paper, it reveal similar performance for factors between 2 and 3.
  • Sample initial starting point uniformly at each restart.

TODO:

This PR is ready for review.

• Benchmark.
  • Script: https://github.com/CyberAgent/cmaes/blob/master/benchmark/runner.sh
  • Command: BUDGET=2500 REPEATS=10 DIM=2 ./benchmark/runner.sh rastrigin ./kurobako-report.json
• Add test case.
• Merge Avoid to sample the value which equals to upper bound #1558 into master.
• Release the new version of cmaes to PyPI and conda-forge.
  • https://github.com/CyberAgent/cmaes/releases/tag/v0.6.0
• Update install_requires.

[c-bata]

Refs: https://github.com/CyberAgent/cmaes/blob/master/examples/bipop_cmaes.py

[c-bata]

I implemented IPOP-CMA-ES and BIPOP-CMA-ES on Goptuna. A benchmark result of two-dimensional Rastringin function with 10000 trials is below.

It achieved almost the same performance on this benchmark.

[c-bata]

memo: I comment the reason why I choose this design here.

In pycma's implementation, x0 accepts not only the float values. It also accepts callable object (e.g. np.random.rand) and string (e.g. "np.random.rand" which is evaluated by eval()). But I guess it's enough for almost every usecases to sample x0 uniformly at each restart.

[HideakiImamura]

Thanks for the update! LGTM!

[hvy]

Thanks a lot for this very interesting PR.

The implementation is simple and the logic looks good for the most part. I left one question though regarding the system attribute if you could take a look.

[hvy]

We decided to set experimental decorator at our off-line discussion.
Thank you @HideakiImamura!

Might be good for future record to leave a comment on why. 🙂

[c-bata]

Might be good for future record to leave a comment on why. 🙂

We set experimental decorator because there are some room for discussion about API (e.g. it might be better to make inc_popsize float). And we might find the room for improvements after dogfooding this feature.

[c-bata]

PTAL.

[hvy]

Thanks for your update. I thought that it basically looked good but noticed that the current logic will when collecting previous trial history include trials from "previous restarts". This is because the generation count is reset on restart (since a CMA object is re-instantiated) and we simply collect all trials belonging to the "same generation" and don't care if they stem from different restarts. Is this intended? Maybe we have to include the restart count after all and filter trials based on it in addition to the generation?

[c-bata]

Oh, thanks. You are completely correct. It is not intended behavior.
To fix the bug, we need to add n_restarts or something optimizer unique information into the key of system_attrs.

[hvy]

I'm really sorry after having asked you to remove n_restarts but reverting 4947c97 and also filtering based on this count when constructing solution_trials could be a reasonable fix?

[c-bata]

I'm really sorry after having asked you to remove n_restarts but reverting 4947c97

No. It's not your fault. It's just my mistake.

reverting 4947c97 and also filtering based on this count when constructing solution_trials could be a reasonable fix?

Yeah, sounds reasonable. I'll fix the problem. Thank you!

[c-bata]

I fixed the bug at cd2e10c.

[hvy]

Thanks for the quick fix and the long running effort. Changes LGTM!

Also leaving the results from a small experiment run locally. It's clear that the restart makes it more explorative escaping a local minimum.

Vanilla

IPOP

import optuna

# Himmelblau
def objective(trial):
    x = trial.suggest_float("x", -6, 6)
    y = trial.suggest_float("y", -6, 6)
    return (x ** 2 + y - 11) ** 2 + (x + y ** 2 - 7) ** 2


if __name__ == "__main__":
    study = optuna.create_study(sampler=optuna.samplers.CmaEsSampler(restart_strategy="ipop"))
    study.optimize(objective, n_trials=1000)
    optuna.visualization.plot_slice(study).show()

[hvy]

@HideakiImamura might want to have another look as well, so I'll leave the final word to him.

[c-bata]

I re-execute Rastrigin (2D) benchmark function at d35a3dc.

[HideakiImamura]

Thanks for all of your efforts. LGTM, again!

[c-bata]

Thank you for your thorough review!