Add BIPOP-CMA-ES support in CmaEsSampler

[cross32768]

Motivation

Current CmaEsSampler has IPOP-CMA-ES as the restart strategy.
BIPOP-CMA-ES is a kind of extension of IPOP-CMA-ES to improve performance in some problems by adding device about the increment of population size.
I want to consider adding BIPOP-CMA-ES as new option.

Description of the changes

• Add BIPOP-CMA-ES option in optuna/samplers/_cmaes.py . Users can use this new feature by optuna.samplers.CmaEsSampler(restart_strategy="bipop")
  • Add population increment logic of BIPOP-CMA-ES in sampler_relative method of CmaEsSampler
  • Add some system_attrs to manage counter variable required for BIPOP-CMA-ES
  • Change the return format of _restore_optimizer method of CmaEsSampler and add code to load counter variable like n_restarts other place
  • Change the instance variable name _popsize to _initial_popsize because its more adequate
  • Add comments and docstring to describe about BIPOP-CMA-ES
• Add test for BIPOP-CMA-ES in tests/sampler_tests/test_cmaes.py

[cross32768]

@c-bata Could you review this PR?

[cross32768]

This gif shows behavior of BIPOP-CMA-ES on himmelblau function.

[github-actions]

This pull request has not seen any recent activity.

[HideakiImamura]

@Alnusjaponica Could you review this PR?

[Alnusjaponica]

Sorry for my delayed reply. I made some small suggestions mainly for code comments. If you don't agree with them, please disregard them.

[cross32768]

@Alnusjaponica Thanks you for comments. I fixed all points.

[codecov-commenter]

Codecov Report

Merging #4464 (bc3f550) into master (caf38ee) will increase coverage by 0.00%.
The diff coverage is 94.59%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@           Coverage Diff           @@
##           master    #4464   +/-   ##
=======================================
  Coverage   90.89%   90.89%           
=======================================
  Files         184      184           
  Lines       13953    14009   +56     
=======================================
+ Hits        12682    12733   +51     
- Misses       1271     1276    +5     

Impacted Files	Coverage Δ
optuna/samplers/_cmaes.py	97.20% <94.59%> (-1.06%)	⬇️

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[github-actions]

This pull request has not seen any recent activity.

[c-bata]

Sorry for the delay in my review. It works fine, but there seems to be room for improvement on how to restore the optimizer. I left one suggestion.

[github-actions]

This pull request has not seen any recent activity.

[github-actions]

This pull request has not seen any recent activity.

[c-bata]

I checked the performance with Rastrigin-2D function, which is the similar settings as c-bata/goptuna#136, and confirmed that BIPOP-CMA-ES works as expected.

https://gist.github.com/c-bata/0e739d661e21a5bc4c2ddf2141bf6a9e

[c-bata]

Thanks for the update! The update looks great to me.

Just for the record, I confirmed that BIPOP-CMA-ES does not take over the optimization history of IPOP-CMA-ES with the following Python script.

import optuna
from optuna_dashboard import run_server


seed = 1
storage = optuna.storages.InMemoryStorage()


def objective(trial: optuna.Trial) -> float:
    x = trial.suggest_float('x', -10, 10)
    y = trial.suggest_float('y', -10, 10)
    system_attrs = trial.storage.get_trial_system_attrs(trial._trial_id)
    for key, value in system_attrs.items():
        if "optimizer" in key:
            trial.set_user_attr(key, f"{value[:5]}...")
        else:
            trial.set_user_attr(key, value)
    return x ** 2 + y ** 2


if __name__ == '__main__':

    study1 = optuna.create_study(sampler=optuna.samplers.RandomSampler(seed=seed), storage=storage, study_name="random-to-bipop")
    study1.optimize(objective, n_trials=100)
    study1 = optuna.load_study(sampler=optuna.samplers.CmaEsSampler(restart_strategy="bipop", seed=seed), storage=storage, study_name="random-to-bipop")
    study1.optimize(objective, n_trials=100)

    study2 = optuna.create_study(sampler=optuna.samplers.CmaEsSampler(restart_strategy="ipop", seed=seed), storage=storage, study_name="ipop-to-bipop")
    study2.optimize(objective, n_trials=100)
    study2 = optuna.load_study(sampler=optuna.samplers.CmaEsSampler(restart_strategy="bipop", seed=seed), storage=storage, study_name="ipop-to-bipop")
    study2.optimize(objective, n_trials=100)

    for t1, t2 in zip(study1.trials[100:], study2.trials[100:]):
        assert t1.params == t2.params and t1.value == t2.value
    run_server(storage, port=8080)