New integration to EvoJax

[dietmarwo]

See https://github.com/google/evojax/tree/main/evojax/algo

I added two versions of the algo to EvoJax:

• a wrapper of the fcmaes C++ implementation
• a new JAX-based implementation

The algorithm performs exceptionally well for the EvoJax benchmarks.

Additionally there are quite interesting results when using it as part of an QD-algorithm,
see google/evojax#52 (not yet merged)

It also has been added as QD-Emitter in fcmaes, see
https://github.com/dietmarwo/fast-cma-es/blob/master/tutorials/Diversity.adoc

Finally it was applied at
https://www.esa.int/gsp/ACT/projects/spoc-2023/ (Surface Exploration with Morphing Rovers) and finally ranked 3rd (Team fcmaes).

[nomuramasahir0]

Thank you for adding it to EvoJax and sharing your interesting results!
I am particularly pleased to see that CR-FM-NES is performing well across a range of problems, as I have been focused on its practical performance.
Please let me know if there are any improvements needed in the algorithm 😎

[dietmarwo]

Hallo, attached are my slided from the ESA optimization workshop were I talked about  CR-FM-NES. CR-FM-NES is one of the very few algorithms where I don't have a good alternative for a specific class of problems. For CR-FM-NES these are mainly problems with a huge number of decision variables. Similar to CMA-ESit can be combined with Map-Elites (see https://arxiv.org/abs/1912.02400) for quality diversity optimization,if there are many variables.  For the task https://optimise.esa.int/challenge/spoc-2-morphing-rovers/p/morphing-roversthe others created a simplified rover driving model by hand, and trained the NN from that. A huge amount of work, and essentially it makes no sense, because their hand-crafted driving modelwas better that the trained NN in the end.  I just wrote these 5 lines of code  pfit = fcmaes.evaluator.parallel(wrapper(fitness), workers=mp.cpu_count())opt = fcmaes.crfmnescpp.CRFMNES_C(dim = dim, bounds=bounds, x0=x0, input_sigma=0.01, popsize=48)for i in range(1000):    xs = opt.ask()    ys = pfit(xs)    opt.tell(ys and this was sufficient to lead the competition most of the time. Thanks to CR-FM-NES. Regarding improvements: Do you have any ideas in mind? I saw https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7354820/ for CMA-ES, not sure if such an idea is applicable.   "Improvements" I have implemented in fcmaes are parallel function evaluation + parallel restart, anask/tell interface and the integration with Map-Elites, but these are straightforward.  Best Regards, Dietmar Am Donnerstag, 16. November 2023 um 19:39:14 MEZ hat Masahiro Nomura ***@***.***> Folgendes geschrieben: Thank you for sharing your interesting results! I am particularly pleased to see that CR-FM-NES is performing well across a range of problems, as I have been focused on its practical performance. Please let me know if there are any improvements needed in the algorithm 😎 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[nomuramasahir0]

It may not be easy, but tuning learning rates in CR-FM-NES may be advantageous, if the problem is multimodal and/or noisy.
We recently developed learning rate adaptation for CMA-ES (https://arxiv.org/abs/2304.03473; GECCO'23 best paper nominated). Although applying this method to CR-FM-NES may be a bit tedious, I believe simply changing learning rate (i.e. decreasing it by a scalar factor) is effective if the problem is difficult.

[nomuramasahir0]

Note that, however, a small learning rate (and learning rate adaptation) may need to take a sufficient evaluation budget. So if the evaluation of the objective function is not cheap, this is not a very attractive option.