Definition of "compositional" in "compositional optimizers"

[sgbaird]

At one point, I may have understood this, but I find myself often wondering and asking again: what does "compositional" in "compositional optimizer" refer to? Part of my confusion probably stems from my materials informatics background, where composition often refers to the chemical make-up (i.e. chemical formula) of a particular compound. In other cases, it just implies that the contribution of individual components sums to one.

https://github.com/huawei-noah/HEBO/tree/master/CompBO

[AntGro]

Hi! "Compositional" in "compositional optimizer" refers to the nested form of the objective function.
Consider the following optimisation problem: max_x E[f(E[g(x)])] where E denotes expected value.
A "simple" optimiser (e.g. Adam, Adagrad...) would deal with that problem by considering f(E[g(x)]) := h(x) and solve: max_x E[h(x)], while a "compositional" optimiser (e.g. CAdam) will exploit the nested form explicitly (working with f and g) to perform the optimisation.

Compositional optimiser --> exploits compositional form of the objective function.

[sgbaird]

@AntGro wonderful! Thanks so much for clarifying this for me