Enh/differential cgp

[jakobj]

This PR add a new node CGPParameter that, in contrast to the CGPConstantFloat can store arbitrary values. This is achieved by keeping (and passing on) a dictionary in each individual which for each position in the graph stores a corresponding CGPParameter value that can be changed, e.g., by copying values from an optimized torch class.
It's fairly clean in terms of implementation, untested with regards to performance and questionable whether this implementation supports efficient evolution. We should hence discuss it in detail @mschmidt87

In particular: does it make sense to store the values in an individual, where values are only passed on from one individual to the next? this means a) good values can get lost if the individual doesn't manage to survive b) bad values can lead to evolution disregarding constant nodes if as soon as they are switched on they lead to terrible fitness values; maybe it would make more sense to store the values at the population level?

[mschmidt87]

Sorry for the long delay in replying. Indeed, it is a good question how differentiation would be most useful in the CGP framework. I think that the approach you're suggesting makes most sense. In fact, I am not sure how storing the values at the population level would work in practice. The problem here seems to be that the optimized parameter values are only optimal within a certain graph, i.e., an individual, don't you think?

Regarding the use cases, I think that the usefulness of gradient-based optimization lies in judging the "potential" of an individual: a parametrized individual can yield very different fitness values depending on its parametrization. Thus, to judge the fitness of an individual, we should optimize these parameters. Whether this should be done excessively in each generation (i.e. a full optimization with many iterations) or just with few iterations to get some estimate, would be left to the user.
As far as I understand, your implementation would support this use case, wouldn't it?

[jakobj]

I agree, the tuned parameters only make sense within a particular computational graph. when tuned parameters are passed on from one individual to the next, mutations can optimize that graph given those parameters. this seems intuitive.

Also fully agree on introducing local search, i.e., grad descent here, lies in judging the potential of an individual. and how much this is explored is a hyperparameter that can be chosen by the user (e.g., https://github.com/jakobj/python-gp/pull/57/files#diff-5fc60d716681a75e11aabee3debd4e12R60 ff could proceed for more than 10 steps). should we provide a "template" local search function that performs gradient descent using a particular torch optimizer?

@mschmidt87 could you confirm that you agree with the general approach? in that case i would prepare this PR for detailed review.

[mschmidt87]

@mschmidt87 could you confirm that you agree with the general approach? in that case i would prepare this PR for detailed review.

Yes, your suggested approach makes sense to me.

[jakobj]

closing in favor of #90. discussion should be continued there.