adding evolutionary algorithm optimizer

[mikkokotila]

@x94carbone thanks very much for suggesting this. Any contributions you would like to make in this respect are definitely most welcome.

Do you already have an idea what signals we would need to feed into an evolutionary algorithm and if yes, in which format those signals are needed out of Talos as part of the Scan() stage.

[matthewcarbone]

Possibly. There is s minor concern about them. If Talos is to be the kind of HP tuning program you can run on your desktop, it might not be entirely feasible to implement them. Evolutionary algorithms will improve performance but are incredibly expensive.

Part of my current work involves training on multiple GPU's where such an implementation will be possible. For work done on a standard computer CPU it might not be.

Does your view of the Talos project potentially involve this kind of expansion? I likely will be writing code for this regardless (kind of slammed right now but I'll get to it). Perhaps a package that runs on top of Talos would be nice for that?

[mikkokotila]

I think that Talos should meet both needs; a single lowish end laptop and multi GPU environments. The evolutionary algorithm would fall under "reduction" methods in the current architecture. My thinking is that there would be several methods to choose from. In other words, it would be totally ok to have something that is restrictive in terms of machine spec.

A package that runs on top of Talos sounds perfect! ps. I took some time last night to study the topic enough to understand clearly the approach and it does seem great indeed :)

[mikkokotila]

This is connected with #40 and #45

[mikkokotila]

From the next major release onwards v.0.6 it will be very easy to add new optimizers. Just a heads up for now, more in the docs later.

[mikkokotila]

Closing this, as now anyone can add their own custom optimizers easily. See more in the corresponding docs section.