Cross Entropy?

[Zelex]

Looking through the code, not sure if its using this. If not, why not?

[codeplea]

Hi Zelex,

No, Genann doesn't use the cross entropy method. Genann implements very basic, standard back-propagation.

Why? Three reasons.

1. New features should be justified. i.e. if I'm going to add a new feature, I need a good reason to add it, not a good reason to leave it out.

2. I personally don't use back-prop at all in my personal applications for Genann (I'm doing reinforcement learning). The back-propagation was added mostly as a sanity check and because others expect it.

3. One of Genann strengths is that it's small and hackable. If we implement cross entropy, then why not also implement the 500 other techniques and optimizations in common use?

Anyway, I'm not very familiar with cross entropy, so if I'm missing a compelling argument for it, please let me know. I'd like to hear your input.

Best,
Lewis