Long Short-Term Memory

[Lunderberg]

https://en.wikipedia.org/wiki/Long_short-term_memory

This looks pretty cool. I'll see if I can think of a way to implement it while still keeping the "single loop over connections" that we have.

[Lunderberg]

I can imagine a new type of node, which retains its current value, rather than being reset when it is used as an output. There would also be a new type of connection, which is used to reset the memory nodes.

What I'm not sure about would be the ordering of the MemoyReset connection in the list of connections. It would need to occur before any of the other inputs to the memory node. That way, the new value to be remembered gets all of the inputs. What I'll have to think about is whether this additional restriction in the ordering can cause the list to be un-orderable.