27 5th March, Monday

Experiments from yesterday have given me my first set of interesting results. Section 0.5.3 of my thesis shows a table of results for the first generation of my third experiment, where I have trained 10 simple networks, each with ~670K connections and half of those removed. The results show networks with similar effective capacities but different structures have hugely varying capacities to memorise the random data fed into the network, ranging from 11% to 86% (where you get 10% from randomly guessing one of the 10 classes). This is just over 10 random networks.

I have been using a genetic algorithm to evolve the networks but looking at these results it is now clear that capacity doesn't seem to be the deciding factor and that the layout of connections is a lot more important. So an obvious follow up to this would be to retain a lot more information from parents to try to preserve this, ie entire weight layer structures. Given there are small mutations this seems to make sense as a next step.

I am still get to get actual results from the first experiment on blue crystal, not sure why I think I am try to run too many scripts in the same directory in Blue Crystal at once.