29 9th March, Friday

Last night I got my first interesting set of results. I discovered that my testing a range of different initialisations of the networks I was able to improve them slightly over many generations.

The results where as I had hoped for, I used:

10 generations, populations size 5, 20 initialisations per network - 1000 networks, 24H on BC4 GPU

And saw performance on malicious data go from ~62 to ~59%, and in addition to this accuracy on MNIST data saw an increase of ~91.5 to ~92% over the same period. From my research I would have liked to perform the experiment with:

100 generations, populations size 20, 100 initialisations per network - 200,000 networks, estimated ~200 Days BC GPU

But I established that such an experiment was infeasible, but given that my margins of improvement are relatively low I have decided to rerun the experiment for longer to see if the gains in performance increase further. Currently running is:

30 generations, populations size 5, 20 initialisations per network - 3000 networks, ~72H on BC4 GPU

I will be writing up about my findings in the experiment section of my thesis, and very much hope to see a continuation of the first experiment in the longer second one. Interestingly, the graphs of original and synthetic accuracy seem to be almost mirrored - suggesting that in rewriting the network to perform worse on synth data we are causing improvements in the original data too, which is an interesting result as networks are not being selected based on performance on original set. Perhaps the network is being poisoned to some extent by the random data?