26 3rd March, Saturday

I've been running additional experiments today, I also found out today that because my synthetic images were based on the MNIST images the data labels were heavily skewed. I've implemented pseudo random labels now and the experimental data gained as a result should be a lot more valuable. I currently have a 12h job running on BC, should be able to report back tomorrow with whether that was successful or not. See report for details, this is currently experiment 2 running. I've put some of the results from the slightly broken code in the thesis but will tidy this up tomorrow if I can get some better results to put in.

Quick update here about experiment 2:

My code crashed after an hour as it tried to generate a second generation, but that is now fixed and the test is rerunning. The results I got out showed me that for the first 10 random architectures, so each with random connections removed as in the previous optimisation paper, the malicious accuracies were as follows:

76%, 35%, 45%, 52%, 70%, 51%, 31%, 80%, 96%, 57%, with all of the original training at 97%

From this the next generation is taken from the best 4, so 31%, 35%, 45%, 52% with 1 in 20 connections randomly either added back or removed to allow for mutation.

Although this is exciting, these results could be more correlated with the number of connections left in the network, so I will need to plot synthetic accuracy vs training parameters to distinguish if whether the network architecture or the capacity is the cause. Either way these results are interesting.