04 31th January, Wednesday

I have managed to get t-SNE working for the traffic sign network and have some nice figures showing the clusterings of the various signs in the weight space. Since this network is much smaller, I was able to train it up to 90% accuracy to get the figures.

My reasoning for getting these figure is that I want to attempt to develop a method of visualising what the weight space is learning. In my example I just have images that have been run through the network to generate a 64-dimensional vector that represents the output of the last layer before softmax, and then have mapped all the 64D points from 300 images into 2-dimentional space using t-SNE.

My reasoning is that if I can now also map though extra 64-D vectors with the images that represent the dimensions that correspond to the highest weights in the last layer I can get an idea visually of where the most weighting is and so where the features are that the network is learning.

Then, if the weight points generated from the images don't sit on the areas that the network has learned "the most" then we could then suspect that these other areas that our test set doesn't match up with are the locations of maliciously trained features.

Whether or not this is a sensible/reasonable idea I am unsure, but I've going to look to implement it myself as see what results I get from it.