21 25th February, Sunday

Today I'm outlining the experiments that will need doing to analyse the effect of capacity on memoized data.

The key decisions to make are:

What network architecture do I want to train on and how am I going to scale it in breadth/depth?

What increments of scaling are suitable for this?

What dataset should I use and how many synthetic images should be used? MNIST first, then CIFAR10 if MNIST experiments are success. I think if I do one set of experiments with a mal set ~5% of original and one set with the mal set being the same size this could be interesting.

I'm aiming to come up with sensible choices for these today, and by the end of tomorrow would like to have some automated tests set up so that I can start gathering data, as it is possible that I answer some of these questions wrong, so allowing time to adjust is necessary.