- This experiments are being implimented in Keras by using Google colab GPU.
- To experiment with 2 hidden layes architecture we choose Neuron size of 512 and 256
2 hidden layers ==> 784-> 512,256 -->10
- To experiment with 3 hidden layes architecture we choose Neuron size of 512,256 and 128
3 hidden layers ==> 784-> 512,256,128 -->10
- To experiment with 5 hidden layes architecture we choose Neuron size of 512,256,128,64 and 32
5 hidden layers ==> 784-> 512,256,128,64,32 -->10
- As we know till now, ReLU is one of the best activation function. Hence, we set our acivation function as ReLU.
- And best optimizer is Adam. When we train our 2 layers, 3 layers and 5 layers NN with ReLU activation function on Adam Optimizer with Dropout, we got a very good Test score as compaire to other models.
- As MNIST is not a very large dataset, we are not able to see far difference in the Test accuracy. This can be observe quite when we try our hand with large scale datasets.