Tweak MNIST example to get rid of first dropout layer

[gigasquid]

This modification will yield better training results for this example which I think is important to show to new comers to the project.

Here are the results on my computer with and without gpu, before and after the change:

before with gpu

Parameter count: 849780
Saving network to trained-network.nippy
Classification accuracy: 0.8026
Saving network to trained-network.nippy
Classification accuracy: 0.7925
Classification accuracy: 0.7751
Classification accuracy: 0.7625
Classification accuracy: 0.7672
Classification accuracy: 0.7555
Classification accuracy: 0.7595
Classification accuracy: 0.7655
Classification accuracy: 0.7707
Classification accuracy: 0.7767
Classification accuracy: 0.7765
Classification accuracy: 0.778
Classification accuracy: 0.7961
Classification accuracy: 0.7854
Classification accuracy: 0.7873
Classification accuracy: 0.7818
Classification accuracy: 0.7845
Classification accuracy: 0.7919
Classification accuracy: 0.795
Classification accuracy: 0.7847
Classification accuracy: 0.7887
Classification accuracy: 0.7883
Classification accuracy: 0.7879
Classification accuracy: 0.7926
Classification accuracy: 0.7952
Classification accuracy: 0.7945
Classification accuracy: 0.7926
Classification accuracy: 0.7911
Classification accuracy: 0.7952
Classification accuracy: 0.7846
Classification accuracy: 0.7941
Classification accuracy: 0.7858
Classification accuracy: 0.7878
Classification accuracy: 0.7971
Classification accuracy: 0.7863
Classification accuracy: 0.7933
Classification accuracy: 0.7936
Classification accuracy: 0.7933
Classification accuracy: 0.7918
Classification accuracy: 0.7849
Classification accuracy: 0.7878
Classification accuracy: 0.7879
Classification accuracy: 0.7983
Classification accuracy: 0.7925
Classification accuracy: 0.7937
Classification accuracy: 0.788
Classification accuracy: 0.7922
Classification accuracy: 0.7919
Classification accuracy: 0.7933
Classification accuracy: 0.7856
Classification accuracy: 0.7851
Classification accuracy: 0.7948

before with cpu

Parameter count: 849780
Saving network to trained-network.nippy
Classification accuracy: 0.7937
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.7962
Saving network to trained-network.nippy
Classification accuracy: 0.7143
Classification accuracy: 0.7196
Classification accuracy: 0.7348
Classification accuracy: 0.7497
Classification accuracy: 0.7549
Classification accuracy: 0.7479
Classification accuracy: 0.7471
Classification accuracy: 0.7486
Classification accuracy: 0.7422
Classification accuracy: 0.7489
Classification accuracy: 0.7387
Classification accuracy: 0.7439
Classification accuracy: 0.7456

after without first dropout with gpu

Parameter count: 849780
Saving network to trained-network.nippy
Classification accuracy: 0.9127
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9386
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9469
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9518
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9559
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9609
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9665
Saving network to trained-network.nippy
Classification accuracy: 0.9658
Classification accuracy: 0.9651
Saving network to trained-network.nippy
Classification accuracy: 0.9695
Saving network to trained-network.nippy
Classification accuracy: 0.9692
Saving network to trained-network.nippy
Classification accuracy: 0.97
Saving network to trained-network.nippy
Classification accuracy: 0.9613
Saving network to trained-network.nippy
Classification accuracy: 0.9716
Saving network to trained-network.nippy
Classification accuracy: 0.9683
Classification accuracy: 0.9704
Classification accuracy: 0.9278
Classification accuracy: 0.9637
Saving network to trained-network.nippy
Classification accuracy: 0.9721
Saving network to trained-network.nippy
Classification accuracy: 0.9708
Saving network to trained-network.nippy
Classification accuracy: 0.9742
Saving network to trained-network.nippy
Classification accuracy: 0.9726
Saving network to trained-network.nippy
Classification accuracy: 0.9744
Saving network to trained-network.nippy
Classification accuracy: 0.9616
Classification accuracy: 0.8867
Classification accuracy: 0.9727
Saving network to trained-network.nippy
Classification accuracy: 0.976
Saving network to trained-network.nippy
Classification accuracy: 0.9757
Saving network to trained-network.nippy
Classification accuracy: 0.9766
Saving network to trained-network.nippy
Classification accuracy: 0.9753
Saving network to trained-network.nippy
Classification accuracy: 0.9777
Saving network to trained-network.nippy
Classification accuracy: 0.9762
Saving network to trained-network.nippy
Classification accuracy: 0.9779
Saving network to trained-network.nippy
Classification accuracy: 0.9764
Saving network to trained-network.nippy
Classification accuracy: 0.9782
Saving network to trained-network.nippy
Classification accuracy: 0.9782
Saving network to trained-network.nippy
Classification accuracy: 0.9784
Saving network to trained-network.nippy
Classification accuracy: 0.9779
Saving network to trained-network.nippy
Classification accuracy: 0.9809
Saving network to trained-network.nippy

After change with cpu only

Parameter count: 849780
Saving network to trained-network.nippy
Classification accuracy: 0.8843
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9276
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9305
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9476
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9512
Saving network to trained-network.nippy
Classification accuracy: 0.9423
Saving network to trained-network.nippy
Classification accuracy: 0.9525
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.955
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.9553
Saving network to trained-network.nippy
Saving network to trained-network.nippy
Classification accuracy: 0.959
Saving network to trained-network.nippy

[harold]

Oh, I didn't realize this was outside the unit test suite. This is great! Thanks.

[harold]

This is the line I was thinking of:
https://github.com/thinktopic/cortex/blob/master/src/cortex/verify/nn/train.clj#L29

Removing that would take training through droput out of the test suite, which would be a mistake imo.

hth.