Classifying MNIST Dataset using Different Classification Algorithms

This repository is about some Classification Algorithms for MNIST

All the basic machine learning models are made from scratch except 2 Layer Neural Network and Basic CNN model.

Requirements

• Python(3.52 or 2.7.14)
• Numpy(for all the matrix operations)
• Matplotlib(to plot graph and for visualization)
• Sklearn(Just to shuffle the data)
• Keras(2.0.8)(Required in 2_layer_neural_network and Simple_CNN_Model)
• tensorflow-gpu(1.3.0)(Required in 2_layer_neural_network and Simple_CNN_Model)

Algorithm with their Papers

• K-Nearest Neighbours:

  • KNN Model-Based Approach in Classification

• Multiclass Sofmax Classification(1 Layer Neural Network):

  • Gated Softmax Classification

• Radial Basis Analysis Over Linear Classifier:

  • Efficient and Accurate Gaussian Image Filtering Using Running Sums

• 2 Layer Neural Network (Relu Activation in all layer except Softmax at the top):

  • Convergence Analysis of Two-layer Neural Networks with ReLU Activation

• Simple CNN Model(The model in not exact Le_Net Architecture.):

  • Le_Net

Details

train.csv contains 42000 labeled images of digits from 0 to 9. The image is 28*28(784) in dimensions. Each image is a black and white image. For most algorithms we have taken 41000 images for training and rest 1000 for testing. Except a simple filter no preprocessing is done because the aim of the project was to understand the different classification techniques.

Accuracy over test set

• Simple_CNN: 99.4%
• 2 Layer NN: 98%
• RBA over Linear Classifier: 91.3%
• Sofmax Classification(1 layer nn): 90.8%
• KNN: 89.5%

Please feel free to contact me if you have any questions or suggestions.