-
Notifications
You must be signed in to change notification settings - Fork 0
/
mnist.html
169 lines (162 loc) · 7.9 KB
/
mnist.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
<!DOCTYPE HTML>
<!--
-->
<html>
<head>
<title>Xiao Tianyou Theo Portfolio</title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no" />
<link rel="stylesheet" href="assets/css/main.css" />
<noscript><link rel="stylesheet" href="assets/css/noscript.css" /></noscript>
</head>
<body class="is-preload">
<!-- Wrapper -->
<div id="wrapper">
<!-- Header -->
<header id="header">
<a href="index.html" class="logo">Theo</a>
</header>
<!-- Nav -->
<nav id="nav">
<ul class="links">
<li><a href="index.html">Projects</a></li>
<li><a href="handwriting_recognition.html">Handwriting</a></li>
<li><a href="binary_search.html">Binary_Search</a></li>
<li><a href="investment.html">Investment</a></li>
<li class = "active"><a href="mnist.html">MNIST</a></li>
<li><a href="passion_computing.html">Passion_Programming</a></li>
<li><a href="myself.html">Myself</a></li>
</ul>
<ul class="icons">
<li><a href="https://github.com/theo-xiao-sg" class="icon brands fa-github" target="_blank"><span class="label">GitHub</span></a></li>
</ul>
</nav>
<!-- Main -->
<div id="main">
<!-- Post -->
<section class="post">
<header class="major">
<h1>COMPARING KNN, NN AND CNN USING MNIST AND Fashion-MNIST DATASETS</h1>
<p><br /></p>
<a href="https://github.com/theo-xiao-sg/mnist_recognition" target="_blank" class="button">Source code</a>
<p><br /></p>
</header>
<!-- Lists -->
<h2>Data</h2>
<p>The <b>MNIST dataset</b> is a very well-known image dataset of handwritten digits (<a href="https://en.wikipedia.org/wiki/MNIST_database" style="color: blue;">link</a>). It has 60,000 training images and 10,000 testing images, each of which is a grayscale 28 x 28 sized image (as shown below). Although it is a good beginner’s dataset to try different machine learning models, many researchers found this dataset is a bit simple for many machine learning algorithms.</p>
<img src="images/mnist_illustration.jpg" style="width: 70%; height: auto;">
<p>The <b>Fashion-MNIST dataset</b> is a collection of images of fashion items, like T-shirts, dresses, shoes etc (as shown below)(<a href="https://en.wikipedia.org/wiki/Fashion_MNIST" style="color: blue;">link</a>). Since it represents much more complex computer vision task, many researchers use it as a replacement for the original MNIST digits dataset. It shares the same image size (28x28 pixels and grayscale) and has 60,000 training and 10,000 testing images. </p>
<img src="images/fashion_mnist_illustration.jpg" style="width: 70%; height: auto;">
<p>Fashion-MNIST dataset has 10 categories of output labels: 0: T-shirt/top, 1: Trouser, 2: Pullover, 3: Dress, 4: Coat, 5: Sandal, 6: Shirt, 7: Sneaker, 8: Bag, 9: Ankle Boot.</p>
<h2>Results</h2>
<h3>MNIST dataset Results</h3>
<p>I compared three models: KNN, Neural Network (NN), and Convolutional Neural Network (CNN), by classifying the MNIST dataset. I stated the accuracies on the testing dataset for all the three models as follows:</p>
<h3>a. KKN model:</h3>
<!-- Preformatted Code -->
<style>
pre {
tab-size: 4;
}
</style>
<pre><code>model training for num_neighbors: 1; accuracy: 96.91%
model training for num_neighbors: 3; accuracy: 97.05%
model training for num_neighbors: 5; accuracy: 96.88%
model training for num_neighbors: 7; accuracy: 96.94%
model training for num_neighbors: 9; accuracy: 96.59%
</code></pre>
<h3>b. Neural Network model:</h3>
<!-- Preformatted Code -->
<style>
pre {
tab-size: 4;
}
</style>
<pre><code>model training for hidden_layer_sizes: 100; accuracy: 96.85%
model training for hidden_layer_sizes: 300; accuracy: 97.79%
model training for hidden_layer_sizes: 500; accuracy: 96.98%
model training for hidden_layer_sizes: 700; accuracy: 97.85%
model training for hidden_layer_sizes: 900; accuracy: 97.00%
</code></pre>
<h3>c. CNN model:</h3>
<!-- Preformatted Code -->
<style>
pre {
tab-size: 4;
}
</style>
<pre><code>CNN with 2 convolutional layers; accuracy: 99.06%
CNN with 4 convolutional layers; accuracy: 99.28%
</code></pre>
<p>For MNIST dataset, KNN had accuracy of 96.59% -97.05%, and Neural network model achieved 96.85% - 97.85%. As expected, CNN model achieved the highest accuracy of 99.06% - 99.28% among all three models. </p>
<h3>Fashion-MNIST dataset Results</h3>
<p>I compared three models: KNN, Neural Network (NN), and Convolutional Neural Network (CNN), by classifying the Fashion-MNIST dataset. I stated the accuracy on the testing dataset for all three models as below:</p>
<h3>a. KKN model:</h3>
<!-- Preformatted Code -->
<style>
pre {
tab-size: 4;
}
</style>
<pre><code>model training for num_neighbors: 1; accuracy: 84.97%
model training for num_neighbors: 3; accuracy: 85.41%
model training for num_neighbors: 5; accuracy: 85.54%
model training for num_neighbors: 7; accuracy: 85.40%
model training for num_neighbors: 9; accuracy: 85.19%
</code></pre>
<h3>b. Neural Network model:</h3>
<!-- Preformatted Code -->
<style>
pre {
tab-size: 4;
}
</style>
<pre><code>model training for hidden_layer_sizes: 100; accuracy: 88.68%
model training for hidden_layer_sizes: 300; accuracy: 89.45%
model training for hidden_layer_sizes: 500; accuracy: 89.67%
model training for hidden_layer_sizes: 700; accuracy: 89.34%
model training for hidden_layer_sizes: 900; accuracy: 89.83%
</code></pre>
<h3>c. CNN model:</h3>
<p>During the training of CNN with 4 hidden convolutional layers, I observed that the accuracy of training data was about 98% while the accuracy of validation data was about 91%. It was a clear sign that the model was overfitted. To avoid overfitting during iterations, I added several dropout layers to the model.</p>
<!-- Preformatted Code -->
<style>
pre {
tab-size: 4;
}
</style>
<pre><code>CNN with 2 hidden convolutional layers; accuracy: 91.18%
CNN with 4 hidden convolutional layers; accuracy: 91.40%
CNN with 4 hidden convolutional layers and dropout layers; accuracy: 92.14%
</code></pre>
<p>After comparing these results of Fashion-MNIST dataset with the results of MNIST dataset, the Fashion-MNIST dataset was indeed a lot more challenging to all the three models tested here. The accuracies on the testing data dropped for all three models. KNN had accuracy of 84.97% -85.54%, and Neural network model achieved 88.68% - 89.83%. Still, CNN model achieved the highest accuracy of 91.18% - 92.14% among all three models.</p>
</section>
</div>
<!-- Footer -->
<footer id="footer">
<section class="split contact">
<section class="alt">
<h3>Location</h3>
<p>Singapore</p>
</section>
<section>
<h3>Email</h3>
<p><a href="#">theo.xiao.sg@gmail.com</a></p>
</section>
<section>
<h3>GitHub</h3>
<ul class="icons alt">
<li><a href="https://github.com/theo-xiao-sg" class="icon brands alt fa-github"><span class="label">GitHub</span></a></li>
</ul>
</section>
</section>
</footer>
<!-- Scripts -->
<script src="assets/js/jquery.min.js"></script>
<script src="assets/js/jquery.scrollex.min.js"></script>
<script src="assets/js/jquery.scrolly.min.js"></script>
<script src="assets/js/browser.min.js"></script>
<script src="assets/js/breakpoints.min.js"></script>
<script src="assets/js/util.js"></script>
<script src="assets/js/main.js"></script>
</body>
</html>