adds method to load MNIST data in learning notebook

learning.ipynb

@@ -53,6 +53,145 @@
     "**Example**: Let's talk about an agent to play the popular Atari game—[Pong](http://www.ponggame.org). We will reward a point for every correct move and deduct a point for every wrong move from the agent. Eventually, the agent will figure out its actions prior to reinforcement were most responsible for it."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": true
+   },
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Practical Machine Learning Task\n",
+    "\n",
+    "### MNIST hand-written digits calssification\n",
+    "\n",
+    "The MNIST database, available from [this page](http://yann.lecun.com/exdb/mnist/) is a large database of handwritten digits that is commonly used for training & testing/validating in Machine learning.\n",
+    "\n",
+    "The dataset has **60,000 training images** each of size 28x28 pixels with labels and **10,000 testing images** of size 28x28 pixels with labels.\n",
+    "\n",
+    "In this section, we will use this database to compare performances of these different learning algorithms:\n",
+    "* kNN (k-Nearest Neighbour) classifier\n",
+    "* Single-hidden-layer Neural Network classifier\n",
+    "* SVMs (Support Vector Machines)\n",
+    "\n",
+    "It is estimates that humans have an error rate of about **0.2%** on this problem. Let's see how our algorithms perform!\n",
+    "\n",
+    "Let's start by loading MNIST data into numpy arrays."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "import os, struct\n",
+    "import array\n",
+    "import numpy as np\n",
+    "\n",
+    "def load_MNIST(path=\"aima-data/MNIST\"):\n",
+    "    \"helper function to load MNIST data\"\n",
+    "    train_img_file = open(os.path.join(path, \"train-images-idx3-ubyte\"), \"rb\")\n",
+    "    train_lbl_file = open(os.path.join(path, \"train-labels-idx1-ubyte\"), \"rb\")\n",
+    "    test_img_file = open(os.path.join(path, \"t10k-images-idx3-ubyte\"), \"rb\")\n",
+    "    test_lbl_file = open(os.path.join(path, 't10k-labels-idx1-ubyte'), \"rb\")\n",
+    "    \n",
+    "    magic_nr, tr_size, tr_rows, tr_cols = struct.unpack(\">IIII\", train_img_file.read(16))\n",
+    "    tr_img = array.array(\"B\", train_img_file.read())\n",
+    "    train_img_file.close()    \n",
+    "    magic_nr, tr_size = struct.unpack(\">II\", train_lbl_file.read(8))\n",
+    "    tr_lbl = array.array(\"b\", train_lbl_file.read())\n",
+    "    train_lbl_file.close()\n",
+    "    \n",
+    "    magic_nr, te_size, te_rows, te_cols = struct.unpack(\">IIII\", test_img_file.read(16))\n",
+    "    te_img = array.array(\"B\", test_img_file.read())\n",
+    "    test_img_file.close()\n",
+    "    magic_nr, te_size = struct.unpack(\">II\", test_lbl_file.read(8))\n",
+    "    te_lbl = array.array(\"b\", test_lbl_file.read())\n",
+    "    test_lbl_file.close()\n",
+    "\n",
+    "#     print(len(tr_img), len(tr_lbl), tr_size)\n",
+    "#     print(len(te_img), len(te_lbl), te_size)\n",
+    "    \n",
+    "    train_img = np.zeros((tr_size, tr_rows*tr_cols), dtype=np.uint8)\n",
+    "    train_lbl = np.zeros((tr_size,), dtype=np.int8)\n",
+    "    for i in range(tr_size):\n",
+    "        train_img[i] = np.array(tr_img[i*tr_rows*tr_cols : (i+1)*tr_rows*tr_cols]).reshape((tr_rows*te_cols))\n",
+    "        train_lbl[i] = tr_lbl[i]\n",
+    "        \n",
+    "    test_img = np.zeros((te_size, te_rows*te_cols), dtype=np.uint8)\n",
+    "    test_lbl = np.zeros((te_size,), dtype=np.int8)\n",
+    "    for i in range(te_size):\n",
+    "        test_img[i] = np.array(te_img[i*te_rows*te_cols : (i+1)*te_rows*te_cols]).reshape((te_rows*te_cols))\n",
+    "        test_lbl[i] = te_lbl[i]\n",
+    "        \n",
+    "    return(train_img, train_lbl, test_img, test_lbl)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The function `load_MNIST()` loads MNIST data from files saved in `aima-data/MNIST`. It returns four numpy arrays that we are gonna use to train & classify hand-written digits in various learning approaches."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "train_img, train_lbl, test_img, test_lbl = load_MNIST()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Check the shape of these NumPy arrays to make sure we have loaded the database correctly.\n",
+    "\n",
+    "Each 28x28 pixel image is flattened to 784x1 array and we should have 60,000 of them in training data. Similarly we should have 10,000 of those 784x1 arrays in testing data. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Training images size: (60000, 784)\n",
+      "Training labels size: (60000,)\n",
+      "Testing images size: (10000, 784)\n",
+      "Training labels size: (10000,)\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\"Training images size:\", train_img.shape)\n",
+    "print(\"Training labels size:\", train_lbl.shape)\n",
+    "print(\"Testing images size:\", test_img.shape)\n",
+    "print(\"Training labels size:\", test_lbl.shape)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Let's visualize some of the images from training & testing datasets."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,