CNN Classification MNIST-Copy2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classification of the MNIST data using CNNs\n",
    "\n",
    "The MNIST Dataset has 28 x 28 black and white images of digits from 0 to 9. It is one of the most common datasets for starting up with deep learning. It comes in built in the keras package.\n",
    "\n",
    "This notebook will walk you through the developing a classification model for the dataset using a <b>Convolutional Neural Network</b> with the help of the <b>Model API</b> from Keras.\n",
    "\n",
    "### Importing required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "os.environ['KERAS_BACKEND'] = 'tensorflow'\n",
    "\n",
    "import numpy as np\n",
    "from sklearn.metrics import confusion_matrix, classification_report\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "from keras.layers import Input, Flatten, Dense, Dropout, MaxPooling2D\n",
    "#from keras.layers.advanced_activations import LeakyReLU\n",
    "from keras.layers import BatchNormalization, Activation, ZeroPadding2D\n",
    "from keras.layers.convolutional import Conv2D\n",
    "\n",
    "from keras.activations import *\n",
    "from keras.optimizers import *\n",
    "from keras.datasets import mnist\n",
    "from keras.utils import to_categorical\n",
    "from keras.models import Model, Sequential"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the dataset\n",
    "\n",
    "MNIST comes as a part of the keras datasets. It contains 60,000 training images while 10,000 test images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "\n",
    "# Display one image?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((60000, 28, 28), (10000, 28, 28))"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Preprocessing\n",
    "\n",
    "The labels refer to the expected classification output for a given image. They are converted from singular to one-hot encoded values from 0 to 9. \n",
    "\n",
    "That is, if a given image corresponds to 5, its encoding will be [0,0,0,0,0,1,0,0,0,0]\n",
    "\n",
    "- Since we are using a convolutional network we do not need to flatten the array into 1D.\n",
    "- rescale pixel values between 0 and 1\\n\n",
    "- Input type should be float\\n\n",
    "- There are 10 classes so in order to compute the cross entropy loss function we need to one-hot encoded vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((60000, 28, 28, 1), (10000, 28, 28, 1))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train = (X_train.astype(np.float32) - 127.5)/127.5\n",
    "X_test = (X_test.astype(np.float32) - 127.5)/127.5\n",
    "\n",
    "X_train = X_train[:,:,:,np.newaxis]\n",
    "X_test = X_test[:,:,:,np.newaxis]\n",
    "\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_classes = 10\n",
    "\n",
    "# convert class vectors to binary class matrices\n",
    "y_train = to_categorical(y_train, num_classes)\n",
    "#y_test = to_categorical(y_test, num_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a Keras Model\n",
    "\n",
    "Construct a model using the sequential API with the following instructions:\n",
    "\n",
    "- Inputs are normalized using BatchNormaliation followed by a Dropout layer with a rate of 0.3\n",
    "- Then add a 2D convolutional layer with a kernel of 3x3 with 32 filters\n",
    "- Activate the output using an Activation layer\n",
    "- Output from the convolutional layer goes through a MaxPooling layer\n",
    "- Add another 2D convolutional layer but ensure that the shape remains 'same' as previous layer having 64 filters\n",
    "- Output from the convolutional layer goes through a MaxPooling layer\n",
    "- Then Flatten the output and add a Dropout layer with a rate of 0.3\n",
    "- Connect the output to a Dense layer containing 100 neurons followed by a BatchNormalization \n",
    "- Add 'relu' activation function\n",
    "- and finally connect to the output layer with an softmax activation function\n",
    "\n",
    "Print the model summary to see the network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 28, 28, 1)         0         \n",
      "_________________________________________________________________\n",
      "conv2d_1 (Conv2D)            (None, 24, 24, 4)         104       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 12, 12, 4)         0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 576)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 10)                5770      \n",
      "=================================================================\n",
      "Total params: 5,874\n",
      "Trainable params: 5,874\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "In = Input(shape=(28,28,1))\n",
    "conv = Conv2D(4, kernel_size=5, strides=1, padding=\"valid\",activation='relu')\n",
    "pool = MaxPooling2D(pool_size=2,strides=2, padding=\"same\")\n",
    "fc = Dense(num_classes, activation='softmax')\n",
    "\n",
    "fmap1 = conv(In)\n",
    "fmap2 = pool(fmap1)\n",
    "dense = Flatten()(fmap2)\n",
    "y = Dense(num_classes, activation='softmax')(dense)\n",
    "# x = BatchNormalization()(x)\n",
    "\n",
    "# x = Conv2D(64, kernel_size=3, strides=2, padding=\"same\",activation='relu')(x)\n",
    "# x = MaxPooling2D(pool_size=2,strides=2, padding=\"same\")(x)\n",
    "# x = BatchNormalization()(x)\n",
    "\n",
    "# x = Flatten()(x)\n",
    "# x = Dense(64, activation='relu')(x)\n",
    "# y = Dense(num_classes, activation='softmax')(x)\n",
    "                                                                                    \n",
    "\n",
    "\"\"\"\n",
    "model.add(layers.Dropout(0.3, noise_shape=None, seed=None))\n",
    "model.add(layers.Dense(50, activation = \"relu\")\n",
    "model.add(layers.Dropout(0.2, noise_shape=None, seed=None))\n",
    "model.add(layers.Dense(50, activation = \"relu\"))\n",
    "\"\"\"                                                                                \n",
    "\n",
    "model = Model(inputs=In,outputs=y)                                                                                   \n",
    "                                                                                   \n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final model is compiled using an optimizer, a loss function and a metric for performance improvement. \n",
    "- The loss function is used to depict how far is the current model from the ideal answer\n",
    "- The optimizer refers to the method that will be used to minimize the loss\n",
    "- The metrics correspond to how we want to measure the performance of the network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss='categorical_crossentropy', optimizer=SGD(), metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training and testing\n",
    "\n",
    "- Training is done using the function fit(). We train out network for 5 epochs.\n",
    "- Testing is done using the predict() function. We can also use evaluate() but since we want to later generate a classificiation report, we will use the former\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 21s 351us/step - loss: 0.4212 - acc: 0.8795\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 19s 319us/step - loss: 0.1582 - acc: 0.9541\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 18s 296us/step - loss: 0.1222 - acc: 0.9645\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 18s 299us/step - loss: 0.1037 - acc: 0.9700\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 18s 297us/step - loss: 0.0928 - acc: 0.9725\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fd5879a3f90>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(X_train,y_train, epochs=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7, 2, 1, ..., 4, 5, 6], dtype=uint8)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_check = model.predict(X_test)\n",
    "\n",
    "y_pred = np.array([np.argmax(y_check[j]) for j in range(len(y_check))])\n",
    "y_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 971    0    1    0    0    1    3    1    2    1]\n",
      " [   0 1125    4    0    0    1    3    0    2    0]\n",
      " [   2    3 1014    3    1    0    1    2    6    0]\n",
      " [   1    0    4  993    0    3    0    4    3    2]\n",
      " [   1    0    4    0  967    0    1    1    2    6]\n",
      " [   1    1    0    5    0  872    3    2    6    2]\n",
      " [   5    3    2    0    2    1  943    0    2    0]\n",
      " [   0    4   13    2    0    0    0  998    3    8]\n",
      " [   7    2    5    3    1    0    2    6  944    4]\n",
      " [   6    6    1    1    6    2    0   11    3  973]]\n"
     ]
    }
   ],
   "source": [
    "print(confusion_matrix(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.98      0.99      0.98       980\n",
      "          1       0.98      0.99      0.99      1135\n",
      "          2       0.97      0.98      0.97      1032\n",
      "          3       0.99      0.98      0.98      1010\n",
      "          4       0.99      0.98      0.99       982\n",
      "          5       0.99      0.98      0.98       892\n",
      "          6       0.99      0.98      0.99       958\n",
      "          7       0.97      0.97      0.97      1028\n",
      "          8       0.97      0.97      0.97       974\n",
      "          9       0.98      0.96      0.97      1009\n",
      "\n",
      "avg / total       0.98      0.98      0.98     10000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#conv.get_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = []\n",
    "for layer in model.layers: \n",
    "    x.append(layer.get_weights())\n",
    "    #print(layer.get_weights())\n",
    "    #print(\"\\n\\n\\n\\n\")\n",
    "    \n",
    "wmap1 = np.array(x[1][0])\n",
    "bmap1 = np.array(x[1][1])\n",
    "\n",
    "wmap2 = np.array(x[4][0])\n",
    "bmap2 = np.array(x[4][1])\n",
    "\n",
    "#layer.get_config(), "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# f1 = open(\"wmap1.txt\",\"w\")\n",
    "# f = wmap1.T.flatten('C')\n",
    "# for i in range(11520/5):\n",
    "#     try:\n",
    "#         x =  str(f[i])+\", \"+str(f[i+1])+\", \"+str(f[i+2])+\", \"+str(f[i+3])+\", \"+str(f[i+4])+\", \\n\"\n",
    "#         f1.write(x)\n",
    "#     except:\n",
    "#         break\n",
    "# f1.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[[ 0.188182  ,  0.07620052, -0.14894305, -0.39629802,\n",
       "          -0.3198553 ],\n",
       "         [ 0.25531337,  0.0342965 , -0.2747088 , -0.28007376,\n",
       "          -0.2903393 ],\n",
       "         [ 0.3933249 ,  0.34005094, -0.00532881, -0.2629006 ,\n",
       "          -0.51142466],\n",
       "         [ 0.45010862,  0.44546333,  0.15801878, -0.2590417 ,\n",
       "          -0.51174414],\n",
       "         [ 0.06208478,  0.5496921 ,  0.42084926,  0.39437827,\n",
       "          -0.28109783]]],\n",
       "\n",
       "\n",
       "       [[[ 0.05863997,  0.29105607,  0.46185175,  0.42564487,\n",
       "           0.1958066 ],\n",
       "         [ 0.25320193,  0.4064024 ,  0.4276098 ,  0.15823913,\n",
       "           0.04433811],\n",
       "         [ 0.38967368,  0.09549404,  0.0308607 , -0.27944162,\n",
       "          -0.3492783 ],\n",
       "         [ 0.19999553, -0.14460343, -0.07421932, -0.28957462,\n",
       "          -0.3618506 ],\n",
       "         [-0.1664491 , -0.21875365, -0.3681912 , -0.43953162,\n",
       "          -0.49415025]]],\n",
       "\n",
       "\n",
       "       [[[-0.36640587, -0.34695625, -0.47125834, -0.2334613 ,\n",
       "          -0.0904766 ],\n",
       "         [-0.5578055 , -0.15011507, -0.21447098, -0.14635916,\n",
       "          -0.03407099],\n",
       "         [ 0.01473147,  0.05696957,  0.01340087,  0.14022921,\n",
       "           0.32948712],\n",
       "         [ 0.33963305,  0.23085773,  0.4595229 ,  0.14261252,\n",
       "           0.2519366 ],\n",
       "         [ 0.6375735 ,  0.46023613,  0.36825648,  0.34860995,\n",
       "           0.23666893]]],\n",
       "\n",
       "\n",
       "       [[[-0.4759945 , -0.4240235 ,  0.2956146 ,  0.44827035,\n",
       "           0.30587927],\n",
       "         [-0.57447976, -0.44474477,  0.29146305,  0.569725  ,\n",
       "           0.07351349],\n",
       "         [-0.61606187, -0.5081376 ,  0.38178194,  0.47644547,\n",
       "           0.32054916],\n",
       "         [-0.38520274, -0.40773672,  0.37526006,  0.37031785,\n",
       "           0.1298082 ],\n",
       "         [-0.6004572 , -0.08480015,  0.1586136 ,  0.31532723,\n",
       "           0.22301948]]]], dtype=float32)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# wmap1 = np.array(x[1][0])\n",
    "# wmap2 = np.array(x[4][0])\n",
    "# #writer = wmap1.flatten()\n",
    "# #wmap1\n",
    "wmap1.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.16742313e-01,  5.55352792e-02, -7.61901140e-02,\n",
       "         5.39284408e-01, -2.09182113e-01, -1.78037643e-01,\n",
       "         3.90237421e-01, -2.15333253e-01,  1.54524758e-01,\n",
       "         7.55098881e-05, -1.38261288e-01,  3.63211483e-01,\n",
       "        -2.87931636e-02, -1.39477476e-01,  6.52614057e-01,\n",
       "        -3.22264999e-01,  2.29107872e-01, -9.85072404e-02,\n",
       "        -1.35328501e-01,  2.28702769e-01, -2.62545258e-01,\n",
       "        -1.42032087e-01,  6.27099693e-01, -2.08277598e-01,\n",
       "         5.27418740e-02,  6.07124828e-02, -1.14080630e-01,\n",
       "        -3.47269386e-01, -1.37009859e-01,  3.95289570e-01,\n",
       "         6.24461353e-01, -1.60556898e-01, -3.19887698e-02,\n",
       "        -1.52072012e-01,  1.43811330e-01, -4.33153123e-01,\n",
       "        -1.65634707e-01,  3.77474278e-01,  5.36893547e-01,\n",
       "        -2.01450378e-01,  1.82521775e-01,  5.54543138e-02,\n",
       "        -1.26140311e-01,  5.00761449e-01, -2.15790182e-01,\n",
       "        -8.37143883e-02,  2.80732781e-01, -3.14082891e-01,\n",
       "         1.02859356e-01, -7.25525692e-02, -6.92386776e-02,\n",
       "         4.81122464e-01, -5.88767827e-02, -2.48904422e-01,\n",
       "         3.13644648e-01, -1.70443878e-01, -7.71002099e-02,\n",
       "         1.07371947e-02, -1.60146669e-01, -1.61897868e-01,\n",
       "        -5.33686802e-02, -1.62925705e-01,  3.03194791e-01,\n",
       "        -3.91360074e-01, -1.33307993e-01, -9.00263041e-02,\n",
       "         2.06153616e-01, -4.08388942e-01, -1.67653069e-01,\n",
       "         2.50184208e-01,  3.30217987e-01, -1.44399136e-01,\n",
       "        -1.24499179e-01, -1.05039939e-01,  3.04657314e-02,\n",
       "        -4.53844190e-01,  8.65486115e-02,  4.56919342e-01,\n",
       "         2.69054025e-01,  1.15359060e-01,  8.53309184e-02,\n",
       "        -5.09911552e-02, -8.66918564e-02,  4.98535037e-01,\n",
       "        -2.83405215e-01, -1.03542045e-01, -4.98201519e-01,\n",
       "        -1.49954140e-01, -2.59427428e-01, -1.53119892e-01,\n",
       "        -2.22009957e-01,  2.25099817e-01, -1.85409904e-01,\n",
       "        -4.03908283e-01, -2.34908834e-01, -1.02654204e-01,\n",
       "        -3.05824250e-01, -1.87309816e-01,  1.29241645e-01,\n",
       "        -9.27751288e-02, -7.03870654e-02, -2.42342338e-01,\n",
       "        -2.84082711e-01,  1.06737860e-01, -1.77181229e-01,\n",
       "        -8.61675590e-02,  1.90890402e-01, -3.13127249e-01,\n",
       "        -9.80491415e-02,  1.18567862e-01, -3.27970982e-01,\n",
       "         2.09460989e-01,  5.33855669e-02, -7.65234679e-02,\n",
       "         4.22273539e-02, -2.16758683e-01,  3.30297142e-01,\n",
       "         5.87600946e-01,  2.57528126e-02,  2.06473127e-01,\n",
       "        -1.51690766e-01, -8.44349787e-02, -1.63021460e-01,\n",
       "         1.68629244e-01, -1.63548961e-01, -3.71799678e-01,\n",
       "        -4.04105425e-01,  3.67971212e-01,  1.28623629e-02,\n",
       "        -1.56516992e-02, -5.00188023e-03,  2.54778296e-01,\n",
       "        -1.34690642e-01, -2.55419642e-01, -3.07900757e-01,\n",
       "         3.70606214e-01,  4.90354709e-02, -1.00623891e-01,\n",
       "         6.32507587e-03,  3.16848159e-02, -1.01325035e-01,\n",
       "        -2.84442510e-02, -3.02834928e-01,  3.77570421e-01,\n",
       "         3.08477223e-01, -1.80530474e-01,  1.48762196e-01,\n",
       "        -3.54645133e-01,  2.74067968e-01,  3.25903714e-01,\n",
       "        -3.37053657e-01,  1.99971974e-01,  2.42513716e-01,\n",
       "        -1.40964821e-01, -3.17769982e-02, -2.95885980e-01,\n",
       "         2.76653975e-01,  3.54026765e-01, -1.90263599e-01,\n",
       "         5.16823307e-03, -1.05779367e-02, -1.10472500e-01,\n",
       "        -1.37778759e-01,  1.52537704e-01, -9.08578094e-03,\n",
       "        -1.84011534e-01, -3.99783164e-01,  3.34027082e-01,\n",
       "         2.15627402e-01, -2.17455208e-01,  7.54587278e-02,\n",
       "         1.20683476e-01, -1.99714359e-02, -1.11206464e-01,\n",
       "        -3.93495053e-01,  2.71492362e-01,  6.32800385e-02,\n",
       "        -1.26578689e-01,  1.22427091e-01, -1.97163060e-01,\n",
       "         4.07128513e-01,  1.19683810e-03, -3.34652007e-01,\n",
       "         1.38952687e-01, -7.94616640e-02, -2.57794373e-02,\n",
       "         1.02404974e-01, -1.57679170e-01,  4.58639115e-01,\n",
       "         3.37356240e-01, -1.02596804e-01,  5.96958511e-02,\n",
       "         9.36099365e-02,  3.47870179e-02, -6.09834902e-02,\n",
       "        -6.93253875e-02,  4.24372017e-01,  2.84871250e-01,\n",
       "        -1.39722258e-01, -2.09334254e-01]])"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wmap = np.zeros((200,1))\n",
    "dx = 0\n",
    "for i in range(5):\n",
    "    for j in range(5):\n",
    "        for k in range(8):\n",
    "            wmap[dx] = wmap1[i][j][0][k]\n",
    "            dx += 1\n",
    "wmap.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "#X_train[99].reshape((28,28,1))\n",
    "#== wmap1.T.flatten(order='C')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#wmap1 = np.array(x[1][0])\n",
    "writer = wmap1.T.flatten(order='C')\n",
    "wmap1_base = 0x20000C54\n",
    "f=open(\"init2.ini\", \"w\")\n",
    "for entry in writer:\n",
    "    str1=\"_WFLOAT (\"+str(wmap1_base)+\",\"+str(entry)+\") \\n\"\n",
    "    f.write(str1)\n",
    "    wmap1_base += 4\n",
    "f.close()\n",
    "\n",
    "# wmap = np.zeros((11520,1))\n",
    "# dx = 0\n",
    "# for i in range(1152):\n",
    "#     for j in range(10):\n",
    "#         #print i,j, dx\n",
    "#         wmap[dx] = wmap2[i][j]\n",
    "#         dx += 1\n",
    "# wmap2 = wmap.flatten(order='C')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "wmap2_base = 0x20000F74\n",
    "#wmap2 = np.array(x[4][0])\n",
    "writer = wmap2.T.flatten(order='C')\n",
    "f=open(\"init2.ini\", \"a\")\n",
    "for entry in writer:\n",
    "    str1=\"_WFLOAT (\"+str(wmap2_base)+\",\"+str(entry)+\") \\n\"\n",
    "    f.write(str1)\n",
    "    wmap2_base += 4\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "784"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_base = 0x20000014\n",
    "input_map = X_train[455].T\n",
    "writer = input_map.flatten(order='C')\n",
    "f=open(\"init2.ini\", \"a\")\n",
    "idx=0\n",
    "for entry in writer:\n",
    "    str1=\"_WFLOAT (\"+str(input_base)+\",\"+str(entry)+\") \\n\"\n",
    "    f.write(str1)\n",
    "    #print str1\n",
    "    input_base += 4\n",
    "    idx+=1\n",
    "f.close()\n",
    "idx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "bmap1 = np.array(x[1][1])\n",
    "#print bmap1.shape\n",
    "writer = bmap1.flatten(order='C')\n",
    "bmap1_base = 0x2000C374\n",
    "f=open(\"init2.ini\", \"a\")\n",
    "for entry in writer:\n",
    "    str1=\"_WFLOAT (\"+str(bmap1_base)+\",\"+str(entry)+\") \\n\"\n",
    "    f.write(str1)\n",
    "    bmap1_base += 4\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "bmap2 = np.array(x[4][1])\n",
    "#print bmap2.shape\n",
    "writer = bmap2.flatten(order='C')\n",
    "bmap2_base = 0x2000C394 #020\n",
    "f=open(\"init2.ini\", \"a\")\n",
    "for entry in writer:\n",
    "    str1=\"_WFLOAT (\"+str(bmap2_base)+\",\"+str(entry)+\") \\n\"\n",
    "    f.write(str1)\n",
    "    bmap2_base += 4\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([0., 1., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32),\n",
       " array([[1.0310464e-06, 9.9448830e-01, 6.5358420e-04, 7.2428084e-05,\n",
       "         2.0706819e-04, 7.8716447e-08, 8.8869126e-07, 4.3997401e-03,\n",
       "         1.4227125e-04, 3.4638411e-05]], dtype=float32))"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[455], model.predict(X_train[455][np.newaxis,:,:,:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.6313726, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.58431375, 0.33333334, 0.9529412, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.5372549, 0.78039217, 0.9764706, 0.9764706, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.29411766, 0.60784316, 0.90588236, 0.9764706, 0.9764706, 0.6627451, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.92156863, -0.23921569, -0.05882353, 0.8980392, 0.9764706, 0.9764706, 0.6862745, -0.18431373, -0.8509804, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.13725491, 0.4117647, 0.9764706, 0.9764706, 0.9764706, 0.9764706, 0.69411767, -0.75686276, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, 0.2627451, 0.8901961, 0.9843137, 0.9764706, 0.9764706, 0.9764706, 0.9764706, -0.011764706, -0.9372549, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.92156863, -0.12941177, 0.85882354, 0.9843137, 0.9843137, 1.0, 0.9764706, 0.3019608, -0.050980393, -0.2784314, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.79607844, 0.13725491, 0.13725491, 0.13725491, 0.78039217, 0.9764706, 0.9764706, 0.9764706, 0.8901961, 0.79607844, 0.16078432, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.8039216, 0.654902, 0.9764706, 0.9764706, 0.9764706, 0.9764706, 0.9764706, 0.9764706, 0.7176471, -0.5686275, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, 0.67058825, 0.9764706, 0.9764706, 0.9764706, 0.9764706, 0.8666667, 0.2784314, -0.34117648, -0.9372549, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, 0.75686276, 0.9764706, 0.9764706, 0.6627451, 0.5058824, -0.37254903, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -0.7490196, 0.003921569, 0.27058825, -0.84313726, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0,\n"
     ]
    }
   ],
   "source": [
    "for i in input_map.flatten():\n",
    "    print str(i)+\",\",\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mo = 255\n",
    "np.argmax(y_train[mo])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[8.9544105e-10, 1.2597758e-07, 1.5826465e-05, 9.9981433e-01,\n",
       "        3.3035694e-10, 5.5585137e-05, 1.8169173e-10, 3.3826855e-10,\n",
       "        1.1387560e-04, 2.5390958e-07]], dtype=float32)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict(X_train[mo][np.newaxis,:,:,:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = X_train[200].flatten('C')\n",
    "\n",
    "f1 = open(\"image.txt\",\"w\")\n",
    "for i in range(len(p)-1):\n",
    "    x = str(p[i])+\",\"\n",
    "    f1.write(x)\n",
    "x = str(p[i])\n",
    "f1.write(x)\n",
    "f1.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 247,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 1., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)"
      ]
     },
     "execution_count": 247,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[99]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0x27f00000'"
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hex(0x1FF00000 + 128*1024*1024)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}