Lesson 2. Training a Neural Network.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training a Neural Network\n",
    "In this notebook, we'll build on principles we learned in [Lesson 1: Basic Neural Network](Lesson%201.%20Basic%20Neural%20Network.ipynb). We'll build the same network, and run synthetic data through it to look at the error rate and training accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import mxnet as mx\n",
    "from data_iter import SyntheticData # You'll need data_iter.py from this repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create a simple two layer network like we did in `Basic Neural Network`\n",
    "net = mx.sym.Variable('data')\n",
    "net = mx.sym.FullyConnected(net, name='fc1', num_hidden=64)\n",
    "net = mx.sym.Activation(net, name='relu1', act_type='relu')\n",
    "net = mx.sym.FullyConnected(net, name='fc2', num_hidden=10)\n",
    "net = mx.sym.SoftmaxOutput(net, name='softmax')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Now synthesize some data with 10 classes and 128 features/dimensions\n",
    "data = SyntheticData(num_classes = 10, num_features = 128)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modules\n",
    "In MXNet, the Module API provides an intermediate and high-level interface for performing computation with neural networks. A module simply wraps a Symbol (like the network created above) and one or more Executors. The module accepts the symbol as input:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mod = mx.mod.Module(symbol=net, \n",
    "                    context=mx.cpu(), # Note we're using CPU here. We will evaluate with GPU later in this notebook.\n",
    "                    data_names=['data'], \n",
    "                    label_names=['softmax_label'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fitting Data to the Module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Already binded, ignoring bind()\n",
      "WARNING:root:optimizer already initialized, ignoring...\n",
      "INFO:root:Epoch[0] Train-accuracy=1.000000\n",
      "INFO:root:Epoch[0] Time cost=0.035\n",
      "INFO:root:Epoch[0] Validation-accuracy=1.000000\n",
      "INFO:root:Epoch[1] Train-accuracy=1.000000\n",
      "INFO:root:Epoch[1] Time cost=0.027\n",
      "INFO:root:Epoch[1] Validation-accuracy=1.000000\n",
      "INFO:root:Epoch[2] Train-accuracy=1.000000\n",
      "INFO:root:Epoch[2] Time cost=0.031\n",
      "INFO:root:Epoch[2] Validation-accuracy=1.000000\n",
      "INFO:root:Epoch[3] Train-accuracy=1.000000\n",
      "INFO:root:Epoch[3] Time cost=0.028\n",
      "INFO:root:Epoch[3] Validation-accuracy=1.000000\n",
      "INFO:root:Epoch[4] Train-accuracy=1.000000\n",
      "INFO:root:Epoch[4] Time cost=0.030\n",
      "INFO:root:Epoch[4] Validation-accuracy=1.000000\n"
     ]
    }
   ],
   "source": [
    "import logging\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "batch_size=32\n",
    "mod.fit(data.get_iter(batch_size),\n",
    "        eval_data = data.get_iter(batch_size),\n",
    "        optimizer = 'sgd',\n",
    "        optimizer_params = {'learning_rate':0.1},\n",
    "        eval_metric = 'acc',\n",
    "        num_epoch = 5\n",
    "       )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('mse', 26.168893623352051), ('accuracy', 1.0)]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = mod.predict(data.get_iter(batch_size))\n",
    "'shape of predict: %s' % (y.shape,)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('mse', 27.361479949951171), ('accuracy', 1.0)]"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the MSE and accuracy\n",
    "mod.score(data.get_iter(batch_size), ['mse', 'acc'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}