notebooks/TensorFlow MNIST tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Demo: MNIST for ML Beginners\n",
    "Before start using this, please select `Cell` - `All Output` - `Clear` to clear the old results. See [TensorFlow Tutorial](https://www.tensorflow.org/versions/master/tutorials/mnist/beginners/index.html) for details of the tutorial.\n",
    "\n",
    "# Loading MNIST training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# import MNIST data\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training Images\n",
    "![mnist.train.xs](https://www.tensorflow.org/versions/master/images/mnist-train-xs.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# check MNIST training images matrix shape\n",
    "mnist.train.images.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# check MNIST training images matrix data\n",
    "sample_img = mnist.train.images[5].reshape(28, 28)\n",
    "sample_img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# plot the image\n",
    "plt.imshow(sample_img).set_cmap('Greys')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training Labels\n",
    "![mnist.train.ys](https://www.tensorflow.org/versions/master/images/mnist-train-ys.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# check MNIST labels shape\n",
    "mnist.train.labels.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# show MNIST label data\n",
    "sample_label = mnist.train.labels[5]\n",
    "sample_label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Defining a Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## in a graph:\n",
    "![](https://www.tensorflow.org/versions/master/images/softmax-regression-scalargraph.png)\n",
    "\n",
    "## in a vector equation:\n",
    "![](https://www.tensorflow.org/versions/master/images/softmax-regression-vectorequation.png)\n",
    "\n",
    "## so that we'll have the weights like this:\n",
    "blue: positive weights, red: negative weights\n",
    "![](https://www.tensorflow.org/versions/master/images/softmax-weights.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# define a neural network (softmax logistic regression)\n",
    "import tensorflow as tf\n",
    "x = tf.placeholder(tf.float32, [None, 784])\n",
    "W = tf.Variable(tf.zeros([784, 10]))\n",
    "b = tf.Variable(tf.zeros([10]))\n",
    "y = tf.nn.softmax(tf.matmul(x, W) + b) # the equation\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Defining the Train Step"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# define the train step to minimize the cross entropy with SGD\n",
    "y_ = tf.placeholder(tf.float32, [None, 10])\n",
    "cross_entropy = -tf.reduce_sum(y_*tf.log(y))\n",
    "train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)\n",
    "train_step"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use Gradient Decent to find the optimal weights\n",
    "![](http://blog.datumbox.com/wp-content/uploads/2013/10/gradient-descent.png)\n",
    "From: [Machine Learning Blog & Software Development News](http://blog.datumbox.com/tuning-the-learning-rate-in-gradient-descent/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Do 1000 times of mini-batch training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# initialize variables and session\n",
    "init = tf.initialize_all_variables()\n",
    "sess = tf.Session()\n",
    "sess.run(init)\n",
    "\n",
    "# train the model mini batch with 100 elements, for 1K times\n",
    "for i in range(1000):\n",
    "    batch_xs, batch_ys = mnist.train.next_batch(100)\n",
    "    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# evaluate the accuracy of the model\n",
    "correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n",
    "print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualizing with TensorBoard\n",
    "\n",
    "See [TensorBoard tutorial](https://www.tensorflow.org/versions/master/how_tos/summaries_and_tensorboard/index.html) for details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "![](https://www.tensorflow.org/versions/master/images/mnist_tensorboard.png)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11+"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}