cifar-10.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example of TFLearn to implement a Convolutional Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Have you ever developed a program that is able to find an object in an image? This was science fiction not so long ago, but machine learning is evolving so fast that now any developer can do it. Here is what you will need:\n",
    "1. Training data: We will be using the public data from the [CIFAR-10 Dataset](https://www.cs.toronto.edu/~kriz/cifar.html) which is automatically downloaded by the program.\n",
    "1. A powerful machine (ideally with GPUs). But if not, it's ok, it will just be quite a bit slower ;)\n",
    "1. A Neural Nets Library: We will be using Google's TensorFlow and it's high level abstraction called TFLearn\n",
    "1. The algorithms and network topology known to produce good results. In this case we will be using a Convolutional Neural Network (or CNN for short). As you learn more and more about machine learning you will slowly be able to do this on your own. For now, trust what is shown below and play with it, tweeking the parameters and observing if the resulting model is more or less accurate.\n",
    "\n",
    "This notebook has been created from the original source code at: https://github.com/tflearn/tflearn/blob/master/examples/images/convnet_cifar10.py\n",
    "\n",
    "An excellent article that provides an intuitive explanation of how CNN's work is https://t.co/Q7EcxtJcU\n",
    "\n",
    "Go ahead and execute the program cell by cell (remember that the shortcut to execute a single cell is Shift+Enter)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# -*- coding: utf-8 -*-\n",
    "from __future__ import division, print_function, absolute_import\n",
    "\n",
    "import tflearn\n",
    "from tflearn.data_utils import shuffle, to_categorical\n",
    "from tflearn.layers.core import input_data, dropout, fully_connected\n",
    "from tflearn.layers.conv import conv_2d, max_pool_2d\n",
    "from tflearn.layers.estimator import regression\n",
    "from tflearn.data_preprocessing import ImagePreprocessing\n",
    "from tflearn.data_augmentation import ImageAugmentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading CIFAR 10, Please wait...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100.0% 170500096 / 170498071\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Succesfully downloaded', 'cifar-10-python.tar.gz', 170498071, 'bytes.')\n",
      "File Extracted in Current Directory\n"
     ]
    }
   ],
   "source": [
    "# Data loading and preprocessing\n",
    "from tflearn.datasets import cifar10\n",
    "(X, Y), (X_test, Y_test) = cifar10.load_data()\n",
    "X, Y = shuffle(X, Y)\n",
    "Y = to_categorical(Y, 10)\n",
    "Y_test = to_categorical(Y_test, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Real-time data preprocessing\n",
    "img_prep = ImagePreprocessing()\n",
    "img_prep.add_featurewise_zero_center()\n",
    "img_prep.add_featurewise_stdnorm()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Real-time data augmentation\n",
    "img_aug = ImageAugmentation()\n",
    "img_aug.add_random_flip_leftright()\n",
    "img_aug.add_random_rotation(max_angle=25.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Convolutional network building\n",
    "network = input_data(shape=[None, 32, 32, 3],\n",
    "                     data_preprocessing=img_prep,\n",
    "                     data_augmentation=img_aug)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network = conv_2d(network, 32, 3, activation='relu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network = max_pool_2d(network, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network = conv_2d(network, 64, 3, activation='relu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network = conv_2d(network, 64, 3, activation='relu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network = max_pool_2d(network, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network = fully_connected(network, 512, activation='relu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network = dropout(network, 0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network = fully_connected(network, 10, activation='softmax')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network = regression(network, optimizer='adam',\n",
    "                     loss='categorical_crossentropy',\n",
    "                     learning_rate=0.001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Train using classifier\n",
    "model = tflearn.DNN(network, tensorboard_verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our network is now ready and it time to train it with our sample data (our our validation set).\n",
    "\n",
    "The code below does 10 iterations (n_epoch=10) which will be reasonably fast (a few mins) and will allow you to see how it works. However it will not produce a very accurate model. Increase the number of iterations to 40 for better results but beware that it will take longer and might even not work inside this docker image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Step: 5210  | total loss: \u001b[1m\u001b[32m0.68284\u001b[0m\u001b[0m\n",
      "| Adam | epoch: 010 | loss: 0.68284 - acc: 0.7679 | val_loss: 0.64109 - val_acc: 0.7824 -- iter: 50000/50000\n",
      "Training Step: 5210  | total loss: \u001b[1m\u001b[32m0.68284\u001b[0m\u001b[0m\n",
      "| Adam | epoch: 010 | loss: 0.68284 - acc: 0.7679 | val_loss: 0.64109 - val_acc: 0.7824 -- iter: 50000/50000\n",
      "--\n"
     ]
    }
   ],
   "source": [
    "model.fit(X, Y, n_epoch=10, shuffle=True, validation_set=(X_test, Y_test),\n",
    "          show_metric=True, batch_size=96, run_id='cifar10_cnn')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.save(\"cifar-10.tfl\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}