2.3 Deep Convolutional Neural Networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Deep CNN Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Constructing and training your own ConvNet from scratch can be Hard and a long task.\n",
    "\n",
    "A common trick used in Deep Learning is to use a **pre-trained** model and finetune it to the specific data it will be used for. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Famous Models with Keras\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This notebook contains code and reference for the following Keras models (gathered from [https://github.com/fchollet/keras/tree/master/keras/applications]())\n",
    "\n",
    "- VGG16\n",
    "- VGG19\n",
    "- ResNet50\n",
    "- Inception v3\n",
    "- Xception\n",
    "- ... more to come\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "## References\n",
    "\n",
    "- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/abs/1409.1556) - please cite this paper if you use the VGG models in your work.\n",
    "- [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385) - please cite this paper if you use the ResNet model in your work.\n",
    "- [Rethinking the Inception Architecture for Computer Vision](http://arxiv.org/abs/1512.00567) - please cite this paper if you use the Inception v3 model in your work.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "All architectures are compatible with both TensorFlow and Theano, and upon instantiation the models will be built according to the image dimension ordering set in your Keras configuration file at `~/.keras/keras.json`. \n",
    "\n",
    "For instance, if you have set `image_data_format=\"channels_last\"`, then any model loaded from this repository will get built according to the TensorFlow dimension ordering convention, \"Width-Height-Depth\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# VGG16"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<img src=\"imgs/vgg16.png\" >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# VGG19"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<img src=\"imgs/vgg19.png\" >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "# `keras.applications`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.applications import VGG16\n",
    "from keras.applications.imagenet_utils import preprocess_input, decode_predictions\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# -- Jupyter/IPython way to see documentation\n",
    "# please focus on parameters (e.g. include top)\n",
    "VGG16??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels.h5\n"
     ]
    }
   ],
   "source": [
    "vgg16 = VGG16(include_top=True, weights='imagenet')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you're wondering **where** this `HDF5` files with weights is stored, please take a look at `~/.keras/models/`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### HandsOn VGG16 - Pre-trained Weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "IMAGENET_FOLDER = 'imgs/imagenet'  #in the repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mapricot_565.jpeg\u001b[m\u001b[m     \u001b[31mapricot_787.jpeg\u001b[m\u001b[m     \u001b[31mstrawberry_1174.jpeg\u001b[m\u001b[m\r\n",
      "\u001b[31mapricot_696.jpeg\u001b[m\u001b[m     \u001b[31mstrawberry_1157.jpeg\u001b[m\u001b[m \u001b[31mstrawberry_1189.jpeg\u001b[m\u001b[m\r\n"
     ]
    }
   ],
   "source": [
    "!ls imgs/imagenet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<img src=\"imgs/imagenet/strawberry_1157.jpeg\" >"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input image shape: (1, 224, 224, 3)\n",
      "Downloading data from https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json\n",
      "Predicted: [[('n07745940', 'strawberry', 0.98483676), ('n07836838', 'chocolate_sauce', 0.0073711565), ('n07614500', 'ice_cream', 0.0030998574), ('n04332243', 'strainer', 0.0025101686), ('n04476259', 'tray', 0.00060175249)]]\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing import image\n",
    "import numpy as np\n",
    "\n",
    "img_path = os.path.join(IMAGENET_FOLDER, 'strawberry_1157.jpeg')\n",
    "img = image.load_img(img_path, target_size=(224, 224))\n",
    "x = image.img_to_array(img)\n",
    "x = np.expand_dims(x, axis=0)\n",
    "x = preprocess_input(x)\n",
    "print('Input image shape:', x.shape)\n",
    "\n",
    "preds = vgg16.predict(x)\n",
    "print('Predicted:', decode_predictions(preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<img src=\"imgs/imagenet/apricot_696.jpeg\" >"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input image shape: (1, 224, 224, 3)\n",
      "Predicted: [[('n07747607', 'orange', 0.87526792), ('n07749582', 'lemon', 0.03620464), ('n07717556', 'butternut_squash', 0.021843448), ('n03937543', 'pill_bottle', 0.0126132), ('n03942813', 'ping-pong_ball', 0.0054204506)]]\n"
     ]
    }
   ],
   "source": [
    "img_path = os.path.join(IMAGENET_FOLDER, 'apricot_696.jpeg')\n",
    "img = image.load_img(img_path, target_size=(224, 224))\n",
    "x = image.img_to_array(img)\n",
    "x = np.expand_dims(x, axis=0)\n",
    "x = preprocess_input(x)\n",
    "print('Input image shape:', x.shape)\n",
    "\n",
    "preds = vgg16.predict(x)\n",
    "print('Predicted:', decode_predictions(preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<img src=\"imgs/imagenet/apricot_565.jpeg\" >"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input image shape: (1, 224, 224, 3)\n",
      "Predicted: [[('n07718472', 'cucumber', 0.29338178), ('n07716358', 'zucchini', 0.2383192), ('n04596742', 'wok', 0.042132568), ('n07716906', 'spaghetti_squash', 0.038422), ('n07711569', 'mashed_potato', 0.036552209)]]\n"
     ]
    }
   ],
   "source": [
    "img_path = os.path.join(IMAGENET_FOLDER, 'apricot_565.jpeg')\n",
    "img = image.load_img(img_path, target_size=(224, 224))\n",
    "x = image.img_to_array(img)\n",
    "x = np.expand_dims(x, axis=0)\n",
    "x = preprocess_input(x)\n",
    "print('Input image shape:', x.shape)\n",
    "\n",
    "preds = vgg16.predict(x)\n",
    "print('Predicted:', decode_predictions(preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Hands On:\n",
    "\n",
    "### Try to do the same with VGG19 Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# from keras.applications import VGG19"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Residual Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"imgs/resnet_bb.png\" >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ResNet 50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"imgs/resnet34.png\" >"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## from keras.applications import ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}