Horse-or-Human-WithDropouts.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "view-in-github"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/lmoroney/dlaicourse/blob/master/Horse-or-Human-WithDropouts.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "rX8mhOLljYeM"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "BZSlp3DAjdYf"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "RXZT2UsyIVe_"
      },
      "outputs": [],
      "source": [
        "!wget --no-check-certificate \\\n",
        "    https://storage.googleapis.com/learning-datasets/horse-or-human.zip \\\n",
        "    -O /tmp/horse-or-human.zip\n",
        "\n",
        "!wget --no-check-certificate \\\n",
        "    https://storage.googleapis.com/learning-datasets/validation-horse-or-human.zip \\\n",
        "    -O /tmp/validation-horse-or-human.zip\n",
        "  \n",
        "import os\n",
        "import zipfile\n",
        "\n",
        "local_zip = '/tmp/horse-or-human.zip'\n",
        "zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
        "zip_ref.extractall('/tmp/horse-or-human')\n",
        "local_zip = '/tmp/validation-horse-or-human.zip'\n",
        "zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
        "zip_ref.extractall('/tmp/validation-horse-or-human')\n",
        "zip_ref.close()\n",
        "# Directory with our training horse pictures\n",
        "train_horse_dir = os.path.join('/tmp/horse-or-human/horses')\n",
        "\n",
        "# Directory with our training human pictures\n",
        "train_human_dir = os.path.join('/tmp/horse-or-human/humans')\n",
        "\n",
        "# Directory with our training horse pictures\n",
        "validation_horse_dir = os.path.join('/tmp/validation-horse-or-human/horses')\n",
        "\n",
        "# Directory with our training human pictures\n",
        "validation_human_dir = os.path.join('/tmp/validation-horse-or-human/humans')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "5oqBkNBJmtUv"
      },
      "source": [
        "## Building a Small Model from Scratch\n",
        "\n",
        "But before we continue, let's start defining the model:\n",
        "\n",
        "Step 1 will be to import tensorflow."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "qvfZg3LQbD-5"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "BnhYCP4tdqjC"
      },
      "source": [
        "We then add convolutional layers as in the previous example, and flatten the final result to feed into the densely connected layers."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "gokG5HKpdtzm"
      },
      "source": [
        "Finally we add the densely connected layers. \n",
        "\n",
        "Note that because we are facing a two-class classification problem, i.e. a *binary classification problem*, we will end our network with a [*sigmoid* activation](https://wikipedia.org/wiki/Sigmoid_function), so that the output of our network will be a single scalar between 0 and 1, encoding the probability that the current image is class 1 (as opposed to class 0)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "PixZ2s5QbYQ3"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.Sequential([\n",
        "    # Note the input shape is the desired size of the image 300x300 with 3 bytes color\n",
        "    # This is the first convolution\n",
        "    tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(300, 300, 3)),\n",
        "    tf.keras.layers.MaxPooling2D(2, 2),\n",
        "    # The second convolution\n",
        "    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
        "    tf.keras.layers.Dropout(0.5),\n",
        "    tf.keras.layers.MaxPooling2D(2,2),\n",
        "    # The third convolution\n",
        "    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
        "    tf.keras.layers.Dropout(0.5),\n",
        "    tf.keras.layers.MaxPooling2D(2,2),\n",
        "    # The fourth convolution\n",
        "    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
        "    tf.keras.layers.MaxPooling2D(2,2),\n",
        "    # The fifth convolution\n",
        "    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
        "    tf.keras.layers.MaxPooling2D(2,2),\n",
        "    # Flatten the results to feed into a DNN\n",
        "    \n",
        "    tf.keras.layers.Flatten(),\n",
        "    tf.keras.layers.Dropout(0.5),\n",
        "    # 512 neuron hidden layer\n",
        "    tf.keras.layers.Dense(512, activation='relu'),\n",
        "    # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')\n",
        "    tf.keras.layers.Dense(1, activation='sigmoid')\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "8DHWhFP_uhq3"
      },
      "outputs": [],
      "source": [
        "from tensorflow.keras.optimizers import RMSprop\n",
        "\n",
        "model.compile(loss='binary_crossentropy',\n",
        "              optimizer=RMSprop(lr=1e-4),\n",
        "              metrics=['accuracy'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "ClebU9NJg99G"
      },
      "outputs": [],
      "source": [
        "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
        "\n",
        "# All images will be rescaled by 1./255\n",
        "train_datagen = ImageDataGenerator(\n",
        "      rescale=1./255,\n",
        "      rotation_range=40,\n",
        "      width_shift_range=0.2,\n",
        "      height_shift_range=0.2,\n",
        "      shear_range=0.2,\n",
        "      zoom_range=0.2,\n",
        "      horizontal_flip=True,\n",
        "      fill_mode='nearest')\n",
        "\n",
        "validation_datagen = ImageDataGenerator(rescale=1/255)\n",
        "\n",
        "# Flow training images in batches of 128 using train_datagen generator\n",
        "train_generator = train_datagen.flow_from_directory(\n",
        "        '/tmp/horse-or-human/',  # This is the source directory for training images\n",
        "        target_size=(300, 300),  # All images will be resized to 150x150\n",
        "        batch_size=128,\n",
        "        # Since we use binary_crossentropy loss, we need binary labels\n",
        "        class_mode='binary')\n",
        "\n",
        "# Flow training images in batches of 128 using train_datagen generator\n",
        "validation_generator = validation_datagen.flow_from_directory(\n",
        "        '/tmp/validation-horse-or-human/',  # This is the source directory for training images\n",
        "        target_size=(300, 300),  # All images will be resized to 150x150\n",
        "        batch_size=32,\n",
        "        # Since we use binary_crossentropy loss, we need binary labels\n",
        "        class_mode='binary')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "Fb1_lgobv81m"
      },
      "outputs": [],
      "source": [
        "history = model.fit(\n",
        "      train_generator,\n",
        "      steps_per_epoch=8,  \n",
        "      epochs=100,\n",
        "      verbose=1,\n",
        "      validation_data = validation_generator,\n",
        "      validation_steps=8)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "7zNPRWOVJdOH"
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "acc = history.history['accuracy']\n",
        "val_acc = history.history['val_accuracy']\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs = range(len(acc))\n",
        "\n",
        "plt.plot(epochs, acc, 'bo', label='Training accuracy')\n",
        "plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n",
        "plt.title('Training and validation accuracy')\n",
        "\n",
        "plt.figure()\n",
        "\n",
        "plt.plot(epochs, loss, 'bo', label='Training Loss')\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation Loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "include_colab_link": true,
      "name": "Horse-or-Human-WithDropouts.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}