C1/W4/ungraded_labs/C1_W4_Lab_1_image_generator_no_validation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5M0E2zlTZI7g"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/https-deeplearning-ai/tensorflow-1-public/blob/master/C1/W4/ungraded_labs/C1_W4_Lab_1_image_generator_no_validation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-74XLLwqPlcw"
   },
   "source": [
    "# Ungraded Lab: Training with ImageDataGenerator\n",
    "\n",
    "In this lab, you will build a train a model on the [Horses or Humans](https://www.tensorflow.org/datasets/catalog/horses_or_humans) dataset. This contains over a thousand images of horses and humans with varying poses and filesizes. You will use the [ImageDataGenerator](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator) class to prepare this dataset so it can be fed to a convolutional neural network.\n",
    "\n",
    "**IMPORTANT NOTE:** This notebook is designed to run as a Colab. Running it on your local machine might result in some of the code blocks throwing errors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qYFguQkJvpV3"
   },
   "source": [
    "Run the code below to download the compressed dataset `horse-or-human.zip`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RXZT2UsyIVe_"
   },
   "outputs": [],
   "source": [
    "!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/horse-or-human.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9brUxyTpYZHy"
   },
   "source": [
    "You can then unzip the archive using the [zipfile](https://docs.python.org/3/library/zipfile.html) module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PLy3pthUS0D2"
   },
   "outputs": [],
   "source": [
    "import zipfile\n",
    "\n",
    "# Unzip the dataset\n",
    "local_zip = './horse-or-human.zip'\n",
    "zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
    "zip_ref.extractall('./horse-or-human')\n",
    "zip_ref.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o-qUPyfO7Qr8"
   },
   "source": [
    "The contents of the .zip are extracted to the base directory `./horse-or-human`, which in turn each contain `horses` and `humans` subdirectories.\n",
    "\n",
    "In short: The training set is the data that is used to tell the neural network model that 'this is what a horse looks like' and 'this is what a human looks like'.\n",
    "\n",
    "One thing to pay attention to in this sample: We do not explicitly label the images as horses or humans. You will use the ImageDataGenerator API instead -- and this is coded to automatically label images according to the directory names and structure. So, for example, you will have a 'training' directory containing a 'horses' directory and a 'humans' one. `ImageDataGenerator` will label the images appropriately for you, reducing a coding step. \n",
    "\n",
    "You can now define each of these directories:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NR_M9nWN-K8B"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Directory with our training horse pictures\n",
    "train_horse_dir = os.path.join('./horse-or-human/horses')\n",
    "\n",
    "# Directory with our training human pictures\n",
    "train_human_dir = os.path.join('./horse-or-human/humans')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LuBYtA_Zd8_T"
   },
   "source": [
    "Now see what the filenames look like in the `horses` and `humans` training directories:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4PIP1rkmeAYS"
   },
   "outputs": [],
   "source": [
    "train_horse_names = os.listdir(train_horse_dir)\n",
    "print(train_horse_names[:10])\n",
    "\n",
    "train_human_names = os.listdir(train_human_dir)\n",
    "print(train_human_names[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HlqN5KbafhLI"
   },
   "source": [
    "You can also find out the total number of horse and human images in the directories:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "H4XHh2xSfgie"
   },
   "outputs": [],
   "source": [
    "print('total training horse images:', len(os.listdir(train_horse_dir)))\n",
    "print('total training human images:', len(os.listdir(train_human_dir)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "C3WZABE9eX-8"
   },
   "source": [
    "Now take a look at a few pictures to get a better sense of what they look like. First, configure the `matplotlib` parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "b2_Q0-_5UAv-"
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as mpimg\n",
    "\n",
    "# Parameters for our graph; we'll output images in a 4x4 configuration\n",
    "nrows = 4\n",
    "ncols = 4\n",
    "\n",
    "# Index for iterating over images\n",
    "pic_index = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xTvHzGCxXkqp"
   },
   "source": [
    "Now, display a batch of 8 horse and 8 human pictures. You can rerun the cell to see a fresh batch each time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Wpr8GxjOU8in"
   },
   "outputs": [],
   "source": [
    "# Set up matplotlib fig, and size it to fit 4x4 pics\n",
    "fig = plt.gcf()\n",
    "fig.set_size_inches(ncols * 4, nrows * 4)\n",
    "\n",
    "pic_index += 8\n",
    "next_horse_pix = [os.path.join(train_horse_dir, fname) \n",
    "                for fname in train_horse_names[pic_index-8:pic_index]]\n",
    "next_human_pix = [os.path.join(train_human_dir, fname) \n",
    "                for fname in train_human_names[pic_index-8:pic_index]]\n",
    "\n",
    "for i, img_path in enumerate(next_horse_pix+next_human_pix):\n",
    "  # Set up subplot; subplot indices start at 1\n",
    "  sp = plt.subplot(nrows, ncols, i + 1)\n",
    "  sp.axis('Off') # Don't show axes (or gridlines)\n",
    "\n",
    "  img = mpimg.imread(img_path)\n",
    "  plt.imshow(img)\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5oqBkNBJmtUv"
   },
   "source": [
    "## Building a Small Model from Scratch\n",
    "\n",
    "Now you can define the model architecture that you will train.\n",
    "\n",
    "Step 1 will be to import tensorflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "qvfZg3LQbD-5"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BnhYCP4tdqjC"
   },
   "source": [
    "You then add convolutional layers as in the previous example, and flatten the final result to feed into the densely connected layers. Note that because this is a two-class classification problem, i.e. a *binary classification problem*, you will end your network with a [*sigmoid* activation](https://wikipedia.org/wiki/Sigmoid_function). This makes the output value of your network a single scalar between 0 and 1, encoding the probability that the current image is class 1 (as opposed to class 0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PixZ2s5QbYQ3"
   },
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "    # Note the input shape is the desired size of the image 300x300 with 3 bytes color\n",
    "    # This is the first convolution\n",
    "    tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(300, 300, 3)),\n",
    "    tf.keras.layers.MaxPooling2D(2, 2),\n",
    "    # The second convolution\n",
    "    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # The third convolution\n",
    "    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # The fourth convolution\n",
    "    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # The fifth convolution\n",
    "    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # Flatten the results to feed into a DNN\n",
    "    tf.keras.layers.Flatten(),\n",
    "    # 512 neuron hidden layer\n",
    "    tf.keras.layers.Dense(512, activation='relu'),\n",
    "    # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')\n",
    "    tf.keras.layers.Dense(1, activation='sigmoid')\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s9EaFDP5srBa"
   },
   "source": [
    "You can review the network architecture and the output shapes with `model.summary()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7ZKj8392nbgP"
   },
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DmtkTn06pKxF"
   },
   "source": [
    "The \"output shape\" column shows how the size of your feature map evolves in each successive layer. As you saw in an earlier lesson, the convolution layers removes the outermost pixels of the image, and each pooling layer halves the dimensions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PEkKSpZlvJXA"
   },
   "source": [
    "Next, you'll configure the specifications for model training. You will train the model with the [`binary_crossentropy`](https://www.tensorflow.org/api_docs/python/tf/keras/losses/BinaryCrossentropy) loss because it's a binary classification problem, and the final activation is a sigmoid. (For a refresher on loss metrics, see this [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course/descending-into-ml/video-lecture).) You will use the `rmsprop` optimizer with a learning rate of `0.001`. During training, you will want to monitor classification accuracy.\n",
    "\n",
    "**NOTE**: In this case, using the [RMSprop optimization algorithm](https://wikipedia.org/wiki/Stochastic_gradient_descent#RMSProp) is preferable to [stochastic gradient descent](https://developers.google.com/machine-learning/glossary/#SGD) (SGD), because RMSprop automates learning-rate tuning for us. (Other optimizers, such as [Adam](https://wikipedia.org/wiki/Stochastic_gradient_descent#Adam) and [Adagrad](https://developers.google.com/machine-learning/glossary/#AdaGrad), also automatically adapt the learning rate during training, and would work equally well here.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8DHWhFP_uhq3"
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras.optimizers import RMSprop\n",
    "\n",
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer=RMSprop(learning_rate=0.001),\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Sn9m9D3UimHM"
   },
   "source": [
    "### Data Preprocessing\n",
    "\n",
    "Next step is to set up the data generators that will read pictures in the source folders, convert them to `float32` tensors, and feed them (with their labels) to the model. You'll have one generator for the training images and one for the validation images. These generators will yield batches of images of size 300x300 and their labels (binary).\n",
    "\n",
    "As you may already know, data that goes into neural networks should usually be normalized in some way to make it more amenable to processing by the network (i.e. It is uncommon to feed raw pixels into a ConvNet.) In this case, you will preprocess the images by normalizing the pixel values to be in the `[0, 1]` range (originally all values are in the `[0, 255]` range).\n",
    "\n",
    "In Keras, this can be done via the `keras.preprocessing.image.ImageDataGenerator` class using the `rescale` parameter. This `ImageDataGenerator` class allows you to instantiate generators of augmented image batches (and their labels) via `.flow(data, labels)` or `.flow_from_directory(directory)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ClebU9NJg99G"
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "# All images will be rescaled by 1./255\n",
    "train_datagen = ImageDataGenerator(rescale=1/255)\n",
    "\n",
    "# Flow training images in batches of 128 using train_datagen generator\n",
    "train_generator = train_datagen.flow_from_directory(\n",
    "        './horse-or-human/',  # This is the source directory for training images\n",
    "        target_size=(300, 300),  # All images will be resized to 300x300\n",
    "        batch_size=128,\n",
    "        # Since we use binary_crossentropy loss, we need binary labels\n",
    "        class_mode='binary')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mu3Jdwkjwax4"
   },
   "source": [
    "### Training\n",
    "\n",
    "You can start training for 15 epochs -- this may take a few minutes to run.\n",
    "\n",
    "Do note the values per epoch.\n",
    "\n",
    "The `loss` and `accuracy` are great indicators of progress in training. `loss` measures the current model prediction against the known labels, calculating the result. `accuracy`, on the other hand, is the portion of correct guesses. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Fb1_lgobv81m"
   },
   "outputs": [],
   "source": [
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=8,  \n",
    "      epochs=15,\n",
    "      verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o6vSHzPR2ghH"
   },
   "source": [
    "### Model Prediction\n",
    "\n",
    "Now take a look at actually running a prediction using the model. This code will allow you to choose 1 or more files from your file system, upload them, and run them through the model, giving an indication of whether the object is a horse or a human.\n",
    "\n",
    "_**Note:** Old versions of the Safari browser might have compatibility issues with the code block below. If you get an error after you select the images(s) to upload, you can consider updating your browser to the latest version. If not possible, please comment out or skip the code block below, uncomment the next code block and run it._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "DoWp43WxJDNT"
   },
   "outputs": [],
   "source": [
    "## NOTE: If you are using Safari and this cell throws an error,\n",
    "## please skip this block and run the next one instead.\n",
    "\n",
    "import numpy as np\n",
    "from google.colab import files\n",
    "from tensorflow.keras.utils import load_img, img_to_array\n",
    "\n",
    "uploaded = files.upload()\n",
    "\n",
    "for fn in uploaded.keys():\n",
    " \n",
    "  # predicting images\n",
    "  path = '/content/' + fn\n",
    "  img = load_img(path, target_size=(300, 300))\n",
    "  x = img_to_array(img)\n",
    "  x /= 255\n",
    "  x = np.expand_dims(x, axis=0)\n",
    "\n",
    "  images = np.vstack([x])\n",
    "  classes = model.predict(images, batch_size=10)\n",
    "  print(classes[0])\n",
    "    \n",
    "  if classes[0]>0.5:\n",
    "    print(fn + \" is a human\")\n",
    "  else:\n",
    "    print(fn + \" is a horse\")\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WkLydXVZr30K"
   },
   "source": [
    "If you're using Safari and the cell above throws an error, you will need to upload the images(s) manually in their workspace.\n",
    "\n",
    "Instructions on how to upload image(s) manually in a Colab:\n",
    "\n",
    "1. Select the `folder` icon on the left `menu bar`.\n",
    "2. Click on the `folder with an arrow pointing upwards` named `..`\n",
    "3. Click on the `folder` named `tmp`.\n",
    "4. Inside of the `tmp` folder, `create a new folder` called `images`. You'll see the `New folder` option by clicking the `3 vertical dots` menu button next to the `tmp` folder.\n",
    "5. Inside of the new `images` folder, upload an image(s) of your choice, preferably of either a horse or a human. Drag and drop the images(s) on top of the `images` folder.\n",
    "6. Uncomment and run the code block below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1_vVstx-r4jy"
   },
   "outputs": [],
   "source": [
    "# # CODE BLOCK FOR OLD VERSIONS OF SAFARI\n",
    "\n",
    "# import numpy as np\n",
    "# from tensorflow.keras.utils import load_img, img_to_array\n",
    "# import os\n",
    "\n",
    "# images = os.listdir(\"/tmp/images\")\n",
    "\n",
    "# print(images)\n",
    "\n",
    "# for i in images:\n",
    "#  print()\n",
    "#  # predicting images\n",
    "#  path = '/tmp/images/' + i\n",
    "#  img = load_img(path, target_size=(300, 300))\n",
    "#  x = img_to_array(img)\n",
    "#  x /= 255\n",
    "#  x = np.expand_dims(x, axis=0)\n",
    "\n",
    "#  images = np.vstack([x])\n",
    "#  classes = model.predict(images, batch_size=10)\n",
    "#  print(classes[0])\n",
    "#  if classes[0]>0.5:\n",
    "#    print(i + \" is a human\")\n",
    "#  else:\n",
    "#    print(i + \" is a horse\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-8EHQyWGDvWz"
   },
   "source": [
    "### Visualizing Intermediate Representations\n",
    "\n",
    "To get a feel for what kind of features your CNN has learned, one fun thing to do is to visualize how an input gets transformed as it goes through the model.\n",
    "\n",
    "You can pick a random image from the training set, and then generate a figure where each row is the output of a layer, and each image in the row is a specific filter in that output feature map. Rerun this cell to generate intermediate representations for a variety of training images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-5tES8rXFjux"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "from tensorflow.keras.utils import img_to_array, load_img\n",
    "\n",
    "# Define a new Model that will take an image as input, and will output\n",
    "# intermediate representations for all layers in the previous model after\n",
    "# the first.\n",
    "successive_outputs = [layer.output for layer in model.layers[1:]]\n",
    "visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)\n",
    "\n",
    "# Prepare a random input image from the training set.\n",
    "horse_img_files = [os.path.join(train_horse_dir, f) for f in train_horse_names]\n",
    "human_img_files = [os.path.join(train_human_dir, f) for f in train_human_names]\n",
    "img_path = random.choice(horse_img_files + human_img_files)\n",
    "\n",
    "img = load_img(img_path, target_size=(300, 300))  # this is a PIL image\n",
    "x = img_to_array(img)  # Numpy array with shape (300, 300, 3)\n",
    "x = x.reshape((1,) + x.shape)  # Numpy array with shape (1, 300, 300, 3)\n",
    "\n",
    "# Scale by 1/255\n",
    "x /= 255\n",
    "\n",
    "# Run the image through the network, thus obtaining all\n",
    "# intermediate representations for this image.\n",
    "successive_feature_maps = visualization_model.predict(x)\n",
    "\n",
    "# These are the names of the layers, so you can have them as part of the plot\n",
    "layer_names = [layer.name for layer in model.layers[1:]]\n",
    "\n",
    "# Display the representations\n",
    "for layer_name, feature_map in zip(layer_names, successive_feature_maps):\n",
    "  if len(feature_map.shape) == 4:\n",
    "\n",
    "    # Just do this for the conv / maxpool layers, not the fully-connected layers\n",
    "    n_features = feature_map.shape[-1]  # number of features in feature map\n",
    "\n",
    "    # The feature map has shape (1, size, size, n_features)\n",
    "    size = feature_map.shape[1]\n",
    "    \n",
    "    # Tile the images in this matrix\n",
    "    display_grid = np.zeros((size, size * n_features))\n",
    "    for i in range(n_features):\n",
    "      x = feature_map[0, :, :, i]\n",
    "      x -= x.mean()\n",
    "      x /= x.std()\n",
    "      x *= 64\n",
    "      x += 128\n",
    "      x = np.clip(x, 0, 255).astype('uint8')\n",
    "    \n",
    "      # Tile each filter into this big horizontal grid\n",
    "      display_grid[:, i * size : (i + 1) * size] = x\n",
    "    \n",
    "    # Display the grid\n",
    "    scale = 20. / n_features\n",
    "    plt.figure(figsize=(scale * n_features, scale))\n",
    "    plt.title(layer_name)\n",
    "    plt.grid(False)\n",
    "    plt.imshow(display_grid, aspect='auto', cmap='viridis')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tuqK2arJL0wo"
   },
   "source": [
    "You can see above how the pixels highlighted turn to increasingly abstract and compact representations, especially at the bottom grid. \n",
    "\n",
    "The representations downstream start highlighting what the network pays attention to, and they show fewer and fewer features being \"activated\"; most are set to zero. This is called _representation sparsity_ and is a key feature of deep learning. These representations carry increasingly less information about the original pixels of the image, but increasingly refined information about the class of the image. You can think of a convnet (or a deep network in general) as an information distillation pipeline wherein each layer filters out the most useful features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j4IBgYCYooGD"
   },
   "source": [
    "## Clean Up\n",
    "\n",
    "You will continue with a similar exercise in the next lab but before that, run the following cell to terminate the kernel and free memory resources:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "651IgjLyo-Jx"
   },
   "outputs": [],
   "source": [
    "from google.colab import runtime\n",
    "runtime.unassign()"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "C1_W4_Lab_1_image_generator_no_validation.ipynb",
   "private_outputs": true,
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}