C1/W4/ungraded_labs/C1_W4_Lab_3_compacted_images.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/https-deeplearning-ai/tensorflow-1-public/blob/master/C1/W4/ungraded_labs/C1_W4_Lab_3_compacted_images.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qR8Am0lBtRAx"
   },
   "source": [
    "# Ungraded Lab: Effect of Compacted Images in Training\n",
    "\n",
    "In this notebook, you will see how reducing the target size of the generator images will affect the architecture and performance of your model. This is a useful technique in case you need to speed up your training or save compute resources. Let's begin!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "D1iD7DhP2NWt"
   },
   "source": [
    "**IMPORTANT NOTE:** This notebook is designed to run as a Colab. Running it on your local machine might result in some of the code blocks throwing errors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qxY7KvGQ2Qdr"
   },
   "source": [
    "As before, start downloading the train and validation sets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RXZT2UsyIVe_"
   },
   "outputs": [],
   "source": [
    "# Download the training set\n",
    "!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/horse-or-human.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "0mLij6qde6Ox"
   },
   "outputs": [],
   "source": [
    "# Download the validation set\n",
    "!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/validation-horse-or-human.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9brUxyTpYZHy"
   },
   "source": [
    "Then unzip them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PLy3pthUS0D2"
   },
   "outputs": [],
   "source": [
    "import zipfile\n",
    "\n",
    "# Unzip training set\n",
    "local_zip = './horse-or-human.zip'\n",
    "zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
    "zip_ref.extractall('./horse-or-human')\n",
    "\n",
    "# Unzip validation set\n",
    "local_zip = './validation-horse-or-human.zip'\n",
    "zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
    "zip_ref.extractall('./validation-horse-or-human')\n",
    "\n",
    "zip_ref.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o-qUPyfO7Qr8"
   },
   "source": [
    "Then define the directories containing the images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NR_M9nWN-K8B"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Directory with training horse pictures\n",
    "train_horse_dir = os.path.join('./horse-or-human/horses')\n",
    "\n",
    "# Directory with training human pictures\n",
    "train_human_dir = os.path.join('./horse-or-human/humans')\n",
    "\n",
    "# Directory with validation horse pictures\n",
    "validation_horse_dir = os.path.join('./validation-horse-or-human/horses')\n",
    "\n",
    "# Directory with validation human pictures\n",
    "validation_human_dir = os.path.join('./validation-horse-or-human/humans')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z1wrZCxTPw4m"
   },
   "source": [
    "You can check that the directories are not empty and that the train set has more images than the validation set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_horse_names = os.listdir(train_horse_dir)\n",
    "print(f'TRAIN SET HORSES: {train_horse_names[:10]}')\n",
    "\n",
    "train_human_names = os.listdir(train_human_dir)\n",
    "print(f'TRAIN SET HUMANS: {train_human_names[:10]}')\n",
    "\n",
    "validation_horse_hames = os.listdir(validation_horse_dir)\n",
    "print(f'VAL SET HORSES: {validation_horse_hames[:10]}')\n",
    "\n",
    "validation_human_names = os.listdir(validation_human_dir)\n",
    "print(f'VAL SET HUMANS: {validation_human_names[:10]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ZTpdVrBg2LZC"
   },
   "outputs": [],
   "source": [
    "print(f'total training horse images: {len(os.listdir(train_horse_dir))}')\n",
    "print(f'total training human images: {len(os.listdir(train_human_dir))}')\n",
    "print(f'total validation horse images: {len(os.listdir(validation_horse_dir))}')\n",
    "print(f'total validation human images: {len(os.listdir(validation_human_dir))}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5oqBkNBJmtUv"
   },
   "source": [
    "## Build the Model\n",
    "\n",
    "The model will follow the same architecture as before but they key difference is in the `input_shape` parameter of the first `Conv2D` layer. Since you will be compacting the images later in the generator, you need to specify the expected image size here. So instead of 300x300 as in the previous two labs, you specify a smaller 150x150 array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PixZ2s5QbYQ3"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "model = tf.keras.models.Sequential([\n",
    "    # Note the input shape is the desired size of the image 150x150 with 3 bytes color\n",
    "    # This is the first convolution\n",
    "    tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
    "    tf.keras.layers.MaxPooling2D(2, 2),\n",
    "    # The second convolution\n",
    "    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # The third convolution\n",
    "    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "#     # The fourth convolution (You can uncomment the 4th and 5th conv layers later to see the effect)\n",
    "#     tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "#     tf.keras.layers.MaxPooling2D(2,2),\n",
    "#     # The fifth convolution\n",
    "#     tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "#     tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # Flatten the results to feed into a DNN\n",
    "    tf.keras.layers.Flatten(),\n",
    "    # 512 neuron hidden layer\n",
    "    tf.keras.layers.Dense(512, activation='relu'),\n",
    "    # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')\n",
    "    tf.keras.layers.Dense(1, activation='sigmoid')\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s9EaFDP5srBa"
   },
   "source": [
    "You can see the difference from previous models when you print the `model.summary()`. As expected, there will be less inputs to the `Dense` layer at the end of the model compared to the previous labs. This is because you used the same number of max pooling layers in your model. And since you have a smaller image to begin with (150 x 150), then the output after all the pooling layers will also be smaller."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7ZKj8392nbgP"
   },
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PEkKSpZlvJXA"
   },
   "source": [
    "You will use the same settings for training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8DHWhFP_uhq3"
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras.optimizers import RMSprop\n",
    "\n",
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer=RMSprop(learning_rate=0.001),\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Sn9m9D3UimHM"
   },
   "source": [
    "### Data Preprocessing\n",
    "\n",
    "Now you will instantiate the data generators. As mentioned before, you will be compacting the image by specifying the `target_size` parameter. See the simple change below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ClebU9NJg99G"
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "# All images will be rescaled by 1./255\n",
    "train_datagen = ImageDataGenerator(rescale=1/255)\n",
    "validation_datagen = ImageDataGenerator(rescale=1/255)\n",
    "\n",
    "# Flow training images in batches of 128 using train_datagen generator\n",
    "train_generator = train_datagen.flow_from_directory(\n",
    "        './horse-or-human/',  # This is the source directory for training images\n",
    "        target_size=(150, 150),  # All images will be resized to 150x150\n",
    "        batch_size=128,\n",
    "        # Since you used binary_crossentropy loss, you need binary labels\n",
    "        class_mode='binary')\n",
    "\n",
    "# Flow training images in batches of 128 using train_datagen generator\n",
    "validation_generator = validation_datagen.flow_from_directory(\n",
    "        './validation-horse-or-human/',  # This is the source directory for training images\n",
    "        target_size=(150, 150),  # All images will be resized to 150x150\n",
    "        batch_size=32,\n",
    "        # Since you used binary_crossentropy loss, you need binary labels\n",
    "        class_mode='binary')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mu3Jdwkjwax4"
   },
   "source": [
    "### Training\n",
    "\n",
    "Now you're ready to train and see the results. Note your observations about how fast the model trains and the accuracies you're getting in the train and validation sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Fb1_lgobv81m"
   },
   "outputs": [],
   "source": [
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=8,  \n",
    "      epochs=15,\n",
    "      verbose=1,\n",
    "      validation_data = validation_generator,\n",
    "      validation_steps=8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o6vSHzPR2ghH"
   },
   "source": [
    "### Model Prediction\n",
    "\n",
    "As usual, it is also good practice to try running your model over some handpicked images. See if you got better, worse, or the same performance as the previous lab.\n",
    "\n",
    "_**Note:** Old versions of the Safari browser might have compatibility issues with the code block below. If you get an error after you select the images(s) to upload, you can consider updating your browser to the latest version. If not possible, please comment out or skip the code block below, uncomment the next code block and run it._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "DoWp43WxJDNT"
   },
   "outputs": [],
   "source": [
    "## NOTE: If you are using Safari and this cell throws an error,\n",
    "## please skip this block and run the next one instead.\n",
    "\n",
    "import numpy as np\n",
    "from google.colab import files\n",
    "from tensorflow.keras.utils import load_img, img_to_array\n",
    "\n",
    "uploaded = files.upload()\n",
    "\n",
    "for fn in uploaded.keys():\n",
    " \n",
    "  # predicting images\n",
    "  path = '/content/' + fn\n",
    "  img = load_img(path, target_size=(150, 150))\n",
    "  x = img_to_array(img)\n",
    "  x /= 255\n",
    "  x = np.expand_dims(x, axis=0)\n",
    "\n",
    "  images = np.vstack([x])\n",
    "  classes = model.predict(images, batch_size=10)\n",
    "  print(classes[0])\n",
    "  if classes[0]>0.5:\n",
    "    print(fn + \" is a human\")\n",
    "  else:\n",
    "    print(fn + \" is a horse\")\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ckps9Sw4657d"
   },
   "source": [
    "If you're using Safari and the cell above throws an error, you will need to upload the images(s) manually in their workspace.\n",
    "\n",
    "Instructions on how to upload image(s) manually in a Colab:\n",
    "\n",
    "1. Select the `folder` icon on the left `menu bar`.\n",
    "2. Click on the `folder with an arrow pointing upwards` named `..`\n",
    "3. Click on the `folder` named `tmp`.\n",
    "4. Inside of the `tmp` folder, `create a new folder` called `images`. You'll see the `New folder` option by clicking the `3 vertical dots` menu button next to the `tmp` folder.\n",
    "5. Inside of the new `images` folder, upload an image(s) of your choice, preferably of either a horse or a human. Drag and drop the images(s) on top of the `images` folder.\n",
    "6. Uncomment and run the code block below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "v_GgQjRT65oM"
   },
   "outputs": [],
   "source": [
    "# # CODE BLOCK FOR OLDER VERSIONS OF SAFARI\n",
    "\n",
    "# import numpy as np\n",
    "# from tensorflow.keras.utils import load_img, img_to_array\n",
    "# import os\n",
    "\n",
    "# images = os.listdir(\"/tmp/images\")\n",
    "\n",
    "# print(images)\n",
    "\n",
    "# for i in images:\n",
    "#  print()\n",
    "#  # predicting images\n",
    "#  path = '/tmp/images/' + i\n",
    "#  img = load_img(path, target_size=(150, 150))\n",
    "#  x = img_to_array(img)\n",
    "#  x /= 255\n",
    "#  x = np.expand_dims(x, axis=0)\n",
    "\n",
    "#  images = np.vstack([x])\n",
    "#  classes = model.predict(images, batch_size=10)\n",
    "#  print(classes[0])\n",
    "#  if classes[0]>0.5:\n",
    "#    print(i + \" is a human\")\n",
    "#  else:\n",
    "#    print(i + \" is a horse\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-8EHQyWGDvWz"
   },
   "source": [
    "### Visualizing Intermediate Representations\n",
    "\n",
    "You can also look again at the intermediate representations. You will notice that the output at the last convolution layer is even more abstract because it contains fewer pixels than before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-5tES8rXFjux"
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import random\n",
    "from tensorflow.keras.utils import img_to_array, load_img\n",
    "\n",
    "# Define a new Model that will take an image as input, and will output\n",
    "# intermediate representations for all layers in the previous model after\n",
    "# the first.\n",
    "successive_outputs = [layer.output for layer in model.layers[1:]]\n",
    "visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)\n",
    "\n",
    "# Prepare a random input image from the training set.\n",
    "horse_img_files = [os.path.join(train_horse_dir, f) for f in train_horse_names]\n",
    "human_img_files = [os.path.join(train_human_dir, f) for f in train_human_names]\n",
    "img_path = random.choice(horse_img_files + human_img_files)\n",
    "img = load_img(img_path, target_size=(150, 150))  # this is a PIL image\n",
    "x = img_to_array(img)  # Numpy array with shape (150, 150, 3)\n",
    "x = x.reshape((1,) + x.shape)  # Numpy array with shape (1, 150, 150, 3)\n",
    "\n",
    "# Scale by 1/255\n",
    "x /= 255\n",
    "\n",
    "# Run the image through the network, thus obtaining all\n",
    "# intermediate representations for this image.\n",
    "successive_feature_maps = visualization_model.predict(x)\n",
    "\n",
    "# These are the names of the layers, so you can have them as part of the plot\n",
    "layer_names = [layer.name for layer in model.layers[1:]]\n",
    "\n",
    "# Display the representations\n",
    "for layer_name, feature_map in zip(layer_names, successive_feature_maps):\n",
    "  if len(feature_map.shape) == 4:\n",
    "\n",
    "    # Just do this for the conv / maxpool layers, not the fully-connected layers\n",
    "    n_features = feature_map.shape[-1]  # number of features in feature map\n",
    "\n",
    "    # The feature map has shape (1, size, size, n_features)\n",
    "    size = feature_map.shape[1]\n",
    "    \n",
    "    # Tile the images in this matrix\n",
    "    display_grid = np.zeros((size, size * n_features))\n",
    "    for i in range(n_features):\n",
    "      x = feature_map[0, :, :, i]\n",
    "      x -= x.mean()\n",
    "      x /= x.std()\n",
    "      x *= 64\n",
    "      x += 128\n",
    "      x = np.clip(x, 0, 255).astype('uint8')\n",
    "    \n",
    "      # Tile each filter into this big horizontal grid\n",
    "      display_grid[:, i * size : (i + 1) * size] = x\n",
    "    \n",
    "    # Display the grid\n",
    "    scale = 20. / n_features\n",
    "    plt.figure(figsize=(scale * n_features, scale))\n",
    "    plt.title(layer_name)\n",
    "    plt.grid(False)\n",
    "    plt.imshow(display_grid, aspect='auto', cmap='viridis')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j4IBgYCYooGD"
   },
   "source": [
    "## Clean Up\n",
    "\n",
    "Please run the following cell to terminate the kernel and free memory resources:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "651IgjLyo-Jx"
   },
   "outputs": [],
   "source": [
    "from google.colab import runtime\n",
    "runtime.unassign()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tFnBvcIrXWW2"
   },
   "source": [
    "## Wrap Up\n",
    "\n",
    "In this lab, you saw how compacting images affected your previous model. This is one technique to keep in mind especially when you are still in the exploratory phase of your own projects. You can see if a smaller model behaves just as well as a large model so you can have faster training. You also saw how easy it is to customize your images for this adjustment in size by simply changing a parameter in the `ImageDataGenerator` class."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "C1_W4_Lab_3_compacted_images.ipynb",
   "private_outputs": true,
   "provenance": [
    {
     "file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C1/C1/W4/ungraded_labs/C1_W4_Lab_3_compacted_images.ipynb",
     "timestamp": 1639112249467
    }
   ],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}