C2/W4/ungraded_lab/C2_W4_Lab_1_multi_class_classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/https-deeplearning-ai/tensorflow-1-public/blob/master/C2/W4/ungraded_lab/C2_W4_Lab_1_multi_class_classifier.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0UD-1_xY-h2u"
   },
   "source": [
    "# Ungraded Lab: Multi-class Classifier\n",
    "\n",
    "In this lab, you will look at how to build a model to distinguish between more than two classes. The code will be similar to the ones you've been using before with a few key changes in the model and in the training parameters. Let's dive in!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MZvja3XzDQsX"
   },
   "source": [
    "**IMPORTANT NOTE:** This notebook is designed to run as a Colab. Running the notebook on your local machine might result in some of the code blocks throwing errors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FvwVR5lHA8q_"
   },
   "source": [
    "## Download and Prepare the Dataset\n",
    "\n",
    "You will be using the [Rock-Paper-Scissors dataset](https://www.tensorflow.org/datasets/catalog/rock_paper_scissors), a gallery of hands images in Rock, Paper, and Scissors poses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download the train set\n",
    "!wget https://storage.googleapis.com/tensorflow-1-public/course2/week4/rps.zip\n",
    "    \n",
    "# Download the test set\n",
    "!wget https://storage.googleapis.com/tensorflow-1-public/course2/week4/rps-test-set.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PnYP_HhYNVUK"
   },
   "outputs": [],
   "source": [
    "import zipfile\n",
    "\n",
    "# Extract the archive\n",
    "local_zip = './rps.zip'\n",
    "zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
    "zip_ref.extractall('tmp/rps-train')\n",
    "zip_ref.close()\n",
    "\n",
    "local_zip = './rps-test-set.zip'\n",
    "zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
    "zip_ref.extractall('tmp/rps-test')\n",
    "zip_ref.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "k3vqjYrpB0hI"
   },
   "source": [
    "As usual, you will assign the directory names into variables and look at the filenames as a sanity check."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MrxdR83ANgjS"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "base_dir = 'tmp/rps-train/rps'\n",
    "\n",
    "rock_dir = os.path.join(base_dir, 'rock')\n",
    "paper_dir = os.path.join(base_dir, 'paper')\n",
    "scissors_dir = os.path.join(base_dir, 'scissors')\n",
    "\n",
    "print('total training rock images:', len(os.listdir(rock_dir)))\n",
    "print('total training paper images:', len(os.listdir(paper_dir)))\n",
    "print('total training scissors images:', len(os.listdir(scissors_dir)))\n",
    "\n",
    "rock_files = os.listdir(rock_dir)\n",
    "print(rock_files[:10])\n",
    "\n",
    "paper_files = os.listdir(paper_dir)\n",
    "print(paper_files[:10])\n",
    "\n",
    "scissors_files = os.listdir(scissors_dir)\n",
    "print(scissors_files[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7t_CNSs6B-8y"
   },
   "source": [
    "You can also inspect some of the images to see the variety in your model inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jp9dLel9N9DS"
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as mpimg\n",
    "\n",
    "pic_index = 2\n",
    "\n",
    "next_rock = [os.path.join(rock_dir, fname) \n",
    "                for fname in rock_files[pic_index-2:pic_index]]\n",
    "next_paper = [os.path.join(paper_dir, fname) \n",
    "                for fname in paper_files[pic_index-2:pic_index]]\n",
    "next_scissors = [os.path.join(scissors_dir, fname) \n",
    "                for fname in scissors_files[pic_index-2:pic_index]]\n",
    "\n",
    "for i, img_path in enumerate(next_rock+next_paper+next_scissors):\n",
    "  img = mpimg.imread(img_path)\n",
    "  plt.imshow(img)\n",
    "  plt.axis('Off')\n",
    "  plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ufa0YF5oCpYw"
   },
   "source": [
    "## Build the model\n",
    "\n",
    "You will then build your CNN. You will use 4 convolution layers with 64-64-128-128 filters then append a `Dropout` layer to avoid overfitting and some Dense layers for the classification. The output layer would be a 3-neuron dense layer activated by [Softmax](https://www.tensorflow.org/api_docs/python/tf/nn/softmax). You've seen this in Course 1 when you were training with Fashion MNIST. It scales your output to a set of probabilities that add up to 1. The order of this 3-neuron output would be `paper`-`rock`-`scissors` (e.g. a `[0.8 0.2 0.0]` output means the model is prediciting 80% probability for paper and 20% probability for rock.\n",
    "\n",
    "You can examine the architecture with `model.summary()` below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GgvGg2nsCj-0"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "model = tf.keras.models.Sequential([\n",
    "    # Note the input shape is the desired size of the image 150x150 with 3 bytes color\n",
    "    # This is the first convolution\n",
    "    tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
    "    tf.keras.layers.MaxPooling2D(2, 2),\n",
    "    # The second convolution\n",
    "    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # The third convolution\n",
    "    tf.keras.layers.Conv2D(128, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # The fourth convolution\n",
    "    tf.keras.layers.Conv2D(128, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # Flatten the results to feed into a DNN\n",
    "    tf.keras.layers.Flatten(),\n",
    "    tf.keras.layers.Dropout(0.5),\n",
    "    # 512 neuron hidden layer\n",
    "    tf.keras.layers.Dense(512, activation='relu'),\n",
    "    tf.keras.layers.Dense(3, activation='softmax')\n",
    "])\n",
    "\n",
    "# Print the model summary\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4P1iuHGiFrPV"
   },
   "source": [
    "You will then compile the model. The key change here is the `loss` function. Whereas before you were using `binary_crossentropy` for 2 classes, you will change it to [categorical_crossentropy](https://keras.io/api/losses/probabilistic_losses/#categoricalcrossentropy-function) to extend it to more classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "OskuZ2ThFqmg"
   },
   "outputs": [],
   "source": [
    "# Set the training parameters\n",
    "model.compile(loss = 'categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7Ps7kIRaFRIC"
   },
   "source": [
    "## Prepare the ImageDataGenerator\n",
    "\n",
    "You will prepare the generators as before. You will set the training set up for data augmentation so it can mimick other poses that the model needs to learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "LWTisYLQM1aM"
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "TRAINING_DIR = \"tmp/rps-train/rps\"\n",
    "training_datagen = ImageDataGenerator(\n",
    "      rescale = 1./255,\n",
    "\t    rotation_range=40,\n",
    "      width_shift_range=0.2,\n",
    "      height_shift_range=0.2,\n",
    "      shear_range=0.2,\n",
    "      zoom_range=0.2,\n",
    "      horizontal_flip=True,\n",
    "      fill_mode='nearest')\n",
    "\n",
    "VALIDATION_DIR = \"tmp/rps-test/rps-test-set\"\n",
    "validation_datagen = ImageDataGenerator(rescale = 1./255)\n",
    "\n",
    "train_generator = training_datagen.flow_from_directory(\n",
    "\tTRAINING_DIR,\n",
    "\ttarget_size=(150,150),\n",
    "\tclass_mode='categorical',\n",
    "  batch_size=126\n",
    ")\n",
    "\n",
    "validation_generator = validation_datagen.flow_from_directory(\n",
    "\tVALIDATION_DIR,\n",
    "\ttarget_size=(150,150),\n",
    "\tclass_mode='categorical',\n",
    "  batch_size=126\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Orf1QQlGGyOe"
   },
   "source": [
    "## Train the model and evaluate the results\n",
    "\n",
    "You will train for 25 epochs and evaludate the results afterwards. Observe how both the training and validation accuracy are trending upwards. This is a good indication that the model is not overfitting to only your training set.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1mHX5L7HFXQ7"
   },
   "outputs": [],
   "source": [
    "# Train the model\n",
    "history = model.fit(train_generator, epochs=25, steps_per_epoch=20, validation_data = validation_generator, verbose = 1, validation_steps=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "aeTRVCr6aosw"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Plot the results\n",
    "acc = history.history['accuracy']\n",
    "val_acc = history.history['val_accuracy']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'r', label='Training accuracy')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend(loc=0)\n",
    "plt.figure()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y3ps8Q1tpYMG"
   },
   "source": [
    "# Model Prediction\n",
    "\n",
    "You should be able to upload an image here and have it classified without crashing. This code block will only work in Google Colab, however. You can use your own images or use the ones available [here](https://storage.googleapis.com/tensorflow-1-public/course2/week4/rps-validation.zip)\n",
    "\n",
    "_**Note:** Old versions of the Safari browser might have compatibility issues with the code block below. If you get an error after you select the images(s) to upload, you can consider updating your browser to the latest version. If not possible, please comment out or skip the code block below, uncomment the next code block and run it._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ZABJp7T3VLCU"
   },
   "outputs": [],
   "source": [
    "## NOTE: If you are using Safari and this cell throws an error,\n",
    "## please skip this block and run the next one instead.\n",
    "\n",
    "import numpy as np\n",
    "from google.colab import files\n",
    "from tensorflow.keras.utils import load_img, img_to_array\n",
    "\n",
    "uploaded = files.upload()\n",
    "\n",
    "for fn in uploaded.keys():\n",
    " \n",
    "  # predicting images\n",
    "  path = fn\n",
    "  img = load_img(path, target_size=(150, 150))\n",
    "  x = img_to_array(img)\n",
    "  x = np.expand_dims(x, axis=0)\n",
    "\n",
    "  images = np.vstack([x])\n",
    "  classes = model.predict(images, batch_size=10)\n",
    "  print(fn)\n",
    "  print(classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6Rsy_ebEDQsc"
   },
   "source": [
    "If you're using Safari and the cell above throws an error, you will need to upload the images(s) manually in their workspace.\n",
    "\n",
    "Instructions on how to upload image(s) manually in a Colab:\n",
    "\n",
    "1. Select the `folder` icon on the left `menu bar`.\n",
    "2. Click on the `folder with an arrow pointing upwards` named `..`\n",
    "3. Click on the `folder` named `tmp`.\n",
    "4. Inside of the `tmp` folder, `create a new folder` called `images`. You'll see the `New folder` option by clicking the `3 vertical dots` menu button next to the `tmp` folder.\n",
    "5. Inside of the new `images` folder, upload an image(s) of your choice. Drag and drop the images(s) on top of the `images` folder.\n",
    "6. Uncomment and run the code block below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MBt6ae3YDQsc"
   },
   "outputs": [],
   "source": [
    "# # CODE BLOCK FOR OLDER VERSIONS OF SAFARI\n",
    "\n",
    "# import os\n",
    "# import numpy as np\n",
    "# from tensorflow.keras.utils import load_img, img_to_array\n",
    "\n",
    "# images = os.listdir(\"/tmp/images\")\n",
    "\n",
    "# print(images)\n",
    "\n",
    "# for i in images:\n",
    "#     print()\n",
    "#     # predicting images\n",
    "#     path = '/tmp/images/' + i\n",
    "#     img = load_img(path, target_size=(150, 150))\n",
    "#     x = img_to_array(img)\n",
    "#     x = np.expand_dims(x, axis=0)\n",
    "    \n",
    "#     images = np.vstack([x])\n",
    "#     classes = model.predict(images, batch_size=10)\n",
    "#     print(path)\n",
    "#     print(classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JHRufhQYJJLU"
   },
   "source": [
    "## Wrap Up\n",
    "\n",
    "That concludes this short exercise on the multi-class classifiers. You saw that with just a few changes, you were able to convert your binary classifiers to predict more classes. You used the same techniques for data and model preparation and were able to get relatively good results in just 25 epochs. For practice, you can search for other datasets (e.g. [here](https://archive.ics.uci.edu/datasets)) with more classes and revise the model to accomodate it. Try to experiment with different layers and data augmentation techniques to improve your metrics."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "C2_W4_Lab_1_multi_class_classifier.ipynb",
   "private_outputs": true,
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}