site/en/tutorials/images/data_augmentation_tmp.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Augmentation.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "both",
        "colab_type": "code",
        "id": "tuOe1ymfHZPu",
        "colab": {}
      },
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "35lZ8kr3UcsB",
        "colab_type": "text"
      },
      "source": [
        "#Tutorial on Data Augmentation using tf.image\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "MfBg1C5NB3X0"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/not_a_real_link\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/drive/1skGIQhwifJY6HWO6ZnbFe4un-VuJ3VW5\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/tools/templates/notebook.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/tools/templates/notebook.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BZP72A6eFw74",
        "colab_type": "text"
      },
      "source": [
        "## Overview"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VeyiaI_ZXfUd",
        "colab_type": "text"
      },
      "source": [
        "When training a Machine Learning model, especially Deep Learning models, We require a lot of data. If there is a scarcity of data, then the model is prone to underfitting. Unfortunately, scarcity of data is very common. So, we have to make use of the handful amount of data we have. Also, creating a variation in data helps in preventing overfitting as well.\n",
        "\n",
        "In this scenario, using data augmentation, We could increase the number of examples of a dataset. We take the provided data and augment it i.e. crop the image, rotate it, change colors, flip, etc. Doing this increases the number of samples and increase variation and helps us to create a good model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-dn3eTWFpPFH",
        "colab_type": "text"
      },
      "source": [
        "Now, Let's Learn data augmentation using tf.image"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8sZIVqk7HvnC",
        "colab_type": "text"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rdP8EQbPsyRA",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "52494cf2-830e-46ed-eab1-22b1271ad880"
      },
      "source": [
        "%tensorflow_version 2.x\n",
        "\n",
        "from __future__ import absolute_import, division, print_function, unicode_literals\n",
        "\n",
        "import tensorflow as tf\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "from tensorflow.keras.datasets import mnist\n",
        "from tensorflow.keras import layers\n",
        "import urllib"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "TensorFlow 2.x selected.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "__EuXwM-8uth",
        "colab_type": "text"
      },
      "source": [
        "We will be checking the data augmentation feautres on an image and we will be augmenting a whole dataset later to train a model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "frBSdODBLOOI",
        "colab_type": "text"
      },
      "source": [
        "Downloading an image from internet for augmentation."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s5ThIwG8KqzI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "urllib.request.urlretrieve(\"https://static.makeuseof.com/wp-content/uploads/2016/10/camera-photo-lens-stock-images-670x335.jpg\", \"a.jpg\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-Ec3bGonGDCF",
        "colab_type": "text"
      },
      "source": [
        "Reading and decoding the image to tensor format."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cdCoB8b-uZjf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "image_path = 'a.jpg'\n",
        " \n",
        "image_string=tf.io.read_file(image_path)\n",
        " \n",
        "image=tf.image.decode_jpeg(image_string,channels=3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "isGwyT0386yi",
        "colab_type": "text"
      },
      "source": [
        "A function to visualize and compare the original and augmented image side by side."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FKnRfw2dvyql",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def visualize(original, augmented):\n",
        "  fig = plt.figure()\n",
        "  original_plt=fig.add_subplot(1,2,1)\n",
        "  original_plt.set_title('original image')\n",
        "  original_plt.imshow(original)\n",
        "\n",
        "  augmented_plt=fig.add_subplot(1,2,2) \n",
        "  augmented_plt.set_title('augmented image')\n",
        "  augmented_plt.imshow(augmented)\n",
        "  plt.show(block=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jYLzpEOhGqWY",
        "colab_type": "text"
      },
      "source": [
        "## Augmenting single image"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8IiXghY99Bo6",
        "colab_type": "text"
      },
      "source": [
        "### Flipping the image\n",
        "You could flip the image either vertically or horizontally."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X14VjLlFxnvZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "flipped = tf.image.flip_left_right(image)\n",
        "visualize(image,flipped)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ObsvSmu99MfC",
        "colab_type": "text"
      },
      "source": [
        "### Grayscale the image\n",
        "You could grayscale an image."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mnqQA2ubyo6O",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "grayscaled = tf.image.rgb_to_grayscale(image)\n",
        "visualize(image, tf.squeeze(grayscaled))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "juI4A4HF9gYc",
        "colab_type": "text"
      },
      "source": [
        "### Saturate the image\n",
        "you could saturate an image by providing a saturation factor."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tiTUhw-gzCJW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "saturated = tf.image.adjust_saturation(image, 3)\n",
        "visualize(image, saturated)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E82CqomP9qcR",
        "colab_type": "text"
      },
      "source": [
        "### Change image brightness\n",
        "You could change the brightness of image by providing a brightness factor."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "05dA6uEtzfyd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bright = tf.image.adjust_brightness(image, 0.4)\n",
        "visualize(image, bright)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5_0kMbmS91x6",
        "colab_type": "text"
      },
      "source": [
        "### Rotate the image\n",
        "You could rotate an image to your desired angles."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "edNoQzhszxo8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "rotated = tf.image.rot90(image)\n",
        "visualize(image,rotated)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bomBnFWp9895",
        "colab_type": "text"
      },
      "source": [
        "### Center crop the image\n",
        "You could crop the image from center upto the image part you desire."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fvgz_6t21dq2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "cropped = tf.image.central_crop(image, central_fraction=0.5)\n",
        "visualize(image,cropped)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8W5E_c7o-H96",
        "colab_type": "text"
      },
      "source": [
        "There are lots of other augmentation options available. You could check them by seeing the available attributes of tf.image. For now, We will be moving to next step."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "92lBGZSQ-1Tx",
        "colab_type": "text"
      },
      "source": [
        "## Augmenting a Dataset and training a model on it"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lrDez4xIX9Ss",
        "colab_type": "text"
      },
      "source": [
        "We will be training our model on mnist dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SDjIFBH4_ctU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow_datasets as tfds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mazlEonS_gTR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dataset, info =  tfds.load('mnist', as_supervised=True, with_info=True)\n",
        "train_dataset, test_dataset = dataset['train'], dataset['test']\n",
        "\n",
        "num_train_examples= info.splits['train'].num_examples"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "011caOa0YCz5",
        "colab_type": "text"
      },
      "source": [
        "In this scale function, we will be augmenting our image. The dataset will be passed to this function and it will augment the data and return the dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3oaSV5QcDS8p",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def scale(image,label):\n",
        "  image = tf.image.resize(image, (28, 28))/255.0 #normalizing the image\n",
        "  image = tf.image.random_crop(image, size=[28,28,1]) #providing random crop to image\n",
        "  image = tf.image.random_brightness(image, max_delta=0.5) #providing random brightness to image\n",
        "  image = tf.image.random_flip_left_right(image) #providing random flip to image\n",
        "\n",
        "  return image,label\n",
        "\n",
        "BATCH_SIZE = 64\n",
        "\n",
        "train_batches      = train_dataset.shuffle(num_train_examples//4).map(scale).batch(BATCH_SIZE).prefetch(1) #A batch dataset that can be directly passed to model for training\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yi9TIwR-ZIOi",
        "colab_type": "text"
      },
      "source": [
        "Creating and compiling the model. The model will be a 2 layered Fully connected neural network without convolution."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hHhkA4Q0CsHx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = tf.keras.Sequential([\n",
        "            layers.Flatten(input_shape=(28, 28,1)),\n",
        "            layers.Dense(256, activation='relu'),\n",
        "            layers.Dense(128, activation='relu'),\n",
        "            layers.Dense(10, activation='softmax')\n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "17wqPAAoNe3N",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.compile(optimizer = 'adam',\n",
        "                    loss='sparse_categorical_crossentropy',\n",
        "                    metrics=['accuracy'])\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P0rciou3ZWwy",
        "colab_type": "text"
      },
      "source": [
        "Training the model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z8X8CpqvNhG9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.fit(train_batches, epochs=5)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UEqeeNsHZaC5",
        "colab_type": "text"
      },
      "source": [
        "## Conclusion:\n",
        "As we can see, the model provided 96.31% accuracy on training set. This is slightly higher than the model trained without data augmentation. So, Data Augmentation is good. It didn't provide much significance on this model because the dataset already had a large number of samples. But, on a small dataset, you could see a huge difference."
      ]
    }
  ]
}