Image_Processing_with_OpenCV.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://cognitiveclass.ai/\">\n",
    "    <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/CV0101/Logo/SNLogo.png\" width=\"200\" align=\"center\">\n",
    "</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Lab - Image Processing and Analysis in Python with OpenCV</h1>\n",
    "<b>Welcome!</b> After completing this lab you will:\n",
    "<h5> 1. Learn to download, read and display images using Python, OpenCV and Matplotlib </h5>\n",
    "<h5> 2. Learn to apply various Image Processing techniques using Python and OpenCV </h5>\n",
    "<h5> 3. Understand about the importance of various Image Processing techniques </h5>\n",
    "<h5> 4. Analyze images using Histograms </h5>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "    <br>\n",
    "    <br>\n",
    "    <h2>Table of Contents</h2>\n",
    "    <ul>\n",
    "        <li><a href=\"#ref0\">OpenCV</a></li>\n",
    "        <li><a href=\"#ref1\">Downloading and plotting images in Python using OpenCV and Matplotlib</a></li>\n",
    "        <li><a href=\"#ref2\">Image Processing using OpenCV</a></li>\n",
    "        <li><a href=\"#ref3\">Analyzing Images using Histograms</a></li>\n",
    "    <li><a href=\"#ref4\">Exercises</a></li>\n",
    "    </ul>\n",
    "    <br>\n",
    "    <p>Estimated Time Needed: <strong>1 hr</strong></p>\n",
    "</div>\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/CV0101/Images/CV0101-image-processing-lab.png\" width=\"640\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref0\"></a>\n",
    "\n",
    "## OpenCV [Homepage](http://opencv.org/)\n",
    "- OpenCV stands for: Open Source Computer Vision Library\n",
    "- includes C++, C, Python and Java interfaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import the OpenCV package"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Did you know?** The package is called **`cv2`** in Python. The `2` in `cv2` doesn't refer to a particular package version."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "print(cv2.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref1\"></a>\n",
    "\n",
    "## Downloading and plotting images in Python using OpenCV and Matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Download an image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first download an image, using the `urllib` package. We'll need to specify a `url` of an image and a `filename`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll be downloading the following image of Donald Trump and Justin Trudeau, the President of the U.S. and Prime Minister of Canada, respectively.\n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/7/76/Donald_Trump_Justin_Trudeau_2017-02-13_02.jpg\" width=400>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "\n",
    "#Set the url and filename\n",
    "trumptrudeau_url = \"https://upload.wikimedia.org/wikipedia/commons/7/76/Donald_Trump_Justin_Trudeau_2017-02-13_02.jpg\"\n",
    "trumptrudeau_filename = \"trumptrudeau.jpg\"\n",
    "\n",
    "\n",
    "urllib.request.urlretrieve(trumptrudeau_url, trumptrudeau_filename) # downloads file as \"trumptrudeau.jpg\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Verify that the image has downloaded**\n",
    "\n",
    "At this point, you should now see the file listed in the files directory in the left-sidebar of the JupyterLab environment. If this side menu is hidden, you can go to `View`>`View Left-Sidebar`.\n",
    "\n",
    "Alternatively, you can run the command below to check the files in your current directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.listdir(os.curdir) #shows all files in current directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Is trump_filename in your directory?\n",
    "print(trumptrudeau_filename in os.listdir(os.curdir))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Plotting images in Jupyter Notebooks**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's display the image into this notebook, using OpenCV. We'll also be borrowing from the plotting library, `matplotlib`, to help display the images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "trumptrudeau = cv2.imread(trumptrudeau_filename)\n",
    "\n",
    "plt.imshow(trumptrudeau)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref2\"></a>\n",
    "## Image Processing using OpenCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hm... the Presidents are looking a little _blue_ in the above photo, wouldn't you say?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fix colors when displaying images**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "img_corrected = cv2.cvtColor(trumptrudeau, cv2.COLOR_BGR2RGB)\n",
    "\n",
    "plt.imshow(img_corrected)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Remove the axes around the image**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.axis(\"off\") #remove axes ticks\n",
    "plt.imshow(img_corrected)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Change the size of the displayed image in the notebook**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that this simply changes the display size of the image, not the actual image dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pylab import rcParams\n",
    "\n",
    "rcParams['figure.figsize'] = 10, 12\n",
    "\n",
    "plt.axis(\"off\") #remove axes ticks\n",
    "plt.imshow(img_corrected)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Convert to Grayscale**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gray_trumptrudeau = cv2.cvtColor(trumptrudeau, cv2.COLOR_BGR2GRAY)\n",
    "\n",
    "plt.imshow(gray_trumptrudeau, cmap = 'gray')\n",
    "plt.axis(\"off\") #remove axes ticks\n",
    "plt.title('Grayscale Image')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Canny Edge Detection**\n",
    "\n",
    "Canny Edge Detection is an algorithm used to detect edges in an image, and was developed by John F. Canny in 1986.  \n",
    "Full OpenCV documentation on Canny Edge Detection: [here](https://docs.opencv.org/master/da/d22/tutorial_py_canny.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rcParams['figure.figsize'] = 10, 12\n",
    "\n",
    "\n",
    "edges = cv2.Canny(img_corrected, \n",
    "                  threshold1=100, \n",
    "                  threshold2=200)\n",
    "\n",
    "plt.imshow(edges,cmap = 'gray')\n",
    "plt.title('Edge Image'), plt.xticks([]), plt.yticks([])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Try different levels of threshold:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rcParams['figure.figsize'] = 10, 12\n",
    "\n",
    "edges = cv2.Canny(img_corrected, \n",
    "                  threshold1=1,   ## try different values here\n",
    "                  threshold2=200) ## try different values here\n",
    "\n",
    "plt.imshow(edges,cmap = 'gray')\n",
    "plt.title('Edge Image'), plt.xticks([]), plt.yticks([])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rcParams['figure.figsize'] = 10, 12\n",
    "\n",
    "edges = cv2.Canny(img_corrected, \n",
    "                  threshold1=100,   ## try different values here\n",
    "                  threshold2=500) ## try different values here\n",
    "\n",
    "plt.imshow(edges,cmap = 'gray')\n",
    "plt.title('Edge Image'), plt.xticks([]), plt.yticks([])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Want to know the parameters of a function?**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try using a question mark `?` before a method to display the help doc for the method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "?cv2.Canny"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref3\"></a>\n",
    "## Analyzing Images using Histograms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Histograms - Grayscale**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Histograms allow you to quickly determine the number of pixels that are on a scale \"black pixel\" (0) to \"white pixel\" (255)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Execute the code cell below to see the histogram of the image of President Trump and President Trudeau:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "rcParams['figure.figsize'] = 8,4\n",
    "\n",
    "plt.hist(gray_trumptrudeau.ravel(),256,[0,256])\n",
    "plt.title('Histogram of Grayscale trumptrudeau.jpg')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What's in the histogram?**\n",
    "\n",
    "Like any other histogram, this graph shows the **count** of something. In this case, it's the _number of pixels in the image_ that are either _dark_ (towards the left of the graph) or _lighter_ (towards the right of the graph)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Why are histograms important**?\n",
    "- If you're concerned about whether the image is properly exposed (not too dark and not too bright), then histograms are a great way of checking the distribution of your lighter and darker pixels!\n",
    "- If your images are overexposed (too bright) or underexposed (too dark), you can fix exposure with OpenCV in Python (although you may lose some image quality). \n",
    "- This might be important if you're developing an app where users are taking photos and your app is trying to classify the objects inside the image. If it's too dark or too light, the objects may not be recognizable. \n",
    "\n",
    "\n",
    "**How do you read the histogram?**\n",
    "- On the **x-axis**, the values normally range from 0 (black) to 255 (white). So darker pixels are on the left, and whiter pixels are to the right. In the `matplotlib` chart above, the x-axis goes from 0 to 256, since each pixel is represented as a bin start from 0 to 0.99, and ends with 255 to 255.99.   \n",
    "- The **y-axis** shows the number of pixels found in the image on the scale of black to white.\n",
    "\n",
    "**Observations:**\n",
    "The grayscale histogram above shows that: \n",
    "- lots of pixels around `x = 30`, which are **darker pixels**.\n",
    "- lots of pixels around `x = 200`, which are **lighter pixels**.\n",
    "\n",
    "**Interpretation:**\n",
    "- the image appears to have lots of very dark, and very light pixels, but not a lot of pixels in between. Looking at the image of the Presidents, this is readily apparent -- the darker pixels seem to come from the black suits and the fireplace. A lot of the lighter pixels come from the furniture and walls."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Histograms - Color scale (RGB)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Interpreting RGB histograms\n",
    "Like grayscale histograms, RGB also shows the count distribution of pixels from black (`x=0`) to full color (`x=255`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a color image of the presidents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rcParams['figure.figsize'] = 10, 12\n",
    "\n",
    "plt.axis(\"off\") #remove axes ticks\n",
    "plt.imshow(img_corrected)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the following is an RGB histogram for the above image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### RGB histogram"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rcParams['figure.figsize'] = 8, 4\n",
    "\n",
    "color = ('b','g','r')\n",
    "for i,col in enumerate(color):\n",
    "    histr = cv2.calcHist([trumptrudeau],[i],None,[256],[0,256])\n",
    "    plt.plot(histr,color = col)\n",
    "    plt.xlim([0,256])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the histogram above, you can observe that:\n",
    "- of the lighter pixels (around `x >= 200`), there are more red pixels than green or blue pixels. This is likely because of the redness in the faces of the presidents.\n",
    "- there seems to be a similar amount of dark red, dark blue, and dark green pixels (around `x<50`)\n",
    "- in terms of middle-tones, there seems to be more blue pixels than green or red. This is likely because of the neckties worn by the presidents."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"ref4\"></a>\n",
    "## Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise 1**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1.1. Download an image of your choosing, and display it  \n",
    "1.2. Convert the image to grayscale, and display it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1.1. Download an image of your choosing and display it\n",
    "\n",
    "# Write your code below:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <font color=\"red\"><b><u>here</b></u></font> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "You should specify the download_image_url and download_image_filename\n",
    "\n",
    "\n",
    "download_image_url = \"\"\n",
    "download_image_filename = \"\"\n",
    "urllib.request.urlretrieve(download_image_url, download_image_filename)\n",
    "\n",
    "img = cv2.imread(download_image_filename)\n",
    "plt.imshow(img)\n",
    "\n",
    "--->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1.2. Convert the image to grayscale and display it\n",
    "\n",
    "# Write your code below:\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <font color=\"red\"><b><u>here</b></u></font> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n",
    "plt.imshow(gray_img, cmap = \"gray\")\n",
    "plt.axis(\"off\") #remove axes ticks\n",
    "plt.title('Grayscale Image')\n",
    "\n",
    "--->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise 2**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2.1. Using your grayscale image from Exercise 1, display its grayscale histogram  \n",
    "2.2. Using your color image from Exercise 1, display its color histogram  \n",
    "2.3. Look at your histogram and think about how its pixels are distributed. What are some observations that you can make?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2.1. Using your grayscale image from Exercise 1, display its grayscale histogram  \n",
    "\n",
    "# Write your code below:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <font color=\"red\"><b><u>here</b></u></font> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "rcParams['figure.figsize'] = 8,4\n",
    "plt.hist(gray_img.ravel(),256,[0,256])\n",
    "plt.title('Histogram of Grayscale Image')\n",
    "plt.show()\n",
    "\n",
    "--->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2.2. Using your color image from Exercise 1, display its color histogram  \n",
    "\n",
    "# Write your code below:\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <font color=\"red\"><b><u>here</b></u></font> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "colors = ('b','g','r')\n",
    "for i,col in enumerate(colors):\n",
    "    histr_color = cv2.calcHist([download_image_filename],[i],None,[256],[0,256])\n",
    "    plt.plot(histr_color,color = col)\n",
    "    plt.xlim([0,256])\n",
    "plt.show()\n",
    "\n",
    "--->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Thank you for completing this lab!</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "<h2>Get IBM Watson Studio free of charge!</h2>\n",
    "    <p><a href=\"https://cocl.us/NotebooksPython101bottom\"><img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/CV0101/Logo/BottomAd.png\" width=\"750\" align=\"center\"></a></p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>About the Authors:</h3>\n",
    "<p>This lab was written by <a href=\"https://www.linkedin.com/in/sacchitchadha/\" target=\"_blank\" >Sacchit Chadha</a> and revised by Nayef Abou Tayoun</p>\n",
    "<p><a href=\"https://www.linkedin.com/in/sacchitchadha/\" target=\"_blank\">Sacchit Chadha</a> is a Software Engineer at IBM, and is a rising senior pursuing a Bachelors Degree in Computer Science from the University of Waterloo. His work at IBM is focused on Computer Vision, Cloud Computing and Blockchain.</p>\n",
    "<p>Nayef Abou Tayoun is a Cognitive Data Scientist at IBM, and is pursuing a Master's Degree in Artificial Intelligence.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "\n",
    "Copyright &copy; 2019 [IBM Developer Skills Network](https://cognitiveclass.ai/?utm_source=cccopyrightlink&utm_medium=labs&utm_campaign=cc). This notebook and its source code are released under the terms of the [MIT License](https://cognitiveclass.ai/mit-license/)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}