Task_01_HoG_Histogram_of_gradients.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 1: Histogram of Gradients (HoG)\n",
    "-----------------------------------------------------------------------------------\n",
    "Author: Rajesh Siraskar\n",
    "Created: 01-Jan-2019\n",
    "- 01-Jan-2019: First version- Create skeleton for function\n",
    "- 02-Jan-2019: Test grads and magnitude/angles with a test image\n",
    "    - Pre-processing. Add bi-lateral filtering to preserve edges\n",
    "- 03-Jan-2019: Step.3.a.: Scan image and gather Im and Ia in num_bins\n",
    "- 04-Jan-2019: Step.3.a.: Understanding what to bin! Created code for creating 20 deg. bins\n",
    "    - Study the orginal thesis by Dalal. Understand recommendations\n",
    "    - Developing binning logic. Cell based Im Ia calculated\n",
    "    - Binned using naive logic. If Ia is within bin, then Im goes into that bin. Issue: Need to be lists of Im? Right now Im is over-writing\n",
    "    - 5 for-loops for the binning. Need to use list-comprehension if logic correct\n",
    "- 05-Jan-2019: Looked at the original CPP code, difficult to understand - however the **original .cpp** version shows 4 for loops!\n",
    "    - It is confirmed that the pixel level loop is required. Approach looks good so far.\n",
    "- 06-Jan-2019: Need to HOG_LAB.py to really understand Dalal N., several concepts solidified\n",
    "    - Move from lab to main notebook\n",
    "- 06-Jan-2019: Non vectorized code: Test images: 8, 4.86MB: Persp. map time: 13.3 mins.  \n",
    "- 10-Jan-2019: Cleaned version\n",
    "\n",
    "\n",
    "**References:**\n",
    "- Algorithm: Klette 'Concise Computer Vision'. Pg.382\n",
    "- Original source: Ph.D. Thesis, Navneet DALAL, 2006: 'Finding People in Images and Videos'\n",
    "- Explanation: Satya Mallick, Big Vision LLC, 2018\n",
    "- Dec. 2016, https://www.learnopencv.com/histogram-of-oriented-gradients/\n",
    "- (Mordvintsev A. and Abid K., 2014), cv2.bilateralFilter() is highly effective at noise removal while still preserving edges at the cost of the operation being slower compared to other filters\n",
    "- https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_filtering/py_filtering.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Jupyter specific command to plot images inline with document\n",
    "%matplotlib inline\n",
    "\n",
    "# Import modules\n",
    "import cv2\n",
    "from matplotlib import pyplot as plt # for displaying images and plots\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Function to compute histogram of gradients for an image\n",
    "\n",
    "Main function applies the following steps:\n",
    "    \n",
    "**Step 1.: Pre-processing**\n",
    "    [a] Intensity normalization: Simple normalization applied w.r.t. max intensity 255\n",
    "    [b] Smoothing filter: **Bilateral Filtering** applied\n",
    "    \n",
    "The algorithm from the Klette 'Concise Computer Vision'. Pg.382\n",
    "\n",
    "The original thesis (Dalal N., 2006: pg. 37 - 4.3.2 Gradient Computation) suggests recommends sigma=0 i.e. _NO_ smoothing\n",
    "\n",
    "However tests showed that the edges were more prominient with use of the bi-lateral filter and might prove more effective for pedestrian detection. The student-author therefore proposes and has implemented **bilateral filtering** instead. The application of course can be controlled using the parameter \n",
    "\n",
    "As stated in OpenCV-Python Tutorials (Mordvintsev A. and Abid K., 2014), cv2.bilateralFilter() is highly effective at noise removal while still preserving edges at the cost of the operation being slower compared to other filters\n",
    "\n",
    "- Ref.: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_filtering/py_filtering.html\n",
    "- Recommended values ref https://docs.opencv.org/3.1.0/d4/d86/group__imgproc__filter.html#ga9d7064d478c95d60003cf839430737ed\n",
    "\n",
    "**Step 2.a.: Calculate gradients for each pixel**\n",
    "\n",
    "The Sobel filter with a kernel size 1 pixel x 1 pixel is used.\n",
    "- Ref.: (Dalal N., 2006: pg. 37 - 4.3.2 Gradient Computation) suggests Sobel operator [−1, 0, 1]\n",
    "\n",
    "Also absolute values of gradients is used   \n",
    "- Use un-signed gradients pg. 48. '4.6 Experiments on Other Classes'\n",
    "\n",
    "**Step 2.b.: Compute the magnitude and angles of the gradient vectors**\n",
    "- Magnitude and angle of a 2D vector can be computed using Polar conversion  \n",
    "\n",
    "**Step.3: Spatial binning**\n",
    "- Group pixels in non-overlapping cells (e.g. 8×8 pixels).\n",
    "- See function _HoG_histogram_binning() below. The outline of algorithm based on original thesis (Dalal N., 2006: Appendix D) \n",
    "\n",
    "Final step is the loop that traverses the image using blocks of cells and moving one block-stride at a time, starting at the top-left corner and scanning.\n",
    "\n",
    "### Known issue:\n",
    "- The student has attempted an implementation by studying the algorithm provided (Klette, R. 2014) and orginal thesis (Dalal, N. 2006)\n",
    "- Although the student has used list comprehension to eliminate 3 inner loops from the starting implementation of 5 for-loops, the implementation has still turned out to be **extremely slow**\n",
    "- Approx. time to scan one test image: **48.05 secs**\n",
    "\n",
    "**Parameters:**\n",
    "\n",
    "- Cells: A unit block of pixels\n",
    "      cell_size = size of cell in nxn pixels  (default = 8x8)\n",
    "- Block cells: n cells\n",
    "      block_cells = multiplier n i.e. n cells HxW  (default = 2)\n",
    "- Block step: Cells by which steps are made - scanning bounding box left to right and top to bottom\n",
    "      step_cells: n cells (default = 1)        \n",
    "- Number of 'directional' (bins of direction i.e. angle of the gradient vectors) 20 deg. bins => 180 degs./20 = 9 bins\n",
    "      num_bins (default = 9)\n",
    "- Apply smoothing filter?\n",
    "      apply_filter (default = None)\n",
    "- If filter applied then supply an additional parameter\n",
    "      filter_param (default = None)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Function to compute histogram of gradients for an image\n",
    "\n",
    "def safe_HoG_V2 (image, cell_size = (8, 8), block_cells = 2, step_cells = 1, num_bins = 9, apply_filter = None, filter_param = None):\n",
    "\n",
    "    # Resize image to recommended size 64×128 [(Dalal N., 2006) Table 4.2. Key HOG parameters :pg 47]\n",
    "    \n",
    "    # Size of bounding box     \n",
    "    image_size = (image.shape[0], image.shape[1])\n",
    "    \n",
    "    # Warning: cv2.resize in cols x rows \n",
    "    recommended_size = (64, 128)\n",
    "    image = cv2.resize(image, recommended_size)\n",
    "    \n",
    "    # Block size nxn cells. n = block_cells\n",
    "    block_size =  (block_cells*cell_size[0], block_cells*cell_size[1])\n",
    "\n",
    "    # Block stride - can move say one cell at a time, scanning left to right and top to bottom\n",
    "    block_stride = step_cells*cell_size[0]\n",
    "\n",
    "    ### Compute Histogram of Gradients\n",
    "    # Algorithm: Klette 'Concise Computer Vision'. Pg.382\n",
    "    # Refinements based on the original thesis (Dalal N., 2006)\n",
    "    \n",
    "    # Step 1.: Pre-processing\n",
    "    #          [a] Intensity normalization\n",
    "    #          [b] Smoothing filter:\n",
    "    \n",
    "    # Step 1.a: Normalize with highest value of intensity\n",
    "    #    Simplest form of normalization used\n",
    "    #    Thesis suggests 'square root gamma compression'\n",
    "    \n",
    "    image = np.float32(image)/255.0\n",
    "    \n",
    "    # Step 1.b: Bilateral Filtering\n",
    "    #    Normal smoothing including Gaussian blur do remove noise but affect the edges too.\n",
    "    #    For pedestrian detection it might be beneficial to preserve the edges - hence we use a bilateral filter instead\n",
    "    #    As stated in OpenCV-Python Tutorials (Mordvintsev A. and Abid K., 2014), cv2.bilateralFilter() is highly effective \n",
    "    #      at noise removal while still preserving edges at the cost of the operation being slower compared to other filters\n",
    "    #      https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_filtering/py_filtering.html\n",
    "    #    Recommended values ref https://docs.opencv.org/3.1.0/d4/d86/group__imgproc__filter.html#ga9d7064d478c95d60003cf839430737ed\n",
    "    # \n",
    "    #    The original thesis (Dalal N., 2006: pg. 37 - 4.3.2 Gradient Computation) suggests\n",
    "    #      recommends sigma=0 i.e. NO smoothing\n",
    "    #\n",
    "    #    However tests showed that the edges were more prominient with use of the bi-lateral filter\n",
    "    \n",
    "    if (apply_filter == 'GAUSSIAN'): \n",
    "        if (filter_param is None): filter_param = 5\n",
    "        # All three parameters (ksize.width, ksize.height) and sigma assumed to take the value filter_param\n",
    "        image = cv2.GaussianBlur(image, (filter_param, filter_param), filter_param)\n",
    "        \n",
    "    if (apply_filter == 'BILATERAL'): \n",
    "        if (filter_param is None): filter_param = 80\n",
    "        image = cv2.bilateralFilter(image, 7, filter_param, filter_param)\n",
    "    \n",
    "    # Step 2.a.: Calculate gradients for each pixel\n",
    "    #   Use Sobel filter with a kernel size 1 pixel x 1 pixel\n",
    "    #   Ref.: (Dalal N., 2006: pg. 37 - 4.3.2 Gradient Computation) suggests Sobel operator [−1, 0, 1]\n",
    "    #   Use un-signed gradients pg. 48. '4.6 Experiments on Other Classes'\n",
    "    \n",
    "    grad_x = abs(cv2.Sobel(image, cv2.CV_32F, 1, 0, ksize=1))\n",
    "    grad_y = abs(cv2.Sobel(image, cv2.CV_32F, 0, 1, ksize=1))\n",
    "\n",
    "\n",
    "    # Step 2.b.: Compute the magnitude and angles of the gradient vectors \n",
    "    Im, Ia = cv2.cartToPolar(grad_x, grad_y, angleInDegrees=True)\n",
    "    \n",
    "\n",
    "    # Step.3: Spatial binning\n",
    "    # Group pixels in non-overlapping cells (e.g. 8×8 pixels)\n",
    "    # Use maps Im and Ia to accumulate magnitude values into direction bins \n",
    "    #  (e.g., nine bins for intervals of 20◦ each, for covering a full 180◦ range) \n",
    "    #  to obtain a voting vector (e.g. of length 9) for each cell\n",
    "    \n",
    "    # 1 histogram per cell with 9 bins\n",
    "    # Create a collection of bins of num_bins length\n",
    "    rows = int(image.shape[0]/cell_size[0])\n",
    "    cols = int(image.shape[1]/cell_size[1])\n",
    "    Im_bins =  np.zeros((rows, cols, num_bins))\n",
    "    \n",
    "    # Step 3.a.: Create bin centers (e.g. 0, 20, 40...160 deg., if num_bins=9)\n",
    "    angle_slices = int(180.0/num_bins)\n",
    "    Ia_bin_centers = [angle for angle in range(0, 180, angle_slices)]\n",
    "\n",
    " \n",
    "    ## Traverse the image starting at top, then left to right, then next row...\n",
    "\n",
    "    # Keep track of cells\n",
    "    n_cell = 0\n",
    "    n_row = 0\n",
    "    n_col = 0\n",
    "\n",
    "    # Histogram bin band-width\n",
    "    b = Ia_bin_centers[1] - Ia_bin_centers[0]\n",
    "\n",
    "    # Move T-B\n",
    "    n_row = 0\n",
    "    for y in range(0, image.shape[0], block_stride):\n",
    "        # Move L-R, intialize column counter\n",
    "        n_col = 0\n",
    "        for x in range(0, image.shape[1], block_stride):\n",
    "            Im_bins = _HoG_histogram_binning (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col, x, y, cell_size, num_bins)\n",
    "            # Complete cell covered, move to next cell \n",
    "            n_col += 1\n",
    "        # 1 row of cells L-R covered, proceed to next row\n",
    "        n_row+=1\n",
    "    # All rows covered L-R, complete image processed\n",
    "\n",
    "    # Create a feature vector\n",
    "    hog_features = Im_bins.ravel().reshape(-1, 1)\n",
    "                    \n",
    "    return (grad_x, grad_y, Im, hog_features)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Function to compute histogram of gradients for an image\n",
    "\n",
    "def HoG_V2 (image, cell_size = (8, 8), block_cells = 2, step_cells = 1, num_bins = 9, apply_filter = None, filter_param = None):\n",
    "\n",
    "    # Resize image to recommended size 64×128 [(Dalal N., 2006) Table 4.2. Key HOG parameters :pg 47]\n",
    "    \n",
    "    # Size of bounding box     \n",
    "    image_size = (image.shape[0], image.shape[1])\n",
    "    \n",
    "    # Warning: cv2.resize in cols x rows \n",
    "    recommended_size = (64, 128)\n",
    "    image = cv2.resize(image, recommended_size)\n",
    "    \n",
    "    # Block size nxn cells. n = block_cells\n",
    "    block_size =  (block_cells*cell_size[0], block_cells*cell_size[1])\n",
    "\n",
    "    # Block stride - can move say one cell at a time, scanning left to right and top to bottom\n",
    "    block_stride = step_cells*cell_size[0]\n",
    "\n",
    "    ### Compute Histogram of Gradients\n",
    "    # Algorithm: Klette 'Concise Computer Vision'. Pg.382\n",
    "    # Refinements based on the original thesis (Dalal N., 2006)\n",
    "    \n",
    "    # Step 1.: Pre-processing\n",
    "    #          [a] Intensity normalization\n",
    "    #          [b] Smoothing filter:\n",
    "    \n",
    "    # Step 1.a: Normalize with highest value of intensity\n",
    "    #    Simplest form of normalization used\n",
    "    #    Thesis suggests 'square root gamma compression'\n",
    "    \n",
    "    image = np.float32(image)/255.0\n",
    "    \n",
    "    # Step 1.b: Bilateral Filtering\n",
    "    #    Normal smoothing including Gaussian blur do remove noise but affect the edges too.\n",
    "    #    For pedestrian detection it might be beneficial to preserve the edges - hence we use a bilateral filter instead\n",
    "    #    As stated in OpenCV-Python Tutorials (Mordvintsev A. and Abid K., 2014), cv2.bilateralFilter() is highly effective \n",
    "    #      at noise removal while still preserving edges at the cost of the operation being slower compared to other filters\n",
    "    #      https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_filtering/py_filtering.html\n",
    "    #    Recommended values ref https://docs.opencv.org/3.1.0/d4/d86/group__imgproc__filter.html#ga9d7064d478c95d60003cf839430737ed\n",
    "    # \n",
    "    #    The original thesis (Dalal N., 2006: pg. 37 - 4.3.2 Gradient Computation) suggests\n",
    "    #      recommends sigma=0 i.e. NO smoothing\n",
    "    #\n",
    "    #    However tests showed that the edges were more prominient with use of the bi-lateral filter\n",
    "    \n",
    "    if (apply_filter == 'GAUSSIAN'): \n",
    "        if (filter_param is None): filter_param = 5\n",
    "        # All three parameters (ksize.width, ksize.height) and sigma assumed to take the value filter_param\n",
    "        image = cv2.GaussianBlur(image, (filter_param, filter_param), filter_param)\n",
    "        \n",
    "    if (apply_filter == 'BILATERAL'): \n",
    "        if (filter_param is None): filter_param = 80\n",
    "        image = cv2.bilateralFilter(image, 7, filter_param, filter_param)\n",
    "    \n",
    "    # Step 2.a.: Calculate gradients for each pixel\n",
    "    #   Use Sobel filter with a kernel size 1 pixel x 1 pixel\n",
    "    #   Ref.: (Dalal N., 2006: pg. 37 - 4.3.2 Gradient Computation) suggests Sobel operator [−1, 0, 1]\n",
    "    #   Use un-signed gradients pg. 48. '4.6 Experiments on Other Classes'\n",
    "    \n",
    "    grad_x = abs(cv2.Sobel(image, cv2.CV_32F, 1, 0, ksize=1))\n",
    "    grad_y = abs(cv2.Sobel(image, cv2.CV_32F, 0, 1, ksize=1))\n",
    "\n",
    "\n",
    "    # Step 2.b.: Compute the magnitude and angles of the gradient vectors \n",
    "    Im, Ia = cv2.cartToPolar(grad_x, grad_y, angleInDegrees=True)\n",
    "    \n",
    "\n",
    "    # Step.3: Spatial binning\n",
    "    # Group pixels in non-overlapping cells (e.g. 8×8 pixels)\n",
    "    # Use maps Im and Ia to accumulate magnitude values into direction bins \n",
    "    #  (e.g., nine bins for intervals of 20◦ each, for covering a full 180◦ range) \n",
    "    #  to obtain a voting vector (e.g. of length 9) for each cell\n",
    "    \n",
    "    # 1 histogram per cell with 9 bins\n",
    "    # Create a collection of bins of num_bins length\n",
    "    rows = int(image.shape[0]/cell_size[0])\n",
    "    cols = int(image.shape[1]/cell_size[1])\n",
    "    Im_bins =  np.zeros((rows, cols, num_bins))\n",
    "    \n",
    "    # Step 3.a.: Create bin centers (e.g. 0, 20, 40...160 deg., if num_bins=9)\n",
    "    angle_slices = int(180.0/num_bins)\n",
    "    Ia_bin_centers = [angle for angle in range(0, 180, angle_slices)]\n",
    "\n",
    " \n",
    "    ## Traverse the image starting at top, then left to right, then next row...\n",
    "\n",
    "    # Keep track of cells\n",
    "    n_cell = 0\n",
    "    n_row = 0\n",
    "    n_col = 0\n",
    "\n",
    "    # Histogram bin band-width\n",
    "    b = Ia_bin_centers[1] - Ia_bin_centers[0]\n",
    "\n",
    "    n_cols = range(0, image.shape[1], block_stride)\n",
    "    \n",
    "    \n",
    "    # Move T-B\n",
    "    n_row = 0\n",
    "    for y in range(0, image.shape[0], block_stride):\n",
    "        # Move L-R, intialize column counter\n",
    "        n_col = 0        \n",
    "        \n",
    "        Im_bins = [_HoG_histogram_binning (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col, x, y, cell_size, num_bins) \n",
    "                          for n_col in enumerate(n_cols)]\n",
    "        \n",
    "        #for x in range(0, image.shape[1], block_stride):\n",
    "        #    Im_bins = _HoG_histogram_binning (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col, x, y, cell_size, num_bins)\n",
    "        #    # Complete cell covered, move to next cell \n",
    "        #    n_col += 1\n",
    "        # 1 row of cells L-R covered, proceed to next row\n",
    "        n_row+=1\n",
    "    # All rows covered L-R, complete image processed\n",
    "\n",
    "    # Create a feature vector\n",
    "    hog_features = Im_bins.ravel().reshape(-1, 1)\n",
    "                    \n",
    "    return (grad_x, grad_y, Im, hog_features)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def _HoG_process_roi_across (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col, x, y, cell_size, num_bins):\n",
    "    \n",
    "    [_HoG_process_pixel(Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col,  x, y, px_row, px_col, num_bins) \n",
    "     for px_row in range(y, y+cell_size[1]) for px_col in range(x, x+cell_size[0])]\n",
    "\n",
    "    n_col += 1\n",
    "    return (Im_bins, n_col)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function: _HoG_histogram_binning\n",
    "\n",
    "#### HoG Vector based on thesis (Dalal N., 2006: Appendix D)\n",
    "\n",
    "        Here the bins are based on 9 centers:\n",
    "        Ia_bins:  [0, 20, 40, 60, 80, 100, 120, 140, 160]\n",
    "        \n",
    "        Ph.D. Thesis, Navneet DALAL, 2006: 'Finding People in Images and Videos', Appendix 'D'\n",
    "        \n",
    "        If x is value to be interpolated into bins and x is such that it lies between x1 and x2,\n",
    "        where x1 and x2 are centers of two bins and b is the band-width of the bins:\n",
    "        \n",
    "        x1 <= x < x2\n",
    "        \n",
    "        h(x1) <- h(x1) + w * [1 - (x-x1)/b]\n",
    "        h(x2) <- h(x1) + w * (x-x1)/b\n",
    "        \n",
    "        **Note**: The thesis pg: 131, Appendix D, could there be an error? h(x1) mentioned in the eqn. for h(x2)?\n",
    "        \n",
    "        Example:\n",
    "        If Im = 32.00, the bin centers are 20 <= x < 40\n",
    "        h(x1) <- h(x1) + w * [1 - (32-20)/20] = h(x1) + w * 0.9\n",
    "        h(x2) <- h(x2) + w * [(32-20)/20] = h(x2) + w * 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### Access each pixel in the cell and call _HoG_process_pixel for each pixel\n",
    "    ### Binning logic: Dalal N., 2006: Appendix D\n",
    "    # Place the pixel magnitude Im in the appropriate bin\n",
    "    # x1 <= x < x2\n",
    "    # h(x1) <- h(x1) + w * [1 - (x-x1)/b]\n",
    "    # h(x2) <- h(x1) + w * (x-x1)/b\n",
    "    \n",
    "    # For this implementation it is assumed that w = Im at that pixel\n",
    "\n",
    "def _HoG_histogram_binning (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col, x, y, cell_size, num_bins):\n",
    "        \n",
    "    [_HoG_process_pixel(Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col,  x, y, px_row, px_col, num_bins) \n",
    "     for px_row in range(y, y+cell_size[1]) for px_col in range(x, x+cell_size[0])]\n",
    "\n",
    "    n_col += 1\n",
    "\n",
    "    return (Im_bins, n_col)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function: _HoG_process_pixel\n",
    "\n",
    "Process each pixel and if the Ia is within the bin-centers' range then call  _HoG_process_bin "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### Process each pixel and if the Ia is within the bin-centers' range then call  _HoG_process_bin \n",
    "def _HoG_process_pixel (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col,  x, y, px_row, px_col, num_bins):\n",
    "    \n",
    "    \n",
    "    [_HoG_process_bin (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col, px_row, px_col, n_bin, num_bins) \n",
    "     if (Ia[px_row, px_col] >= Ia_bin_centers[n_bin] and Ia[px_row, px_col] < Ia_bin_centers[n_bin+1]) else None \n",
    "     for n_bin in range(9)] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function: _HoG_process_bin\n",
    "\n",
    "Process Im and bin it. This function handles the logic of splitting the 'vote' if the bin is the last one and sharing it with the first bin "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### Process Im and bin it\n",
    "def _HoG_process_bin (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col, px_row, px_col, n_bin, num_bins):\n",
    "    \n",
    "    if (n_bin > -1):\n",
    "        # Allow if we are at last-but-one bin, but if last bin then split share with 0 deg. (1st) bin:\n",
    "        next_bin = (n_bin+1) if (n_bin+1 < num_bins) else 0\n",
    "\n",
    "        Im_bins[n_row][n_col][n_bin] += Im[px_row,px_col] * (1 - (Ia[px_row,px_col] - Ia_bin_centers[n_bin])/b)\n",
    "        Im_bins[n_row][n_col][next_bin] += Im[px_row,px_col] * (Ia[px_row,px_col] - Ia_bin_centers[n_bin])/b\n",
    "    \n",
    "    return(Im_bins)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test the HoG Vector\n",
    "\n",
    "Read a sample image and test the HoG vector output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read image\n",
    "img_bgr  = cv2.imread('images/Lab/HoG_lab_2_pedestrian.PNG')\n",
    "img_rgb  = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2RGB)  \n",
    "img_gray = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2GRAY)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'x' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m----------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[1;32m<timed exec>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;32m<ipython-input-10-661b245a5552>\u001b[0m in \u001b[0;36mHoG_V2\u001b[1;34m(image, cell_size, block_cells, step_cells, num_bins, apply_filter, filter_param)\u001b[0m\n\u001b[0;32m    104\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    105\u001b[0m         Im_bins = [_HoG_histogram_binning (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col, x, y, cell_size, num_bins) \n\u001b[1;32m--> 106\u001b[1;33m                           for n_col in enumerate(n_cols)]\n\u001b[0m\u001b[0;32m    107\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    108\u001b[0m         \u001b[1;31m#for x in range(0, image.shape[1], block_stride):\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-10-661b245a5552>\u001b[0m in \u001b[0;36m<listcomp>\u001b[1;34m(.0)\u001b[0m\n\u001b[0;32m    104\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    105\u001b[0m         Im_bins = [_HoG_histogram_binning (Im, Ia, Im_bins, Ia_bin_centers, b, n_row, n_col, x, y, cell_size, num_bins) \n\u001b[1;32m--> 106\u001b[1;33m                           for n_col in enumerate(n_cols)]\n\u001b[0m\u001b[0;32m    107\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    108\u001b[0m         \u001b[1;31m#for x in range(0, image.shape[1], block_stride):\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mNameError\u001b[0m: name 'x' is not defined"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "(grad_x, grad_y, Im, hog_features) = HoG_V2(img_gray)\n",
    "fig, axes = plt.subplots(1, 5, figsize=(8, 8), gridspec_kw=dict(hspace=0.1, wspace=0.1))\n",
    "\n",
    "axes[0].imshow(img_rgb)\n",
    "axes[0].set(xticks=[], yticks=[], xlabel='org.')\n",
    "\n",
    "axes[1].imshow(img_gray, cmap='bone')\n",
    "axes[1].set(xticks=[], yticks=[], xlabel='gray img.')\n",
    "\n",
    "axes[2].imshow(grad_x, cmap='bone')\n",
    "axes[2].set(xticks=[], yticks=[], xlabel='grad.x')\n",
    "     \n",
    "axes[3].imshow(grad_y, cmap='bone')\n",
    "axes[3].set(xticks=[], yticks=[], xlabel='grad.y')\n",
    "\n",
    "axes[4].imshow(Im, cmap='bone')\n",
    "axes[4].set(xticks=[], yticks=[], xlabel='magnitudes')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Approx. time to scan one test image: 48.05 secs\n",
      " Approx. time for 26 test images 20.82 mins\n"
     ]
    }
   ],
   "source": [
    "## Approximate time estimation:\n",
    "    \n",
    "# Time required to run HoG on a single ROI\n",
    "roi_size = (64, 128)\n",
    "t = 0.383\n",
    "\n",
    "# For testing it was determined that the size for reasonable performance was\n",
    "(test_image_H, test_image_W) = (400, 500)\n",
    "\n",
    "# View-port sizing\n",
    "(viewport_H, viewport_W) =  (216, 82)\n",
    "\n",
    "# View-port stride as it scans the test image\n",
    "STEP_SIZE_FACTOR = 0.30\n",
    "\n",
    "bounding_boxes_across = test_image_W/(STEP_SIZE_FACTOR*viewport_W)\n",
    "bounding_boxes_down = test_image_H/(STEP_SIZE_FACTOR*viewport_H)\n",
    "\n",
    "total_bbs = bounding_boxes_across*bounding_boxes_down\n",
    "\n",
    "total_time = t*total_bbs\n",
    "\n",
    "print('\\n Approx. time to scan one test image: {:4.2f} secs'.format(total_time))\n",
    "print(' Approx. time for {} test images {:4.2f} mins'.format(26, 26*total_time/60))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}