The goals / steps of this project are the following:
- Perform a Histogram of Oriented Gradients (HOG) feature extraction on a labeled training set of images and train a classifier Linear SVM classifier
- Optionally, you can also apply a color transform and append binned color features, as well as histograms of color, to your HOG feature vector.
- Note: for those first two steps don't forget to normalize your features and randomize a selection for training and testing.
- Implement a sliding-window technique and use your trained classifier to search for vehicles in images.
- Run your pipeline on a video stream (start with the test_video.mp4 and later implement on full project_video.mp4) and create a heat map of recurring detections frame by frame to reject outliers and follow detected vehicles.
- Estimate a bounding box for vehicles detected.
Histogram of Oriented Gradients (HOG)
1. Explain how (and identify where in your code) you extracted HOG features from the training images.
The code for this step is contained in the first code cell of the IPython notebook
VehicleDetection.ipynb under cell #6 .
I started by reading in all the
non-vehicle images. Here is an example of one of each of the
I then explored different color spaces and different
skimage.hog() parameters (
cells_per_block). I grabbed random images from each of the two classes and displayed them to get a feel for what the
skimage.hog() output looks like.
Here is an example using the
YUV color space and HOG parameters of
pixels_per_cell=(8, 8) and
2. Explain how you settled on your final choice of HOG parameters.
I tried various combinations of parameters and finally settled for the following values
colorspace = 'YUV' # Can be RGB, HSV, LUV, HLS, YUV, YCrCb orient = 11 pix_per_cell = 16 cell_per_block = 2 hog_channel = 'ALL' # Can be 0, 1, 2, or "ALL"
based upon the performance of the SVM classifier using the above values.
56.74 Seconds to extract HOG features... Using: 11 orientations 16 pixels per cell and 2 cells per block Feature vector length: 1188
3. Describe how (and identify where in your code) you trained a classifier using your selected HOG features (and color features if you used them).
I trained a linear SVM with the default classifier parameters and using HOG features alone. spatial intensity or channel intensity histogram features can also be tried in the future.
1.11 Seconds to train SVC... Test Accuracy of SVC = 0.9806 My SVC predicts: [ 0. 1. 0. 0. 1. 0. 1. 0. 0. 0.] For these 10 labels: [ 0. 1. 0. 0. 1. 0. 1. 0. 0. 0.] 0.00302 Seconds to predict 10 labels with SVC
Sliding Window Search
1. Describe how (and identify where in your code) you implemented a sliding window search. How did you decide what scales to search and how much to overlap windows?
I explored several configurations of window sizes and positions, with various overlaps in the X and Y directions. The duplicates and false positives are avoided based on the repition of the bounding boxes in the subsequent frames.
2. Show some examples of test images to demonstrate how your pipeline is working. What did you do to optimize the performance of your classifier?
Here are some example images:
1. Provide a link to your final video output. Your pipeline should perform reasonably well on the entire project video (somewhat wobbly or unstable bounding boxes are ok as long as you are identifying the vehicles most of the time with minimal false positives.)
Here's a link to my video result
2. Describe how (and identify where in your code) you implemented some kind of filter for false positives and some method for combining overlapping bounding boxes.
I recorded the positions of positive detections in each frame of the video. From the positive detections I created a heatmap and then thresholded that map to identify vehicle positions. I then used
scipy.ndimage.measurements.label() to identify individual blobs in the heatmap. I then assumed each blob corresponded to a vehicle. I constructed bounding boxes to cover the area of each blob detected.
Here's an example result showing the heatmap from a series of frames of video, the result of
scipy.ndimage.measurements.label() and the bounding boxes then overlaid on the last frame of video:
A threshold is applied to the heatmap with a value of 1, setting all pixels that don't exceed the threshold to zero. The result is below:
Here is the output of
scipy.ndimage.measurements.label() on the integrated heatmap from all six frames:
Here the resulting bounding boxes are drawn onto the last frame in the series:
1. Briefly discuss any problems / issues you faced in your implementation of this project. Where will your pipeline likely fail? What could you do to make it more robust?
- I trained a linear SVM with the default classifier parameters and using HOG features alone. Spatial intensity or channel intensity histogram features can also be tried in the future.
- CNN can be used as a classifier instead of a simple SVC to have even more generalise classifier.
- Integrating detections from previous frames solved the problem of misclassification. Window overlap has to be maximised to improve the classifier accuracy with the help of GPU and advanced classifiers.