This document is the reflection of the development of the pipeline to find lane lines in a real road, in this case, in the Highway 280 of California.
This document is divided in 3 sections: the first one, is the pipeline whit its corresponding experiments. The second one, is the limitations of the pipeline. And the last one is the possible improvements of the pipeline.
This approach consist in a sequence of steps to obtain the lane lines of a roadway driving image. As examples, in this work I show the pipeline with two different images, the first one with white mark lanes, and the other with a yellow lane mark.
One of the most important steps is how detect properly the lane marks, and this is crucial in the challenge video overall. The different day conditions affect the detection, so I have tried to do my best in this step.
Gray scale is basic in this problem, it allows to get the canny edges. But to make more robust the pipeline I have change the gamma of the gray image darken it, thus, I get better detection the white colors.
Using HLS space with a color threshold allows the algorithm to detect better yellow and white. The difficulty lies in find the correct thresholds. Finally I have merge both methods to get the best of them.
1. Gray Scale, Darken and Color space
2. HLS Colour threshold and final mask
To detect the edges I have used Canny algorithm. There are other edge detectors like Sobel or Prewitt, but, for this problem, canny work good. Internally, Canny applies a gaussian blur with 5x5 kernel.
I have found that the best thresholds for Canny are between 50 to 150, but it can be reduced to 70 and 140 and works even better.
The image have so much more information that is needed, and it can confound the algorithm, so to simplify the problem I have focused the vision in the front area of the car.
To don't have problems with the image resolution I have defined this points: (0,imshape[0]),(0.47*imshape[1], 0.6*imshape[0]), (0.54*imshape[1], 0.6*imshape[0]), (imshape[1],imshape[0])
As you can see on the images, one the are inside the region of interest keeps the edges.
The last step is the Hough Transform. As a highway is almost a straight line, with the edges and the hough algorithm you can detect shapes, as lines. The left images show the detection of the lines.
But, to get the left and right lines (right images) there are other steps to consider. First, you need to get the slope and intercept of each line. Second, group the lines in clusters, in this case, 2 clusters (left and right) erasing the horizontal and too vertical lines. Third, filter the lines that have a slope and intercept too different to the mean value of all the lines (it prevent outliers, but in most of cases is not necessary), and use a low-pass filter to obtain more stability in the lines using previously information, because the lines should not have a very quick change in slope. Fourth, draw the line with the new slope and intercept.
I have found two important limitations:
- Different light conditions and weather makes difficult to see the lane marks.
- Approach just valid for straight lanes and some roads are curvy.
Deep learning is taking relevance in this field. With Convolutionals Neural Networks and Semantic Segmentation we can improve the detection of the road in several conditions, even at night. Or even using End-to-end learning to "teach" the car to drive itself.
More classical methods to improve the pipeline could be the use os RANSAC or other methods to detect curvy lines.