3D road line extraction, bird's-eye view projection, and junction detection for ROS stereo images
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
example_results initial import Aug 15, 2015
.gitignore
README.md fixed image size parameter and readme typo Aug 16, 2015
birdseyesnapshot.jpg
depth.dat
geometry_utils.py initial import Aug 15, 2015
gtpos.dat fixed image size parameter and readme typo Aug 16, 2015
parameters.py fixed image size parameter and readme typo Aug 16, 2015
road_out.jpg
roadlinesegments.py removed inelegant constant Aug 17, 2015
rosutils.py initial import Aug 15, 2015
savesensordata_node.py
snapshot.jpg fixed image size parameter and readme typo Aug 16, 2015

README.md

3D road line extraction, bird's-eye view projection, and junction detection for ROS stereo images

Finds the middle line of a road, and its junctions, in a stereo image. Tested with ROS+Gazebo (DRC Atlas robot), but should work with any stereo camera as input.

A Convolutional Neural Network, CN24, is used for road detection. Subsequently, a birds-eye view is calculated using OpenCV's findHomography and warpPerspective. Line segments are obtained by thinning the detected road, simplifying with a voxel grid filter, and calculating a minimum spanning tree of the resulting points.

Usage

Requires:

  • CN24 - the environment variable 'CN24PATH' has to be set to the path to the cn24 build directory (e.g. 'export CN24PATH=/home/USER/cn24/build')
  • numpy, scipy, matplotlib
  • skimage
  • cv2
  • pcl (optional)
  • ROS and rospy to capture image data (optional)

Usage example:

import matplotlib
import matplotlib.pyplot as plt
from roadlinesegments import *

## uncomment to obtain images from ROS topic
#from savesensordata_node import *
#import time
#savesensordata()
#time.sleep(2)

## load images
camimg = getcameraimg()
roadimg = getroadimg()
depthmap, points, notnanindices = getdepthimg()

## get birds-eye view
warpedimg, H = getbirdseyeview(camimg, roadimg, depthmap, points, notnanindices)

## run road detection   
points, edge_list, junctionpointidx, allpoints = getroadlinesegments(camimg, warpedimg, H)

## plot results
plt.subplot(1,2,1)
camimg[roadimg!=0] += [0,100,0] # paint road green for visualization
plt.imshow(camimg)
plt.title("Original image and road detection")

plt.subplot(1,2,2)
plt.imshow(warpedimg)
for edge in edge_list:
    plt.plot([points[edge[0], 0], points[edge[1], 0]], [points[edge[0], 1], points[edge[1], 1]], c='r')
for i in junctionpointidx:
    plt.scatter(points[i, 0], points[i, 1], c='red', s=300)
plt.title('Birds-eye warped image, road skeleton and junctions')
plt.show()

Examples

Detected road in first-person camera image (left), obtained by a simulated robot, and the same image warped to bird's eye perspective (right) Example 1

Actual environment Example 1

Detected road in first-person camera image (left), obtained by a simulated robot, and the same image warped to bird's eye perspective (right) Example 1

Actual environment Example 1