Skip to content
Code for the paper "EKLT: Asynchronous, Photometric Feature Tracking using Events and Frames" (IJCV'19)
C++ CMake
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config first commit Sep 10, 2019
images first commit Sep 10, 2019
include first commit Sep 10, 2019
launch first commit Sep 10, 2019
src first commit Sep 10, 2019
CMakeLists.txt first commit Sep 10, 2019
LICENSE GPL license Sep 10, 2019
README.md added link to feature tracking evalutation Sep 21, 2019
dependencies.yaml first commit Sep 10, 2019
package.xml first commit Sep 10, 2019

README.md

EKLT: Asynchronous, Photometric Feature Tracking using Events and Frames

This repository implements the work in the 2019 IJCV paper EKLT: Asynchronous, Photometric Feature Tracking using Events and Frames by Daniel Gehrig, Henri Rebecq, Guillermo Gallego, and Davide Scaramuzza. Also check out our feature tracking evaluation package here.

Citation

A pdf of the paper is available here. If you use any of this code, please cite this publication as follows:

@Article{Gehrig19ijcv,
  author        = {Daniel Gehrig and Henri Rebecq and Guillermo Gallego and
                  Davide Scaramuzza},
  title         = {{EKLT}: Asynchronous, Photometric Feature Tracking using Events and Frames},
  journal       = "Int. J. Comput. Vis.",
  year          = 2019,
}

Overview

EKLT works by leveraging the complementarity between events and frames for feature tracking. Using the events, it manages to track in the blind-time between two frames. First features are extracted on the frames and then tracked using only the events. The tracker then produces asynchronous feature tracks with high temporal resolution. More details can be found in the paper and video.

Installation

This code was tested on ROS melodic and ROS kinetic. Install catkin tools and vcstool if needed:

sudo apt install python-catkin-tools python-vcstool

Create a new catkin workspace if needed:

mkdir -p catkin_ws/src
cd catkin_ws
catkin config --init --mkdirs --extend /opt/ros/melodic  --cmake-args -DCMAKE_BUILD_TYPE=Release

Then clone the repo and all its dependencies using vcs-import

cd src
git clone git@github.com:uzh-rpg/rpg_eklt.git
vcs-import < rpg_eklt/dependencies.yaml

Finally, build the project and then source the workspace

catkin build eklt
source ~/catkin_ws/devel/setup.bash

Running Example

Download boxes_6dof from the Event Camera Dataset which was recorded using the DVS ROS driver into a new directory

cd /tmp/
wget http://rpg.ifi.uzh.ch/datasets/eklt_example.zip
unzip eklt_example.zip
rm eklt_example.zip

Run EKLT with the following command:

roslaunch eklt eklt.launch tracks_file_txt:=/tmp/eklt_example/tracks.txt v:=1

In a separate terminal play the rosbag:

rosbag play /tmp/eklt_example/boxes_6dof.bag

Configuration parameters: Configuration parameters for eklt can be viewed by running the following command in a sourced terminal:

rosrun eklt eklt_node --help

The individual parameters can be changed in config/eklt.conf. In particular, the parameter min_corners has been set to 0 which makes it such that no new features are initialized after the first image. This configuration was used in the paper. However, for continuous tracking you can set this value higher (50 is a good number). Additional optional parameters can be set through the launch file eklt.launch:

  • v:=1 verbosity level of VLOG from the glog library
  • bag:=/path/to.bag: path to the bag which will be used to perform evalution
  • tracks_file_txt:=/path/to/file.txt: path to output file where feature tracks will be stored.

Evaluation

The tracks file generated by the prevous command contains all of the information necessary to reconstruct feature tracks. The feature tracks are stored in the following format:

feature id timestamp x y
0 1468940293.922985274 187.032 132.671
2 1468940293.958816290 204.603 105.36
1 1468940293.957388878 222.378 104.176
0 1468940293.964686394 223.596 19.1384
1 1468940293.960546732 182.122 52.1019
2 1468940293.960608244 172.227 47.1469

These feature tracks can be used directly for evaluation with this feature tracking analysis package. First clone the repo into your workspace

cd ~/catkin_ws/src
git clone git@github.com:uzh-rpg/rpg_feature_tracking_analysis.git

and run the following command

python ~/catkin_ws/src/rpg_feature_tracking_analysis/evaluate_tracks.py --error_threshold 10 --tracker_type KLT --file /tmp/eklt_example/tracks.txt --dataset /tmp/eklt_example/dataset.yaml --root /tmp/eklt_example/ --plot_errors

A preview of the output (stored in /tmp/eklt_example/results/errors.pdf) is shown below. In addition, the tracking error and feature age are stored in /tmp/eklt_example/results/errors.txt and /tmp/eklt_example/results/feature_age.txt respectively.

Visualization

When running the example a visualization of the feature tracks is publisher to the topic /feature_tracks

The visualization shows the features projected on to the current frames together with their flow angle estimate (blue arrow). Changing the flags display_features, display_feature_id and display_feature_patches toggle the display of the features, feature ids and feature patches respectively.

Additional Resources on Event Cameras

You can’t perform that action at this time.