FusionTracking

Feature tracking with standard and event-based camera fusion.

This repository provides a novel event-based feature tracking algorithm. This algorithm fuse data from both a standard camera and an event camera. Features are extracted, and updated by frames, and tracked from events.

Full paper avaliable: @ACM DL @Research Gate

Citaion

Please cite this paper if you use this code in an academic publication.

@inproceedings{10.1145/3459066.3459075,
author = {Dong, Yan and Zhang, Tao},
title = {Standard and Event Cameras Fusion for Feature Tracking},
year = {2021},
isbn = {9781450389556},
publisher = {Association for Computing Machinery},
url = {https://doi.org/10.1145/3459066.3459075},
doi = {10.1145/3459066.3459075},
pages = {55–60},
}

Build and Run

1. Dependency

First you need to install gflags, from here

OpenCV is also required (OpenCV 3.2.0 is tested).

2. Build

mkdir -p ~/tracking_ws/src
cd ~/tracking_ws/src
git clone git@github.com:LarryDong/FusionTracking.git
cd ..
catkin build

3. Run

We provide two methods. The first one we implemented based on the "ICP method" by Beat Kueng. The code of this method (only feature tracking part) is not released by the author so far.

@INPROCEEDINGS{ICP_Method,
  author={B. {Kueng} and E. {Mueggler} and G. {Gallego} and D. {Scaramuzza}},
  booktitle={2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
  title={Low-latency visual odometry using event-based feature tracks}, 
  year={2016},
  volume={},
  number={},
  pages={16-23},
  doi={10.1109/IROS.2016.7758089}}

We improve this method by adding frame information.

Before launch, you should source the setup.bash and modify the --flagfile path in each .launch file.

To run the "ICP method":

roslaunch fusion_tracker icp.launch

or our fusion tracking method:

roslaunch fusion_tracker uft.launch

Then run the dataset.rosbag in a new terminal. Bags are from the Event Camera Dataset.

rosbag play shape_6dof.bag

4. Trouble shooting

Tested under Ubuntu 18.04, ROS melodic, and OpenCV 3.2.0.

Useful link

[1] Event Camera Dataset: http://rpg.ifi.uzh.ch/davis_data.html

[2] Celex_MP ROS driver: https://github.com/kehanXue/CeleX5-ROS

[3] Event feature tracking evaluation codes: https://github.com/uzh-rpg/rpg_feature_tracking_analysis This amazing tool can draw beautiful curves. The following pictures show the "feature age" and "tracking error" of 4 different method: ours, ICP method, EKLT, and EM-ICP. Please check the paper for more details.