This repository implements an extension of the algorithm described in the 2019 IJCV paper EKLT: Asynchronous, Photometric Feature Tracking using Events and Frames by Daniel Gehrig, Henri Rebecq, Guillermo Gallego, and Davide Scaramuzza using a multithreading approach.
This algorithm is based on EKLT, that works by leveraging the complementarity between events and frames for feature tracking. Using the events, it manages to track in the blind-time between two frames. First features are extracted on the frames and then tracked using only the events. The tracker then produces asynchronous feature tracks with high temporal resolution. More details can be found in the paper and in the repository. Our job was to make the algorithm more efficient, making it multithreaded, and making it more suitable for a possible real application of visual odometry. For more details look at the report of our work. It is written in Italian for didactic reasons.
For the installation refer to what is described in the eklt repository. However, there are some differences. Inside folder config, there is a file named eklt.conf, which now contains a parameter that allows you to specify the number of threads, and two parameters that enable you to handle the memory used by the events vector. In addition, in order to build the project now you have to use the command catkin build eklt_multithreading instead of catkin build eklt. And finally, to launch it you should run roslaunch eklt eklt_multithreading.launch instead of roslaunch eklt eklt.launch
The tests were performed with different thread numbers and using two different computers. One that has an i7-4720HQ with Ubuntu 16.04 and ROS Kinetic, and the other that has a Ryzen 5 3600 with Ubuntu 20.04 and ROS Noetic. For the sake of simplicity, the algorithm was tested only on the first 10 seconds of the rosbag data. The tracks file generated by the execution of the algorithm contains all the information needed to reconstruct the tracks of the features. The feature tracks are stored using the following format:
| feature id | timestamp | x | y | thread |
|---|---|---|---|---|
| 0 | 1468940293.922985274 | 187.032 | 132.671 | 0 |
| 2 | 1468940293.958816290 | 204.603 | 105.36 | 1 |
| 1 | 1468940293.957388878 | 222.378 | 104.176 | 2 |
| 0 | 1468940293.964686394 | 223.596 | 19.1384 | 0 |
| 1 | 1468940293.960546732 | 182.122 | 52.1019 | 1 |
| 2 | 1468940293.960608244 | 172.227 | 47.1469 | 2 |
The performances observed using different thread numbers are summarized in the following table.
| Threads | 1 (Original EKLT) | 2 | 4 | 6 | 8 | 12 | 16 |
|---|---|---|---|---|---|---|---|
| Ryzen 5 seconds | 609 | 382 | 180 | 132 | 115 | 93 | 90 |
| i7 seconds | 982 | 681 | 418 | - | 267 | - | - |
The following graph shows the trend for the i7 configuration of the seconds in relation to the number of threads.
And there we have the Ryzen one.
