Estimation of a global optical flow vector with an event camera. Using the Contrast Maximization method, the implemented algorithm estimates an optical flow vector (u,v) shared by all the events on the image during a short time window.
Input:
- Events (topic)
Output:
- Image velocity (a 2D vector) obtained by maximizing the contrast of the image of warped events (optimization algorithm).
- Image of warped event IWE (i.e., event image), without and with motion compensation using the estimated flow vector.
Parameters (in the launch file):
- Number of events to process together in the "sliding window".
- Number of events to shift in the "sliding window" for the next packet of events.
- Type of objective function to be optimized (variance, mean square).
- Whether to use polarity or not.
- Amount of Gaussian smoothing for voting in the IWE (typically 1 pixel).
- Verbosity level (>=0), for printing and debugging purposes.
Create a catkin workspace (if there is none yet). For example, from your home folder:
cd
mkdir -p catkin_ws/src
cd catkin_ws
catkin config --init --mkdirs --extend /opt/ros/melodic --merge-devel --cmake-args -DCMAKE_BUILD_TYPE=Release
Depending on the ROS distribution you installed, you might have to use kinetic instead of melodic in the previous command.
Clone this repository into the src folder of your catkin workspace.
The catkin package dependencies are:
- catkin simple
- ROS messages for DVS (rpg_dvs_ros)
- Google Logging Library (glog)
- Gflags (formerly Google Commandline Flags)
The above dependencies are specified in the dependencies.yaml file. They can be installed with the following commands from the src folder of your catkin workspace:
cd catkin_ws/src
sudo apt-get install python3-vcstool
vcs-import < dvs_global_flow/dependencies.yaml
The previous command should clone the repositories into folders catkin_simple, rpg_dvs_ros, etc. inside the src/ folder of your catkin workspace, at the same level as this repository dvs_global_flow. They should NOT be inside the dvs_global_flow folder.
The GSL library is a scientific library that can be installed with the command:
sudo apt-get install libgsl-dev
Preliminaries:
First, build catkin simple and the davis_ros_driver. The most important commands are:
catkin build catkin_simple
catkin build davis_ros_driver
Then, incorporate the packages to the path, so that ROS finds them:
source ~/catkin_ws/devel/setup.bash
Compile this package:
catkin build dvs_global_flow --force-cmake
The flag --force-cmake is optional.
After building, at least the first time, remember to run:
source ~/catkin_ws/devel/setup.bash
Sometimes (in case of strange errors) it is useful to delete the build folder before compilation:
rm -rf build/dvs_global_flow/
An alternative command to start from scratch (cleaning all catkin packages) is (to be used with caution): catkin clean
Download a ROS bag, for example slider_close, to use as source of data (as if an event camera was connected to the computer).
ROS bags with linear motion from the Event Camera Dataset IJRR 2017:
You may also test on the first seconds of sequences with dominantly linear motion and approximately constant depth, for example, the first 7 seconds of boxes_translation.bag.
In a terminal:
roslaunch dvs_global_flow ijrr_translation.launch
Please make sure that the launch file has the correct path to the data (e.g., ROS bag).
End the program execution with Ctrl + C keyboard shortcut.
- Gallego et al., A Unifying Contrast Maximization Framework for Event Cameras..., CVPR 2018. PDF, Poster, YouTube, Spotlight presentation.
- Gallego et al. Focus Is All You Need: Loss Functions For Event-based Vision, CVPR 2019. PDF arXiv, Poster, YouTube