📝 aanet_stereo_matching_ros

---

ROS package for stereo matching and point cloud projection using AANet

🎉 TODO

---

• create ros nodes to get disparity map from stereo matching with AANet
• create pseudo-lidar point cloud from disparity map with camera info

🎛 Dependencies

---

• create stereo_matching conda environment

    conda env create --file environment.yml

• Fetch AANet

    git submodule update --init --recursive

• Activate conda environment and build deformable convolution for aanet

    conda activate stereo_matching
    cd ./scripts/aanet/nets/deform_conv && bash build.sh

• python3: this package (also the original implementation) uses python3, which is taboo for ROS1. One solution to avoid confliction between ROS1 and python3 is to run roslaunch under the directory with stereo_matching env activated.

🔨 How to Build

---

Better to build with catkin or colcon Before building with ROS build tool, config current ROS workspace to use python in stereo_matching conda environment with

    catkin config -DPYTHON_EXECUTABLE=`which python` -DPYTHON_INCLUDE_DIR=`python -c "from __future__ import print_function; from distutils.sysconfig import get_python_inc; print(get_python_inc())"` -DPYTHON_LIBRARY=`python -c "from __future__ import print_function; import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR') + '/' + sysconfig.get_config_var('LDLIBRARY'))"`
    catkin build

🏃 How to Run

---

• A sample rosbag of stereo images created from DrivingStereo Dataset with this tool has been prepared HERE. Please download and extract into ./data/output.bag

• Download AANet+ KITTI 2015 model from MODEL ZOO and put into ./data/aanet+_kitti15-2075aea1.pth

• Start sample roslaunch

    roslaunch aanet_stereo_matching_ros aanet_stereo_matching_ros_rviz_sample.launch

Please take a look at the sample roslaunch to know how to use the parameters. Tune img_scale with bigger value (1.0 means original image size) would yeild better point cloud at the cost of speed. Currently the perspective projection from disparity to point cloud in python seems costly. Might be better to just propagate disparity message and do the projection in C++ package.

💎 References

---

• [1] AANet: Adaptive Aggregation Network for Efficient Stereo Matching, CVPR 2020
• [2] Original Implementation

Steps to run this code:

• after extracting the zip file: download the model and the rosbag as descrived above and but them in data folder.
• cd /path/to/aanet_stereo_matching
• cd docker
• ./docker_build.sh
• After building docker "successfully!" : ./docker_start.sh
• ./docker_into.sh
• inside docker : cd src/aanet_ros_node/
• cd ./scripts/aanet/nets/deform_conv && sudo bash build.sh (PASSWORD IS "user")
• cd .. (go back to catkin_ws)
• catkin_make (if it doesn't work run : source /opt/ros/noetic/setup.bash)
• source devel/setup.bash
• roslaunch aanet_stereo_matching_ros aanet_stereo_matching_ros_rviz_sample.launch
• open a new terminal and run : rviz
• In rviz open file->open config and navigate to this repo -> rviz and choose sample.rviz