This is the implementation of the work: Depth-aware Volume Attention for Texture-less Stereo Matching.
Stereo matching in large texture-less scenarios with perspective effect is challenging. In this paper, taking road surface as a typical scenario, we reveal the performance decrease due to the texture deterioration in natural images. We introduce the depth-aware texture hierarchy attention and target-aware disparity attention modules to focus on the texture hierarchy. We propose a noval metric named Weighted Relative Depth Error (WRDE), which provides comprehensive assessment of depth-wise performance.
- Python 3.9
- Pytorch 1.11.0
conda create -n dvanet python=3.9
conda activate dvanet
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch -c nvidia
pip install opencv-python
pip install pillow
pip install tqdm
Download RSRD(the dense subset with half resolution), Scene Flow, KITTI 2012, KITTI 2015
Use the following command to train DVANet on Scene Flow
python train.py --dataset 'sceneflow'
Use the following command to train DVANet on KITTI (using pretrained model on Scene Flow)
python train.py --dataset 'kitti' --loadckpt 'xxx.ckpt'
python test.py --dataset 'rsrd' --loadckpt 'xxx.ckpt'
Method | EPE | >1px (%) | >2px (%) |
---|---|---|---|
IGEV-Stereo | 0.19 | 0.54 | 0.22 |
CFNet | 0.18 | 0.90 | 0.17 |
PSMNet | 0.17 | 0.63 | 0.16 |
RAFT-Stereo | 0.17 | 0.43 | 0.17 |
ACVNet | 0.16 | 0.59 | 0.15 |
GwcNet | 0.16 | 0.57 | 0.14 |
DVANet(Ours) | 0.15 | 0.34 | 0.14 |
Method | Noc (%) | All (%) | # Params (M) |
---|---|---|---|
HITNet | 1.41 | 1.89 | - |
GANet-15 | 1.36 | 1.80 | - |
RAFT-Stereo | 1.30 | 1.66 | 11.1 |
CFNet | 1.23 | 1.58 | 22.2 |
AcfNet | 1.17 | 1.54 | 5.5 |
ACVNet | 1.13 | 1.47 | 7.1 |
IGEV-Stereo | 1.12 | 1.44 | 12.6 |
DVANet(Ours) | 1.09 | 1.52 | 5.1 |
If you find this project helpful in your research, welcome to cite the paper.
@misc{zhao2024depthaware,
title={Depth-aware Volume Attention for Texture-less Stereo Matching},
author={Tong Zhao and Mingyu Ding and Wei Zhan and Masayoshi Tomizuka and Yintao Wei},
year={2024},
eprint={2402.08931},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Thanks to the excellent work GwcNet and ACVNet. Our work is inspired by these works and part of codes are migrated from GwcNet and ACVNet.