Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

Pytorch Implementation of the Wide Stride Multi-Classification Stereo Matching Network

citation{

Wang Yufeng,Wang Hongwei,Yu Guang,Yang Mingquan,Yuan Yuwei,Quan Jicheng. Stereo Matching Algorithm Based on Three-Dimensional Convolutional Neural Network[J]. Acta Optica Sinica, 2019, 39(11): 1115001

王玉锋,王宏伟,于光,杨明权,袁昱纬,全吉成. 基于三维卷积神经网络的立体匹配算法[J]. 光学学报, 2019, 39(11): 1115001

url=http://www.opticsjournal.net/Articles/Abstract?aid=OJc47fda4ba4b4ce65 }

Contents

  1. Introduction
  2. Usage
  3. Results
  4. Contacts

Introduction

For the stereo matching method based on deep learning, the network architecture is critical for the accuracy of the algorithm, while the efficiency is also an important factor to consider in practical applications. A stereo matching method with spare cost volume in disparity dimension is proposed. The spare cost volume is created by shifting right-view features with a wide stride to reduce greatly the memory and computational resources in three-dimension convolution module. The matching cost is nonlinearly sampled by means of multi-classification in disparity dimension, and model is trained with merging two kind of loss function, so that the accuracy is improved without notably lowering the efficiency.

Usage

Dependencies

Usage of KITTI and SceneFlow dataset in stereo/dataloader/README.md

Train

As an example, use the following command to train a WSMCnet on SceneFlow

dir_save="./results"
LOG="${dir_save}/log_`date +%Y-%m-%d_%H-%M-%S`.txt"
mkdir -p "${dir_save}"

python main.py --mode Train --arch WSMCnetEB_S2C3F32 --maxdisp 192 --bn 4 \
               --loadmodel None \
               --datas_train "sf-tr" --dir_datas_train (dir_root_sf) \
               --datas_val "sf-val" --dir_datas_val (dir_root_sf) \
               --crop_width 512 --crop_height 256 \
               --epochs 20 --nloop 1 --freq_print 20 \
               --freq_optim 4 \
               --lr 0.001 --lr_epoch0 16 \
               --lr_stride 10 --lr_delay 0.1 \
               --dir_save $dir_save \
               2>&1 | tee -a "$LOG"

As another example, use the following command to finetune a WSMCnet on KITTI

dir_save="./results"
LOG="${dir_save}/log_`date +%Y-%m-%d_%H-%M-%S`.txt"
mkdir -p "${dir_save}"

python main.py --mode Finetune --arch WSMCnetEB_S2C3F32 --maxdisp 192 --bn 4 \
               --loadmodel (filepath of pretrained weight) \
               --datas_train "k15-tr,k12-tr" --dir_datas_train (dir_root_kitti) \
               --datas_val "k15-val,k12-val" --dir_datas_val (dir_root_kitti) \
               --crop_width 512 --crop_height 256 \
               --epochs 20 --nloop 30 --freq_print 20 \
               --freq_optim 4 \
               --lr 0.005 --lr_epoch0 16 \
               --lr_stride 10 --lr_delay 0.2 \
               --dir_save $dir_save \
               2>&1 | tee -a "$LOG"

You can also see those examples in [demos/train_*.sh] for details.

Evaluation

Use the following command to evaluate the trained WSMCnet on KITTI 2015 test data

dir_save="./results"
LOG="${dir_save}/log_`date +%Y-%m-%d_%H-%M-%S`.txt"
mkdir -p "${dir_save}"

python main.py --mode Submission --arch WSMCnetEB_S2C3F32 --maxdisp 192 --bn 1 \
               --loadmodel (filepath of pretrained weight) \
               --datas_val "k15-te" --dir_datas_val (dir_root_kitti) \
               --freq_print 1 \
               --dir_save $dir_save \
               2>&1 | tee -a "$LOG"

You can also see the example in demos/kitti_submission.sh for details.

Pretrained Model

Baidu-pan Extraction code:jz05

Results on KITTI 2015 leaderboard

Method D1-all (All) D1-all (Noc) Runtime (s) Environment
WSMCnetEB-S2C3 2.13 % 1.85 % 0.39 Nvidia GTX 1070 (pytorch)
PSMNet 2.32 % 2.14 % 0.41 Nvidia GTX Titan Xp (pytorch)
iResNet-i2 2.44 % 2.19 % 0.12 Nvidia GTX Titan X (Pascal) (Caffe)
GC-Net 2.87 % 2.61 % 0.90 Nvidia GTX Titan X (TensorFlow)
MC-CNN 3.89 % 3.33 % 67 Nvidia GTX Titan X (CUDA, Lua/Torch7)

Contacts

wangyf_1991@163.com

Any discussions or concerns are welcomed!

About

Pytorch Implementation of the Wide Stride Multi-Classification Matching Network

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages