Dual Correlation Network for Efficient Video Semantic Segmentation
This repository is the official implementation of "Dual Correlation Network for Efficient Video Semantic Segmentation” ( This paper is under submission, we will show it later)
Requirements: PyTorch >= 1.4.0, CUDA >= 10.0, and Python==3.8
To Install weightingFunction
cd $DCNVSS_ROOT/Local-Attention-master
python setup.py build
To Install Correlation
cd $DCNVSS_ROOT/correlation
python setup.py build
Please follow Cityscapes to download Cityscapes dataset. After correctly downloading, the file system is as follows:
$DCNVSS_ROOT/data
├── Cityscapes_video
│ ├── gtFine
│ │ ├── train
│ │ └── val
│ └── leftImg8bit_sequence
│ ├── train
│ └── val-
Download pretrained PSP101 models BaiduYun(Access Code:ghk4) on Cityscapes dataset, and put them in a folder
./ckpt. -
Training requires 4 Nvidia GPUs.
# training Dual Correlation Network
bash ./train.sh
# training key frame selection module
bash ./train_KDM.sh-
Download the trained weights from BaiduYun(Access Code:bay9) and put them in a folder
./ckpt. -
Run the following commands:
bash ./eval_multipro.shThe code is heavily based on the following repositories:
- https://github.com/CSAILVision/sceneparsing
- https://github.com/zzd1992/Image-Local-Attention
- https://github.com/WeilunWang/correlation-layer
Thanks for their amazing works.
We will show it later.