Deep Feature Flow for Video Semantic Segmentation
Branch: master
Clone or download
Latest commit eba533c Aug 9, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
DockerConfigs add docker file Aug 9, 2018
data
deeplab update head Aug 9, 2018
experiments update head Aug 9, 2018
lib update head Aug 9, 2018
model/pretrained_model update main structure Aug 9, 2018
.gitignore
DeeplabCityscapes.tgn update main structure Aug 9, 2018
LICENSE update main structure Aug 9, 2018
README.md add scirpt for setup Aug 9, 2018
init.sh update main structure Aug 9, 2018
requirements.txt update main structure Aug 9, 2018

README.md

Deep Feature Flow for Video Semantic Segmentation

Based on Deeplab V2

1. Setup environment

  • If you use our dockerfile, you can run the code easily.
  • If you want to set up your own env, please follow these steps:
    • We only support python2.7 now
    • Install tk: sudo apt-get -y install python-tk
    • Install OpenCV 3.4.1
    • Install needed python packages with pip install -r requirements.txt
      • If you are in China Mainland, you can use these to speedup pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
  • Then sh init.sh to build the lib for faster-rcnn Because we use the code from Deformable ConvNets and the dataloader has some dependencies on faster-rcnn, so you need to build the lib first.

2. Prepare Data and Pretrained Model

Cityscapes Data

You need to download the cityscapes data from the official webpapge and unzip the data Put the data into data/cityscapes, you can use soft link to set the data path as the following: ln -s Dataset_path ./data/cityscapes

If you want to try DFF, you should download cityscapes video data and put it into data/cityscapes_video

Pretrained Model

Download pretrained resnet model flow net from Onedrive, and put the model into mode/pretrained_model/

./model/pretrained_model/resnet_v1_101-0000.params
./model/pretrained_model/flownet-0000.params

3. Train and Test

Training Deeplab V2

python ./experiments/deeplab/deeplab_train_test.py --cfg ./experiments/deeplab/cfgs/deeplab_resnet_v1_101_cityscapes_segmentation_base.yaml

Training Deeplab V2 Deformable

python ./experiments/deeplab/deeplab_train_test.py --cfg ./experiments/deeplab/cfgs/deeplab_resnet_v1_101_cityscapes_segmentation_dcn.yaml

Training DFF Deeplab V2

python ./experiments/deeplab_dff/deeplab_dff_train.py --cfg ./experiments/deeplab_dff/cfgs/deeplab_resnet_v1_101_cityscapes_segmentation_video.yaml

4. Performance

TBD

5. TODO List

  • Add Scripts
  • Add experiment results
  • Add support for Deeplab V3+
  • Add BiSeNet

6. FAQ

  • Program hang if your system opencv is 2.x and your opencv-python is 3.x

7. Acknowledgement

Thanks for the official deep featuere flow implementation and deeplab implementation from MSRACVER