Skip to content
Segmentation Prediction framework from WACV 2019
Branch: master
Clone or download
Latest commit 96606d1 May 25, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs update May 24, 2019
external
utils update segpred May 24, 2019
.gitignore
README.md
segpred_val.py update segpred May 24, 2019
write_psp_feat.py update May 24, 2019

README.md

Recurrent Flow-Guided Semantic Forecasting

Adam M. Terwilliger, Garrick Brazil, Xiaoming Liu

Overview

We are releasing our segmentation prediction framework for the Cityscapes dataset as detailed in our arXiv report, accepted at WACV 2019. Segpred Overview Our approach aggregates past optical flow features using a convolutional LSTM to predict future optical flow, which is used by an learnable warp layer to produce future segmentation.

For additional details, please visit our project page.

If you utilize our framework in your work, please include this citation:

@inproceedings{recurrent-flow-guided-semantic-forecasting, 
  author = {Adam M. Terwilliger and Garrick Brazil and Xiaoming Liu},
  title = {Recurrent Flow-Guided Semantic Forecasting},
  booktitle = {Proc. IEEE Winter Conference on Application of Computer Vision},
  month = {January},
  year = {2019},
}

Implementation

Our framework was tested with Debian GNU/Linux 8, Python 2.7, CUDA 8.0, cuDNN v5.1, OpenCV 2.4.9.1, NVIDIA Titan X GPUs, and a modified version of Caffe as provided. The included Caffe contains custom layers for PSPNet, ConvLSTM, FlowNet, and our warp layer, so it is necessary to utilize this version to successfully reproduce our results.

Setup

For specific details regarding setup, please refer to Installation.md.

Testing

We release our top performing models for short-term and mid-term prediction, which achieve 67.2 mIOU and 52.5 mIOU, respectively on Cityscapes validation set. These results can be reproduced, after completing the setup, as follows:

python segpred_val.py t3 0
python segpred_val.py t10 0

The pre-trained models are available for download here.

Contact

For any questions regarding the framework, feel free to post an issue on this github.

With any follow-up questions regarding the paper, don't hesitate to contact the authors at {adamtwig, brazilga, liuxm}@msu.edu.

You can’t perform that action at this time.