Skip to content
Demo code of the paper "SegFlow: Joint Learning for Video Object Segmentation and Optical Flow", in ICCV 2017
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
caffe Update resample_layer.cu Mar 29, 2018
data first_commit Sep 15, 2017
demo release SegFlow Sep 26, 2017
fcn_python first_commit Aug 7, 2017
model release SegFlow Sep 15, 2017
python_layers
training Create train.sh Feb 1, 2018
util first_commit Aug 16, 2017
README.md Update README.md Feb 22, 2018
iccv17_segflow_git.png Add files via upload Dec 24, 2017

README.md

SegFlow: Joint Learning for Video Object Segmentation and Optical Flow

Alt Text

Project webpage: https://sites.google.com/site/yihsuantsai/research/iccv17-segflow
Contact: Jingchun Cheng (chengjingchun at gmail dot com)

Paper

SegFlow: Joint Learning for Video Object Segmentation and Optical Flow
Jingchun Cheng, Yi-Hsuan Tsai, Shengjin Wang and Ming-Hsuan Yang
IEEE International Conference on Computer Vision (ICCV), 2017.

This is the authors' demo code described in the above paper. Please cite our paper if you find it useful for your research.

@inproceedings{Cheng_ICCV_2017,
  author = {J. Cheng and Y.-H. Tsai and S. Wang and M.-H. Yang},
  booktitle = {IEEE International Conference on Computer Vision (ICCV)},
  title = {SegFlow: Joint Learning for Video Object Segmentation and Optical Flow},
  year = {2017}
}

SegFlow Results

Segmentation Comparisons with Unsupervised Method

Segmentation Comparisons with Semi-supervised Method

Optical Flow Comparisons

Requirements

  • Install caffe and pycaffe (opencv is required).
    cd caffe
    make all -j8 (paths are needed to change in the configuration file)
    make pycaffe

  • Download the DAVIS 2016 dataset and put it in the data folder.

  • Download our pre-trained caffe model here and put it in the model folder.

Demo on DAVIS 2016

cd demo
python infer_DAVIS.py VIDEO_NAME
For example, run python infer_DAVIS.py dog

This code provides a demo for the parent net (Ours_OL) in SegFlow. The output contains both the segmentation and optical flow results.

Test on your own Videos

cd demo
python infer_video.py VIDEO_FILE
For example, run python infer_video.py ../data/video_example.mp4

Training code on DAVIS 2016 (Ours_OL)

Download the segmentation and flow pre-trained weights, and put them in the model folder.
cd training
sh train.sh

Download Our Segmentation Results on DAVIS 2016

  • SegFlow without online training step (Ours_OL) here
  • SegFlow without optical flow branch (Ours_FLO) here
  • Final SegFlow results here

Note

The model and code are available for non-commercial research purposes only.

  • 09/2017: demo code released
  • 01/2018: the pre-trained caffe model is updated
  • 02/2018: training code for the parent net (unsupervised setting) is released
You can’t perform that action at this time.