This repo contains the code for the CVPR 2018 paper "Semantic Video Segmentation by Gated Recurrent Flow Propagation" by David Nilsson and Cristian Sminchisescu. [pdf]
Check config.py. Download all data from the cityscapes dataset and change the paths in config.py. Check that you can run python config.py without any errors.
Run misc/compile.sh to compile the bilinear warping operator. Change the include directory on line 9 if you get errors related to libcudart.
Download all pretrained models from here and unpack them under ./checkpoints/. For instance, the file ./checkpoints/flownet1.index should exist.
Evaluate the GRFP(LRR-4x, FlowNet2) setup on the validation set by running:
python evaluate.py --static lrr --flow flownet2
Evalutate GRFP(Dilation10, FlowNet2) for various number of frames, as in Table 3 and 4 in the paper:
python evaluate.py --static dilation --flow flownet2 --frames 1
python evaluate.py --static dilation --flow flownet2 --frames 5
The values in table 9 can be reproduced by running the following:
python evaluate.py --static lrr --flow flownet2
python evaluate.py --static lrr --flow flownet1
python evaluate.py --static lrr --flow farneback
Train and evaluate a model with the following commands:
python train.py --static lrr --flow flownet2
python evaluate.py --static lrr --flow flownet2 --ckpt lrr_flownet2_it10000
This should match the performance of the pre-trained LRR model above. See the ./checkpoints directory where parameters are saved during the training procedure. Only LRR is supported at the moment.
If you use the code in your own research, please cite
@InProceedings{Nilsson_2018_CVPR,
author = {Nilsson, David and Sminchisescu, Cristian},
title = {Semantic Video Segmentation by Gated Recurrent Flow Propagation},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}
Depending on the setup you use, consider also citing PSP, LRR, Dilation, FlowNet1, FlowNet2 or Farnebäck.