Skip to content
Joint-task Self-supervised Learning for Temporal Correspondence (NeurIPS 2019)
Python
Branch: master
Clone or download
Sifei Liu
Sifei Liu clean codes
Latest commit b4f4e8f Jan 8, 2020
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs release Sep 8, 2019
libs clean codes Jan 8, 2020
match_track_switch
weights release Sep 8, 2019
README.md Update README.md Sep 8, 2019
model.py
test.py
test_with_track.py release Sep 8, 2019
track_match_v1.py

README.md

Joint-task Self-supervised Learning for Temporal Correspondence

Project | Paper

Overview

Joint-task Self-supervised Learning for Temporal Correspondence

Xueting Li*, Sifei Liu*, Shalini De Mello, Xiaolong Wang, Jan Kautz, Ming-Hsuan Yang.

(* equal contributions)

In Neural Information Processing Systems (NeurIPS), 2019.

Citation

If you use our code in your research, please use the following BibTex:

@inproceedings{uvc_2019,
    Author = {Xueting Li and Sifei Liu and Shalini De Mello and Xiaolong Wang and Jan Kautz and Ming-Hsuan Yang},
    Title = {Joint-task Self-supervised Learning for Temporal Correspondence},
    Booktitle = {NeurIPS},
    Year = {2019},
}

Instance segmentation propagation on DAVIS2017

Method J_mean J_recall J_decay F_mean F_recall F_decay
Ours 0.563 0.650 0.289 0.592 0.641 0.354
Ours - track 0.577 0.683 0.263 0.613 0.698 0.324

Prerequisites

The code is tested in the following environment:

  • Ubuntu 16.04
  • Pytorch 1.1.0, tqdm, scipy 1.2.1

Testing on DAVIS2017

Testing without tracking

To test on DAVIS2017 for instance segmentation mask propagation, please run:

python test.py -d /workspace/DAVIS/ -s 480

Important parameters:

  • -c: checkpoint path.
  • -o: results path.
  • -d: DAVIS 2017 dataset path.
  • -s: test resolution, all results in the paper are tested on 480p images, i.e. -s 480.

Please check the test.py file for other parameters.

Testing with tracking

To test on DAVIS2017 by tracking & propagation, please run:

python test_with_track.py -d /workspace/DAVIS/ -s 480

Similar parameters as test.py, please see the test_with_track.py for details.

Training on Kinetics

Dataset

We use the kinetics dataset for training.

Training command

python track_match_v1.py --wepoch 10 --nepoch 30 -c match_track_switch --batchsize 40 --coord_switch 0 --lc 0.3

Acknowledgements

You can’t perform that action at this time.