Skip to content
code for CVPR-2019 paper: Self-supervised Spatio-temporal Representation Learning for Videos by Predicting Motion and Appearance Statistics
Branch: master
Clone or download
Latest commit 14c943a Jun 13, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
list Upload data list Jun 14, 2019
LICENSE Initial commit Apr 4, 2019
README.md Update README.md Jun 14, 2019
comput_motion_statistics_fast.py Initial submit Jun 14, 2019
input_data.py Initial submit Jun 14, 2019
model.py Initial submit Jun 14, 2019
train.py Initial submit Jun 14, 2019

README.md

video_repres_mas

Tensorflow implementation of our CVPR 2019 paper Self-Supervised Spatio-Temporal Representation Learning for Videos by Predicting Motion and Appearance Statistics.

Overview

We realease partial of our training code on UCF101 dataset. It contains the self-supervised learning based on motion statistics (see more details in our paper).

Requirements

  1. tensorflow >= 1.9.0
  2. Python 3
  3. cv2
  4. scipy

Data preparation

You can download the original UCF101 dataset from the official website. And then extarct RGB images from videos and finally extract optical flow data using TVL1 method. But I recommend you to direclty download the pre-processed RGB and optical flow data of UCF101 provided by feichtenhofer.

Train

Here we provide the first version of our training code with "placeholder" as data reading pipeline, so you don't need to write RGB/Optical flow data into tfrecord format. We also rewrite the training code using Dataset API, but currently we think the placeholder version is enough for you to get to understand motion statsitics.

Before python train.py, remember to set right dataset directory in the list file, and then you can play with the motion statistics!

Citation

If you find this repository useful in your research, please consider citing:

@inproceedings{wang2019self,
  title={Self-Supervised Spatio-Temporal Representation Learning for Videos by Predicting Motion and Appearance Statistics},
  author={Wang, Jiangliu and Jiao, Jianbo and Bao, Linchao and He, Shengfeng and Liu, Yunhui and Liu, Wei},
  booktitle={CVPR},
  pages={4006--4015},
  year={2019}
}
You can’t perform that action at this time.