Skip to content
Tensorflow implementation of the ICLR 2017 paper: Decomposing Motion and Content for Natural Video Sequence Prediction
Python Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data Update download.sh Jun 26, 2017
models/paper_models Delete ._.DS_Store Jun 23, 2017
src test with trained model path fix Jul 2, 2017
LICENSE iclr2017mcnet code Jun 23, 2017
README.md Update README.md Apr 29, 2019

README.md

MCnet

This is the code for the ICLR 2017 paper Decomposing Motion and Content for Natural Video Sequence Prediction by Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin and Honglak Lee.

Please follow the instructions to run the code.

Requirements

MCnet works with

  • Linux
  • NVIDIA Titan X GPU
  • Python2
  • Tensorflow version 1.1.0

Installing Dependencies (Anaconda installation is recommended)

FFMPEG needs to be installed as well to generate gif videos. If using anaconda, ffmpeg can be installed as follows:

  • conda install -c menpo ffmpeg=3.1.3

Downloading Data

  • KTH
./data/KTH/download.sh
  • UCF101
./data/UCF101/download.sh
  • Sporst1M (will take a very long time --> 70,000 videos are downloaded only)
./data/S1M/download.sh

Downloading Pre-trained models

  • Download the models trained on KTH and Sports1M
./models/paper_models/download.sh

KTH training/testing

  • Training:
python train_kth.py --gpu=0 --batch_size=8 --K=10 --T=10 --alpha=1.0 --beta=0.02
  • Testing with model used in paper:
python test_kth.py --gpu=0 --prefix=paper_models
  • Testing with model from training command above:
python test_kth.py --gpu=0 --prefix=KTH_MCNET_image_size=128_K=10_T=10_batch_size=8_alpha=1.0_beta=0.02_lr=0.0001

Sports1M training / UCF101 testing

  • Training:
python train_s1m.py --gpu=0 --batch_size=8 --K=4 --T=1 --alpha=1.0 --beta=0.001
  • Testing with model used in paper:
python test_ucf101.py --gpu=0 --prefix=paper_models
  • Testing with model from training command above:
python test_ucf101.py --gpu=0 --prefix=S1M_MCNET_K=4_T=1_batch_size=8_alpha=1.0_beta=0.001_lr_0.0001

Results

The generated gifs will be located in

./results/images/<dataset>

The quantative results will be in

./results/quantitative/<dataset>

The quantitative results for each video will be stored as dictionaries, and the mean results for all test data instances at every timestep can be displayed as

import numpy as np
results = np.load('<results_file_name>')
print(results['psnr'].mean(axis=0))
print(results['ssim'].mean(axis=0))

Citation

If you find this useful, please cite our work as follows:

@article{villegas17mcnet,
  author = {Ruben Villegas and Jimei Yang and Seunghoon Hong and Xunyu Lin and Honglak Lee},
  title = {Decomposing Motion and Content for Natural Video Sequence Prediction},
  journal = {ICLR},
  year = {2017},
}

Please contact "ruben.e.villegas@gmail.com" if you have any questions.

You can’t perform that action at this time.