Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Latest commit 5c5bff0 May 31, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
data add training code May 30, 2019
LICENSE init May 20, 2019 fix typo May 30, 2019 add training code May 30, 2019 init May 20, 2019 add training code May 30, 2019 add training code May 30, 2019 add training code May 30, 2019 add training code May 30, 2019

Spatio-temporal Video Re-localization by Warp LSTM

by Yang Feng, Lin Ma, Wei Liu, and Jiebo Luo


We formulate a new task named spatio-temporal video re-localization. Given a query video and a reference video, spatio-temporal video re-localization aims to localize tubelets in the reference video such that the tubelets semantically correspond to the query. For more details, please refer to our paper.

alt text


  author = {Feng, Yang and Ma, Lin and Liu, Wei and Luo, Jiebo},
  title = {Spatio-temporal Video Re-localization by Warp LSTM},
  booktitle = {CVPR},
  year = {2019}


pip install tensorflow-gpu
sudo apt install python-opencv

In case you are only interested in the proposed Warp LSTM, please find the implementation in the link.


  1. Generate the dataset for STVR.

Install Tensorflow Object Detection API.

  1. mkdir ~/workspace
    cd ~/workspace
    git clone
    cd models
    git remote add yang
    git fetch yang
    git checkout warp_c
    export PYTHONPATH=${HOME}/workspace/models/research/object_detection:\

Then follow the instructions in Installation

Extract video clips.

  1. We cut the videos to one-second clip for loading into Tensorflow.
    cd ~/workspace
    git clone
    cd STVR
    python --data_dir PATH_TO_VIDEOS --subset train
    python --data_dir PATH_TO_VIDEOS --subset val


  1. python --data_dir PATH_TO_VIDEOS --batch_size 8 --i3d_ckpt CKPT_PATH


  1. python --data_dir PATH_TO_VIDEOS --i3d_ckpt CKPT_PATH


Part of the code is from kinetics-i3d, ActivityNet, and Tensorflow Object Detection API.

You can’t perform that action at this time.