R(2+1)D and Mixed-Convolutions for Action Recognition.
Clone or download
Latest commit 9c49f45 Jan 14, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data combined commit - shuffle list May 15, 2018
lib Enable CPU for test_net Jul 26, 2018
scripts adding copyright headers Oct 18, 2018
tools Fix GPU selection Jan 11, 2019
tutorials Update Installation_guide.md Jun 14, 2018
.gitignore Initial commit May 2, 2018
CODE_OF_CONDUCT.md Initial commit May 2, 2018
CONTRIBUTING.md Initial commit May 2, 2018
LICENSE Initial commit May 2, 2018
README.md Update README.md to specify a Caffe2 source build is required Oct 11, 2018

README.md

R(2+1)D and Mixed-Convolutions for Action Recognition

r2plus1d1

[project page] [paper]

If you find this work helpful for your research, please cite our following paper:

D. Tran, H. Wang, L. Torresani, J. Ray, Y. LeCun and M. Paluri. A Closer Look at Spatiotemporal Convolutions for Action Recognition. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

@inproceedings{r2plus1d_cvpr18,
    title = {A Closer Look at Spatiotemporal Convolutions for Action Recognition},
    author = {Du Tran and Heng Wang and Lorenzo Torresani and Jamie Ray and Yann LeCun and
               Manohar Paluri},
    booktitle = {CVPR},
    year = 2018
}

If you have any question or feedback about the code, please contact: trandu@fb.com, hengwang@fb.com.

Requirements

R2Plus1D requires the following dependencies:

  • OpenCV (tested on 3.4.1) and ffmpeg.
  • Caffe2 and its dependencies.
    • You will need to build from source and install with USE_OPENCV=1 USE_FFMPEG=1 USE_LMDB=1 python setup.py install for OpenCV, ffmpeg, and lmdb support.
  • And lmdb, python-lmdb, and pandas.

Installation

  • You need to install ffmpeg, OpenCV, and caffe2. Caffe2 source build instructions can be found here but make sure you install with USE_OPENCV=1 USE_FFMPEG=1 USE_LMDB=1 python setup.py install. You also need to install lmdb, python-lmdb, and pandas.

Tutorials

We provide some basic tutorials for you to get familar with the code and tools.

License

R2Plus1D is Apache 2.0 licensed, as found in the LICENSE file.

Acknowledgements

The authors would like to thank Ahmed Taei, Aarti Basant, Aapo Kyrola, and the Facebook Caffe2 team for their help in implementing ND-convolution, in optimizing video I/O, and in providing support for distributed training. We are grateful to Joao Carreira for sharing I3D results on the Kinetics validation set.