A fork of Caffe with OpenMPI-based Multi-GPU (mainly data parallel) support for action recognition and more. More documentation please see the original readme.
C++ Python Cuda CMake Protocol Buffer Matlab Other
Clone or download
Pull request Compare This branch is 1137 commits ahead, 18 commits behind CUHK-MMLAB:master.
yjxiong Merge pull request #177 from vra/action_recog
Update readme.md to make table render correctly
Latest commit 9831a2b Jul 22, 2017
Permalink
Failed to load latest commit information.
action_matlab fix rgb_mean.mat Aug 11, 2015
action_python Update VideoSpatialPrediction.py Feb 29, 2016
cmake CMake: link with ${HDF5_HL_LIBRARIES} Oct 27, 2016
data [example] image classification web demo Jul 12, 2014
docs List protobuf-compiler dependency in the correct place (it is in the … Jul 7, 2015
examples add segmentation example Dec 7, 2016
include/caffe Merge branch 'action_recog' of https://github.com/yjxiong/caffe into … May 26, 2017
matlab Update ilsvrc_2012_mean.mat to W x H x C, update demo and add comments May 30, 2015
models Update readme.md Jun 29, 2017
python add a handy method to disable logging in pycaffe May 26, 2017
scripts Update travis (#168) May 25, 2017
src Remove unnecessary frozen-check in bn_layer Jun 21, 2017
tools Split MPI comm by specified groups (#101) Sep 2, 2016
.Doxyfile update doxygen config to stop warnings Sep 3, 2014
.gitignore ignore pycharm files Feb 22, 2015
.travis.yml Update travis (#168) May 25, 2017
CHANGELOG.md typo Dec 25, 2016
CMakeLists.txt Merge branch 'master' into data_parallel Jul 10, 2015
CONTRIBUTORS.md clarify the license and copyright terms of the project Aug 7, 2014
INSTALL.md replace bundled install instructions with link to site Feb 10, 2014
LICENSE copyright 2015 Jun 23, 2015
Makefile Fix automatic header file dependency for MatCaffe May 29, 2015
Makefile.config.example Add commented out helpers for homebrew users Apr 2, 2015
README.md Update README.md Feb 14, 2017
caffe.cloc [fix] stop cloc complaint about cu type Sep 4, 2014

README.md

Action Recognition with Deep Learning

Build Status

This branch hosts the code for the technical report "Towards Good Practices for Very Deep Two-stream ConvNets", and more.

Updates

  • Dec, 2016
    • Major updates of the codebase. New features include memory optimization and dilated convolution.
  • Aug 23, 2016
  • Aug 1, 2016
    • New working example: "Actionness Estimation Using Hybrid FCNs" on CVPR 2016.
  • Jul 16, 2016
    • New working example: "Real-time Action Recognition with Enhanced Motion Vector CNNs" on CVPR 2016.
  • Apr 27, 2016
    • cuDNN v5 support, featuring the super fast WINOGrad Convolution and cuDNN implementation of BatchNormalization.
  • Dec 23, 2015
    • Refactored cudnn wrapper to control overall memory consumption. Will automatically find the best algorithm combination under memory constraint.
  • Dec 17, 2015
    • cuDNN v4 support: faster convolution and batch normalization (around 20% performance gain).
  • Nov 22, 2015
    • Now python layer can expose a prefetch() method, which will be run in parallel with network processing.

Full Change Log

Features

  • VideoDataLayer for inputing video data.
  • Training on optical flow data.
  • Data augmentation with fixed corner cropping and multi-scale cropping.
  • Parallel training with multiple GPUs.
  • Newest cuDNN integration.
  • Slim memory footprints in both training and testing,

Usage

See more in Wiki.

Generally it's the same as the original caffe. Please see the original README. Please see following instruction for accessing features above. More detailed documentation is on the way.

  • Video/optic flow data
    • First use the optical flow extraction tool to convert videos to RGB images and opitcal flow images.
    • A new data layer called VideoDataLayer has been added to support multi-frame input. See the UCF101 sample for how to use it.
    • Note: The VideoDataLayer can only input the optical-flow images generated by the tool listed above.
  • Fixed corner cropping augmentation
    • Set fix_crop to true in tranform_param of network's protocol buffer definition.
  • "Multi-scale" cropping augmentation
    • Set multi_scale to true in transform_param
    • In transform_param, specify scale_ratios as a list of floats smaller than one, default is [1, .875, .75, .65]
    • In transform_param, specify max_distort to an integer, which will limit the aspect ratio distortion, default to 1
  • cuDNN v5
  • The cuDNN v5 wrapper has optimized engines for convolution and batch normalization.
  • The solver protobuf config has a parameter richness which specifies the total GPU memory in MBs available to the cudnn convolution engine as workspaces. Default richness is 300 (300MB). Using this parameter you can control the GPU memory consumption of training, the system will find the best setup under the memory limit for you.
  • Training with multiple GPUs
    • Requires OpenMPI > 1.7.4 (Why?). Remember to compile your OpenMPI with option --with-cuda
    • Specify list of GPU IDs to be used for training, in the solver protocol buffer definition, like device_id: [0,1,2,3]
    • Compile using cmake and use mpirun to launch caffe executable, like
mkdir build && cd build
cmake .. -DUSE_MPI=ON
make && make install
mpirun -np 4 ./install/bin/caffe train --solver=<Your Solver File> [--weights=<Pretrained caffemodel>]

Note: actual batch_size will be num_device times batch_size specified in network's prototxt.

  • Runtime memory optimization
    • Memory optimization drastically reduces memory usage (half for training and almost all for testing) by safely sharing underlying storage of a series of blobs.
    • For usage and the mechanism behind the scene, see the Wiki Page

Working Examples

Extension

Currently all existing data layers sub-classed from BasePrefetchingDataLayer support parallel training. If you have newly added layer which is also sub-classed from BasePrefetchingDataLayer, simply implement the virtual method

inline virtual void advance_cursor();

Its function should be forwarding the "data cursor" in your data layer for one step. Then your new layer will be able to provide support for parallel training.

Questions

Contact

Citation

You are encouraged to also cite one of the following papers if you find this repo helpful

@inproceedings{TSN2016ECCV,
  author    = {Limin Wang and
               Yuanjun Xiong and
               Zhe Wang and
               Yu Qiao and
               Dahua Lin and
               Xiaoou Tang and
               Luc {Val Gool}},
  title     = {Temporal Segment Networks: Towards Good Practices for Deep Action Recognition},
  booktitle   = {ECCV},
  year      = {2016},
}

@article{MultiGPUCaffe2015,
  author    = {Limin Wang and
               Yuanjun Xiong and
               Zhe Wang and
               Yu Qiao},
  title     = {Towards Good Practices for Very Deep Two-Stream ConvNets},
  journal   = {CoRR},
  volume    = {abs/1507.02159},
  year      = {2015},
  url       = {http://arxiv.org/abs/1507.02159},
}

Following is the original README of Caffe.

Caffe

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and community contributors.

Check out the project site for all the details like

and step-by-step examples.

Join the chat at https://gitter.im/BVLC/caffe

Please join the caffe-users group or gitter chat to ask questions and talk about methods and models. Framework development discussions and thorough bug reports are collected on Issues.

Happy brewing!

License and Citation

Caffe is released under the BSD 2-Clause license. The BVLC reference models are released for unrestricted use.

Please cite Caffe in your publications if it helps your research:

@article{jia2014caffe,
  Author = {Jia, Yangqing and Shelhamer, Evan and Donahue, Jeff and Karayev, Sergey and Long, Jonathan and Girshick, Ross and Guadarrama, Sergio and Darrell, Trevor},
  Journal = {arXiv preprint arXiv:1408.5093},
  Title = {Caffe: Convolutional Architecture for Fast Feature Embedding},
  Year = {2014}
}