Matlab Shell
Switch branches/tags
Nothing to show
Clone or download
Latest commit 96b91af Jan 10, 2018
Permalink
Failed to load latest commit information.
Datasets Add missing files Dec 14, 2017
Layers Minor Jan 10, 2018
dicnn some little mistakes Dec 25, 2017
matconvnet @ 14223b8 Add matconvnet Sep 24, 2017
utils Rm matconvnet Sep 24, 2017
.gitmodules Add matconvnet Sep 24, 2017
README.md Update README.md Jan 3, 2018
main_train.m Minor Nov 6, 2017

README.md

Dynamic Image Networks for Action Recognition

Improved Results (see the extended version of CVPR paper)

ResNeXt-50 HMDB51 (%) UCF101 (%)
SI 53.5 87.6
DI 57.3 86.6
OF 55.8 84.9
DOF 58.9 86.6
SI+OF 67.5 93.9
SI+DI 61.3 90.6
OF+DOF 62.6 89.1
SI+DI+OF+DOF 71.5 95.0
SI+DI+OF+DOF+iDT 74.2 95.4
  • Results are in the standard average multi-class accuracy (%)
  • SI: RGB image
  • DI: dynamic RBG image
  • OF: optical flow
  • DOF: dynamic optical flow
  • iDT: improved trajectory features

Installation

  1. Clone the Dynamic Image Net repository:

    git clone --recursive  https://github.com/hbilen/dynamic-image-nets
  2. Compile matconvnet toolbox: (see http://www.vlfeat.org/matconvnet/install/)

  3. Install additional matconvnet packages

  run matconvnet/matlab/vl_setupnn.m ;
  vl_contrib install mcnExtraLayers ; vl_contrib setup mcnExtraLayers ;
  vl_contrib install autonn ; vl_contrib setup autonn ;
  1. Download your dataset : (e.g. UCF101 from http://crcv.ucf.edu/data/UCF101.php)

  2. Convert videos to frames, resize them to 256x256 and store them in such a directory structure: Alternatively, you can download RGB and precomputed optical flow frames from Christoph Feichtenhofer and copy RGB frames under "UCF101/frames" and optical flow frames under "UCF101/tvl1_flow".

    data/UCF101/ucfTrainTestlist/
    ├── classInd.txt
    ├── testlist01.txt
    ├── testlist02.txt
    ├── testlist03.txt
    ├── trainlist01.txt
    ├── trainlist02.txt
    └── trainlist03.txt
    data/UCF101/frames/
    ├── v_ApplyEyeMakeup_g01_c01
    │   ├── 00001.jpg
    │   ├── 00002.jpg
    │   ├── 00003.jpg
    │   ├── 00004.jpg
    │   ├── 00005.jpg

Compute and Visualise Approximate Dynamic Images

  1. If you want to compute approximate dynamic images, get a list of ordered frames from a video and try
di = compute_approximate_dynamic_images(images) ;
  1. If you want to visualise approximate dynamic images, get a list of ordered frames from a video and try
visualize_approximate_dynamic_images(images)

Train a Dynamic Image Net

You can modify the options in main_train.m and train your model by running matlab main_train

Note: If you want to train a model on a different dataset than UCF101 or HMDB51, you need to write a custom script cnn_dataset_setup_data to build your database (imdb).

Evaluation

  1. Download the CNN Models for the UCF101 dataset, that are used in the journal, from here.
  2. Choose the right model, split and input type (e.g.)
    net = load('resnext50-rgb-arpool-split1.mat') ;
    net = dagnn.DagNN.loadobj(net) ;
    net.addLayer('errMC',ErrorMultiClass(),{'prediction','label'},'mcerr') ;
    opts.network = net ;
    opts.split = 1 ;
    opts.train.gpus = 1 ;
    opts.epochFactor = 0 ; 
    [net, info] = cnn_dicnn_rgb(opts)

Citing Dynamic Image Networks

If you find the code useful, please cite:

    @inproceedings{Bilen2016a,
      author    = "Bilen, H. and Fernando, B. and Gavves, E. and Vedaldi, A. and Gould, S.",
      title     = "Dynamic Image Networks for Action Recognition",
      booktitle = "CVPR",
      year      = "2016"
    }
    @journal{Bilen2017a,
      author    = "Bilen, H. and Fernando, B. and Gavves, E. and Vedaldi, A.",
      title     = "Action Recognition with Dynamic Image Networks",
      journal   = " IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)",
      year      = "2017"
    }

License

The analysis work performed with the program(s) must be non-proprietary work. Licensee and its contract users must be or be affiliated with an academic facility. Licensee may additionally permit individuals who are students at such academic facility to access and use the program(s). Such students will be considered contract users of licensee. The program(s) may not be used for commercial competitive analysis (such as benchmarking) or for any commercial activity, including consulting.