Pytorch implementation of Non-local Neural Networks.
After downloading the Kinetics dataset version from DATASET.md, use the following script to create a classes.txt file and a dir_name_list.txt file. The first one maps each class name found in the directory to an ID, while the second lists each video of the directory with its respective class ID.
python prepare_dataset.py --dir_path [path_to_video_dir]For instance, you should call this script for the folders: 'train_clips', 'test_clips' and 'val_clips'; generating the files: 'train_clips_list.txt', 'test_clips_list.txt', 'val_clips_list.txt' and 'classes.txt'.
Pretrained Weights
Download the pretrained weights from https://github.com/facebookresearch/video-nonlocal-net repo:
- i3d_baseline_32x2_IN_pretrain_400k.pkl
- i3d_nonlocal_32x2_IN_pretrain_400k.pkl
- i3d_baseline_8x8_IN_pretrain_400k.pkl
- i3d_nonlocal_8x8_IN_pretrain_400k.pkl
Convert these weights from caffe2 to pytorch:
python nonlocal_net.py --type [baseline, nonlocal] --frame_num [8, 32]Remember to change the path to the downloaded weights in the code to match yours.
Evaluate
The evaluation script saves the top5 predictions for each video as they are evaluated and provide at the end of the evaluation a report with some important metrics.
python eval_model.py --map_file [path_to_map_file] --root_data_path [path_to_videos_dir] --output_file [output_file_path] --mode [test, val] --weights_file [path_to_pretrained_weights] [--baseline]| pre-trained weights file | input frames | non-local? | fully-conv? | top1 / top5 | reported by authors |
|---|---|---|---|---|---|
| i3d_baseline_32x2_IN_pretrain_400k.pkl | 32 | - | - | 72.3 / 90.5 | - |
| i3d_baseline_32x2_IN_pretrain_400k.pkl | 32 | - | Yes | 73.0 / 90.8 | 73.3 / 90.7 (in paper) |
| i3d_nonlocal_32x2_IN_pretrain_400k.pkl | 32 | Yes | - | 73.9 / 91.2 | - |
| i3d_nonlocal_32x2_IN_pretrain_400k.pkl | 32 | Yes | Yes | 74.4 / 91.6 | 74.9 / 91.6 (in paper) |
| i3d_baseline_8x8_IN_pretrain_400k.pkl | 8 | - | - | 72.1 / 90.4 | - |
| i3d_baseline_8x8_IN_pretrain_400k.pkl | 8 | - | Yes | 72.9 / 90.7 | 73.4 / 90.9 (in github page) |
| i3d_nonlocal_8x8_IN_pretrain_400k.pkl | 8 | Yes | - | 73.8 / 91.1 | - |
| i3d_nonlocal_8x8_IN_pretrain_400k.pkl | 8 | Yes | Yes | 74.3 / 91.5 | 74.7 / 91.6 (in github page) |