The training and testing in PRSA-Net is reimplemented in PyTorch for the ease of use.
Other minor Python modules can be installed by running
pip install -r requirements.txtWe support experimenting with two publicly available datasets for temporal action detection: THUMOS14 & ActivityNet v1.3. Here are some steps to download these two datasets.
- THUMOS14: THUMOS14 challenge website.
- ActivityNet v1.3: Using the official ActivityNet downloader to download videos from the YouTube. And the dataset is provided in the form of YouTube URL list.
You can get the TSN features for training and testing from G-TAD GoogleDrive. I3D features will be provided later.
Install Align1D layers
cd aligner/
python setup.py installSet the path of features in config/cfg.yaml
feature_path: $PATH_OF_FEATURES
video_info_path: $PATH_OF_ANNOTATIONSThen, you can use the following commands to train PRSA-Net
python main.py --mode train --cfg $PATH_TO_CONFIG_FILEYou can evaluate the model's action proposal generation performance and action detection performance at the same time by running the following command
python main.py --mode infer --cfg $PATH_TO_CONFIG_FILEWe use the weight of the 4-th epoch by default for model evaluation during the experiment. If you want, you can modify the eval_model field in the config file. This script will report the proposal generation performance in terms of AR(average recall) under various number of proposals, and detection performance in terms of (mean average precision) at different IoU thresholds..
| AR@100 | RGB+Flow |
|---|---|
| PRSA-Net (I3D) | 56.12 |
| mAP@0.5IoU (%) | RGB+Flow |
|---|---|
| PRSA-Net (I3D + two-stream) | 55.0 |
| PRSA-Net (I3D + PGCN) | 58.7 |
My implementations borrow ideas from previous works.
BMN: BMN: Boundary-Matching Network for Temporal Action Proposal Generation.
G-TAD: Sub-Graph Localization for Temporal Action Detection
lishuaicheng@sensetime.com