Skip to content
/ RSKP Public

The official implementation of 'Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation' (CVPR 2022)

License

Notifications You must be signed in to change notification settings

LeonHLJ/RSKP

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

43 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RSKP

Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation (CVPR 2022)
Linjiang Huang (CUHK), Liang Wang (CASIA), Hongsheng Li (CUHK)

arXiv CVPR2022

Overview

The experimental results on THUMOS14 are as below.

Method \ mAP(%) @0.1 @0.2 @0.3 @0.4 @0.5 @0.6 @0.7 AVG
UntrimmedNet 44.4 37.7 28.2 21.1 13.7 - - -
STPN 52.0 44.7 35.5 25.8 16.9 9.9 4.3 27.0
W-TALC 55.2 49.6 40.1 31.1 22.8 - 7.6 -
AutoLoc - - 35.8 29.0 21.2 13.4 5.8 -
CleanNet - - 37.0 30.9 23.9 13.9 7.1 -
MAAN 59.8 50.8 41.1 30.6 20.3 12.0 6.9 31.6
CMCS 57.4 50.8 41.2 32.1 23.1 15.0 7.0 32.4
BM 60.4 56.0 46.6 37.5 26.8 17.6 9.0 36.3
RPN 62.3 57.0 48.2 37.2 27.9 16.7 8.1 36.8
DGAM 60.0 54.2 46.8 38.2 28.8 19.8 11.4 37.0
TSCN 63.4 57.6 47.8 37.7 28.7 19.4 10.2 37.8
EM-MIL 59.1 52.7 45.5 36.8 30.5 22.7 16.4 37.7
BaS-Net 58.2 52.3 44.6 36.0 27.0 18.6 10.4 35.3
A2CL-PT 61.2 56.1 48.1 39.0 30.1 19.2 10.6 37.8
ACM-BANet 64.6 57.7 48.9 40.9 32.3 21.9 13.5 39.9
HAM-Net 65.4 59.0 50.3 41.1 31.0 20.7 11.1 39.8
ACSNet - - 51.4 42.7 32.4 22.0 11.7 -
WUM 67.5 61.2 52.3 43.4 33.7 22.9 12.1 41.9
AUMN 66.2 61.9 54.9 44.4 33.3 20.5 9.0 41.5
CoLA 66.2 59.5 51.5 41.9 32.2 22.0 13.1 40.9
ASL 67.0 - 51.8 - 31.1 - 11.4 -
TS-PCA 67.6 61.1 53.4 43.4 34.3 24.7 13.7 42.6
UGCT 69.2 62.9 55.5 46.5 35.9 23.8 11.4 43.6
CO2-Net 70.1 63.6 54.5 45.7 38.3 26.4 13.4 44.6
D2-Net 65.7 60.2 52.3 43.4 36.0 - - -
FAC-Net 67.6 62.1 52.6 44.3 33.4 22.5 12.7 42.2
Ours 71.3 65.3 55.8 47.6 38.2 25.4 12.5 45.1

Prerequisites

Recommended Environment

  • Python 3.6
  • Pytorch 1.5
  • Tensorboard Logger
  • CUDA 10.1

Note: Our code works with different PyTorch and CUDA versions, for high version of Pytorch, you need to change one line of our code according to this issue.

Data Preparation

  1. Prepare THUMOS'14 dataset.

    • We recommend using features and annotations provided by this repo.
  2. Place the features and annotations inside a dataset/Thumos14reduced/ folder.

Usage

Training

You can easily train the model by running the provided script.

  • Refer to options.py. Modify the argument of dataset-root to the path of your dataset folder.

  • Run the command below.

$ python main.py --run-type 0 --model-id 1

Models are saved in ./ckpt/dataset_name/model_id/

Evaulation

The trained model can be found here. (This saved model's result is slightly different from the one reported in our paper.)

Please put it into ./ckpt/dataset_name/model_id/.

  • Run the command below.
$ python main.py --pretrained --run-type 1 --model-id 1 --load-epoch xxx

Please refer to the log in the same folder of saved models to set the load epoch of the best model. Make sure you set the right model-id that corresponds to the model-id during training.

References

We referenced the repos below for the code.

Citation

@InProceedings{rskp,
  title={Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation},
  author={Huang, Linjiang and Wang, Liang and Li, Hongsheng},
  booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2022}
}

Contact

If you have any question or comment, please contact the first author of the paper - Linjiang Huang (ljhuang524@gmail.com).

About

The official implementation of 'Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation' (CVPR 2022)

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages