Skip to content
MAC: Mining Activity Concepts for Language-based Temporal Localization
Python
Branch: master
Clone or download
Latest commit a35be04 Nov 26, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
img camera ready Nov 21, 2018
ref_info code Nov 15, 2018
results_history code Nov 15, 2018
tf_summary code Nov 15, 2018
trained_save
.gitignore code Nov 15, 2018
LICENSE Initial commit Nov 7, 2018
README.md readme update Nov 26, 2018
acl_model.py
cnn.py code Nov 15, 2018
dataset.py
main.py code Nov 15, 2018
mpu.py code Nov 15, 2018

README.md

MAC

By Runzhou Ge, Jiyang Gao, Kan Chen, Ram Nevatia.

University of Southern California (USC).

Introduction

This repository contains the code for the WACV 2019 paper, MAC: Mining Activity Concepts for Language-based Temporal Localization. arXiv

Requirements

  • Python 2.7
  • Tensorflow 1.0 or higher
  • others

Download

The code is for Charades-STA dataset.

After cloning this repo, please donwload:

ref_info contains Charades-STA annotations, semantic activity concepts, checkpoints and others. After downloading ref_info.tar, untar it and move the folder to the root/ directory of this repo.

Please also change the visual feature and visual activity concepts directories in the main.py.

Training

For the paper results on Charades-STA dataset, run

python main.py --is_only_test True \
--checkpoint_path ./ref_info/charades_sta_wacv_2019_paper_ACL_k_results/trained_model.ckpt-10000 \
--test_name paper_results

You will get similar results listed in the row "ACL-K" of the following table.

Model R@1,IoU=0.7 R@1,IoU=0.5 R@5,IoU=0.7 R@5,IoU=0.5
CTRL 7.15 21.42 26.91 59.11
ACL-K 12.20 30.48 35.13 64.84

To train the model from scratch, run

python main.py

The results and checkpoints will appear in root/results_history/ and root/trained_save/, respectively.

Results Visualization

Citation

If you find this work is helpful, please cite:

@InProceedings{Ge_2019_WACV,
  author = {Ge, Runzhou and Gao, Jiyang and Chen, Kan and Nevatia, Ram},
  title = {MAC: Mining Activity Concepts for Language-based Temporal Localization},
  booktitle = {The IEEE Winter Conference on Applications of Computer Vision (WACV)},
  month = {January},
  year = {2019}
}

License

MIT License

Acknowledgements

This research was supported, in part, by the Office of Naval Research under grant N00014-18-1-2050 and by an Amazon Research Award.

You can’t perform that action at this time.