TALL: Temporal Activity Localization via Language Query
Switch branches/tags
Nothing to show
Clone or download
Latest commit 3df6794 Mar 16, 2018
Permalink
Failed to load latest commit information.
exp_data exp_data Dec 2, 2017
util clean Dec 2, 2017
README.md Update README.md Mar 15, 2018
ctrl_model.py first commit Dec 2, 2017
ctrl_model.pyc first commit Dec 2, 2017
ctrl_test_results.txt first commit Dec 2, 2017
dataset.py first commit Dec 2, 2017
dataset.pyc first commit Dec 2, 2017
main.py first commit Dec 2, 2017
video_allframes_info.pkl first commit Dec 2, 2017
vs_multilayer.py first commit Dec 2, 2017
vs_multilayer.pyc first commit Dec 2, 2017

README.md

TALL: Temporal Activity Localization via Language Query

This is the repository for our ICCV 2017 paper TALL: Temporal Activity Localization via Language Query.

Visual Features on TACoS

Download the C3D features for training set and test set of TACoS dataset. Modify the path to feature folders in main.py

Sentence Embeddings on TACoS

Download the Skip-thought sentence embeddings and sample files from here of TACoS Dataset, and put them under exp_data folder.

Reproduce the results on TACoS

python main.py

Charades-STA anno download

The sentence temporal annotations on Charades dataset are available here: train, test. The format is "[video name] [start time] [end time]##[sentence]". You may want to generate the skip-thought embeddings and C3D features on Charades-STA, and modify the codes slightly to reproduce the experiments.

Updates on Charades-STA performance

I did some anno cleaning for Charades-STA (compared to the version I used in ICCV paper), the updated performance is listed below. Please compare to these results when using Charades-STA.

Model R@1,IoU=0.5 R@1,IoU=0.7 R@5,IoU=0.5 R@5,IoU=0.7
CTRL (aln) 17.69 5.91 55.54 23.79
CTRL (reg-p) 19.22 6.64 57.98 25.22
CTRL (reg-np) 21.42 7.15 59.11 26.91