Skip to content
Switch branches/tags
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

AherNet: Learning to Localize Actions from Moments

This repository includes the codes and configuration files of the transfer setting of "From ActivityNet v1.3 to Kinetics-600, i.e., ANet -> K600" in the paper.

All the training/validation features and related meta files of the dataset are not uploaded in the repository since they are too large, which will be released on google drive or Baidu Yun in the future.

The file k600_val_annotation.txt contains the mannul temporal annotations of 6,459 videos in the validation set of Kinetics-600.


  • 2020.8.25: Repository for AherNet and annotations of sampled validation video in Kinetics-600


Paper Introduction


With the knowledge of action moments (i.e., trimmed video clips that each contains an action instance), humans could routinely localize an action temporally in an untrimmed video. Nevertheless, most practical methods still require all training videos to be labeled with temporal annotations (action category and temporal boundary) and develop the models in a fully-supervised manner, despite expensive labeling efforts and inapplicable to new categories. In this paper, we introduce a new design of transfer learning type to learn action localization for a large set of action categories, but only on action moments from the categories of interest and temporal annotations of untrimmed videos from a small set of action classes. Specifically, we present Action Herald Networks (AherNet) that integrate such design into an one-stage action localization framework. Technically, a weight transfer function is uniquely devised to build the transformation between classification of action moments or foreground video segments and action localization in synthetic contextual moments or untrimmed videos. The context of each moment is learnt through the adversarial mechanism to differentiate the generated features from those of background in untrimmed videos. Extensive experiments are conducted on the learning both across the splits of ActivityNet v1.3 and from THUMOS14 to ActivityNet v1.3. Our AherNet demonstrates the superiority even comparing to most fully-supervised action localization methods. More remarkably, we train AherNet to localize actions from 600 categories on the leverage of action moments in Kinetics-600 and temporal annotations from 200 classes in ActivityNet v1.3.


TensorFlow version: 1.12.3 of GPU

Operation system in docker version: Ubuntu16.04

Python version: 3.6.7

GPU version: NVIDIA Tesla P40 (23GB)

Cuda and Cudnn version: CUDA 9.0 and cudnn 7.3.1

Training of AherNet

The training script is Once you have all the training features and information files, you could run


Testing of AherNet on Kinetics-600

The testing script is Once you have all the testing features and information files, you could run


You can evaluate several snapshots (models) during testing stage to find which model is the best one.

After the evaluation, you can get the results of ".json" file which will be evaluated by ./evaluation/ or ./evaluation/ for temporal action proposal and temporal action localization evaluation.


If you use these models in your research, please cite:

  title={Learning To Localize Actions from Moments},
  author={Fuchen Long, Ting Yao, Zhaofan Qiu, Xinmei Tian, Jiebo Luo and Tao Mei},


No description, website, or topics provided.



No releases published


No packages published