Skip to content
HACS: Human Action Clips and Segments Dataset
Python
Branch: master
Clone or download
Latest commit 79b1a27 Jul 17, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
HACS_v1.1.zip dataset in zip Dec 23, 2018
LICENSE Create LICENSE Jan 16, 2019
README.md Update README.md Jul 17, 2019
check_missing_videos.py minor Jan 28, 2019
dataset_stats.py download script Jan 7, 2019
download_videos.py comment cleanup Mar 1, 2019
teaser.png add teaser Jan 15, 2019

README.md

HACS-dataset

This project introduces a novel video dataset, named HACS (Human Action Clips and Segments). It consists of two kinds of manual annotations. HACS Clips contains 1.55M 2-second clip annotations; HACS Segments has complete action segments (from action start to end) on 50K videos. The large-scale dataset is effective for pretraining action recognition and localization models, and also serves as a new benchmark for temporal action localization. (*SLAC dataset is now part of HACS dataset.)

Project Website: http://hacs.csail.mit.edu/

Paper: https://arxiv.org/abs/1712.09374

Download Annotation Files

  1. Clone this repository:
git clone https://github.com/hangzhaomit/HACS-dataset.git
  1. Unzip annotation files:
unzip HACS_v1.1.zip
  1. Check dataset statistics:
python dataset_stats.py

   You should expect the following output:

====Parsing clips====
[training set]: 492753 videos, 1509497 clips
[validation set]: 5982 videos, 20249 clips
[testing set]: 5988 videos, 20296 clips
====Parsing segments====
[training set]: 37618 videos
[validation set]: 5982 videos
[testing set]: 5988 videos
Done.

Annotation File Format

  1. For HACS Clips, the annotation file is HACS_v1.1/HACS_clips_v1.1.csv. "label": 1/"label": -1 refers to positive/negative sample. The format looks like the following:
classname,youtube_id,subset,start,end,label
Archery,a2X2hz1G6i8,training,15.5,17.5,1
Archery,NUdji_CqvcY,training,77.5,79.5,-1
Archery,0O_qMHxBfXg,training,24.5,26.5,-1
...
  1. For HACS Segments, the annotation file is HACS_v1.1/HACS_segments_v1.1.json, with the same format as ActivityNet dataset:
{
  "database": {
    "--0edUL8zmA": {
        "annotations": [
            {"label": "Dodgeball", "segment": [5.40, 11.60]},
            {"label": "Dodgeball", "segment": [12.60, 88.16]},
        "subset": "training",
        "duration": "92.166667",
        "url": "https://www.youtube.com/watch?v=--0edUL8zmA"
    },
  ...
  },
}

Download Videos

  1. Install the following libraries:
  1. Run the following command to download videos:

python download_videos.py --root_dir ROOT_DIR [--dataset {all,segments}] [--shortside SHORTSIDE]

  • ROOT_DIR is the root path to save the downloaded videos; videos are saved in the following directory structure ROOT_DIR/CLASSNAME/v_ID.mp4;

  • You can either download all videos (default), or only HACS Segments videos with --dataset segments;

  • By default, we resize videos with short side of 256 for less disk usage, you can change with --shortside.

Request testing videos and missing videos: (NEW)

  • To access the full testing videos, please submit a request at https://goo.gl/forms/0STStcLndI32oke22. You will get links to them within 72 hours.

  • YouTube videos can dissapear over time, so you may find the videos you downloaded incomplete, we provide the following solution for you to have access to missing videos.

    (a) Run python check_missing_videos.py to generate text file missing.txt containing missing video IDs. You can also create your own in the following format {VIDEO_ID,CLASS_NAME}:

    WWowbBQB1lU,Ironing_clothes
    Kb08E4K8fg8,Pole_vault
    NKB_CEA5jNQ,Playing_squash
    ...
    

    (b) Submit a video request by agreeing to terms of use at: https://goo.gl/forms/0STStcLndI32oke22. You will get links to the missing videos within 72 hours.

    (c) NOTE: We limit up to 20,000 videos per request, so please send multiple requests if you have more missing videos.

    (d) Use the download script to download missing videos by running python download_videos.py --root_dir rest_of_your_videos --dataset missing --missing_url "http://sample.missing/urls.txt"

Reference

If you use find the dataset helpful, please cite:

@article{zhao2019hacs,
  title={HACS: Human Action Clips and Segments Dataset for Recognition and Temporal Localization},
  author={Zhao, Hang and Yan, Zhicheng and Torresani, Lorenzo and Torralba, Antonio},
  journal={arXiv preprint arXiv:1712.09374},
  year={2019}
}
You can’t perform that action at this time.