Epic-kitchen

Github

Official Pretrained Model

• Path on Srv: s1_md0/v-fanxu/junyidu/github/action-models

TSM Backbone Code

• Borrows data pipeline

Data

• Raw Path: /s1_md0/v-fanxu/Extraction/youcook2/videos
• Extracted Img Path: /s1_md0/v-fanxu/Extraction/youcook2/imgs
• List Path: /s1_md0/v-fanxu/Extraction/youcook2/manifest.txt
• Epic Kitchen Verb List: /s1_md0/v-fanxu/junyidu/github/action-models/EPIC_verb_classes.csv

Inference Result

• Bulk: /s1_md0/v-fanxu/junyidu/github/action-models/predict_verb_bulk.txt

Setting

• Model download link
• Use: TSN, 8seg, Res50, RGB
• Path on Srv: /s1_md0/v-fanxu/junyidu/download/TSN_arch=resnet50_modality=RGB_segments=8-3ecf904f.pth.tar

Preprocessing

Decode raw video

• In dataset's root folder
• python vid2img.py videos imgs

Generate manifest for video (bulk or clip)

• In the corresponding imgs folder
• bash /s1_md0/v-fanxu/junyidu/github/action-models/youcook2/generate_manifest.sh

Fetch framerate for each video

• In videos folder
• bash /s1_md0/v-fanxu/junyidu/github/action-models/youcook2/fetch_framerate.sh

Other Action Dataset

• Kinetics
  • Actions: 400
  • Min Clip for an Action: ~400
  • Tot Clips: 306k
  • Length: 10s
  • Resolution: Variable (Usually normalized to 340px)
• UCF101
  • Actions: 101
  • Min Clip for an Action: ~100
  • Tot Clips: 13k
  • Mean Length: 7.2s
  • Resolution: 320x240

Paper

• TSM: Temporal Shift Module for Efficient Video Understanding
  • Dataset

Experiments

• Official Demo(Rand): bash rundemo.sh
• Extract: bash runextract.sh