This repo contains code for our NeurIPS Datasets and Benchmarks 2022 paper:
EgoTaskQA: Understanding Human Tasks in Egocentric Videos
Baoxiong Jia, Ting Lei, Song-Chun Zhu, Siyuan Huang
For data download, please check our website for instructions and details.
We provide all environment configurations in requirements.txt
. In our experiments, we used NVIDIA CUDA 11.3 on Ubuntu 20.04
and need this additional step for version control on pytorch:
$ conda create -n egotaskqa python=3.8
$ pip install -r requirements.txt
$ pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu113
Similar CUDA version should also be acceptable with corresponding version control for torch
and torchvision
.
We refer the authors to Generation and Experiment for details on quetsion-answer
generation, balancing, data split, and baseline experiments. For these two functionalities, please checkout the corresponding
sub-directory for code and instructions.
If you find our paper and/or code helpful, please consider citing:
@inproceedings{jia2022egotaskqa,
title = {EgoTaskQA: Understanding Human Tasks in Egocentric Videos},
author = {Jia, Baoxiong and Lei, Ting and Zhu, Song-Chun and Huang, Siyuan},
booktitle = {The 36th Conference on Neural Information Processing Systems (NeurIPS 2022) Track on Datasets and Benchmarks},
year = {2022}
}
We thank all colleagues from VCLA and BIGAI for fruitful discussions. We would also like to thank the anonymous reviewers for their constructive feedback.