Skip to content

Ego4d dataset repository. Download the dataset, visualize, extract features & example usage of the dataset

License

Notifications You must be signed in to change notification settings

facebookresearch/Ego4d

Repository files navigation

Important

EGO-EXO4D DATASET ANNOUNCEMENT: Ego-Exo4D V2 is now available to the public. V2 contains 1286.30 video hours (221.26 ego-hours) across 5035 takes with more annotations Please refer to the changelog for details on what has changed.

EGO4D UPDATE: V2.1 has been released due to the addition of the Goal-Step annotations and accompanying "grouped videos". Please refer to the documentation for more information.

Ego4D & Ego-Exo4D

Ego-Exo4D is a large-scale multi-modal multi-view video dataset (including 3D) and benchmark challenge. The dataset consists of time-synchronized videos of participants recorded with at least one first-person (egocentric Aria glasses) and third-person (exocentric GoPro cameras) perspective cameras.

Ego4D is the world's largest egocentric (first person) video ML dataset and benchmark suite, including over 3700 hours of annotated first-person video data.

Getting Started

Summary

The Ego4d repository (ego4d python module) includes:

Please see Structure of the Repository below for details.

Setup

Please follow the below instructions to setup the downloader CLI for Ego4d and to install the ego4d python module.

Option 1: From the PyPi package

Tip

Please ensure you have a conda or pyenv environment created & activated. If you're unsure on how to do so, you can follow Option 2: Step 1.

pip install ego4d --upgrade

NOTE: Please ensure you are on at least Python 3.10

Option 2: Clone/Download the Code

Ensure you have cloned or downloaded the code to your local disk. All instructions assume you are the root of the repository.

Step 1: Create or Use an Environment

Create a conda environment to enable pip installation:

conda create -n ego4d python=3.11 -y
conda activate ego4d

If you are using an existing conda (or pyenv) environment: please ensure you have installed at least Python 3.10.

Step 2:

pip install .  # run from the root of Ego4d

Now you should be able to import ego4d:

python3 -c 'import ego4d; print(ego4d)'

You can check that the ego4d module links to the correct file on your file system from the output of the above command.

Structure of the Repository

The repository contains multiple directories covering a specific theme. Each theme contains an associated README.md file, please refer to them.

All python code is located in the ego4d and associated subdirectories. The goal for each subdirectory is to cover one specific theme.

  • ego4d: the ego4d python module exists
    • cli: The Ego4D CLI for downloading the dataset
    • features: Feature extraction across the dataset
    • research: Everything related to research and usage of the dataset (dataloaders, etc).
      • research/clep: Contrastive Language Ego-centric video Pre-training
  • viz: visualization engine

Visualization and Demo

License

Ego4D is released under the MIT License.