Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.

Pretrained models for Moments in Time Dataset

We release the pre-trained models trained on Moments in Time.

Download the Models

  • Clone the code from Github:
    git clone
    cd moments_models


  • RGB model in PyTorch (ResNet50 pretrained on ImageNet). Run the following script to download and run the test sample. The model is tested sucessfully in PyTorch 1.0 + python36.

We provide a 3D ResNet50 (inflated from 2D RGB model) trained on 16 frame inputs at 5 fps.

The model has been recently updated with 305 classes and the following performance on the MiT-V2 dataset:

Top-1 Top-5
28.4% 54.5%

The 3D model can be downloaded and run using a similar command:

    python --video_file path/to/video.mp4 --arch resnet3d50

If you use any of these files please cite our Moments paper (

We now include the Multi-label Moments (M-MiT) 3D Resnet50 Model, Broden dataset with action regions and loss implementations including wLSEP. If you use any of these files please cite our Multi Moments paper (

The multi-label model has been recently updated with 305 classes and the following performance on the M-MiT-V2 dataset:

Top-1 Top-5 micro mAP macro mAP
59.4% 81.7% 62.4 39.4

The 3D M-MiT model can be downloaded and run using the following command:

    python --video_file path/to/video.mp4 --arch resnet3d50 --multi

We uploaded a python file with our pytorch implementations of the different loss functions used in our Multi Moments paper (

In order to NetDissect Moments models, download the Broden datasets with action regions:

  • Dynamic Image model in Caffe: use the testing script.

  • TRN models is at this repo. To use the TRN model trained on Moments:

Clone the TRN repo and Download the pretrained TRN model

git clone --recursive
cd TRN-pytorch/pretrain
cd ../sample_data

Test the pretrained model on the sample video (Bolei is juggling ;-]!)


python --arch InceptionV3 --dataset moments \
    --weight pretrain/TRN_moments_RGB_InceptionV3_TRNmultiscale_segment8_best.pth.tar \
    --frame_folder sample_data/bolei_juggling

RESULT ON sample_data/bolei_juggling
0.982 -> juggling
0.003 -> flipping
0.003 -> spinning


Mathew Monfort, Alex Andonian, Bolei Zhou, Kandan Ramakrishnan, Sarah Adel Bargal, Tom Yan, Lisa Brown, Quanfu Fan, Dan Gutfruend, Carl Vondrick, Aude Oliva. Moments in Time Dataset: one million videos for event understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. pdf, bib

Mathew Monfort, Kandan Ramakrishnan, Alex Andonian, Barry A McNamara, Alex Lascelles, Bowen Pan, Quanfu Fan, Dan Gutfreund, Rogerio Feris, Aude Oliva. Multi-Moments in Time: Learning and Interpreting Models for Multi-Action Video Understanding. arxiv preprint arXiv:1911.00232, 2019. pdf, bib


The project is supported by MIT-IBM Watson AI Lab, IBM Research, the SystemsThatLearn@CSAIL / Ignite Grant and the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/ Interior Business Center (DOI/IBC) contract number D17PC00341.


The pretrained models trained on Moments in Time Dataset







No releases published


No packages published