Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

A Graph-Based Framework to Bridge Movies and Synopses

This repo holds the code and models for the movie-synopsis association framework presented on ICCV2019:

Title: A Graph-Based Framework to Bridge Movies and Synopses, ICCV19, Seoul, South Korea.

Authors: Yu Xiong, Qingqiu Huang, Lingfeng Guo, Hang Zhou, Bolei Zhou, Dahua Lin.

Useful Links: [Paper] [Arxiv] [Project Page] [Dataset Website]


  • [10/2019] We are still cleaning and expanding this dataset.


  • Update to a newer version of PyTorch
  • Data preprocessing codes
  • Release the whole dataset

Getting Started

The following instructions will get the project set up on your local machine.

I. Prerequisites

All the codes are tested on the following environments:

  • Linux (Ubuntu 16.04)
  • Python >= 3.6
  • PyTorch >= 0.4.1
  • CUDA 9.2

II. Install Python Dependencies

Pip install the following dependencies:

mmcv >= 0.2.0

III. Install Gurobi

Gurobi is a powerful optimization solver. We use gurobi for solving the graph matching problem. Follow the instructions below to install gurobi with Free Academic License. For other license or detailed official instructions, please visit here.

Download Gurobi

  1. Login or register an account using academic email address at here.
  2. Enter this page and download gurobi optimizer (linux, version>=8.1.0)
  3. Issue an academic lisence for your local machine.

Install Gurobipy

Take Gurobi-8.1.0 for example, enter the folder that store the gurobi source file, unzip it

tar -xvzf gurobi8.1.0_linux64.tar.gz

And then install gurobipy by

cd gurobi810/linux64
python install


Q1. Python ImportError: libgurobi{version}.so: cannot open shared object file: No such file or directory

A1. export gurobi lib to your LD_LIBIRARY_PATH by:

export GUROBI_HOME="/path/of/gurobi810(or_other_version)/linux64" # replace the path

IV. Fetch the code

git clone

V. Prepare the data


Download our MSA (Movie Synopsis Association) dataset from Here. Note that we only provide extrated features at this moment due to legal issues. More information of our dataset will be updated at the Dataset Website.

Unzip the dataset by


Create soft link

cd /path/of/MovieSynopsisAssociation
ln -s /path/of/your/msa/dataset/directory data

VI. Training

Train element embedding networks

First we train embedding networks for appearance feature and action feature respectively by

python tools/ config/ --work_dir work_dir/WORK_DIR_NAME --validate

Other Information


author = {Xiong, Yu and Huang, Qingqiu and Guo, Lingfeng and Zhou, Hang and Zhou, Bolei and Lin, Dahua},
title = {A Graph-Based Framework to Bridge Movies and Synopses},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}

Related Projects

This project is part of the Movie Understanding Project. The other related movie projects are:

Actor Recognition

Trailer Analytics

  • Trailer to Movie: From Trailers to Storylines: An Efficient Way to Learn from Movies


Xiong Yu:


Code for "A Graph-Based Framework to Bridge Movies and Synopses", ICCV2019



No releases published


No packages published