Skip to content
Implementation of NAACL'19 Strong and Simple Baselines for Multimodal Utterance Embeddings
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
configs
mosi
pom add pom data Jun 4, 2019
scripts
.gitignore
LICENSE
README.md
losses.py
models.py
sentiment_model.py
sif.py
sif2.py
sif_functions.py
simplesif.py
utils.py

README.md

An implementation of baselines for multimodal utterance embedding, as described in https://www.aclweb.org/anthology/N19-1267.

Based on original SIF implementation) by Arora et al. (2016, 2017).

Requires Python 3.

Data

Processed data for the MOSI and POM datasets used in the code can be obtained from here, and should be saved in the folder data/. Alternatively, you can get the raw data here.

Instructions

configs/ contains JSON files that holds the hyperparameters of the model. To generate some config files, run python configs/make_configs.py. These will be saved in configs/multimodal_search.

Then, to run MMB2,

python simplesif.py configs/multimodal_search/config_0.json $DATASET

where $DATASET is mosi or pom.

For MMB1, set the --unimodal flag:

python simplesif.py configs/multimodal_search/config_0.json $DATASET --unimodal

Run python simplesif.py --help for more options`.

License

This code is released under the MIT License. See LICENSE for more details.

You can’t perform that action at this time.