An implementation of baselines for multimodal utterance embedding, as described in https://www.aclweb.org/anthology/N19-1267.
Based on original SIF implementation) by Arora et al. (2016, 2017).
Requires Python 3.
Processed data for the MOSI and POM datasets used in the code can be obtained from here, and should be saved in the folder data/. Alternatively, you can get the raw data here.
configs/ contains JSON files that holds the hyperparameters of the model. To generate some config files, run python configs/make_configs.py. These will be saved in configs/multimodal_search.
Then, to run MMB2,
python simplesif.py configs/multimodal_search/config_0.json $DATASET
where $DATASET is mosi or pom.
For MMB1, set the --unimodal flag:
python simplesif.py configs/multimodal_search/config_0.json $DATASET --unimodal
Run python simplesif.py --help for more options`.
This code is released under the MIT License. See LICENSE for more details.