The paper is available arxiv
Please follow the instructions here to make the environment and install the libraries.
Use following command to train the model
python main.py --save_dir ./model --lr 2e-5 --batch_size 1024 --max_num_epoch 50 --dim_embed 256 \
--train_path_face <path_to_train_face_features> \
--train_path_voice <path_to_train_voice_features> \
--test_path_face <path_to_test_face_features> \
--test_path_voice <path_to_test_voice_features>
Use following command to test the trained model
python test.py --ckpt <path to checkpoint.pth.tar> --dim_embed 256
Download model weights from here
The codebase is inspired from the FOP repository. We thank them for releasing their valuable codebase.