A PyTorch Implementation of Cross-Modal Interaction Networks for Query-Based Moment Retrieval in Videos

The paper is available here.

The pretrained word vectors can be downloaded here.

The original ActivityNet dataset can be downloaded here.

• Train:

python main.py
--word2vec-path
XXX/glove_model.bin
--dataset
ActivityNet
--feature-path
XXX/activity-c3d
--train-data
data/activity/train_data_gcn.json
--val-data
data/activity/val_data_gcn.json
--test-data
data/activity/test_data_gcn.json
--max-num-epochs
10
--dropout
0.2
--warmup-updates
300
--warmup-init-lr
1e-06
--lr
8e-4
--num-heads
4
--num-gcn-layers
2
--num-attn-layers
2
--weight-decay
1e-7
--train
--model-saved-path
models_activity

The original TACoS dataset can be downloaded here.

• The features are extracted through pre-trained C3D networks.
• Train:

python main.py
--word2vec-path
XXX/glove_model.bin
--dataset
TACOS
--feature-path
XXX
--train-data
data/tacos/TACOS_train_gcn.json
--val-data
data/tacos/TACOS_val_gcn.json
--test-data
data/tacos/TACOS_test_gcn.json
--max-num-epochs
40
--dropout
0.2
--warmup-updates
300
--warmup-init-lr
1e-07
--lr
4e-4
--num-heads
4
--num-gcn-layers
2
--num-attn-layers
2
--weight-decay
1e-8
--train
--model-saved-path
models_tacos
--batch-size
64