Contrastive Masked Autoencoders for Self-Supervised Video Hashing

This repository is the official PyTorch implementation of our AAAI 2023 paper Contrastive Masked Autoencoders for Self-Supervised Video Hashing.

Catalogue

• 1. Getting Started
• 2. Train
• 3. Test
• 4. Trained Models
• 5. Results
• 6. Citation

Getting Started

1. Clone this repository:

git clone https://github.com/haungmozhi9527/ConMH.git
cd ConMH

2. Create a conda environment and install the dependencies:

conda create -n conmh python=3.6
conda activate conmh
conda install pytorch==1.6.0 cudatoolkit=10.1 -c pytorch -c conda-forge
pip install -r requirements.txt

3. Download Datasets: VGG features of FCVID and YFCC are kindly uploaded by the authors of SSVH. ResNet50 features of ActivityNet are kindly provided by the authors of BTH. You can download them from Baiduyun disk.

Dataset	Link
FCVID	Baidu disk
ActivityNet	Baidu disk
YFCC	Baidu disk

4. Set data_root and home_root in config files (e.g., ./configs/conmh_fcv.py).

Train

To train ConMH on FCVID:

python train.py --gpu 0 --config configs/conmh_fcv.py

To train ConMH on ActivityNet:

python train.py --gpu 0 --config configs/conmh_act.py

To train ConMH on YFCC:

python train.py --gpu 0 --config configs/conmh_yfcc.py

Test

To test ConMH on FCVID:

python eval.py --gpu 0 --config configs/conmh_fcv.py

To test ConMH on ActivityNet:

python eval.py --gpu 0 --config configs/conmh_act.py

To test ConMH on YFCC:

python eval.py --gpu 0 --config configs/conmh_yfcc.py

Trained Models

We provide trained ConMH checkpoints. You can download them from Baiduyun disk.

Dataset	16 bits	32 bits	64 bits
FCVID	Baidu disk	Baidu disk	Baidu disk
ActivityNet	Baidu disk	Baidu disk	Baidu disk
YFCC	Baidu disk	Baidu disk	Baidu disk

Results

Quantitative Results

For this repository, the expected performance is:

Dataset	Bits	mAP@5	mAP@20	mAP@40	mAP@60	mAP@80	mAP@100
FCVID	16	0.350	0.252	0.216	0.196	0.181	0.169
FCVID	32	0.476	0.332	0.287	0.263	0.245	0.230
FCVID	64	0.524	0.373	0.326	0.301	0.283	0.267
ActivityNet	16	0.156	0.081	0.050	0.036	0.029	0.024
ActivityNet	32	0.229	0.124	0.075	0.054	0.042	0.035
ActivityNet	64	0.267	0.150	0.092	0.066	0.051	0.042
YFCC	16	0.225	0.146	0.122	0.113	0.108	0.104
YFCC	32	0.341	0.182	0.148	0.135	0.128	0.123
YFCC	64	0.368	0.194	0.158	0.143	0.135	0.130

Citation

If you find this repository useful, please consider citing our work:

@inproceedings{wang2023contrastive,
  title={Contrastive Masked Autoencoders for Self-Supervised Video Hashing},
  author={Wang, Yuting and Wang, Jinpeng and Chen, Bin and Zeng, Ziyun and Xia, Shu-Tao},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={37},
  number={3},
  pages={2733--2741},
  year={2023}
}