Introduction

This is the source code of Efficient Fine-Grained Visual-Text Search Using Adversarially-Learned Hash Codes, IEEE ICME 2021.

This project is modified from the SAEM repo

Requirements

• python 3.6
• pytorch 0.4.1+

Download data

For the roi feature, we use the precomputed image features provided by SCAN. Please download data.zip from SCAN.

For other data like rephrased sentences or adversarial samples as well as concept data which can be download from here.

Bert model

We use the bert code from BERT-pytorch. Please following here to convert the Google bert model to a PyTorch save file.

Adversarial & Rephrase data generation

If you need to generate more adversarial sampels, follow the repo below:

https://github.com/ExplorerFreda/VSE-C

The code of generating rephrase data is in the generate_rephrase folder.

Training

python train_screen.py  # train screen model
python train_reranking.py  # train reranking model

Test

CUDA_VISIBLE_DEVICES=1 python test.py --model_name=screen_model --final_dims=256 --need_concept_label=0 --need_rephrase_data=0 --adversary_num=0 --resume=$CHECKPOINT --remark=$YOUR_REMARK
CUDA_VISIBLE_DEVICES=1 python test.py --model_name=rerank_model --final_dims=2048 --need_concept_label=0 --need_rephrase_data=0 --adversary_num=0 --resume=$CHECKPOINT --remark=$YOUR_REMARK

Citation

If this code is useful for you, please cite our paper:

TBD