CNN Image Retrieval in PyTorch: Training and evaluating CNNs for Image Retrieval in PyTorch

The code is based on the https://github.com/filipradenovic/cnnimageretrieval-pytorch .

We added our attacking method ODFA (https://arxiv.org/abs/1809.02681). The performance drops from 88.2% to 2.24% on Oxford.

The results attacking other datasets can be found at our main repo https://github.com/layumi/U_turn.

Regular Testing

We provide the pretrained networks trained using the same parameters as in our TPAMI 2018 paper, with precomputed whitening. To evaluate them run:

python3 -m cirtorch.examples.test --gpu-id '2' --network-path 'retrievalSfM120k-resnet101-gem' --datasets 'oxford5k,paris6k,roxford5k,rparis6k' --whitening 'retrieval-SfM-120k' --multiscale '[1]'

The table below shows the performance comparison of networks trained with this framework and the networks used in the paper which were trained with our CNN Image Retrieval in MatConvNet:

Model	Oxford	Paris	ROxf (M)	RPar (M)	ROxf (H)	RPar (H)
ResNet101-GeM (PyTorch) Multiple Scale	88.2	92.5	65.4	76.7	40.1	55.2
ResNet101-GeM (PyTorch) Single Scale	86.24	90.66	-	-	-	-
VGG16-GeM (PyTorch) Single Scale	85.24	86.28	-	-	-	-

Testing Adversarial Query

We add one function cirtorch.network.extract_vectors_aq and one py file test_aq.py.

single-scale

python3 -m cirtorch.examples.test_aq --gpu-id '2' --network-path 'retrievalSfM120k-resnet101-gem' --datasets 'oxford5k,paris6k' --whitening 'retrieval-SfM-120k' --multiscale '[1]'

multi-scale

python3 -m cirtorch.examples.test_aq --gpu-id '2' --network-path 'retrievalSfM120k-resnet101-gem' --datasets 'oxford5k,paris6k' --whitening 'retrieval-SfM-120k' --multiscale '[1, 1/2**(1/2), 1/2]'

PIRE / TMA with Adam Optimizer 100 iteration

python3 -m cirtorch.examples.test_pire --gpu-id '2' --network-path 'retrievalSfM120k-resnet101-gem' --datasets 'oxford5k,paris6k' --whitening 'retrieval-SfM-120k' --multiscale '[1]' --iter 100

PIRE / TMA with Adam Optimizer 100 iteration (MS)

python3 -m cirtorch.examples.test_pire --gpu-id '0' --network-path 'retrievalSfM120k-resnet101-gem' --datasets 'oxford5k,paris6k' --whitening 'retrieval-SfM-120k' --multiscale '[1, 1/2**(1/2), 1/2]' --iter 100

You will obtain result around:

Model	Oxford	Paris	ROxf (M)	RPar (M)	ROxf (H)	RPar (H)
ResNet101-GeM (PyTorch) Multiple Scale	2.24 (1.82)	4.78 (3.64)	-	-	-	-
ResNet101-GeM (PyTorch) Single Scale	0.80	3.00	-	-	-	-
VGG16-GeM (PyTorch) Single Scale	0.64	1.85	-	-	-	-

Related publications

Training (fine-tuning) convolutional neural networks

@article{RTC18,
 title = {Fine-tuning {CNN} Image Retrieval with No Human Annotation},
 author = {Radenovi{\'c}, F. and Tolias, G. and Chum, O.}
 journal = {TPAMI},
 year = {2018}
}

@inproceedings{RTC16,
 title = {{CNN} Image Retrieval Learns from {BoW}: Unsupervised Fine-Tuning with Hard Examples},
 author = {Radenovi{\'c}, F. and Tolias, G. and Chum, O.},
 booktitle = {ECCV},
 year = {2016}
}

Revisited benchmarks for Oxford and Paris ('roxford5k' and 'rparis6k')

@inproceedings{RITAC18,
 author = {Radenovi{\'c}, F. and Iscen, A. and Tolias, G. and Avrithis, Y. and Chum, O.},
 title = {Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking},
 booktitle = {CVPR},
 year = {2018}
}

Versions

master (development)

• Migrated code to PyTorch 1.0.0, removed Variable, added torch.no_grad for more speed and less memory at evaluation
• Added rigid grid regional pooling that can be combined with any global pooling method (R-MAC, R-SPoC, R-GeM)
• Added PowerLaw normalization layer
• Added multi-scale testing with any given set of scales, in example test script
• Fix related to precision errors of covariance matrix estimation during whitening learning
• Fixed minor bugs

v1.0 (09 Jul 2018)

• First public version
• Compatible with PyTorch 0.3.0