Query-guided End-to-End-Person-Search
Paper Link: https://arxiv.org/abs/1905.01203
If you are referring this work please cite:
@inproceedings{munjal2019cvpr, author = {Munjal, Bharti and Amin, Sikandar and Tombari, Federico and Galasso, Fabio}, title = {Query-guided End-to-End Person Search}, booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, year = {2019} }
Abstract
Person search has recently gained attention as the novel task of finding a person, provided as a cropped sample, from a gallery of non-cropped images, whereby several other people are also visible. We believe that i. person detection and re-identification should be pursued in a joint optimization framework and that ii. the person search should leverage the query image extensively (e.g. emphasizing unique query patterns). However, so far, no prior art realizes this. We introduce a novel query-guided end-to-end person search network (QEEPS) to address both aspects. We leverage a most recent joint detector and re-identification work, OIM. We extend this with i. a query-guided Siamese squeeze-and-excitation network (QSSE-Net) that uses global context from both the query and gallery images, ii. a query-guided region proposal network (QRPN) to produce query-relevant proposals, and iii. a query-guided similarity subnetwork (QSimNet), to learn a query-guided reidentification score. QEEPS is the first end-to-end queryguided detection and re-id network. On both the most recent CUHK-SYSU and PRW datasets, we outperform the previous state-of-the-art by a large margin.
Results on CUHK-SYSU [3]
| Method @Gallery100 | mAP | top-1 |
|---|---|---|
| Mask-G[2] | 83.0 | 83.7 |
| QEEPS | 88.9 | 89.1 |
Results on PRW [4]
| Method | mAP | top-1 |
|---|---|---|
| Mask-G[2] | 32.6 | 72.1 |
| QEEPS | 37.1 | 76.7 |
PRW-mini
The task of query dependent person search encompasses complexity of O(MN) during benchmarking (M queries, N gallery images). PRW dataset has 2,057 probes and 6,112 gallery images. This means, conditioning on the query requires jointly processing each [query-gallery] pair and the exhaustive evaluation of the product space, i.e. 2, 057 × 6, 112. We introduce the PRW-mini to reduce the evaluation time while maintaining the difficulty. PRW-mini tests 30 query images against the whole gallery. The 30 probes selected for PRW-mini are given in prwmini_query_info.txt
Results on PRW-mini [1]
| Method | mAP | top-1 |
|---|---|---|
| Mask-G[2] | 33.1 | 70.0 |
| QEEPS | 39.1 | 80.0 |
PRW Evaluation
We evaluate on PRW with the same evaluation script (prw_test.py) as adopted by Mask-G [2]. This evaluation is motivated from CUHK-SYSU https://github.com/ShuangLI59/person_search/tree/master/lib/datasets. Each probe image is compared to all gallery images except the probe image itself. We also provide here the script for PRW-mini evaluation (prwmini_test.py). If you are using this evaluation for PRW please cite:
[1] B. Munjal, S. Amin, F. Tombari, F. Galasso. Query-guided End-to-End Person Search. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019
[2] D. Chen, S. Zhang, W. Ouyang, J. Yang, and Y. Tai. Person search via a mask-guided two-stream cnn model. In The European Conference on Computer Vision (ECCV), 2018
[3] T. Xiao, S. Li, B. Wang, L. Lin, and X. Wang. Joint detection and identification feature learning for person search. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[4] L. Zheng, H. Zhang, S. Sun, M. Chandraker, Y. Yang, and Q. Tian. Person re-identification in the wild. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.