Person Retrieval in Surveillance Video using Height, Color and Gender

This code contains a Tensorflow implementation for our paper - Person Retrieval in Surveillance Video using Height, Color and Gender. If you find this code useful in your research, please consider citing:

  title={Person retrieval in surveillance video using height, color and gender},
  author={Galiyawala, Hiren and Shah, Kenil and Gajjar, Vandit and Raval, Mehul S},
  booktitle={2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)},
  pages={1--6},
  year={2018},
  organization={IEEE}
}

This code was initially tested on an Ubuntu 16.04 system using Keras 2.0.8 with Tensorflow 1.12 backend.

The paper proposes a deep learning-based linear filtering approach for person retrieval using height, cloth color, and gender.

Installation

1. Clone this repository.

git clone https://github.com/Vanditg/Description_Based_Person_Identification.git

2. In the repository, execute pip install -r requirements.txt to install all the necessary libraries.

3. Three deep learning models are used inorder to filter out the desired person.

   1. Mask_RCNN:- Used to determine the coordinates of the person and fetch the pixelwise segmentation
   2. gender_model:- Used to determine gender of the person
   3. color_model:- Used to determine torso color of the person

4. Download the pretrained weights.

   1. Mask_RCNN pretrained weights and save it in root directory
   2. gender_model pretrained weights and save it in /modalities/gender/
   3. color_model pretrained weight and save it in /modalities/torso_color/

Usage

To use the GUI version

python app.py

Browse the video from your local disk and enter the soft-biometric features required to locate the desired person.

Important Instructions

1. Currently, Height channel is not used as a filter due to it's dependance on camera parameters.

2. Output frames with bounding box of the desired person are stored in /output/. folder.

Example Output Frames

Retrieving a person using torso color blue and gender female.

Many thanks to Matterport for the Mask R-CNN code.