Skip to content
/ RAP Public

A Richly Annotated Pedestrian Dataset for Person Retrieval in Real Surveillance Scenarios

Notifications You must be signed in to change notification settings

dangweili/RAP

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

A Richly Annotated Pedestrian Dataset for Person Retrieval in Real Surveillance Scenarios

Preparation

Prerequisite: Caffe, Python 2.7 and Matlab.

  1. Install Caffe, Python 2.7 and Maltab.

  2. Download and prepare the dataset as follow:

    RAP Links

    ./data/RAP_annotation/RAP_annotation.mat
    ./data/RAP_dataset/*.png
    

    if you want to use body parts for attribute recognition, please exec this command to generate body part images.

    cd data
    matlab -nodisplay -r 'rap2_part_extraction'
    
  3. Download the imagenet pretrained models. ImageNet pretrained models which is used in finetuning Baiduyun or GoogleDrive.

Pedestrian Attribute Recognition

  1. SVM-based models

    a. feature extraction, including ELF and pretrained CNN features:

    cd features/ELF-v2.0-Descriptor
    matlab -nodisplay -r 'Feature_Extraction_elf'
    matlab -nodisplay -r 'Feature_PCA_elf'
    

    download pretrained CNN models and run the follow commands to extract cnn features.

    cd features/CNN-v1.0-Descriptor
    matlab -nodisplay -r 'imagenet_feature_extraction_caffenet_single'
    matlab -nodisplay -r 'imagenet_feature_extraction_resnet_single'
    matlab -nodisplay -r 'imagenet_feature_extraction_caffenet_parts' [optional]
    matlab -nodisplay -r 'imagenet_feature_extraction_resnet_parts' [optional]
    
    

    compile the liblinear

    cd person-attribute/utils/liblinear-master/matlab
    matlab -nodisplay -r 'make'
    

    b. use svm to train attribute classifiers training: mixture clean and occlusion data, test: mixture clean and occlusion data.

    cd person-attribute/baseline-svm
    matlab -nodisplay -r 'v1_fullbody_analysis_mm' [one types of features]
    matlab -nodisplay -r 'v1_fullbody_analysis_mm_test' [all types of features]
    matlab -nodisplay -r 'v1_fullbody_analysis_mm_statistic' [summary the results]
    

    c. analysis of viewpoint training: maxture clean and occlusion data, test: mixture clean and occlusion data.

    cd person-attribute/baseline-svm
    matlab -nodisplay -r 'v2_fullbody_analysis_mm_viewpoint'
    matlab -nodisplay -r 'v2_fullbody_analysis_mm_viewpoint_statistic'
    
    

    training: clean, test: clean

    cd person-attribute/baseline-svm
    matlab -nodisplay -r 'v2_fullbody_analysis_cc'
    matlab -nodisplay -r 'v2_fullbody_analysis_cc_viewpoint'
    
    

    d. analysis of occlusion occlusion positions and types: training: clean, test: occlusion

    cd person-attribute/baseline-svm
    matlab -nodisplay -r 'v3_fullbody_analysis_co_test'
    matlab -nodisplay -r 'v3_fullbody_analysis_co_test_personvspersons'
    
    

    e. analysis of body parts

    cd person-attribute/baseline-svm
    matlab -nodisplay -r 'v4_parts_analysis_cc'
    matlab -nodisplay -r 'v4_parts_analysis_cc_test'
    
  2. CNN-based models

    a. prepare the data splits.

    cd person-attribute/static
    matlab -nodisplay -r 'prepare_data'
    matlab -nodisplay -r 'prepare_data_parts'
    matlab -nodisplay -r 'prepare_data_binary'
    

    b. train the deep attribute classifiers with deepmar based on CaffeNet. For deepmar, acn, and their single attribute versions, the operators are in similar format.

    cd person-attribute/baseline-deepmar
    sh train_caffenet.sh
    sh test_caffenet.sh
    

Attribute-based Person Retrieval

The product of multiple attributes' prediction probability are used for person retrieval.

  1. generate the attributes for attribute-based person retrieval.
    cd person-attribute/baseline-search
    matlab -nodisplay -r 'generate_multiquery_index'
    matlab -nodisplay -r 'generate_query_names'
    python generate_query_names.py
    
  2. generate the attribute score based on the trained models
    cd person-attribute/baseline-search
    matlab -nodisplay -r 'generate_svm_score'
    python generate_cnn_score.py
    python generate_cnn_score_binary.py
    
  3. evaluate the attribute-based person retrieval
    cd person-attribute/baseline-search
    matlab -nodisplay -r 'evaluate_multiquery_attributes'
    

Person Re-identification

  1. hand-crafted features/pretrained cnn features with L2/XQDA/KISSME

    a. feature extraction, incluidng ELF, LOMO, GOG (window), JSTL

    cd features/ReID_GOG_v1.01
    matlab -nodisplay -r 'Feature_Extraction_gog'
    cd features/CNN-v1.0-Descriptor
    matlab -nodisplay -r 'jstl_feature_extraction_single'
    cd features/LOMO_XQDA/code
    matlab -nodisplay -r 'Feature_Extraction_lomo'
    

    b. feature evaluation

    cd person-reid/evaluation
    matlab -nodisplay -r 'rap2_evaluation_features'
    
  2. end-to-end feature learning

    a. generate the data split file for training.

    cd person-reid/static
    matlab -nodisplay -r 'generate_att_trainval_test'
    matlab -nodisplay -r 'generate_ide_trainval_test'
    matlab -nodisplay -r 'generate_ide_att_trainval_test'
    matlab -nodisplay -r 'generate_ide_att_trainvaltest' 
    

    b. training with only ID classification loss, such as CaffeNet, ResNet50/ResNet101/ResNet152, DenseNet121, MSCAN.

    cd person-reid/baseline-IDE
    sh train_caffenet.sh
    

    c. training with only attribute classification loss

    cd person-reid/baseline-att
    sh train_caffenet.sh
    sh test_caffenet.sh [optional for attribute classification]
    

    d. training with attribute and ID classification losses

    cd person-reid/baseline-IDE-att
    sh train_caffenet.sh
    sh test_caffenet.sh [optional for attribute classification]
    

    e. deep feature extraction and evaluation.

    cd person-reid/evaluation
    sh rap2_feature_extraction_resnet.sh [one model per time]
    matlab -nodisplay -r 'rap2_test' [one model per time]
    
  3. cross-day person retrieval

    a. person retrieval in the same day as query or the different day as query.

    cd person-reid/evaluation
    matlab -nodisplay -r 'rap2_test_control_single_cross'
    

    b. person retrieval from different day as query. The appearance would be partially different for the same person across different days.

    cd person-reid/evaluation
    matlab -nodisplay -r 'rap2_test_control_single_cross_quantively'
    
  4. identity-level attribute vs. instance-level attributes for person re-identification.

    a. generate identity-level attributes from instance-level attributes for training.

    cd person-reid/static
    matlab -nodisplay -r 'generate_ide_att_trainval_test_control'
    

    b. train: instance-level attributes. The default setup.

    cd person-reid/baseline-IDE-att
    sh train_caffenet.sh
    

    c. train: identity-level attributes.

    cd person-reid/baseline-IDE-att-control
    sh train_caffenet.sh
    

    d. feature extraction and evaluation.

    cd person-reid/evaluation
    sh rap2_reid_extraction_resnet_control_identity_instance.sh 
    matlab -nodisplay -r 'rap2_test_control_identity_identity'
    matlab -nodisplay -r 'rap2_test_control_identity_instance'
    matlab -nodisplay -r 'rap2_test_control_instance_identity'
    

Citation

Please cite this paper in your publications if it helps your research:
@article{li2018richly,
    title={A Richly Annotated Pedestrian Dataset for Person Retrieval in Real Surveillance Scenarios},
    author={Li, Dangwei and Zhang, Zhang and Chen, Xiaotang and Huang, Kaiqi},
    journal={IEEE Transactions on Image Processing},
    volume={28},
    number={4},
    pages={1575--1590},
    year={2019},
    publisher={IEEE}
}

About

A Richly Annotated Pedestrian Dataset for Person Retrieval in Real Surveillance Scenarios

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published