KE-RCNN

Official implementation of KE-RCNN for part-level attribute parsing. It based on mmdetection.

Installation

• pytorch 1.10.0
• python 3.7.0
• mmdet 2.25.1
• fashionpeida-API
• einops

Dataset

You need to download the datasets and annotations follwing this repo's formate

• FashionPedia
• knowledge_matrix

Make sure to put the files as the following structure:

  ├─data
  │  ├─fashionpedia
  │  │  ├─train
  │  │  ├─test
  │  │  │─instances_attribute_train2020.json
  │  │  │─instances_attribute_val2020.json
  |  |  |─train_norm_attr_knowledge_matrix.npy
  |
  ├─work_dirs
  |  ├─ke_rcnn_r50_fpn_fashion_1x
  |  |  ├─epoch32.pth

Results and Models

FashionPedia

Backbone	LR	AP_iou+f1	AP_mask_iou+f1	DOWNLOAD
R-50	1x	39.6	36.4	model
R-101	1x	39.9	36.6	model
HRNet-w18	1x	38.0	35.3	model
Swin-tiny	1x	43.7	40.5	model

• This is a reimplementation. Thus, the numbers are slightly different from our original paper.

Evaluation

# inference
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 ./tools/dist_test.sh configs/ke_rcnn/ke_rcnn_r50_fpn_fashion_1x.py work_dirs/ke_rcnn_r50_fpn_fashion_1x/epoch32.pth 8 --format-only --eval-options "jsonfile_prefix=work_dirs/ke_rcnn_r50_fpn_fashion_1x/ke_rcnn_r50_fpn_fashion_1x_val_result"

# eval, noted that should change the json path produce by previous step.
python eval/fashion_eval.py

Training

# training
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 ./tools/dist_train.sh configs/ke_rcnn/ke_rcnn_r50_fpn_fashion_1x.py 8

Citation

@article{wang2022ke,
  title={KE-RCNN: Unifying Knowledge-Based Reasoning Into Part-Level Attribute Parsing},
  author={Wang, Xuanhan and Song, Jingkuan and Chen, Xiaojia and Cheng, Lechao and Gao, Lianli and Shen, Heng Tao},
  journal={IEEE Transactions on Cybernetics},
  year={2022},
  publisher={IEEE}
}