Skip to content



Folders and files

Last commit message
Last commit date

Latest commit



42 Commits

Repository files navigation

Deep Reasoning with Knowledge Graph for Social Relationship Understanding

This repo includes the source code of the paper: "Deep Reasoning with Knowledge Graph for Social Relationship Understanding" (IJCAI 2018) by Zhouxia Wang, Tianshui Chen, Jimmy Ren, Weihao Yu, Hui Cheng, Liang Lin.


The code is implemented using the Pytorch library with Python 2.7 and has been tested on a desktop with the system of Ubuntu 14.04 LTS.


PISC was released by [Li et al. ICCV 2017]. It involves two-level relationship, i.e., coarse-level relationships with 3 categories and fine-level relationships with 6 categories.

PIPA-relation was released by [Sun et al. CVPR 2017]. It covers 5 social domains, which can be further divided into 16 social relationships. On this dataset, we focus on the 16 social relationships.

Models && objects && adjacency matrices

Models, objects and ajacency matrices are in HERE.


usage: [-h] [-j N] [-b N] [--print-freq N] [--weights PATH]
           [--scale-size SCALE_SIZE] [--world-size WORLD_SIZE] [-n N]
           [--write-out] [--adjacency-matrix PATH] [--crop-size CROP_SIZE]
           [--result-path PATH]
           DIR DIR DIR

PyTorch Relationship

positional arguments:
  DIR                       path to dataset
  DIR                       path to objects (bboxes and categories of objects)
  DIR                       path to test list

optional arguments:
  -h, --help                show this help message and exit
  -j N, --workers N         number of data loading workers (defult: 4)
  -b N, --batch-size N      mini-batch size (default: 1)
  --print-freq N, -p N      print frequency (default: 10)
  --weights PATH            path to weights (default: none)
  --scale-size SCALE_SIZE   input size
  --world-size WORLD_SIZE   number of distributed processes
  -n N, --num-classes N     number of classes / categories
  --write-out               write scores
  --adjacency-matrix PATH   path to adjacency-matrix of graph
  --crop-size CROP_SIZE     crop size
  --result-path PATH        path for saving result (default: none)


Modify the path of data before running the script.



PISC: Coarse-level

Methods Intimate Non-Intimate No Relation mAP
Union CNN 72.1 81.8 19.2 58.4
Pair CNN 70.3 80.5 38.8 65.1
Pair CNN + BBox + Union 71.1 81.2 57.9 72.2
Pair CNN + BBox + Global 70.5 80.0 53.7 70.5
Dual-glance 73.1 84.2 59.6 79.7
Ours 81.7 73.4 65.5 82.8

PISC: Fine-level

Methods Friends Family Couple Professional Commercial No Relation mAP
Union CNN 29.9 58.5 70.7 55.4 43.0 19.6 43.5
Pair CNN 30.2 59.1 69.4 57.5 41.9 34.2 48.2
Pair CNN + BBox + Union 32.5 62.1 73.9 61.4 46.0 52.1 56.9
Pair CNN + BBox + Global 32.2 61.7 72.6 60.8 44.3 51.0 54.6
Dual-glance 35.4 68.1 76.3 70.3 57.6 60.9 63.2
Ours 59.6 64.4 58.6 76.6 39.5 67.7 68.7


Methods accuracy
Two stream CNN 57.2
Dual-Glance 59.6
Ours 62.3


    title={Deep Reasoning with Knowledge Graph for Social Relationship Understanding},
    author={Zhouxia Wang, Tianshui Chen, Jimmy Ren, Weihao Yu, Hui Cheng, Liang Lin},
    booktitle={International Joint Conference on Artificial Intelligence},


For any questions, feel free to open an issue or contact us ( &


No description, website, or topics provided.






No releases published


No packages published


  • Python 98.5%
  • Shell 1.5%