Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
dataloaders
lib
models
script
.gitignore
Makefile
README.md
config.py

README.md

Introduction

This repository just provides a baseline for PIC Challenge. Lots of code are from neural-motifs and it is not end-to-end.

@inproceedings{zellers2018scenegraphs,
  title={Neural Motifs: Scene Graph Parsing with Global Context},
  author={Zellers, Rowan and Yatskar, Mark and Thomson, Sam and Choi, Yejin},
  booktitle = "Conference on Computer Vision and Pattern Recognition",  
  year={2018}
}

Usage

  1. Compile everything. run make in the main directory

  2. You should train a instance-segmentation model on PIC. For example, you can use Detectron to generate the boxes, masks and feature maps of train/val/test data.

  3. run ./script/train.sh to train relation model.

  4. run ./scripy/test.sh to generate results.

File Tree

  • obj_feat/xxx.npy: [#instance, 1024+85+4+30*30] 1024: the output feat of fc7. 85: cls_score. 4: (x1,y1,x2,y2). 30*30: mask
  • roi_feat/xxx.npy: [#instance, 256*7*7] the feat after RoIAlign of all instances
  • you can download glove.6B.100d.pt/glove.6B.200d.pt from baiduyun pwd: at8t or official website

----------------------------------------
PIC_PATH
└───categories_list
│   └───label_categories.json 
│   └───relation_categories.json
└───image
│   └───train
│   │      xxx.jpg
│   └───val
│   │      xxx.jpg
│   └───test
│   │      xxx.jpg
└───relation
│   └───relations_train.json 
│   └───relations_val.json
└───segmentation
│   └───train
│   │      └───instance
│   │      │      xxx.png
│   │      └───semantic
│   │      │      xxx.png
│   └───val
│   │      └───instance
│   │      │      xxx.png
│   │      └───semantic
│   │      │      xxx.png
----------------------------------------
PIC_OFFLINE_PATH
└───train
│   └───obj_feat
│   │      xxx.npy
│   └───roi_feat
│   │      xxx.npy
└───val
│   └───obj_feat
│   │      xxx.npy
│   └───roi_feat
│   │      xxx.npy
└───test
│   └───obj_feat
│   │      xxx.npy
│   └───roi_feat
│   │      xxx.npy
----------------------------------------
DATA_PATH
└───glove.6B.100d.pt
└───glove.6B.200d.pt
----------------------------------------

Help

Feel free to push issues if you encounter trouble getting it to work!