2D MOTHODS.md

Awesome Scene Graph

A curated list of 2D scene graph generation grouped in different methods . :-)

Introduction

Markdown format of paper list items:

- [Paper Name](link) - Author 1 _et al_, `Conference Year`. [[code]](link)

Table of Contents

• [Using Message Passing](#Using Message Passing)
• [Attention Mechanisms](#Attention Mechanisms)
• [Using Language Priors](#Using Language Priors)
• [Using External Knowledge Bases](#Using External Knowledge Bases)
• [Visual Embedding Approaches](#Visual Embedding Approaches)
• [Predict Object Pair-wise Relationship Categories Directly](#Predict Object Pair-wise Relationship Categories Directly)
• [Using Depth Information](#Using Depth Information)
• [Other Methods](#Other Methods)

Using Message Passing

• Scene Graph Generation by Iterative Message Passing - Danfei Xu et al, CVPR 2017. [code]
• Scene Graph Generation from Objects, Phrases and Region Captions - Yikang Li et al, ICCV 2017. [code]
• ViP-CNN: Visual Phrase Guided Convolutional Neural Network - Yikang Li et al, CVPR 2017.
• Detecting Visual Relationships with Deep Relational Networks - Bo Dai et al, CVPR 2017. [code]
• Recurrent Visual Relationship Recognition with Triplet Unit - Kento Masui et al, ISM 2017.
• Factorizable Net: An Efficient Subgraph-based Framework for Scene Graph Generation - Yikang Li et al, ECCV 2018. [code]
• Neural Motifs_Scene Graph Parsing with Global Context - Rowan Zellers et al, CVPR 2018. [code]
• Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition - Guojun Yin et al, ECCV 2018. [code]
• Deep Structured Learning for Visual Relationship Detection - Yaohui Zhu et al, AAAI 2018.
• Region-Object Relevance-Guided Visual Relationship Detection - Yusuke Goutsu et al, BMVC 2018.
• Recurrent Visual Relationship Recognition with Triplet Unit for Diversity - Kento Masui et al, IJSC 2018.
• Deep Image Understanding Using Multilayered Contexts - Donghyeop Shin et al, MPE 2018.
• Large-Scale Visual Relationship Understanding - Ji Zhang et al, AAAI 2019. [code]
• Learning to Compose Dynamic Tree Structures for Visual Contexts - Kaihua Tang et al, CVPR 2019 Oral. [code]
• Counterfactual Critic Multi-Agent Training for Scene Graph Generation - Long Chen et al, ICCV 2019 Oral.
• On Exploring Undetermined Relationships for Visual Relationship Detection - Yibing Zhan et al, CVPR 2019. [code]
• Exploring Context and Visual Pattern of Relationship for Scene Graph Generation - Wenbin Wang et al, CVPR 2019.
• Neural Message Passing for Visual Relationship Detection - Yue Hu et al, ICML LRG Workshop 2019. [code]
• PANet: A Context Based Predicate Association Network for Scene Graph Generation - Yunian Chen et al, ICME 2019.
• Visual Relationship Detection with Relative Location Mining - Hao Zhou et al, ACM MM 2019.
• Visual relationship detection based on bidirectional recurrent neural network - Yibo Dai et al, Multimedia Tools and Applications 2019.
• Exploring the Semantics for Visual Relationship Detection - Wentong Liao et al, arXiv 2019.
• A hierarchical recurrent approach to predict scene graphs from a visual‐attention‐oriented perspective - Wenjing Gao et al, Computational Intelligence 2019 SPECIAL ISSUE ARTICLE.
• Relationship-Aware Spatial Perception Fusion for Realistic Scene Layout Generation - Hongdong Zheng et al, arXiv 2019.
• The Limited Multi-Label Projection Layer - Brandon Amos et al, arXiv 2019. [code]

Attention Mechanisms

• Towards Context-Aware Interaction Recognition for Visual Relationship Detection - Bohan Zhuang et al, ICCV 2017.
• Graph R-CNN for Scene Graph Generation - Jianwei Yang et al, ECCV 2018. [code]
• LinkNet_Relational Embedding for Scene Graph - Sanghyun Woo et al, NIPS 2018. [code]
• Generating Triples with Adversarial Networks for Scene Graph Construction - Matthew Klawonn et al, AAAI 2018.
• Referring Relationships - Ranjay Krishna et al, CVPR 2018. [code]
• Scene Graph Generation Based on Node-Relation Context Module - Xin Lin et al, ICONIP 2018.
• Detecting Visual Relationships Using Box Attention - Alexander Kolesnikov et al, ICCVW 2019.
• Visual Relationships as Functions: Enabling Few-Shot Scene Graph Prediction - Apoorva Dornadula et al, ICCVW 2019.
• Attention-Translation-Relation Network for Scalable Scene Graph Generation - Nikolaos Gkanatsios et al, ICCVW 2019.
• Attentive Relational Networks for Mapping Images to Scene Graphs - Mengshi Qi et al, CVPR 2019.
• Expressing Visual Relationships via Language - Hao Tan et al, ACL 2019. [code]
• Visual Spatial Attention Network for Relationship Detection - Chaojun Han, et al, ACM MM 2019.
• Visual Relation Detection with Multi-Level Attention - Sipeng Zheng, et al, ACM MM 2019.
• Visual Relationship Recognition via Language and Position Guided Attention - Hao Zhou, et al, ICASSP 2019.
• Relationship Detection Based on Object Semantic Inference and Attention Mechanisms - Liang Zhang et al, ICMR 2019.
• Leveraging Auxiliary Text for Deep Recognition of Unseen Visual Relationships - Gal Sadeh Kenigsfield et al, Under review as a conference paper at ICLR 2020.

Using Language Priors

• Visual Relationship Detection with Language Priors - Cewu Lu et al, ECCV 2016 Oral. [code]
• Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues - Bryan A et al, ICCV 2017. [code]
• Deep Variation-Structured Reinforcement Learning for Visual Relationship and Attribute Detection - Xiaodan Liang et al, CVPR 2017. [code]
• Improving Visual Relationship Detection using Semantic Modeling of Scene Descriptions - Bryan A et al, ISWC 2017.
• Tensorize, Factorize and Regularize: Robust Visual Relationship Learning - Seong Jae Hwang et al, CVPR 2018. [code]
• Visual Relationship Detection with Language prior and Softmax - Jaewon Jung et al, IPAS 2018. [code]
• Visual Relationship Detection Based on Guided Proposals and Semantic Knowledge Distillation - François Plesse et al, ICME 2018.
• Context-Dependent Diffusion Network for Visual Relationship Detection - Zhen Cui et al, ACM MM 2018. [code]
• Natural Language Guided Visual Relationship Detection - Wentong Liao et al, CVPR 2019.
• Knowledge-Embedded Routing Network for Scene Graph Generation - Tianshui Chen et al, CVPR 2019. [code]
• Soft Transfer Learning via Gradient Diagnosis for Visual Relationship Detection - Diqi Chen et al, WACV 2019.
• Compensating Supervision Incompleteness with Prior Knowledge in Semantic Image Interpretation - Ivan Donadello et al, IJCNN 2019. [code]
• Hierarchical Visual Relationship Detection - Xu Sun et al, ACM MM 2019.
• Visual Relationship Detection with Low Rank Non-Negative Tensor Decomposition - Mohammed Haroon Dupty et al, arXiv 2019.
• Relational Reasoning using Prior Knowledge for Visual Captioning - Jingyi Hou et al, arXiv 2019.

Using External Knowledge Bases

• Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation - Ruichi Yu et al, ICCV 2017.
• Scene Graph Generation with External Knowledge and Image Reconstruction - Jiuxiang Gu et al, CVPR 2019. [code]

Visual Embedding Approaches

• Visual Translation Embedding Network for Visual Relation Detection - Hanwang Zhang et al, CVPR 2017. [code]
• Representation Learning for Scene Graph Completion via Jointly Structural and Visual Embedding - Hai Wan et al, IJCAI 2018. [code]
• Visual Relationship Detection Using Joint Visual-Semantic Embedding - Binglin Li et al, ICPR 2018.
• Object Relation Detection Based on One-shot Learning - Li Zhou et al, arXiv 2018.
• Attention-Translation-Relation Network for Scalable Scene Graph Generation - Nikolaos Gkanatsios et al, ICCV 2019.
• Detecting Unseen Visual Relations Using Analogies - Julia Peyre et al, ICCV 2019.
• Contextual Translation Embedding for Visual Relationship Detection and Scene Graph Generation - Zih-Siou Hung et al, arXiv 2019.
• Deeply Supervised Multimodal Attentional Translation Embeddings for Visual Relationship Detection - Nikolaos Gkanatsios et al, arXiv 2019.

Predict Object Pair-wise Relationship Categories Directly

• Relationship Proposal Networks - Ji Zhang et al, CVPR 2017.
• Visual relationship detection with object spatial distribution - Yaohui Zhu et al, ICME 2017.
• A Problem Reduction Approach for Visual Relationships Detection - Toshiyuki Fukuzawa et al, ECCVW 2018.
• An Interpretable Model for Scene Graph Generation - Ji Zhang et al, arXiv 2018.
• Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features - Xu Yang et al, ECCV 2018. [code]
• Visual Relationship Detection with Deep Structural Ranking - Kongming Liang et al, AAAI 2018. [code]
• Learning Prototypes for Visual Relationship Detection - François Plesse et al, CBMI 2018.
• Visual Relationship Detection Based on Local Feature and Context Feature - Yuping Han et al, ICNIDC 2018.
• VrR-VG: Refocusing Visually-Relevant Relationships - Yuanzhi Liang et al, ICCV 2019.
• SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition - Kaiyu Yang et al, ICCV 2019. [code]
• BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and Visual Relationship Detection - Kaiyu Yang et al, AAAI 2019. [code]
• On Class Imbalance and Background Filtering in Visual Relationship Detection - Alessio Sarullo et al, arXiv 2019.

Using Depth Information

• On Support Relations and Semantic Scene Graphs - Michael Ying Yang et al, ISPRS 2017.
• Visual Relationship Prediction via Label Clustering and Incorporation of Depth Information - Hsuan-Kung Yang et al, ECCVW 2018.
• Support Relation Analysis for Objects in Multiple View RGB-D Images - Peng Zhang et al, IJCAIW QR 2019.
• Improving Visual Relation Detection using Depth Maps - Sahand Sharifzadeh et al, arXiv 2019.

Other Methods

• Weakly-Supervised Learning of Visual Relations - Julia Peyre et al, ICCV 2017. [code]
• PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCN - Hanwang Zhang et al, ICCV 2017. [code]
• Detecting Visual Relationships with Deep Relational Networks - Bo Dai et al, CVPR 2017. [code]
• Pixels to Graphs by Associative Embedding - Alejandro Newell et al, NIPS 2017. [code]
• Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction - Roei Herzig et al, NIPS 2018. [code]
• Scene Graph Parsing as Dependency Parsing - Yu-Siang Wang et al, NAACL 2018. [code]
• Scene Graph Generation via Conditional Random Fields - Weilin Cong et al, arXiv 2018.
• MR-NET: Exploiting Mutual Relation for Visual Relationship Detection - Yi Bin et al, AAAI 2019.
• Scene Graph Prediction with Limited Labels - Vincent S. Chen et al, ICCV 2019. [code]
• Differentiable Scene Graphs - Moshiko Raboh et al, ICCVW 2019.
• Graphical Contrastive Losses for Scene Graph Parsing - Ji Zhang et al, CVPR 2019. [code]
• Generating Expensive Relationship Features from Cheap Objects - Xiaogang Wang et al, BMVC 2019.
• Optimising the Input Image to Improve Visual Relationship Detection - Noel Mizzi et al, arXiv 2019.