Reading List for Machine Learning Research

-- Graph Neural Networks, Probabilistic Modeling, NLP, and Related areas

Content

1. Graph Neural Networks
1.1 Basic Models	1.2 Graph Types
1.3 Pooling Methods	1.4 Analysis
1.5 Graph Generation	1.6 N/A
2. NLP
2.1 BERT and alike	2.2 Information Extraction
2.3 Parsing trees	2.4 N/A
3. Neural Architecture Search
3.1 Comparisions
4. Probabilistic Models
4.1 Theory

Graph Neural Networks

Basic Models

1. Benchmarking Graph Neural Networks. arxiv 2020. paper

Vijay Prakash Dwivedi, Chaitanya K. Joshi , Thomas Laurent , Yoshua Bengio and Xavier Bresson.

2. Graph Agreement Models for Semi-Supervised Learning. NIPS 2019. paper

Otilia Stretcu‡∗, Krishnamurthy Viswanathan†, Dana Movshovitz-Attias†, Emmanouil Antonios Platanios‡, Andrew Tomkins†, Sujith Ravi†

3. Position-aware Graph Neural Networks. ICML 2019. paper

Jiaxuan You, Rex Ying, Jure Leskovec

4. A Flexible Generative Framework for Graph-based Semi-supervised Learning Neurips 2019. paper

Jiaqi Ma, Weijing Tang, Ji Zhu, Qiaozhu Mei

5. Graph Transformer Networks Neurips 2019 paper

Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, Hyunwoo J. Kim

6. MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing ICML 2019 paper

Abu-El-Haija, Sami and Perozzi, Bryan and Kapoor, Amol and Alipourfard, Nazanin and Lerman, Kristina and Harutyunyan, Hrayr and Steeg, Greg Ver and Galstyan, Aram

Graph Types

1. Graph Convolutional Reinforcement Learning ICLR 2020. paper

Jiechuan Jiang, Chen Dun, Tiejun Huang, Zongqing Lu

Pooling Methods

1. Graph U-Nets ICML 2019. paper

Hongyang Gao, Shuiwang Ji

2. Hierarchical graph representation learning with differentiable pooling NEURIPS 2018. paper

Ying, Zhitao and You, Jiaxuan and Morris, Christopher and Ren, Xiang and Hamilton, Will and Leskovec, Jure

3. Self-Attention Graph Pooling ICML 2019 paper

Junhyun Lee, Inyeop Lee, Jaewoo Kang

4. Rethinking pooling in graph neural networks Neurips 2020 paper

Diego Mesquita, Amauri H. Souza, and Samuel Kaski

Analysis

1. Graph Neural Networks Exponentially Lose Expressive Power for Node Classification ICLR 2020. paper

Kenta Oono, Taiji Suzuki

2. The Logical Expressiveness of Graph Neural Networks ICLR 2020. paper

Pablo Barceló, Egor V. Kostylev, Mikael Monet, Jorge Pérez, Juan Reutter, and Juan Pablo Silva

Graph Generation

1. Learning Deep Generative Models of Graphs ICML 2018. paper

*Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, Peter Battaglia *

2. Efficient Graph Generation with Graph Recurrent Attention Networks NIPS 2019. paper

Renjie Liao, Yujia Li, Yang Song, Shenlong Wang, William L. Hamilton, David Duvenaud, Raquel Urtasun, Richard Zemel1

NLP

BERT and alike

1. Xlnet: Generalized autoregressive pretraining for language understanding. NeurIPS 2019. paper

Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R. R., & Le, Q. V.

2. Visualizing and Measuring the Geometry of BERT NeurIPS 2019. paper

Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, Martin Wattenberg

3. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. ACL 2019. paper

Dai, Zihang, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov.

4. BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning ICML 2019. paper

Asa Cooper Stickland, Iain Murray.

5. BERT Rediscovers the Classical NLP Pipeline ACL 2019. paper

Tenney, I., Das, D. and Pavlick, E

6. What does BERT learn about the structure of language? ACL 2019. paper

Jawahar, G., Sagot, B. and Seddah, D

7. Distilling Knowledge Learned in BERT for Text Generation ACL 2020. paper

Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu, Jingjing Liu

8. MASS: Masked Sequence to Sequence Pre-training for Language Generation ICML 2019. paper

Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu

9. Cross-Thought for Sentence Encoder Pre-training EMNLP 2020. paper

Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jingjing Liu, Jing Jiang

Information Extraction

1. Open information extraction with meta-pattern discovery in biomedical literature. BCB2018 paper

Wang, Xuan, Yu Zhang, Qi Li, Yinyin Chen, and Jiawei Han.

2. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. ACL 2019. paper

Ming Ding†, Chang Zhou‡, Qibin Chen†, Hongxia Yang‡, Jie Tang†

3. On the Possibilities and Limitations of Multi-hop Reasoning Under Linguistic Imperfections. paper

Daniel Khashabi1∗ Erfan Sadeqi Azer2 Tushar Khot1 Ashish Sabharwal1 Dan Roth3

4. Semantic Graphs for Generating Deep Questions. ACL 2020. paper

Liangming Pan, Yuxi Xie Yansong Feng Tat-Seng Chua Min-Yen Kan

5. GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction. ACL 2019. paper

Tsu-Jui Fu, Peng-Hsuan Li, Wei-Yun Ma

6. OpenIE-based approach for Knowledge Graph construction from text. paper

Jose L. Martinez-Rodriguez, Ivan Lopez-Arevalo, Ana B. Rios-Alvarado

7. CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training. paper

Qipeng Guo, Zhijing Jin, Xipeng Qiu, Weinan Zhang, David Wipf, Zheng Zhang

Parsing trees

1. Unsupervised Recurrent Neural Network Grammars. NAACL-HLT 2019. paper

Kim, Yoon, Alexander M. Rush, Lei Yu, Adhiguna Kuncoro, Chris Dyer, and Gábor Melis.

Neural Architecture Search

Comparisons

1. NAS evaluation is frustratingly hard. ICLR 2020. paper

Yang, Antoine, Pedro M. Esperança, and Fabio M. Carlucci.

Probabilistic Models

Optimization

1. Computational Separations between Sampling and Optimization Neurips 2019. paper

Kunal Talwar

2. Bayesian Optimization of Combinatorial Structures ICML2018. paper

Ricardo Baptista, Matthias Poloczek