Molecule Generation with Graph Neural Networks and Probabilistic Diffusion

Talk/Tutorial on "Molecule Generation with Graph Neural Networks and Probabilistic Diffusion" as part of the ScaDS.AI Summer School 2024.

Please feel free to reach out at Gerrit Großmann at DSA at DFKI.

• Internet Resources
• Reviews
• Foundations of Diffusion Models
• Foundations of GNNs
• Diffusion Modes for Molecular Graphs
• Diffusion Modes for Molecular Point Clouds
• Diffusion in the Latent Space
• Broader Applications (Proteins, Docking, Force Fields, Antibodies, Retrosynthesis)

Internet Resources

• TeachOpenCADD
• Michael Bronstein: The Road to Biology 2.0 Will Pass Through Black-Box Data
• Learning on Graphs & Geometry
• Awesome Diffusion Models
• Step-by-Step Diffusion: An Elementary Tutorial (Nikkiran et al.)
• Geometric Deep Learning - Grids, Groups, Graphs, Geodesics, and Gauges (Bronstein et al.)
• Flow Matching Tutorial (Ben-Hamu et al.)
• A Visual Dive into Conditional Flow Matching

Reviews

1. 2024 - Diffusion Models in De Novo Drug Design
   Amira Alakhdar, Barnabas Poczos, Newell Washburn

2. 2024 - Diffusion models in protein structure and docking
   Jason Yim, Hannes Stärk, Gabriele Corso, Bowen Jing, Regina Barzilay, Tommi S. Jaakkola

3. 2024 - Machine learning-aided generative molecular design
   Yuanqi Du, Arian R. Jamasb, Jeff Guo, Tianfan Fu, Charles Harris, Yingheng Wang, Chenru Duan, Pietro Liò, Philippe Schwaller, Tom L. Blundell

4. 2023 - A Survey on Graph Diffusion Models: Generative AI in Science for Molecule, Protein and Material
   Mengchun Zhang, Maryam Qamar, Taegoo Kang, Yuna Jung, Chenshuang Zhang, Sung-Ho Bae, Chaoning Zhang

5. 2023 - Diffusion Models: A Comprehensive Survey of Methods and Applications
   Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, Ming-Hsuan Yang

6. 2023 - A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems
   Alexandre Duval, Simon V. Mathis, Chaitanya K. Joshi, Victor Schmidt, Santiago Miret, Fragkiskos D. Malliaros, Taco Cohen, Pietro Lio, Yoshua Bengio, Michael Bronstein

7. 2022 - Diffusion Models: A Comprehensive Survey of Methods and Applications
   Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, Ming-Hsuan Yang

Foundations of Diffusion Models

Here is the converted list in Markdown format with the newest papers listed first:

• 2023 - Universal Guidance for Diffusion Models
  Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, Tom Goldstein

• 2023 - On the Connection Between MPNN and Graph Transformer
  Chen Cai, Truong Son Hy, Rose Yu, Yusu Wang

• 2023 - Improving and Generalizing Flow-Based Generative Models with Minibatch Optimal Transport
  Alexander Tong, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Kilian Fatras, Guy Wolf, Yoshua Bengio

• 2023 - Stochastic Interpolants: A Unifying Framework for Flows and Diffusions
  Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Eijnden

• 2023 - Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models
  Gabriele Corso, Yilun Xu, Valentin de Bortoli, Regina Barzilay, Tommi Jaakkola

• 2023 - RetroBridge: Modeling Retrosynthesis with Markov Bridges
  Ilia Igashov, Arne Schneuing, Marwin Segler, Michael Bronstein, Bruno Correia

• 2023 - Fine-grained Expressivity of Graph Neural Networks
  Jan Böker, Ron Levie, Ningyuan Huang, Soledad Villar, Christopher Morris

• 2022 - Understanding DDPM Latent Codes Through Optimal Transport
  Valentin Khrulkov, Gleb Ryzhakov, Andrei Chertkov, Ivan Oseledets

• 2022 - Generative Modelling With Inverse Heat Dissipation
  Severi Rissanen, Markus Heinonen, Arno Solin

• 2022 - Diffusion Causal Models for Counterfactual Estimation
  Pedro Sanchez, Sotirios A. Tsaftaris

• 2022 - Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise Arpit Bansal, Eitan Borgnia, Hong-Min Chu, Jie S. Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping, Tom Goldstein

• 2021 - Generative AI for designing and validating easily synthesizable and structurally novel antibiotics
  Wenhao Gao, Rocío Mercado, Connor W. Coley

Foundations of GNNs

• 2021 - Weisfeiler and leman go machine learning: The story so far
  Christopher Morris, Yaron Lipman, Haggai Maron, Bastian Rieck, Nils M Kriege, Martin Grohe, Matthias Fey, Karsten Borgwardt

• 2023 - Fine-grained Expressivity of Graph Neural Networks
  Jan Böker, Ron Levie, Ningyuan Huang, Soledad Villar, Christopher Morris

• 2022 - Incompleteness of graph neural networks for points clouds in three dimensions
  Sergey N Pozdnyakov, Michele Ceriotti

Diffusion Modes for Molecular Graphs

• 2023 - Variational Autoencoding Molecular Graphs with Denoising Diffusion Probabilistic Model
  Daiki Koge, Naoaki Ono, Shigehiko Kanaya

• 2023 - Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling
  Xiaohui Chen, Jiaxing He, Xu Han, Li-Ping Liu

• 2022 - DiGress: Discrete Denoising diffusion for graph generation
  Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, Pascal Frossard

• 2022 - Invertible Neural Networks for Graph Prediction
  Chen Xu, Xiuyuan Cheng, Yao Xie

• 2022 - TIDE: Time Derivative Diffusion for Deep Learning on Graphs
  Maximilian Krahn, Maysam Behmanesh, Maks Ovsjanikov

• 2022 - Structure-based Drug Design with Equivariant Diffusion Models
  Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lió, Carla Gomes, Max Welling, Michael Bronstein, Bruno Correia

Diffusion Modes for Molecular Point Clouds

• 2024 - Equivariant 3D-conditional diffusion model for molecular linker design
  Ilia Igashov, Hannes Stärk, Clément Vignac, Arne Schneuing, Victor Garcia Satorras, Pascal Frossard, Max Welling, Michael Bronstein, Bruno Correia

• 2023 - Coarse-to-Fine: a Hierarchical Diffusion Model for Molecule Generation in 3D
  Bo Qiang, Yuxuan Song, Minkai Xu, Jingjing Gong, Bowen Gao, Hao Zhou, Weiying Ma, Yanyan Lan

• 2023 - Geometric Latent Diffusion Models for 3D Molecule Generation
  Minkai Xu, Alexander Powers, Ron Dror, Stefano Ermon, Jure Leskovec

• 2023 - MUDiff: Unified Diffusion for Complete Molecule Generation
  Chenqing Hua, Sitao Luan, Minkai Xu, Rex Ying, Jie Fu, Stefano Ermon, Doina Precup

• 2022 - LION: Latent Point Diffusion Models for 3D Shape Generation
  Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis

• 2022 - Equivariant Diffusion for Molecule Generation in 3D
  Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, Max Welling

• 2022 - Uni-Mol: A Universal 3D Molecular Representation Learning Framework
  Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, Guolin Ke

• 2022 - GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation
  Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, Jian Tang

• 2022 - Torsional Diffusion for Molecular Conformer Generation
  Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, Tommi Jaakkola

Diffusion in the Latent Space

• 2024 - Latent Graph Diffusion: A Unified Framework for Generation and Prediction on Graphs
  Zhou Cai, Xiyuan Wang, Muhan Zhang

• 2024 - Graphusion: Latent Diffusion for Graph Generation
  Ling Yang, Zhilin Huang, Zhilong Zhang, Zhongyi Liu, Shenda Hong, Wentao Zhang, Wenming Yang, Bin Cui, Luxia Zhang

• 2023 - Geometric Latent Diffusion Models for 3D Molecule Generation
  Minkai Xu, Alexander S. Powers, Ron O. Dror, Stefano Ermon, Jure Leskovec

Broader Applications (Proteins, Docking, Force Fields, Antibodies, Retrosynthesis)

• 2024 - 'A landmark moment': scientists use AI to design antibodies from scratch
  Ewen Callaway

• 2024 - Latent Graph Diffusion: A Unified Framework for Generation and Prediction on Graphs
  Zhou Cai, Xiyuan Wang, Muhan Zhang

• 2024 - Graphusion: Latent Diffusion for Graph Generation
  Ling Yang, Zhilin Huang, Zhilong Zhang, Zhongyi Liu, Shenda Hong, Wentao Zhang, Wenming Yang, Bin Cui, Luxia Zhang

• 2023 - SE(3)-Stochastic Flow Matching for Protein Backbone Generation
  Avishek Joey Bose, Tara Akhound-Sadegh, Kilian Fatras, Guillaume Huguet, Jarrid Rector-Brooks, Cheng-Hao Liu, Andrei Cristian Nica, Maksym Korablyov, Michael Bronstein, Alexander Tong

• 2023 - Harmonic Self-Conditioned Flow Matching for Multi-Ligand Docking and Binding Site Design
  Hannes Stärk, Bowen Jing, Regina Barzilay, Tommi Jaakkola

• 2023 - Eigenfold: Generative protein structure prediction with diffusion models B Jing, E Erives, P Pao-Huang, G Corso, B Berger, T Jaakkola

• 2023 - Illuminating protein space with a programmable generative model
  John B. Ingraham, Max Baranov, Zak Costello, Karl W. Barber, Wujie Wang, Ahmed Ismail, Vincent Frappier, Dana M. Lord, Christopher Ng-Thow-Hing, Erik R. Van Vlack, Shan Tie, Vincent Xue, Sarah C. Cowles, Alan Leung, João V. Rodrigues, Claudio L. Morales-Perez, Alex M. Ayoub, Robin Green, Katherine Puentes, Frank Oplinger, Nishant V. Panwar, Fritz Obermeyer, Adam R. Root, Andrew L. Beam, Frank J. Poelwijk, Gevorg Grigoryan

• 2022 - Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures
  Shitong Luo, Yufeng Su, Xingang Peng, Sheng Wang, Jian Peng, Jianzhu Ma

• 2020 - A Review of Deep Learning Methods for Antibodies
  Jordan Graves, Jacob Byerly, Eduardo Priego, Naren Makkapati, S. Vince Parish, Brenda Medellin, Monica Berrondo