awesome-protein-representation-learning

This repository contains a list of papers on the Protein Representation Learning (PRL), we categorize them based on their published years. We will try to make this list updated. If you found any error or any missed paper, please don't hesitate to open issues or pull requests.

However, Since 2023, the number of simple PRL methods has dwindled, and researchers have begun to focus on more divergent and difficult domain problems.

Year 2024

1. [Arxiv 2024] HeMeNet: Heterogeneous Multichannel Equivariant Network for Protein Multi-task Learning[paper]
2. [Arxiv 2024] Clustering for Protein Representation Learning[paper]
3. [ICLR 2024] BioBridge: Bridging Biomedical Foundation Models via Knowledge Graph[paper]
4. [ICLR 2024] Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning[paper]
5. [ICLR 2024] Learning Scalar Fields for Molecular Docking with Fast Fourier Transforms[paper]
6. [ICLR 2024] BEND: Benchmarking DNA Language Models on Biologically Meaningful Tasks[paper]
7. [ICLR 2024] Searching for High-Value Molecules Using Reinforcement Learning and Transformers[paper]
8. [ICLR 2024] Expected flow networks in stochastic environments and two-player zero-sum games[paper]
9. [ICLR 2024] Conversational Drug Editing Using Retrieval and Domain Feedback[paper]
10. [ICLR 2024] The Discovery of Binding Modes Requires Rethinking Docking Generalization[paper]
11. [ICLR 2024] Hierarchical Data-efficient Representation Learning for Tertiary Structure-based RNA Design[paper]
12. [ICLR 2024] Generative Adversarial Policy Network for Modelling Protein Complexes[paper]
13. [ICLR 2024] Protein Multimer Structure Prediction via PPI-guided Prompt Learning[paper]
14. [ICLR 2024] Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment[paper]
15. [ICLR 2024] Protein-Ligand Interaction Prior for Binding-aware 3D Molecule Diffusion Models[paper]
16. [ICLR 2024] Str2Str: A Score-based Framework for Zero-shot Protein Conformation Sampling[paper]
17. [ICLR 2024] Neural Probabilistic Protein-Protein Docking via a Differentiable Energy Model[paper]
18. [ICLR 2024] Robust Model-Based Optimization for Challenging Fitness Landscapes[paper]
19. [ICLR 2024] Protein-ligand binding representation learning from fine-grained interactions[paper]
20. [ICLR 2024] Learning to design protein-protein interactions with enhanced generalization[paper]
21. [ICLR 2024] KW-Design: Pushing the Limit of Protein Design via Knowledge Refinement[paper]
22. [ICLR 2024] Rigid Protein-Protein Docking via Equivariant Elliptic-Paraboloid Interface Prediction[paper]
23. [ICLR 2024] Evaluating Representation Learning on the Protein Structure Universe[paper]
24. [ICLR 2024] Improving protein optimization with smoothed fitness landscapes[paper]
25. [ICLR 2024] Dynamics-Informed Protein Design with Structure Conditioning[paper]
26. [ICLR 2024] Pre-training Sequence, Structure, and Surface Features for Comprehensive Protein Representation Learning[paper]
27. [ICLR 2024 spotlight] SE(3)-Stochastic Flow Matching for Protein Backbone Generation[paper]
28. [ICLR 2024 spotlight] De novo Protein Design Using Geometric Vector Field Networks[paper]
29. [ICLR 2024 spotlight] MAPE-PPI: Towards Effective and Efficient Protein-Protein Interaction Prediction via Microenvironment-Aware Protein Embedding[paper]
30. [ICLR 2024 spotlight] SaProt: Protein Language Modeling with Structure-aware Vocabulary[paper][code]
31. [ICLR 2024 oral] Protein Discovery with Discrete Walk-Jump Sampling[paper]
32. [AAAI 2024] Improving PTM Site Prediction by Coupling of Multi-Granularity Structure and Multi-Scale Sequence Representation[paper][code]

Year 2023

1. [NIPS 2023] DSR: Dynamical Surface Representation as Implicit Neural Networks for Protein[paper][code]
2. [NIPS 2023] Predicting a Protein's Stability under a Million Mutations[paper][code]
3. [NIPS 2023] Protein Design with Guided Discrete Diffusion[paper]
4. [NIPS 2023] ProteinNPT: Improving protein property prediction and design with non-parametric transformers[paper][code]
5. [NIPS 2023] ProteinShake: Building datasets and benchmarks for deep learning on protein structures[paper][code]
6. [NIPS 2023] Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materials[paper][code]
7. [NIPS 2023] Full-Atom Protein Pocket Design via Iterative Refinement[paper][code]
8. [NIPS 2023] Injecting Multimodal Information into Rigid Protein Docking via Bi-level Optimization[paper]
9. [NIPS 2023] Unsupervised Protein-Ligand Binding Energy Prediction via Neural Euler's Rotation Equation[paper][code]
10. [NIPS 2023] ProteinInvBench: Benchmarking Protein Inverse Folding on Diverse Tasks, Models, and Metrics[paper][code]
11. [NIPS 2023] OpenProteinSet: Training data for structural biology at scale[paper]
12. [NIPS 2023] CELLE-2: Translating Proteins to Pictures and Back with a Bidirectional Text-to-Image Transformer[paper][code]
13. [NIPS 2023] FABind: Fast and Accurate Protein-Ligand Binding[paper][code]
14. [NIPS 2023] ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction[paper][code]
15. [NIPS 2023] Graph Denoising Diffusion for Inverse Protein Folding[paper][code]
16. [NIPS 2023] DiffPack: A Torsional Diffusion Model for Autoregressive Protein Side-Chain Packing[paper][code]
17. [NIPS 2023] DrugCLIP: Contrasive Protein-Molecule Representation Learning for Virtual Screening[paper]
18. [NIPS 2023] Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model[paper]
19. [NIPS 2023] PoET: A generative model of protein families as sequences-of-sequences[paper][code]
20. [NIPS 2023] Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Prediction[paper]
21. [ICML 2023] Bidirectional Learning for Offline Model-based Biological Sequence Design[paper][code]
22. [ICML 2023] Reprogramming Pretrained Language Models for Antibody Sequence Infilling[paper][code]
23. [ICML 2023] Learning Subpocket Prototypes for Generalizable Structure-based Drug Design[paper][code]
24. [ICML 2023] Extrapolative Controlled Sequence Generation via Iterative Refinement[paper][code]
25. [ICML 2023] AbODE: Ab initio antibody design using conjoined ODEs[paper][code]
26. [ICML 2023] End-to-End Full-Atom Antibody Design[paper][code]
27. [ICML 2023] Exploring Chemical Space with Score-based Out-of-distribution Generation[paper][code]
28. [ICML 2023] SE(3) diffusion model with application to protein backbone generation[paper][code]
29. [ICML 2023] Importance Weighted Expectation-Maximization for Protein Sequence Design[paper][code]
30. [ICML 2023] Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds[paper][code]
31. [ICML 2023] Chemically Transferable Generative Backmapping of Coarse-Grained Proteins[paper][code]
32. [ICML 2023] Structure-informed Language Models Are Protein Designers[paper]
33. [ICML 2023] ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts [paper][code]
34. [bioRxiv 2023] Retrieved Sequence Augmentation for Protein Representation Learning [paper][code]
35. [Arxiv 2023] Data-Efficient Protein 3D Geometric Pretraining via Refinement of Diffused Protein Structure Decoy [paper]
36. [ICLR 2023] Protein Representation Learning by Geometric Structure Pretraining [paper]
37. [ICLR 2023] Protein Representation Learning via Knowledge Enhanced Primary Structure Reasoning[paper]
38. [ICLR 2023] Multi-level Protein Structure Pre-training via Prompt Learning[paper]
39. [ICLR 2023] Learning Hierarchical Protein Representations via Complete 3D Graph Networks[paper]
40. [ICLR 2023] Rotamer Density Estimator is an Unsupervised Learner of the Effect of Mutations on Protein-Protein Interaction[paper][code]
41. [ICLR 2023] Matching receptor to odorant with protein language and graph neural networks[paper][code]
42. [ICLR 2023] Protein Sequence and Structure Co-Design with Equivariant Translation[paper][code]
43. [ICLR 2023] HotProtein: A Novel Framework for Protein Thermostability Prediction and Editing[paper][code]
44. [ICLR 2023] PiFold: Toward effective and efficient protein inverse folding[paper][code]
45. [ICLR 2023] Continuous-Discrete Convolution for Geometry-Sequence Modeling in Proteins[paper][code]
46. [Nature 2023] De novo design of protein interactions with learned surface fingerprints[paper][code]
47. [Nature Communications 2023] PeSTo: parameter-free geometric deep learning for accurate prediction of protein binding interfaces [paper][code]
48. [IJCAI 2023] SemiGNN-PPI: Self-Ensembling Multi-Graph Neural Network for Efficient and Generalizable Protein-Protein Interaction Prediction [paper]
49. [Science 2023] Top-down design of protein architectures with reinforcement learning[paper][code]
50. [Nature Communications 2023] Hierarchical graph learning for protein–protein interaction [paper][code]
51. [Nature Biotechnol 2023] Efficient evolution of human antibodies from general protein language models [paper][code]
52. [Advanced Science 2023] A Multimodal Protein Representation Framework for Quantifying Transferability Across Biochemical Downstream Tasks [paper][code]
53. [Nature Communications Biology] Learning the protein language of proteome-wide protein-protein binding sites via explainable ensemble deep learning[paper][code]

Year 2022

1. [Nature Communications 2022]ProtGPT2 is a deep unsupervised language model for protein design[paper]
2. [bioRxiv 2022] Codon language embeddings provide strong signals for protein engineering [paper]
3. [Arxiv 2022] When Geometric Deep Learning Meets Pretrained Protein Language Models [paper]
4. [Arxiv 2022] Contrastive Representation Learning for 3D Protein Structures [paper]
5. [Bioinformatics 2022] Structure-aware Protein Self-supervised Learning [paper][video]
6. [KDD 2022] GBPNet: Universal Geometric Representation Learning on Protein Structures [paper][code]
7. [Arxiv 2022] Directed Weight Neural Networks for Protein Structure Representation Learning [paper]
8. [PLOS Computational Biology 2022] Fast protein structure comparison through effective representation learning with contrastive graph neural networks [paper][code]
9. [NeurIPS 2022] Exploring evolution-based &-free protein language models as protein function predictors [paper]
10. [bioRxiv 2022] High-resolution de novo structure prediction from primary sequence [paper][code]
11. [Bioinformatics 2022] ProteinBERT: A universal deep-learning model of protein sequence and function [paper][code]
12. [Communications Biology 2022] Artificial Intelligence Guided Conformational Mining of Intrinsically Disordered Proteins [paper][code]
13. [Cell Systems 2022] Evolutionary velocity with protein language models predicts evolutionary dynamics of diverse proteins [paper][code]
14. [bioRxiv 2022] Convolutions are competitive with transformers for protein sequence pretraining [paper][code]
15. [bioRxiv 2022] Masked inverse folding with sequence transfer for protein representation learning [paper][code]
16. [Nature methods 2022] Self-supervised deep learning encodes high-resolution features of protein subcellular localization [paper][code]
17. [bioRxiv 2022] Language models of protein sequences at the scale of evolution enable accurate structure prediction [paper]
18. [bioRxiv 2022] Atomic protein structure refinement using all-atom graph representations and SE(3)-equivariant graph neural networks [paper]
19. [Bioinformatics 2022] Cross-Modality and Self-Supervised Protein Embedding for Compound–Protein Affinity and Contact Prediction [paper][code]
20. [bioRxiv 2022] COLLAPSE: A representation learning framework for identification and characterization of protein structural sites [paper]
21. [bioRxiv 2022] An Analysis of Protein Language Model Embeddings for Fold Prediction [paper]
22. [ICLR 2022] OntoProtein: Protein Pretraining With Gene Ontology Embedding [paper] [code]
23. [Bioinformatics 2022] DistilProtBert: a distilled protein language model used to distinguish between real proteins and their randomly shuffled counterparts [paper]
24. [Briefings in Bioinformatics 2022] SPRoBERTa: protein embedding learning with local fragment modeling [paper]
25. [NeurIPS 2022] Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures[paper][code]
26. [NeurIPS 2022] PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding[paper][code]
27. [AAAI 2022] Self-Supervised Pre-training for Protein Embeddings Using Tertiary Structures[paper][code]
28. [ICLR 2022] Geometric Transformers for Protein Interface Contact Prediction[paper][code]
29. [ICLR 2022] Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking[paper][code]
30. [ICML 2022] Proximal Exploration for Model-guided Protein Sequence Design[paper][code]
31. [ICML 2022] Generating 3D Molecules for Target Protein Binding[paper][code]
32. [ICML 2022] Tranception: Protein Fitness Prediction with Autoregressive Transformers and Inference-time Retrieval[paper][code]
33. [Arxiv 2022] DiffBP: Generative Diffusion of 3D Molecules for Target Protein Binding[paper]

Year 2021

1. [Arxiv 2021] Pre-training co-evolutionary protein representation via a pairwise masked language model [paper]
2. [NeurIPS 2021] Language models enable zero-shot prediction of the effects of mutations on protein function [paper][code]
3. [ICML 2021] MSA Transformer [paper][code]
4. [TPAMI 2021] ProtTrans: Towards Cracking the Language of Lifes Code Through Self-Supervised Learning [paper][code]
5. [Arxiv 2021] Modeling Protein Using Large-scale Pretrain Language Model [paper][code]
6. [IEEE Access 2021] Pre-Training of Deep Bidirectional Protein Sequence Representations With Structural Information [paper][code]
7. [Bioinformatics 2021] GraphQA: protein model quality assessment using graph convolutional networks [paper][code]
8. [ICLR 2021] Intrinsic-extrinsic convolution and pooling for learning on 3d protein structures [paper][code]
9. [ICLR 2021] Learning from Protein Structure with Geometric Vector Perceptrons [paper][code]
10. [NeurIPS 2021] Multi-Scale Representation Learning on Proteins [paper][code]
11. [PNAS 2021] Neural networks to learn protein sequence–function relationships from deep mutational scanning data [paper][code]
12. [bioRxiv 2021] LM-GVP: A Generalizable Deep Learning Framework for Protein Property Prediction from Sequence and Structure [paper][code]
13. [Cell Systems 2021] Learning the protein language: Evolution, structure, and function [paper][code]
14. [Nature Communications 2021] Structure-based protein function prediction using graph convolutional networks [paper][code]
15. [KDD 2021] Geometric Graph Representation Learning on Protein Structure Prediction [paper]
16. [Arxiv 2021] Adversarial Contrastive Pre-training for Protein Sequences [paper]
17. [Emerg Top Life Sci 2021] Graph representation learning for structural proteomics [paper]
18. [Arxiv 2021] Graph Representation Learning in Biomedicine [paper]
19. [Applied Sciences 2021] GraphMS:Drug Target Prediction Using Graph Representation Learning with Substructures [paper][code]
20. [JMC 2021] InteractionGraphNet: A Novel and Efficient Deep Graph Representation Learning Framework for Accurate Protein−Ligand Interaction Predictions [paper][code]
21. [bioRxiv 2021] Self-Supervised Representation Learning of Protein Tertiary Structures (PtsRep) and Its Implications for Protein Engineering [paper]
22. [Algorithms 2021] Capturing Protein Domain Structure and Function Using Self-Supervision on Domain Architectures [paper][code]
23. [bioRxiv 2021] Combining evolutionary and assay-labelled data for protein fitness prediction [paper]
24. [Science 2021] Accurate prediction of protein structures and interactions using a three-track neural network [paper][code]
25. [Nature 2021] Highly accurate protein structure prediction with AlphaFold [paper][code]
26. [IEEE TCBB 2021] Sequence representations and their utility for predicting protein-protein interactions [paper]
27. [Bioinformatics 2021] Unsupervised protein embeddings outperform hand-crafted sequence and structure features at predicting molecular function [paper][code]
28. [CVPR 2021] Fast end-to-end learning on protein surfaces [paper]
29. [Briefings in Functional Genomics 2021] Pretraining model for biological sequence data [paper]
30. [bioRxiv 2021] Toward More General Embeddings for Protein Design: Harnessing Joint Representations of Sequence and Structure [paper]
31. [NeurIPS 2021] Neural Distance Embeddings for Biological Sequences [paper][code]
32. [Computational Biology and Chemistry 2021] Convolutional neural networks with image representation of amino acid sequences for protein function prediction [paper][code]
33. [bioRxiv 2021] Distillation of MSA Embeddings to Folded Protein Structures with Graph Transformers [paper]
34. [chemRxiv 2021] Identification of Enzymatic Active Sites with Unsupervised Language Modeling [paper]
35. [bioRxiv 2021] Deciphering the language of antibodies using self-supervised learning [paper]
36. [bioRxiv 2021] Hydrogen bonds meet self-attention: all you need for general-purpose protein structure embedding [paper]
37. [bioRxiv 2021] Improving Generalizability of Protein Sequence Models with Data Augmentations [paper]

Year 2020

1. [BCB 2020] Transforming the Language of Life: Transformer Neural Networks for Protein Prediction Tasks [paper][code]
2. [Bioinformatics 2020] UDSMProt: universal deep sequence models for protein classification [paper][code]
3. [bioRxiv 2020] Self-Supervised Contrastive Learning of Protein Representations By Mutual Information Maximization [paper]
4. [bioRxiv 2020] End-to-end multitask learning, from protein language to protein features without alignments [paper]
5. [bioRxiv 2020] Language modelling for biological sequences – curated datasets and baselines [paper][code]
6. [NeurIPS 2020] Is Transfer Learning Necessary for Protein Landscape Prediction? [paper]
7. [PNAS 2020] Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences [paper][code]
8. [Arxiv 2020] Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models [paper]
9. [Arxiv 2020] ProGen: Language Modeling for Protein Generation [paper][code]
10. [bioRxiv 2020] Evaluation of Methods for Protein Representation Learning: A Quantitative Analysis [paper]
11. [NAR Genomics and Bioinformatics 2020] Mutation effect estimation on protein–protein interactions using deep contextualized representation learning [paper][code]
12. [CSBJ 2020] Representation learning applications in biological sequence analysis [paper]
13. [bioRxiv 2020] TripletProt: Deep Representation Learning of Proteins based on Siamese Networks [paper]
14. [RCMB 2020] Evolutionary context-integrated deep sequence modeling for protein engineering [paper]
15. [Arxiv 2020] What is a meaningful representation of protein sequences? [paper][code]
16. [bioRxiv 2020] Transformer protein language models are unsupervised structure learners [paper][code]
17. [bioRxiv 2020] Deep learning-based prediction of protein structure using learned representations of multiple sequence alignments [paper]

Year before-2019

1. [Cell 2019] A High Efficient Biological Language Model for Predicting Protein–Protein Interactions [paper][code]
2. [Nature Method 2019] Unified rational protein engineering with sequence-only deep representation learning [paper][code]
3. [NeurIPS 2019] Evaluating Protein Transfer Learning with TAPE [paper][code]
4. [Nature communications 2019] Deciphering protein evolution and fitness landscapes with latent space models [paper][code]
5. [bioRxiv 2019] DeepPrime2Sec: Deep Learning for Protein Secondary Structure Prediction from the Primary Sequences [paper][code]
6. [ACS Nano 2019] A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence [paper]
7. [bioRxiv 2019] Augmenting protein network embeddings with sequence information [paper]
8. [Nature Machine Intelligence 2019] Leveraging implicit knowledge in neural networks for functional dissection and engineering of proteins. [paper][code]
9. [bioRxiv 2019] Modeling the Language of Life – Deep Learning Protein Sequences [paper][code]
10. [ICLR 2019] Learning protein sequence embeddings using information from structure [paper][code]
11. [BIBM 2019] GraphCPI: Graph Neural Representation Learning for Compound-Protein Interaction [paper]
12. [Bioinformatics 2018] Learned protein embeddings for machine learning [paper][code]
13. [Bioinformatics 2018] Deep convolutional networks for quality assessment of protein folds [paper][code]
14. [bioRxiv 2018] Deep Semantic Protein Representation for Annotation, Discovery, and Engineering [paper][code]
15. [bioRxiv 2017] Predicting Protein Binding Affinity With Word Embeddings and Recurrent Neural Networks [paper][code]
16. [Arxiv 2017] Variational auto-encoding of protein sequences [paper][code]
17. [Arxiv 2016] Distributed Representations for Biological Sequence Analysis [paper]
18. [Bioinformatics 2015] ProFET: Feature engineering captures high-level protein functions [paper]

Related Awesome

• awesome-graph-representation-learning
• awesome-graph-self-supervised-learning
• awesome-self-supervised-gnn
• awesome-self-supervised-learning-for-graphs
• awesome-AI-based-protein-design

Citation

If you find this project useful for your research, please use the following BibTeX entry.

@article{wu2022survey,
  title={A Survey on Protein Representation Learning: Retrospect and Prospect},
  author={Wu, Lirong and Huang, Yufei and Lin, Haitao and Li, Stan Z},
  journal={arXiv preprint arXiv:2301.00813},
  year={2022}
}

Contact

If you have any issue about this work, please feel free to contact me by email:

• Lirong Wu: wulirong@westlake.edu.cn
• Yufei Huang: huangyufei@westlake.edu.cn
• Bozhen Hu: hubozhen@westlake.edu.cn