Skip to content

waleking/Awesome-Transformers

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 

Repository files navigation

Awesome-Transformers

A list of transformers

  1. Attention Is All You Need, NIPS 2017 (paper) originial paper

  2. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context, ACL 2019 (paper) (pytorch & tensorflow code) segment-level recurrence, and relative position encoding

  3. COMET: Commonsense Transformers for Automatic Knowledge Graph Construction, ACL 2019 (paper) (pytorch code) used for KG's tuples

  4. Adaptive Attention Span in Transformers, ACL 2019 (paper) (pytorch code) adaptive attention span

  5. XLNet: Generalized Autoregressive Pretraining for Language Understanding, arxiv 2019 (paper) (tensorflow code) permutation language model

  6. Syntactically Supervised Transformers for Faster Neural Machine Translation, ACL 2019 (paper) (pytorch code) non-autoregressive decoding

  7. Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction, ACL 2019 (paper) (pytorch code) relation extraction

  8. Learning Deep Transformer Models for Machine Translation, ACL 2019 (paper) (pytorch code) residual

  9. Large Batch Optimization for Deep Learning: Training BERT in 76 Minutes, arxiv 2019 (paper) distributed computation

  10. Universal Transformers, ICLR 2019 (paper) (tensorflow code) recurrent Transformer blocks

  11. Lattice Transformer for Speech Translation, ACL 2019 (paper) on lattice (directed acyclic graph, a.k.a., DAG)

  12. ERNIE: Enhanced Language Representation with Informative Entities, ACL 2019 (paper) (code)

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published