Transformers_Models Transformers의 다양한 언어 모델(BERT, GPT 제외)을 공부하기 위한 repository이며, 최대한 논문을 통해 공부하고자 합니다. LLM (Large Language Models)의 발전 역사 Autoencoding (Encoder) 기반 RoBERTa: A Robustly Optimized BERT Pretraining Approach ALBERT: A Lite BERT for Self-supervised Learning of Language Representations ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators XLM: Cross-lingual Language Model Pretraining AutoRegressive (Decoder) 기반 XLNet: Generalized Autoregressive Pretraining for Language Understanding LLaMA: Open and Efficient Foundation Language Models Sequence-to-Sequence (Encoder-Decoder) 기반 BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer 참고할만한 사이트 Mooler0410's LLMsPracticalGuide