toy-transformer-models

A project to elaborate on transformer-based models (e.g., BERT, BART, etc), with simple codes from scratch and useful links to other resources.

Tasks

I test several basic but interesting tasks with the vanilla transformer model

Task	Notebook	Comment
copy	build_a_transformer_from_scratch.ipynb	Output the same string of numbers