How to write a Llama-structured tiny language model from scratch? The detailed step-by-step instructions can found inside this blog, Write A Llama-structured Tiny Language Model (TLM).
Within this folder, you'll find the original Jupyter notebook for blog writing, which allows you to execute the code interactively on your own.
Contained within this folder are the resources to unlock the full capabilities of a tiny language model, along with important considerations and potential pitfalls to be mindful of when training a more deep and complex language model.