This repository is a step-by-step learning project focused on understanding how Large Language Models are built from the ground up.
Each folder covers one stage of the process. The code in every folder is intentionally simple and explained in detail — the goal is understanding, not production use.
| # | Folder | What you learn |
|---|---|---|
| 1 | working with text data/ |
Tokenization, BPE, embeddings, positional encodings, and preparing text as tensors for a transformer |
More chapters coming soon.
Open each folder in order. Inside you will find:
- The implementation files
- A
README.mdexplaining every file and its purpose - A
main.pythat walks through the concepts step by step with printed output
Run any chapter with:
cd "<folder name>"
python main.pyThis project follows the concepts introduced in:
Build a Large Language Model (From Scratch) — Sebastian Raschka A hands-on book that walks through building a GPT-style LLM step by step, from tokenization to training. Highly recommended as a companion to this code.