My Learning Notes on the book "Build a Large Language Model (From Scratch)" by Professor Raschka

My Greatest Thanks to Professor Raschka, the author of this book.

Useful Links

• The GitHub Repository from the Author
• The Author's Blog
• The Author's Free Courses

My Current Progress

• ✅ Set Up (Using uv)
• ✅ Chapter 1: Understanding LLM
• ✅ Appendix A: Introduction to PyTorch
• ✅ Chapter 2: Planning + Preparing Text
• ✅ Chapter 3: Attention Mechanism
• 🧠 Chapter 4: Coding a LLM
• ⏳ Chapter 5: LLM Pretraining
• ⏳ Chapter 6: Classification Fine-tuning
• ⏳ Chapter 7: Instruction Fune-tuning
• ⏳ Appendix D: Enhance the Training Function
• ⏳ Appendix E: Fune-tuning with LoRA

Note

• Run source .venv/bin/activate to activate the virtual environment.
• I am using a MacBook Air A1466 from 2015. It installed PyTorch version 2.2.2 for Intel macOS. My laptop does not have a compatible GPU, and it does not have an Apple Silicon chip.