Repository navigation
How to categorize all stages of building a LLM #258
|
Hi all, I am currently reading the first chapter of this book, and I am confused about how to categorize the different stages of LLM training. We have pre-training and post-training stages. From the book, it seems that post-training only includes instruction tuning and preference tuning, while reasoning is not considered one of them like in Figure 1.5. However, the reasoning methods introduced in the book, such as GRPO or some other RL methods, are commonly categorized as post-training methods, though inference-time scaling is not a "training" method, so it should not be in post-training stage. My question is: how should these stages and methods be categorized correctly? |
Replies: 1 comment 3 replies
|
Hi there, I think you are referring to figure 1.3? That's a figure that depicts the classic pipeline, i.e.,
Figure 1.5 then adds on the reasoning-related methods. |
Ah, so the introduction chapter of this book is for people who are new to reasoning models. I.e., it considers everything before 2025's DeepSeek R1 as "conventional" LLM / "conventional" training stages. And then yes, in these conventional post-training steps, there are no reasoning methods.
This is to explain the difference between "conventional" LLMs and reasoning LLMs (the latter are the topic of this book then)