Skip to content
Discussion options

You must be logged in to vote

the practical test i use: before choosing, sample 8 to 16 outputs per prompt from the model you'd start from and run your reward function on them.

if almost everything scores zero, go SFT first. GRPO learns from the differences between outputs in the same group, so when the whole group fails the same way there's no signal and you burn compute on nothing. small models hit this a lot on new formats.

if a decent share passes, somewhere around 10 to 70 percent, GRPO directly can work, because there's real variation to push on.

with a good instruction/response set like you describe, SFT then GRPO is usually the safe order. SFT gets the format and the basic task right, GRPO then sharpens what y…

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by Advait251206
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants