1.5B coder model reproduction

[yesiam-png]

Hi, thanks for your excellent work! When I tried to train with Qwen-2.5-Coder 1.5B using your 14b_16k recipe, I found that the minimum response length is very small, and the response length & LCB score significantly decrease after certain steps. Could you help me understand the possible reasons? If possible, could you share a recipe for your 1.5B deepseek-qwen coder model?

[michaelzhiluo]

You should try this on top of the Deepscaler-1.5b model, doing it directly from qwen-2.5-coder does not work in my experience.