Local LLM fine‐tuning Blueprint

Based on our experiments in the "Fine‐Tuning Experiments" section, the most promising approach is Reinforcement Learning. The issue we are still facing is the instability in training when training continues for longer than one epoch. For this issue, we suggest 2 approaches.

1. Increasing the stability through the KL divergence. The KL divergence loss constrains the model so that it doesn't diverge too far from it's initial trajectory. This should help with the instability issue.
2. Using a better starting model. With Qwen2.5-14B we managed to double the performance after one epoch of training. Qwen3-Coder-30B-A3B-Instruct achieves over 50% performance before training, so there is the potential for this model that a single epoch will be sufficient for achieving the desired performance. Also, a plethora of newer and probably better open-source LLMs have been released since we did our evaluation of open-source LLMs. So there are probably newer and better LLMs that can be used as a starting point for the fine-tuning.