Add example to fine-tune StarCoder for chat-based applications #17
Conversation
There was a problem hiding this comment.
Looks good overall. Left only a few minor comments.
I am mostly wondering if we could have used the SFTTrainer to reduce a bit the amount of code needed for train.py or run a simple preprocessing script that pushes the formated dataset to the hub for the script to reduce a bit the amount of code. Don't think we need to change anything now but I think we should provide a bit more utility to be able to make such examples more concise.
| # Fine-tuning StarCoder for chat-based applications | ||
|
|
||
| This is an educational example to fine-tune `StarCoderBase` on a corpus of multi-turn dialogues and thus create a coding assistant that is chatty and helpful. Check out our [blog post](ADD LINK) for more details. | ||
|
|
There was a problem hiding this comment.
maybe some examples of what the model can do after tuning would be cool to show here.
Yes I agree this would be nice to do & I'll fix this in a follow-up PR (I ran out of time to test everything with the new API) |
TODO