What the structure of the dataset needs to be ?

[MohamedAliRashad]

I want to finetune MPT-7B-Instruct but i don't know how ?
Is mpttune can help me with this ?

[rmihaylov]

Yes, just added it in the README

[eschaffn]

Will it work with oasst1? What kind of processing needs to be done in order to make it work?

[rmihaylov]

You have to add it in data.py as a separate class Train...

[MohamedAliRashad]

@rmihaylov How to extend the context length of MPT-7B-Instruct ?

[benam2]

@rmihaylov thanks for the great repo.
I want to fine tune the mpt-7b-instruct on both Alpaca and Openassistant. Could you please share your idea regarding this. Do I need to mix the dataset and start fine tuning? Or need to add another script to read from openassistant and start fine tuning that once fine tuning Alpaca finished?

Thanks~