[feat] Add Accelerate SFT Trainer

[maxreciprocate]

This PR gathers supervised finetuning from these two places[1][2] under the shared api

[1] https://github.com/CarperAI/trlx/blob/main/examples/summarize_rlhf/sft/train_gptj_summarize.py
[2] https://github.com/Dahoas/reward-modeling/blob/main/reward-modeling/finetune_base.py

https://wandb.ai/sorry/trlx/reports/Add-Accelerate-SFT-Trainer-280--VmlldzozNTAyNzQ1

[cat-state]

Thanks @reciprocated , this mostly LGTM, could you update the other examples to also match the samples, rewards args instead of dataset?

[maxreciprocate]

@cat-state, I've updated those, but perhaps I was overzealous with making TRLConfig a positional and required argument, since the only example that doesn't work is simulacra.py and by extension also simulacra's colab which git cloned main, since it didn't provide a config of its own. Should I maybe revert those changes? Not sure how much have you progressed with replacing the current configuration, and how much the above still matters

[cat-state]

Maybe model path should still be the first arg? It's also whats still in the readme.

[maxreciprocate]

yeah, reverted that bit

[cat-state]

For the future config changes, we'll still need to keep the ModelConfig part around so keeping model_path makes sense for now. We can update the simulacra colab to give a model_path which should make this work?
I don't think you need to revert the changes made to the examples/config usage.

[cat-state]

okay! This LGTM now