Fine-tune ChatGPT: what are the content and structure of the prompts dataset?

[spig95]

Hello everyone,

I have read this article that shows how Colossal-AI can be used to train ChatGPT. It motivated me to give a try to Colossal-AI to make a virtual assistant on a specific topic, about which I have a lot of text input that I would like to use to fine-tune the training of ChatGPT.

After some research, I came across the train_prompts.py script. If my understanding is correct, I can use this script to train ChatGPT (or to fine tune a pretrained model). However, it is unclear to me how the data/prompts should look like and how they are structured.

In particular, I found this line of code: dataset = pd.read_csv(args.prompt_path)['prompt']. It loads the dataset used in trainer.fit(dataset, ... ).

Could someone kindly tell me what the csv file at args.prompt_path contains and how it should be structured? If it is written somewhere, I was not able to find the documentation on this, and a link to it would suffice!

Thanks for taking the time to read through my question!

[binmakeswell]

Hi @spig95 You can use awesome-chatgpt-prompts as example dataset. It is a small dataset with hundreds of prompts.
https://github.com/hpcaitech/ColossalAI/tree/main/applications/ChatGPT/examples#train-with-real-prompt-data-stage-3

[spig95]

Thank you very much, now it is clear how prompts look like. From my understanding, they are prompts and do not contain knowledge about a specific domain. Therefore, I have a follow-up question: is it possible to fine-tune ChatGPT adding knowledge related to a specific topic?

Let's assume that my starting point is a lot of text containing information related to a topic, but not in the form of prompts. And my end-goal is to train ChatGPT as a query-answering chatbot on the specific topic. If it's possible to fine tune chatGPT for this taks, does anyone have any hints on how to proceed?

As far as I understood, the train_prompts.py script performs the 'stage 3' of training. It would not be as straightforward as I hoped but, to my understanding, to achieve my target I would need to replicate also stage 1 and stage 2. This sounds unfeasible, maybe I am missing some point, or maybe relying ChatGPT to achieve my task is not a good idea.

[raihan0824]

hi, do you have an answer to this question? I have the same goal as you

[spig95]

I am not 100% sure, but reading the paper about instructGPT, I think there is no easy way that does not involve manual work/labeling to re-train chatGPT including new knowledge, at least relying on ColossalAI solutions. What ColossalAI offers is a way to retrain a model that should behave similarly to ChatGPT starting from GPT.