gpt-3-chinese-chatbot

A GPT-3 based Chinese chatbot finetuned by LCCC dataset

Fine-tuning Methods

1. Traditional Fine-tuning with LoRA

I fine-tuned the last nn.Linear layer with PEFT = Huggingface+LoRA and the preprocessing method in this repo

2. Instruction Fine-tuning

To be done