使用甄嬛传剧本数据以LoRA方式训练BLOOMZ模型,实现以甄嬛口吻对不同人物进行不同回答的Bot
![image](https://private-user-images.githubusercontent.com/133947013/239542057-a1ccacf5-93cb-4a5e-b879-d20d14c58be5.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg4MTg0MjQsIm5iZiI6MTcxODgxODEyNCwicGF0aCI6Ii8xMzM5NDcwMTMvMjM5NTQyMDU3LWExY2NhY2Y1LTkzY2ItNGE1ZS1iODc5LWQyMGQxNGM1OGJlNS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNjE5JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDYxOVQxNzI4NDRaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1mY2UzMmY3MGQ1NjM5NjEzZTBiNzI3OGQ0ODNmOTNlNWJiNjA4OTZkYzNjZWQ4ODJlMjllNGE5M2FmNDZkMzAyJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.JXG6evheS9Yq2iJ0GXc-71B-sfFGZxBwsdHYGUp8JDw)
![image](https://private-user-images.githubusercontent.com/133947013/239542156-030b596e-cd86-4129-abf7-4ca1d44e22c7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg4MTg0MjQsIm5iZiI6MTcxODgxODEyNCwicGF0aCI6Ii8xMzM5NDcwMTMvMjM5NTQyMTU2LTAzMGI1OTZlLWNkODYtNDEyOS1hYmY3LTRjYTFkNDRlMjJjNy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNjE5JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDYxOVQxNzI4NDRaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02NmNiYmU5MDg5ZDBjOGEzYTBmZGM3MWNkNGExNDQ2YzJjMzkwMDk4NjJjMjA4MDk1N2I4YzA2MDBjZDlkNzYwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.uqQeL5hoiWIl7dS7BkjWyhwUfagys4nk8d60jSAr2kY)
基础模型使用BLOOMZ-7B,请从Hugging Face下载后放入base_model目录
数据来源于甄嬛传76集剧本(data_dir/scripts_data),对剧本数据进行了清洗
- 过滤出甄嬛回答的问答对
- 保留提问者身份,去除甄嬛回答身份
- 清洗内容(滤除短的无意义的问话)
- 去除对话次数少于10次的人的数据
- 人工过滤(1小时)
采用3块 32G V100 进行训练
依赖已经在requirements.txt中写明