使用甄嬛传剧本数据以LoRA方式训练BLOOMZ模型,实现以甄嬛口吻对不同人物进行不同回答的Bot
![image](https://private-user-images.githubusercontent.com/133947013/239542057-a1ccacf5-93cb-4a5e-b879-d20d14c58be5.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg3NTY4MjQsIm5iZiI6MTcxODc1NjUyNCwicGF0aCI6Ii8xMzM5NDcwMTMvMjM5NTQyMDU3LWExY2NhY2Y1LTkzY2ItNGE1ZS1iODc5LWQyMGQxNGM1OGJlNS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNjE5JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDYxOVQwMDIyMDRaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1jOTgwZjM0N2VhY2VjNDdhM2I0ZmU2MTY3YzM0Zjc0YzUyMmU1MzA1MDdmOTUzMjdlYTc1OWZhMGIzYTJlM2EyJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.Ae-2dEmZj2HW14GRiEewWyx6kTnYfKEIFdQfJvqhNO8)
![image](https://private-user-images.githubusercontent.com/133947013/239542156-030b596e-cd86-4129-abf7-4ca1d44e22c7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg3NTY4MjQsIm5iZiI6MTcxODc1NjUyNCwicGF0aCI6Ii8xMzM5NDcwMTMvMjM5NTQyMTU2LTAzMGI1OTZlLWNkODYtNDEyOS1hYmY3LTRjYTFkNDRlMjJjNy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNjE5JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDYxOVQwMDIyMDRaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iYmM3MjlhYjFlOGNkMGMwNzBhY2UxNjlhZjZjMTE2NDRlYzRhZWFlNjgzNTRjNTI0ODMxOGFiOGViMTIyMTY5JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.z1-w1zgvGKPZ8aOMVkwDM1bAKG2zP-Qeg7oaIFgESJU)
基础模型使用BLOOMZ-7B,请从Hugging Face下载后放入base_model目录
数据来源于甄嬛传76集剧本(data_dir/scripts_data),对剧本数据进行了清洗
- 过滤出甄嬛回答的问答对
- 保留提问者身份,去除甄嬛回答身份
- 清洗内容(滤除短的无意义的问话)
- 去除对话次数少于10次的人的数据
- 人工过滤(1小时)
采用3块 32G V100 进行训练
依赖已经在requirements.txt中写明