使用甄嬛传剧本数据以LoRA方式训练BLOOMZ模型,实现以甄嬛口吻对不同人物进行不同回答的Bot
![image](https://private-user-images.githubusercontent.com/133947013/239542057-a1ccacf5-93cb-4a5e-b879-d20d14c58be5.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQxNzQzMjUsIm5iZiI6MTcyNDE3NDAyNSwicGF0aCI6Ii8xMzM5NDcwMTMvMjM5NTQyMDU3LWExY2NhY2Y1LTkzY2ItNGE1ZS1iODc5LWQyMGQxNGM1OGJlNS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODIwJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgyMFQxNzEzNDVaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT05N2NlMWRhNTk4MWJkYjFiYzFkMDYxYmZjMmEyMzQ0OWNiMWQ5YmVkNWRkYjY2Njc3ZjM2MDA5NWRhMzBjOTU2JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.1jWU0K35Knqyb-e5IHlj7z9-KmqytA0xdO98cXlHKjY)
![image](https://private-user-images.githubusercontent.com/133947013/239542156-030b596e-cd86-4129-abf7-4ca1d44e22c7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQxNzQzMjUsIm5iZiI6MTcyNDE3NDAyNSwicGF0aCI6Ii8xMzM5NDcwMTMvMjM5NTQyMTU2LTAzMGI1OTZlLWNkODYtNDEyOS1hYmY3LTRjYTFkNDRlMjJjNy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODIwJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgyMFQxNzEzNDVaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iNTZlNzdjZGJlMzVhNTdjNWEyNGZjN2UwMWE0MDU2YWM0NjU5Yjk5OTczNGQzNDg1OTMyMGNkMTI0MDRkMjQxJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.gZU1JBb0wIOKcU77jqiHhvKtItoMubR3ng7XOUZ0VW0)
基础模型使用BLOOMZ-7B,请从Hugging Face下载后放入base_model目录
数据来源于甄嬛传76集剧本(data_dir/scripts_data),对剧本数据进行了清洗
- 过滤出甄嬛回答的问答对
- 保留提问者身份,去除甄嬛回答身份
- 清洗内容(滤除短的无意义的问话)
- 去除对话次数少于10次的人的数据
- 人工过滤(1小时)
采用3块 32G V100 进行训练
依赖已经在requirements.txt中写明