民间版中文羊驼模型 v4.1

[ymcui]

本版本以功能性更新为主，包括更新技术报告、添加C-Eval预测脚本、添加低资源模型转换脚本等。

📚 更新技术报告（arXiv-v2）

• 更新模型构建的相关描述，添加Plus版本模型训练参数等
• 生成效果评测部分替换为Alpaca-Plus-7B、Alpaca-Plus-13B、Alpaca-33B对比
• 添加C-Eval完整评测结果，包括原版LLaMA以及本项目所有模型的结果与分析
• 添加不同量化层级（2-bit ~ 8-bit, FP16）的效果对比和分析
• 详细技术报告请查阅：https://arxiv.org/abs/2304.08177

🔬 添加C-Eval预测脚本（#601）

• 添加了论文中使用的C-Eval预测脚本，轻松复现论文中相关模型效果
• 提供了基于上述脚本的Colab notebook，用户可参考相关运行过程
• 详细的介绍请参见Wiki教程

🚀 添加低资源模型转换脚本（#608）

• 添加了新版低资源模型转换脚本merge_llama_with_chinese_lora_low_mem.py，大幅降低内存消耗
• 7B需要15GB，13B需要18GB，33B需要22GB（不同机器可能略有不同）
• 使用方法与旧脚本一致，后续将作为主要使用的模型合并脚本

其他更新和修复

• 修复了预训练脚本中文件名的一处错误（Fixed data duplication issue caused by special pattern in file names #540），Contribution by @alkaideemo
• 修复了模型保存问题，支持LoRA权重和tokenizer的保存 （Support lora weight and tokenizer saving in training #555）
• 在预训练和精调脚本中增加了检查项，避免后续模型合并出现异常问题（Add tokenizer assertion #595）

For English release note, please refer to Discussion.

---

This discussion was created from the release 民间版中文羊驼模型 v4.1.

[ymcui]

English Release Note v4.1

This version mainly includes functional updates, including updating technical reports, adding C-Eval prediction scripts, adding low-resource model conversion scripts, etc.

📚 Updated Technical Report (arXiv-v2)

• Updated the related descriptions of model construction, added training parameters for the Plus version model, etc.
• The generation effect evaluation part was replaced with the comparison of Alpaca-Plus-7B, Alpaca-Plus-13B, Alpaca-33B
• Added the complete C-Eval evaluation results, including the original version of LLaMa and the results and analysis of all models in this project
• Added the comparison and analysis of the effects of different quantization levels (2-bit ~ 8-bit, FP16)
• For a detailed technical report, please check: https://arxiv.org/abs/2304.08177

🔬 Added C-Eval Prediction Script (#601)

• Added the C-Eval prediction script used in the paper, easily reproduce the related model effects in the paper
• Provided a Colab notebook based on the above script, users can refer to the related running process
• For a detailed introduction, please refer to the Wiki tutorial

🚀 Added Low-Resource Model Conversion Script (#608)

• Added a new low-resource model conversion script merge_llama_with_chinese_lora_low_mem.py, which significantly reduces memory consumption
• 7B requires 15GB, 13B requires 18GB, 33B requires 22GB (may slightly vary on different machines)
• The usage method is consistent with the old script and will be used as the main model merge script in the future

Other Updates and Fixes

• Fixed a filename error in the pre-training script (Fixed data duplication issue caused by special pattern in file names #540), Contribution by @alkaideemo
• Fixed the model saving issue, supporting the saving of LoRA weights and tokenizer (Support lora weight and tokenizer saving in training #555)
• Added check items in the pre-training and fine-tuning scripts to avoid subsequent model merge problems (Add tokenizer assertion #595)