民间版中文羊驼模型（Plus）v3.1

[ymcui]

本次是针对Plus版本的更新，主要聚焦于以下几方面：

🚀 发布中文LLaMA-Plus, Alpaca-Plus 13B版本

发布中文LLaMA-Plus, Alpaca-Plus 13B版本，改进点如下：

• 相比基础版进一步扩充了训练数据，其中LLaMA扩充至120G文本，Alpaca扩充至4.3M指令数据
• 重点增加了科学领域数据，涵盖：物理、化学、生物、医学、地球科学等
• Alpaca训练时采用了更大的rank，相比基础版具有更低的验证集损失
• Alpaca评测结果：13B获得74.3分，Plus-7B获得78.2分，Plus-13B获得80.8分，具体评测结果请参考效果评测
  • 多轮回复长度相比旧模型提升明显（可适当增大温度系数）
  • 知识问答、写作、翻译等方面效果显著提升

⚠️ 注意：Alpaca-Plus-7B/13B的模型合并流程稍有不同，请参考合并教程。

💯 完善了量化评测

llama.cpp提供了多种量化方案，本次针对Alpaca-Plus-7B以及Alpaca-Plus-13B进行了综合评测。

• 默认的量化方法为q4_0，虽然速度最快但损失也是最大的，其余方法各有利弊，按实际情况选择
• 机器资源够用且对速度要求不是那么苛刻的情况下可以使用q8_0，接近F16模型的效果
• 需要注意的是F16以及q8_0并不会因为增加线程数而提高太多速度
• 线程数-t与物理核心数一致时速度最快，超过之后速度反而变慢

详细结果见关于量化方法选择及推理速度。

For English release note, please refer to Discussion.

---

This discussion was created from the release 民间版中文羊驼模型（Plus）v3.1.

[ymcui]

English Release Note v3.1

This update is for the Plus version, focusing on the following aspects:

🚀 Release of Chinese LLaMA-Plus, Alpaca-Plus 13B version

The Chinese LLaMA-Plus and Alpaca-Plus 13B versions have been released, with the following improvements:

• Further expanded training data compared to the basic version, with LLaMA expanded to 120G of text and Alpaca expanded to 4.25M instruction data
• Emphasis on adding scientific field data covering physics, chemistry, biology, medicine, earth sciences, etc.
• A larger rank was used for Alpaca training, resulting in a lower validation set loss compared to the basic version
• Alpaca evaluation results: 13B scored 74.3, Plus-7B scored 78.2, Plus-13B scored 80.8. For detailed evaluation results, please refer to performance evaluation
  • Significant improvement in multi-turn reply length compared to the old model (can moderately increase temperature coefficient)
  • Significant improvement in knowledge Q&A, writing, translation, and other aspects

⚠️ Note: The model merging process for Alpaca-Plus-7B/13B is slightly different. Please refer to the merging model tutorial.

💯 Improved Quantitative Evaluation

llama.cpp provides multiple quantization schemes. Comprehensive evaluations were performed for Alpaca-Plus-7B and Alpaca-Plus-13B.

• The default quantization method is q4_0, which has the fastest speed but also has a larger loss. Other methods have their pros and cons, so choose according to your actual situation
• If you have sufficient machine resources and do not require high speed, you can use q8_0, which is close to the F16 model's performance
• It should be noted that F16 and q8_0 do not significantly increase speed with an increase in the number of threads
• The fastest speed is achieved when the number of threads -t is equal to the number of physical cores, and the speed decreases when it is exceeded

For detailed results, see About Quantization Method Selection and Inference Speed.

[yangjianxin1]

请问Chinese-LLaMA-Plus-7B预训练时，采用了更大的rank，请问lora_rank是多少？

[ymcui]

LLaMA-Plus并没有增加rank，还是8；Alpaca-Plus增加到64。
过些天会更新技术报告，里面会介绍详细内容。