v1.0: 发布Chinese-LLaMA-2-7B, Chinese-Alpaca-2-7B

[ymcui]

很高兴地向大家宣布中文LLaMA-2-7B、Alpaca-2-7B大模型已正式发布。

• Chinese-LLaMA-2-7B：基座模型，使用120G语料增量训练，与一期Plus系列模型一致
• Chinese-Alpaca-2-7B：指令/chat模型，在Chinese-LLaMA-2-7B的基础上进一步通过指令精调（5M条指令）获得

相比一期项目的主要特点

📖 经过优化的中文词表

• 在一期项目中，我们针对一代LLaMA模型的32K词表扩展了中文字词（LLaMA：49953，Alpaca：49954），以期进一步提升模型对中文文本的编解码效率
• 在本项目中，我们重新设计了新词表（大小：55296），进一步提升了中文字词的覆盖程度，同时统一了LLaMA/Alpaca的词表，避免了因混用词表带来的问题

⚡ 基于FlashAttention-2的高效注意力

• FlashAttention-2是高效注意力机制的一种实现，相比其一代技术具有更快的速度和更优化的显存占用
• 当上下文长度更长时，为了避免显存爆炸式的增长，使用此类高效注意力技术尤为重要
• 本项目的所有模型均使用了FlashAttention-2技术进行训练

🚄 基于NTK的自适应上下文扩展技术

• 在一期项目中，我们实现了基于NTK的上下文扩展技术，可在不继续训练模型的情况下支持更长的上下文
• 在上述基础上，我们进一步设计了方便的自适应经验公式，无需针对不同的上下文长度设置相应超参
• 本项目模型原生支持4K上下文，利用上述技术可扩展至12K，并最高支持扩展至18K+（精度有一定损失）

🤖 简化的中英双语系统提示语

• 在一期项目中，中文Alpaca系列模型使用了Stanford Alpaca的指令模板和系统提示语
• 初步实验发现，Llama-2-Chat系列模型的默认系统提示语未能带来统计显著的性能提升，且其内容过于冗长
• 本项目中的Alpaca-2系列模型简化了系统提示语，同时遵循Llama-2-Chat指令模板，以便更好地适配相关生态

模型效果

主观评测

为了更加直观地了解模型的生成效果，本项目仿照Fastchat Chatbot Arena推出了模型在线对战平台，可浏览和评测模型回复质量。对战平台提供了胜率、Elo评分等评测指标，并且可以查看两两模型的对战胜率等结果。测试模型包括：

• 一期模型：Chinese-Alpaca-Pro系列（7B/13B/33B）、Chinese-Alpaca-Plus系列（7B/13B/33B）
• 二期模型（本项目）：Chinese-Alpaca-2（7B）

📊 模型在线对战：http://chinese-alpaca-arena.ymcui.com/

客观评测

客观评测选择了C-Eval进行评价，结果如下表。可以看到二代模型显著优于一代模型，且部分指标上甚至超过13B版本。

LLaMA系列模型之间对比：

模型	Valid (zero-shot)	Valid (5-shot)	Test (zero-shot)	Test (5-shot)
Chinese-LLaMA-2-7B	28.2	36.0	30.3	34.2
Chinese-LLaMA-Plus-13B	27.3	34.0	27.8	33.3
Chinese-LLaMA-Plus-7B	27.3	28.3	26.9	28.4

Alpaca系列模型之间对比：

模型	Valid (zero-shot)	Valid (5-shot)	Test (zero-shot)	Test (5-shot)
Chinese-Alpaca-2-7B	41.3	42.9	40.3	39.5
Chinese-Alpaca-Plus-13B	43.3	42.4	41.5	39.9
Chinese-Alpaca-Plus-7B	36.7	32.9	36.4	32.3

For English release note, please refer to Discussion.

---

This discussion was created from the release v1.0: 发布Chinese-LLaMA-2-7B, Chinese-Alpaca-2-7B.

[ymcui]

Release Note for v1.0

This part of the text is simply HTML code for formatting purposes. It centers an image (with an alternate text "newlogo") from the provided URL and sets its width to 700 pixels.

---

We are pleased to announce that the Chinese LLaMA-2-7B and Alpaca-2-7B models have been officially released.

• Chinese-LLaMA-2-7B: This is a pedestal model that uses 120GB of corpus for incremental training and is consistent with the first series of Plus models.
• Chinese-Alpaca-2-7B: This is a command/chat model, which is further fine-tuned on the basis of Chinese-LLaMA-2-7B through command tuning (5 million commands).

---

Main features compared to the first project

📖 Optimized Chinese Vocabulary

• In the first generation of the project, we expanded Chinese words and characters for the first-generation Chinese LLaMA model (LLaMA: 49953, Alpaca: 49954) to improve the model's encoding and decoding efficiency of Chinese texts.
• In this project, we redesigned the new vocabulary (size: 55296) to further improve the coverage of Chinese words and characters. We also unified the LLaMA/Alpaca vocabulary to avoid problems due to mixed use.

⚡ Efficient FlashAttention-2

• FlashAttention-2 is an implementation of efficient attention mechanisms, offering faster speed and optimized memory usage compared to its first-generation.
• When the context length is longer, using efficient attention technology is essential to prevent explosive growth in memory usage.

🚄 Adaptive Context Extension based on NTK

• In the first generation of the project, we implemented the context extension based on NTK, which can support longer contexts without further training the model.
• Based on the above, we further designed a convenient adaptive empirical formula that does not require manually setting corresponding hyperparameters for different context lengths.
• The models in this project natively support a 4K context, which can be extended to 12K with the above technology and up to 18K+ at the expense of some accuracy loss.

🤖 Simplified Bilingual System Prompt

• In the first generation of the project, we use Stanford Alpaca template for our Chinese Alpaca models
• Through preliminary experiments, we found that the lengthy system prompt by Llama-2-Chat is not as effective as a simple one
• We use a very simple system prompt while keeping the Llama-2-Chat template to better adapt to relevant ecosystems

---

Model Performance

Subjective evaluation

In order to understand the model's generation effects more intuitively, this project has launched an online model battle platform in imitation of the Fastchat Chatbot Arena. You can browse and evaluate the quality of model responses. The battle platform provides evaluation indicators such as win rate and Elo rating, and you can also view the win rate results of pairwise model battles. The tested models include:

• First generation models: Chinese-Alpaca-Pro series (7B/13B/33B), Chinese-Alpaca-Plus series (7B/13B/33B)
• Second generation models (this project): Chinese-Alpaca-2 (7B)

📊 Model Online Battle: http://chinese-alpaca-arena.ymcui.com/

Objective evaluation

Objective evaluation was conducted using C-Eval. The results are shown in the following table. It can be seen that the second-generation models are significantly better than the first-generation models, and even surpass the 13B version on some indicators.

Comparison between LLaMA series models:

Model	Valid (zero-shot)	Valid (5-shot)	Test (zero-shot)	Test (5-shot)
Chinese-LLaMA-2-7B	28.2	36.0	30.3	34.2
Chinese-LLaMA-Plus-13B	27.3	34.0	27.8	33.3
Chinese-LLaMA-Plus-7B	27.3	28.3	26.9	28.4

Comparison between Alpaca series models:

Model	Valid (zero-shot)	Valid (5-shot)	Test (zero-shot)	Test (5-shot)
Chinese-Alpaca-2-7B	41.3	42.9	40.3	39.5
Chinese-Alpaca-Plus-13B	43.3	42.4	41.5	39.9
Chinese-Alpaca-Plus-7B	36.7	32.9	36.4	32.3

[johnbuddhaking]

请问新的词表在哪里下载啊？

[ymcui]

如果你只要tokenizer file，去huggingface找对应文件下载：https://huggingface.co/ziqingyang/chinese-llama-2-lora-7b

[johnbuddhaking]

Chinese-LLaMA-Alpaca项目，有scripts目录下有一个merge_tokenizer目录，目录下有chinese_sp.model和merge_tokenizers.py来把中文词表合并到llama词表里。
在Chinese-LLaMA-Alpaca-2项目里，词表部分直接去huggingface加载模型tokenizer吗？

[ymcui]

词表不需要你自己合并，tokenizer.model文件就是已经合并好的词表。
一期项目只是给大家一个演示，如何把自己做好的词表和原版LLaMA词表进行合并。

[johnbuddhaking]

收到，多谢啦