Skip to content

Commit 7ea6253

Browse files
authored
【PaddleNLP No.1】 add pre-train.md (#10506)
* Paddle NLP No.1 pretrain.md * add comment to printInfo * fix according to review * fix according to review * 移除模型列表,改为参考根目录列表 * 修改模型列表措辞
1 parent 9bee5b3 commit 7ea6253

File tree

3 files changed

+224
-19
lines changed

3 files changed

+224
-19
lines changed

docs/zh/llm/docs/pretrain.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../../../llm/docs/pretrain.md

llm/README.md

Lines changed: 1 addition & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -13,27 +13,9 @@
1313
- **大模型无损量化**。大模型套件预先集成了 PaddleSlim LLM.PTQ 量化算法,以及业界广泛采用的 GPTQ 和 AWQ 的 W4量化方法,成功实现了对主流大模型的无损量化处理,显著加速了模型的推理速度。
1414
- **高性能推理**。大模型套件的高性能推理模块内置了动态插入和全环节算子融合的高级策略,这极大地提升了并行推理的速度。同时,该模块隐藏了底层技术细节,为用户提供了开箱即用的高性能并行推理功能。
1515

16-
1716
## 🛠️ 支持模型列表 🛠️
1817

19-
| Model | Pretrain | SFT | LoRA | Prefix Tuning | DPO/SimPO/ORPO/KTO | RLHF | Mergekit | Quantization | Torch convert |
20-
|----------------------------------------|----------|-----|------|---------------|----------------|------|-------|--------------|---------------|
21-
| [LLaMA](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/llama) ||||||||||
22-
| [Qwen](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/qwen) |||||| 🚧 || 🚧 ||
23-
| [Mixtral](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/mixtral) |||||| 🚧 || 🚧 | 🚧 |
24-
| [Mistral](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/mistral) |||||| 🚧 || 🚧 ||
25-
| [Baichuan/Baichuan2](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/llama) |||||| 🚧 ||||
26-
| [ChatGLM-6B](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/chatglm) ||||| 🚧 | 🚧 ||||
27-
| [ChatGLM2/ChatGLM3](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/chatglm2) |||||| 🚧 ||||
28-
| [Bloom](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/bloom) ||||| 🚧 | 🚧 ||||
29-
| [GPT-3](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/gpt-3) ||| 🚧 | 🚧 | 🚧 | 🚧 || 🚧 ||
30-
| [OPT](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/opt) |||| 🚧 | 🚧 | 🚧 || 🚧 ||
31-
| [Gemma](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/gemma) |||| 🚧 || 🚧 || 🚧 | 🚧 |
32-
| [Yuan](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/yuan) |||| 🚧 || 🚧 || 🚧 | 🚧 |
33-
34-
- ✅: Supported
35-
- 🚧: In Progress
36-
- ❌: Not Supported
18+
参考 [PaddleNLP 支持模型列表](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/README.md#模型支持)
3719

3820
## 🚀 快速开始 🚀
3921

0 commit comments

Comments
 (0)