Skip to content

Commit

Permalink
cherry pick paddlenlp 2.8 (#8323)
Browse files Browse the repository at this point in the history
* cherry pick paddlenlp 2.8

* Update README
  • Loading branch information
w5688414 committed Apr 26, 2024
1 parent ae06f0c commit a37f53d
Show file tree
Hide file tree
Showing 4 changed files with 5 additions and 3 deletions.
2 changes: 2 additions & 0 deletions README.md
Expand Up @@ -30,6 +30,8 @@

## News 📢

* **2024.04.24 [PaddleNLP v2.8](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.8.0)**:自研极致收敛的RsLoRA+算法,大幅提升PEFT训练收敛速度以及训练效果;引入高性能生成加速到RLHF PPO算法,打破 PPO 训练中生成速度瓶颈,PPO训练性能大幅领先。通用化支持 FastFNN、FusedQKV等多个大模型训练性能优化方式,大模型训练更快、更稳定。

* **2024.01.04 [PaddleNLP v2.7](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.7.1)**: 大模型体验全面升级,统一工具链大模型入口。统一预训练、精调、压缩、推理以及部署等环节的实现代码,到 `PaddleNLP/llm`目录。全新大[模型工具链文档](https://paddlenlp.readthedocs.io/zh/latest/llm/finetune.html),一站式指引用户从大模型入门到业务部署上线。全断点存储机制 Unified Checkpoint,大大提高大模型存储的通用性。高效微调升级,支持了高效微调+LoRA同时使用,支持了QLoRA等算法。

* **2023.08.15 [PaddleNLP v2.6](https://github.com/PaddlePaddle/PaddleNLP/releases/tag/v2.6.0)**: 发布[全流程大模型工具链](./llm),涵盖预训练,精调,压缩,推理以及部署等各个环节,为用户提供端到端的大模型方案和一站式的开发体验;内置[4D并行分布式Trainer](./docs/trainer.md)[高效微调算法LoRA/Prefix Tuning](./llm#33-lora), [自研INT8/INT4量化算法](./llm#6-量化)等等;全面支持[LLaMA 1/2](./llm/llama), [BLOOM](.llm/bloom), [ChatGLM 1/2](./llm/chatglm), [GLM](./llm/glm), [OPT](./llm/opt)等主流大模型
Expand Down
2 changes: 1 addition & 1 deletion paddlenlp/__init__.py
Expand Up @@ -18,7 +18,7 @@
PADDLENLP_STABLE_VERSION = "PADDLENLP_STABLE_VERSION"


__version__ = "2.7.1.post"
__version__ = "2.8.0.post"
if os.getenv(PADDLENLP_STABLE_VERSION):
__version__ = __version__.replace(".post", "")

Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Expand Up @@ -23,4 +23,4 @@ safetensors
tool_helpers
aistudio-sdk>=0.1.3
jinja2
regex
regex
2 changes: 1 addition & 1 deletion setup.py
Expand Up @@ -109,7 +109,7 @@ def show():
f.write(content)


__version__ = "2.7.1.post"
__version__ = "2.8.0.post"
if os.getenv(PADDLENLP_STABLE_VERSION):
__version__ = __version__.replace(".post", "")

Expand Down

0 comments on commit a37f53d

Please sign in to comment.