68 Pull requests merged by 34 people

• [Doc] add xpu doc and update llama xpu scripts

  #10148 merged Mar 21, 2025

• [LLM] fix pir problem

  #10235 merged Mar 21, 2025

• Setting max_seq_length to 4097 when using mtp

  #10230 merged Mar 20, 2025

• Fix arch issue(CUDA 222) in hopper arch

  #10209 merged Mar 20, 2025

• support first_k_dense_replace for dualpipe

  #10226 merged Mar 20, 2025

• [AutoParallel] Add_llama_dpo_auto_parallel_intermediate

  #10044 merged Mar 20, 2025

• [inference cpu]cpu install fix

  #10218 merged Mar 20, 2025

• Default config for dsv3

  #10225 merged Mar 20, 2025

• Optimize gpu memory usage for UnPermute

  #10217 merged Mar 20, 2025

• 【Bug Fix】Fix triton_moe shape error

  #10216 merged Mar 20, 2025

• fix triton moe bugs

  #10222 merged Mar 20, 2025

• support overlap for fusion moe

  #10220 merged Mar 20, 2025

• Remove dependency on FP8 module when using BF16 training mode

  #10219 merged Mar 20, 2025

• Fix typos in multiple files

  #10115 merged Mar 20, 2025

• [LLM] Add Minimax-Text-01

  #9893 merged Mar 20, 2025

• Spliting fusion moe, fix memlory leakage of fusion moe

  #10192 merged Mar 20, 2025

• Save config file for inference in automatic parallel training

  #10175 merged Mar 20, 2025

• Revert "optimize ds3 attention impl"

  #10208 merged Mar 19, 2025

• Fix bug causing incorrect shape propagation in permute gradient

  #10213 merged Mar 19, 2025

• [revert]Add timer for deepep

  #10212 merged Mar 19, 2025

• Refine kitchen use

  #10207 merged Mar 19, 2025

• Add timer for dsv3

  #10211 merged Mar 19, 2025

• Split q for mla to support group_size=128

  #10168 merged Mar 19, 2025

• Merge MoEFlexTokenLayer to MoELayer

  #10205 merged Mar 19, 2025

• Optimize attention output linear fp8 memory

  #10204 merged Mar 19, 2025

• Add several fused quanted ops in support of FP8 training

  #10202 merged Mar 19, 2025

• Adding TokenDispatcherUtils, for MoE token dispatch and regroup, in preparation for GroupedGEMM using in training.

  #10190 merged Mar 19, 2025

• optimize ds3 attention impl

  #10200 merged Mar 19, 2025

• [CI]update upload for ci

  #10135 merged Mar 19, 2025

• [CI]fix bug for deepseek_data & paddleocr2.10_install & paddlenlp_ops whl

  #10183 merged Mar 19, 2025

• Fix dequant bug and dw compute bug

  #10193 merged Mar 19, 2025

• [LLM] add port check and fix bugs

  #10195 merged Mar 19, 2025

• refine fp8 Linear

  #10191 merged Mar 19, 2025

• Delete dead links

  #10188 merged Mar 19, 2025

• Refine fuse ffn fp8 func

  #10189 merged Mar 19, 2025

• Use Fp8 dispatch for moe layer

  #10165 merged Mar 18, 2025

• [BugFix]Fix expert number in DSV3

  #10184 merged Mar 18, 2025

• use training_args to contorl split input

  #9944 merged Mar 18, 2025

• update_docs

  #10180 merged Mar 18, 2025

• modify gpt3-13b config

  #10181 merged Mar 18, 2025

• [AutoParallel] Fix per_device_batch_size in Qwen2VL

  #10170 merged Mar 18, 2025

• update docs

  #10178 merged Mar 18, 2025

• Add auto_parallel_sync_shared_params pass flag

  #10057 merged Mar 18, 2025

• Open recompute for GPT benchmark

  #10160 merged Mar 18, 2025

• Opt dualpipe overlap

  #10173 merged Mar 18, 2025

• Add Contrastive Learning

  #10097 merged Mar 18, 2025

• support deepseek on xpu

  #10060 merged Mar 18, 2025

• Add XLMRoberta in Embedding Train

  #10074 merged Mar 18, 2025

• 【Inference】Migrate MoE Kernel from Paddle Inner

  #10063 merged Mar 18, 2025

• Replace index_select with gather to avoid randomness in controlling speed reduction

  #10158 merged Mar 18, 2025

• [predict] fix ddp bug for predict benchmark.

  #10171 merged Mar 18, 2025

• Add flag DSV3_USE_FP8_GEMM

  #10133 merged Mar 18, 2025

• Fix the bug in tiny_fused_bloom::test_wint8 with pir mode

  #9307 merged Mar 18, 2025

• [CI] fix install issue for requirements-dev.txt

  #10162 merged Mar 18, 2025

• [News] DeepSeek-R1满血版单机部署实测

  #10167 merged Mar 17, 2025

• [Application] Distill and finetune LLM

  #10061 merged Mar 17, 2025

• [Hot Fix] hotfix for param decoder_chunk_size_cpu

  #10111 merged Mar 17, 2025

• optimize target_lists_for_model

  #10139 merged Mar 17, 2025

• [Tools] add tool safetensors_filter

  #10140 merged Mar 17, 2025

• [AutoParallel] Add ci for deepseek-v3

  #10064 merged Mar 17, 2025

• [Pipeline Overlap] Adapt to fix cpu stall

  #10154 merged Mar 17, 2025

• [LLM] add port check and fix bugs

  #10153 merged Mar 17, 2025

• [Hot Fix] hotfix for param decoder_chunk_size_cpu

  #10144 merged Mar 17, 2025

• [AutoParallel]: fix bug because of renaming file

  #10132 merged Mar 17, 2025

• support reshard attention mask in intermediate_api

  #10137 merged Mar 17, 2025

• Delete fp8 gemm warning

  #10134 merged Mar 17, 2025

• Add pid and start timestamp to metrics dump

  #10142 merged Mar 16, 2025

• Fix cpu stall in permute and unpermute

  #10147 merged Mar 16, 2025

33 Pull requests opened by 25 people

• Run banchmark gpt config

  #10163 opened Mar 17, 2025

• Added regroup padded op to perform token regroup using 1D expert_idx with max-token-per-expert padding

  #10164 opened Mar 17, 2025

• Add custom op for Tokens zip and unzip, in preparation of using groupgemm

  #10169 opened Mar 17, 2025

• [Infer] Change groupwise weight quant from cpu to gpu for deepseek_v2 model

  #10174 opened Mar 18, 2025

• [LLM] remove distutils in pdc sdk

  #10176 opened Mar 18, 2025

• [LLM] remove distutils in pdc sdk

  #10179 opened Mar 18, 2025

• support deepseek_vl2 inference

  #10182 opened Mar 18, 2025

• 【Inference Fix BUG】fix mix model save_output

  #10185 opened Mar 18, 2025

• fix rng_state checkpoint error when tp8

  #10187 opened Mar 19, 2025

• [Serving] close dyinput

  #10197 opened Mar 19, 2025

• update_llama_conf_cinn_0319

  #10198 opened Mar 19, 2025

• 【Inference Optimize】Paddle supports MOE model EP parallel

  #10201 opened Mar 19, 2025

• Integrate DeepGEMM into fused_moe op

  #10210 opened Mar 19, 2025

• [fp8]zip and group gemm

  #10214 opened Mar 19, 2025

• optimize expert memory

  #10215 opened Mar 20, 2025

• [Docs] Refactor docs.

  #10223 opened Mar 20, 2025

• Fix typos in multiple files

  #10224 opened Mar 20, 2025

• 【DCU】support dcu paged attention

  #10227 opened Mar 20, 2025

• Optimize upermutation memory

  #10228 opened Mar 20, 2025

• [DCU]support dcu PagedAttention prefix in compute_mla_absorb

  #10229 opened Mar 20, 2025

• Add fp8 support for regroup op.

  #10231 opened Mar 20, 2025

• [WIP] Add deepep timer.

  #10232 opened Mar 20, 2025

• [DCU]add op reshape_and_cache for dcu paged_attention

  #10233 opened Mar 20, 2025

• [AutoParallel] Add auto parallel DPO docs

  #10234 opened Mar 20, 2025

• [BugFix] fix src_length when --benchmark

  #10237 opened Mar 21, 2025

• [LLM]fix dpop loss

  #10239 opened Mar 21, 2025

• Optimize the Model List Docs

  #10240 opened Mar 21, 2025

• cherrpick bug fix dpop & optrimizer

  #10241 opened Mar 21, 2025

• [CI] reopen test for zero-dim tensor

  #10242 opened Mar 21, 2025

• MTP Linear support fp8 gemm

  #10244 opened Mar 21, 2025

• Fix load torch.

  #10246 opened Mar 21, 2025

• 【Infer】MLA matrix absorption separation

  #10249 opened Mar 21, 2025

• Adding Tdu FP32 type support

  #10250 opened Mar 21, 2025

63 Issues closed by 7 people

• 运行pp-uie推理失败

  #10243 closed Mar 21, 2025

• 昇腾推理报 ACL error, the error code is : 500002

  #10194 closed Mar 21, 2025

• [Bug]: 静态图推理predic benchmark报错

  #10149 closed Mar 21, 2025

• pp-uie 自定义代码调用加载llm模块出错

  #10186 closed Mar 19, 2025

• [Bug]: ernie-3.0-nano-zh tokenizer缺一个token

  #6429 closed Mar 18, 2025

• [Question]: 没有找到处理 CHID 和 C3 数据集的脚本

  #6164 closed Mar 18, 2025

• [Question]: 评估脚本报错 evaluate.py

  #6022 closed Mar 18, 2025

• 你们训练GPT中文用的什么数据集啊

  #6079 closed Mar 18, 2025

• [Question]: stablediffusion save_pretrained 设置to_diffusers为True无效

  #6014 closed Mar 18, 2025

• [Question]: uiex抽取速度问题

  #6013 closed Mar 18, 2025

• [Question]: paddleocr训练时发出调用Python对象时超过的最大递归深度的报错

  #6006 closed Mar 18, 2025

• [Bug]: FileNotFoundError: configuration file<config.json> or <model_config.json> not found

  #5927 closed Mar 18, 2025

• [Question]: ernie-3.0 训练数据集和精度咨询

  #5897 closed Mar 18, 2025

• [Question]: 微调后没有模型权重文件生成

  #5851 closed Mar 18, 2025

• [Bug]: 运行python client 出错

  #5850 closed Mar 18, 2025

• [Bug]: taskFlow _prepare_onnx_mode 创建onnx文件夹错误

  #5839 closed Mar 18, 2025

• [Question]:

  #5746 closed Mar 18, 2025

• [Question]: 在飞桨平台finetune之后模型，本地加载，预测结果不一致

  #5754 closed Mar 18, 2025

• [Bug]: dll load failed 动态链接库dll初始化例程失败

  #5732 closed Mar 18, 2025

• [Question]: uie定制化训练时，进度条卡在0%不动

  #5716 closed Mar 18, 2025

• [Bug]: float division by zero when training

  #5679 closed Mar 18, 2025

• [Question]: MAC M1使用PaddleNLP失败

  #5648 closed Mar 18, 2025

• [Question]:

  #5641 closed Mar 18, 2025

• [Question]: 使用这个脚本训练自定义数据集的时候，40g的内存，都会显存溢出吗？

  #5609 closed Mar 18, 2025

• [Question]: When will UIE support English Event Extraction?

  #5578 closed Mar 18, 2025

• [Question]: where to download seen_schema.zip ?

  #5523 closed Mar 18, 2025

• [Question]: Pegasus模型转换为静态图后不启用FastGeneration直接使用GPU推理报错

  #5512 closed Mar 18, 2025

• [Question]: UIE 用 evaluate.py 和 taskflow 识别结果不同（2.5.2版本）

  #5510 closed Mar 18, 2025

• [Question]:

  #5506 closed Mar 18, 2025

• [Question]: 使用 label_studio.py 识别文本有误

  #5500 closed Mar 18, 2025

• [Question]: from paddlenlp.transformers import ElectraConfig引入ElectraConfig报错。paddlenlp已经下载了。

  #5455 closed Mar 18, 2025

• [Bug]: TypeError: forward() got an unexpected keyword argument 'output_attention'

  #5454 closed Mar 18, 2025

• [Question]: 基于unimo文本生成任务，打包成推理模型后，如何设置batch_size?

  #5447 closed Mar 18, 2025

• [Bug]: 模型自动下载到默认root盘下，然后改变其位置，重新加载就报错

  #5319 closed Mar 18, 2025

• [Question]: UIE-X训练的识别模型会把前面一截漏掉是我的标注问题吗，检测框看起来完美覆盖识别区域输出结果却少了

  #5280 closed Mar 18, 2025

• [Question]: 使用PaddleNlp训练在保存模型时报显存不足Out of memory error on GPU 1

  #5267 closed Mar 18, 2025

• [Question]: 使用uie，标好数据再用doccano.py脚本的时候发生错误：cannot import name 'strtobool' from 'paddlenlp.trainer.argparser'

  #5257 closed Mar 18, 2025

• 在官网使用AIstudio跑paddlenlp的一键预测的时候出现导入错误

  #5254 closed Mar 18, 2025

• [Question]: AssertionError: The models name: uie-senta-base is not in task:[sentiment_analysis]

  #5236 closed Mar 18, 2025

• [Bug]: 安装失败

  #5245 closed Mar 18, 2025

• [Question]:

  #5229 closed Mar 18, 2025

• [Question]: Taskflow使用pos_tagging，无法运行

  #5213 closed Mar 18, 2025

• [Question]: ROCM平台不支持paddle.multinomial

  #5202 closed Mar 18, 2025

• 教学的问答系统使用什么技术是比较好的选择?

  #4651 closed Mar 18, 2025

• [Question]: model = ErnieViLModel.from_pretrained("PaddlePaddle/ernie_vil-2.0-base-zh") 模型文件下载识别

  #5187 closed Mar 18, 2025

• [Question]: Taskflow如何指定模型路径，task为information_extraction

  #9698 closed Mar 18, 2025

• [Bug]: 依存句法分析使用出错，提示LAC依赖，但是LAC已经安装

  #9378 closed Mar 17, 2025

• [Bug]: TypeError: 'type' object is not subscriptable

  #9775 closed Mar 17, 2025

• [Bug]: ERNIE 3.0系列词表存在重复token

  #6839 closed Mar 17, 2025

• [Question]: No module named 'paddle.nn.layer.layers'

  #6847 closed Mar 17, 2025

• 通过paddlenlp启动的大模型兼容openai接口

  #6857 closed Mar 17, 2025

• [Question]: ERNIE-3.5-SE的模型问题

  #6851 closed Mar 17, 2025

• [Docs]:

  #6903 closed Mar 17, 2025

• [Question]: 加载paddlenlp的Taskflow报错 cudaErrorNoKernelImageForDevice

  #7251 closed Mar 17, 2025

• [Bug]: cannot import name 'switch_rng_state_tracker' from 'paddle.distributed.fleet.recompute.recompute'

  #9564 closed Mar 17, 2025

• 安装paddlenlp的时候出现错误，paddle3.0b2, paddlenlp3.0

  #9639 closed Mar 17, 2025

• [Question]: 用P800跑微调任务的时候遇到问题

  #9727 closed Mar 17, 2025

• [Question]: msra数据集无法加载

  #9842 closed Mar 17, 2025

• [Question]: Taskflow 在 NPU上运行information_extraction报错

  #9691 closed Mar 17, 2025

• [Question]: 请问UTC模型的其他版本在哪里下载？huggingface上只能搜到large版本的

  #9743 closed Mar 17, 2025

• 建议paddleNLP兼容一下BGE模型

  #10145 closed Mar 17, 2025

• [Question]: Why is the performance of GPTTokenizer much worse than Tiktoken?

  #9999 closed Mar 17, 2025

• [Question]: Taskflow("feature_extraction")功能是否下线？按照Paddle文档中的代码运行报错：RuntimeError

  #9545 closed Mar 17, 2025

8 Issues opened by 8 people

• [Question]: pp-uie推理返回格式

  #10245 opened Mar 21, 2025

• [Question]: 我这边想部署个离线的微型nlp模型到我的exe中

  #10238 opened Mar 21, 2025

• [Question]: PP-UIE系列模型下载地址在哪，本地没有网络需要离线加载

  #10236 opened Mar 21, 2025

• [Bug]: paddle_ops cpu 编译报错

  #10199 opened Mar 19, 2025

• 完成实测、提交测评，赢取奖金！——DeepSeek-R1-MTP 单机部署实战

  #10166 opened Mar 17, 2025

• [Bug]: taskflow uie 动转静报错

  #10159 opened Mar 17, 2025

• [直播稿件] 飞桨框架3.0支持DeepSeek-V3/R1系列模型部署

  #10157 opened Mar 17, 2025

• [Bug]: UIE信息抽取预测时，请求多次，每次返回结果不固定

  #10150 opened Mar 16, 2025

64 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [generation]update generation

  #10080 commented on Mar 21, 2025 • 24 new comments

• [AutoParallel]:update autoparallel format

  #9747 commented on Mar 17, 2025 • 0 new comments

• Add galvatron search engine & hardware bandwidth profile for paddle.

  #9731 commented on Mar 18, 2025 • 0 new comments

• [LLM] Add Aquila2 model

  #9692 commented on Mar 18, 2025 • 0 new comments

• [Tokenizer] Upgrade tokenizer

  #9683 commented on Mar 21, 2025 • 0 new comments

• [MLU] taskflow mlu

  #9635 commented on Mar 18, 2025 • 0 new comments

• [DON'T NEED REVIEW] Mthreads llama 13 b 64 pp16

  #9558 commented on Mar 22, 2025 • 0 new comments

• [DON'T NEED REVIEW] Mthreads llama 13 b 64 pp8

  #9557 commented on Mar 22, 2025 • 0 new comments

• moe auto_parallel load ckpt from dyhand hack fix

  #9457 commented on Mar 23, 2025 • 0 new comments

• [NPU] Add chatglmv3-6b

  #9213 commented on Mar 18, 2025 • 0 new comments

• [DON'T NEED REVIEW] Mthreads llama 13 b 128 pp16

  #9193 commented on Mar 22, 2025 • 0 new comments

• Ckpt quant

  #9178 commented on Mar 18, 2025 • 0 new comments

• support llama2 w8a8 fp8 inference

  #8827 commented on Mar 22, 2025 • 0 new comments

• implemention of lqlora

  #8820 commented on Mar 17, 2025 • 0 new comments

• add fp8 cutlass gemm fused op for llama2 inference

  #8809 commented on Mar 22, 2025 • 0 new comments

• support qwen2 inferece model

  #8806 commented on Mar 22, 2025 • 0 new comments

• support Llama2 fp8c16 inference

  #8762 commented on Mar 22, 2025 • 0 new comments

• 【LLM】solve dynamic to static problem

  #9755 commented on Mar 18, 2025 • 0 new comments

• LinChain

  #9776 commented on Mar 23, 2025 • 0 new comments

• remove monitor_log_file

  #9792 commented on Mar 19, 2025 • 0 new comments

• [Infer] Add the custom_ops for paddlenlp

  #9794 commented on Mar 17, 2025 • 0 new comments

• [DON'T REVIEW][PEFT] Try to migrate peft repo to paddlenlp

  #9799 commented on Mar 22, 2025 • 0 new comments

• Support deepseek v3

  #9835 commented on Mar 21, 2025 • 0 new comments

• [Embedding] Add LLARA-passage and bge-en-icl with benchmark

  #9887 commented on Mar 21, 2025 • 0 new comments

• [Config] Add model configs

  #9981 commented on Mar 21, 2025 • 0 new comments

• update_llama_conf_cinn_0218

  #10006 commented on Mar 19, 2025 • 0 new comments

• [Feature] Sageattn write kv cache with BF16/FP16, INT8, FP8

  #10032 commented on Mar 21, 2025 • 0 new comments

• 2:4 sparse for int8/fp8/bf16/fp16 dtype

  #10081 commented on Mar 20, 2025 • 0 new comments

• GRPO、RF++功能优化

  #10123 commented on Mar 19, 2025 • 0 new comments

• Split predictor.

  #10136 commented on Mar 18, 2025 • 0 new comments

• support dynamic server

  #10143 commented on Mar 19, 2025 • 0 new comments

• LoRA-Pro Implemention

  #10146 commented on Mar 17, 2025 • 0 new comments

• [Question]: paddlenlp 3.0.0b1版本使用uie-m-base报错找不到static/inference.pdmodel

  #9646 commented on Mar 17, 2025 • 0 new comments

• [Question]: Error loading layoutlmv2-base-uncased: Missing model_state.pdparams file

  #9868 commented on Mar 17, 2025 • 0 new comments

• [Bug]: 训练后评估阶段报错

  #9908 commented on Mar 17, 2025 • 0 new comments

• [Question]: doccano=1.6.2，autolabeling无法实现，一直转圈

  #9744 commented on Mar 18, 2025 • 0 new comments

• [Question]: ernie-3-tiny 按文档步骤运行例子报错

  #9896 commented on Mar 18, 2025 • 0 new comments

• [Question]: LogitsProcessorList 缺少__iter__和extend方法

  #9926 commented on Mar 19, 2025 • 0 new comments

• [Question]:多个预训练模型无法下载

  #10042 commented on Mar 21, 2025 • 0 new comments

• [Bug]: BERT蒸馏至BiLSTM的数据增强是不是有问题

  #9816 commented on Mar 22, 2025 • 0 new comments

• Add byt5 Model

  #1742 commented on Mar 22, 2025 • 0 new comments

• Translation of Documentation for Release/2.3

  #2567 commented on Mar 23, 2025 • 0 new comments

• Add question generation example

  #2944 commented on Mar 22, 2025 • 0 new comments

• Refactor training loop

  #6098 commented on Mar 22, 2025 • 0 new comments

• Cyk1337 patch 1

  #6387 commented on Mar 22, 2025 • 0 new comments

• [Not Review] Enable profile with detail nvtx tag.

  #6419 commented on Mar 23, 2025 • 0 new comments

• Moe

  #6515 commented on Mar 23, 2025 • 0 new comments

• Enable to keep sin_cached and cos_cached in fp32 precision and apply rope also in fp32 precision.

  #6752 commented on Mar 23, 2025 • 0 new comments

• [LLM] support bloom fine grained dybatch v1.

  #6878 commented on Mar 22, 2025 • 0 new comments

• [LLM] Support pre_caches input of llama

  #6900 commented on Mar 22, 2025 • 0 new comments

• [DO NOT MERGE] calculate train tokens for LLM training

  #6917 commented on Mar 23, 2025 • 0 new comments

• Add json configuration for llama-13b pretrain on 1, 2, 4 GPUs.

  #7067 commented on Mar 22, 2025 • 0 new comments

• [NPU] adaptation for LLaMA

  #7262 commented on Mar 23, 2025 • 0 new comments

• Optimize the use of no_recompute_layers.

  #7315 commented on Mar 23, 2025 • 0 new comments

• fix bug when use_flas_attention is 0

  #7421 commented on Mar 22, 2025 • 0 new comments

• add run_hybrid_parallel.sh

  #7549 commented on Mar 22, 2025 • 0 new comments

• Dygraph auto

  #7609 commented on Mar 23, 2025 • 0 new comments

• MP2-PP2 hack shared layer to non-sharded layer to Step Alignment

  #7614 commented on Mar 22, 2025 • 0 new comments

• [AutoParallel] Test 3d SP acc

  #7677 commented on Mar 22, 2025 • 0 new comments

• [DO NOT Merge] Test dynamic auto parallel 3d sp acc

  #7683 commented on Mar 22, 2025 • 0 new comments

• [WIP] Implement fast_rms_norm as custom operators.

  #8073 commented on Mar 22, 2025 • 0 new comments

• ceval_quant_eval

  #8220 commented on Mar 22, 2025 • 0 new comments

• xxx. fix_ceval_quant_eval

  #8221 commented on Mar 22, 2025 • 0 new comments

• Allow to pre alloc memory for pretraining for better memory use.

  #8600 commented on Mar 22, 2025 • 0 new comments