llama_with_auto_pp #10648

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

Waynezee wants to merge 7 commits into PaddlePaddle:develop from Waynezee:llama_with_auto_pp

Contributor

Waynezee commented May 23, 2025

Before submitting

Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

PR changes

Description


          llama_with_auto_pp

a8992df

paddle-bot bot commented May 23, 2025

Thanks for your contribution!


          support change n_microbatches in json

d1d8e44

codecov bot commented May 23, 2025 •

edited

Loading

Codecov Report

Attention: Patch coverage is 16.27907% with 288 lines in your changes missing coverage. Please review.

Project coverage is 46.84%. Comparing base (91920a6) to head (d1d8e44).
Report is 54 commits behind head on develop.

❗ Current head d1d8e44 differs from pull request most recent head e42e54c

Please upload reports for the commit e42e54c to get more accurate results.

Files with missing lines	Patch %	Lines
paddlenlp/transformers/llama/modeling_auto_pp.py	21.07%	206 Missing ⚠️
paddlenlp/trainer/auto_trainer.py	0.00%	82 Missing ⚠️

❌ Your patch check has failed because the patch coverage (16.27%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.
❌ Your project check has failed because the head coverage (46.84%) is below the target coverage (58.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop   #10648      +/-   ##
===========================================
- Coverage    46.98%   46.84%   -0.14%     
===========================================
  Files          799      800       +1     
  Lines       132255   132774     +519     
===========================================
+ Hits         62135    62195      +60     
- Misses       70120    70579     +459

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Waynezee added 4 commits

May 23, 2025 19:50


          fix init model


          keep layers which are not in curr rank

41e72a7


          formal llama_with_auto_pp

79052ae


          formal llama_with_auto_pp

5f72a6f

xuxinyi389 reviewed

View reviewed changes

paddlenlp/transformers/llama/modeling_auto_pp.py Outdated

+              # output_attentions、use_cache 可以由config控制且有默认值， delete掉
+              # inputs_embeds、past_key_values 动手PP组网没有使用，delete掉，使用默认值
+              # attn_mask_startend_row_indices 自动并行组网没有使用，不考虑

Collaborator

xuxinyi389 Jun 9, 2025

这些注释可以删除掉

xuxinyi389 reviewed

View reviewed changes

paddlenlp/transformers/llama/modeling_auto_pp.py Outdated

+                      local_chunk_id = stage_idx // pp_degree
+                      if stage_idx == 0: # 第一个model_chunk输入特殊处理
+                          new_model = _Pipeline_model_chunk(layer_lists[:chunk_size])
+                          def forward0(

Collaborator

xuxinyi389 Jun 9, 2025

规范下命名，可以命名为 forward_with_emb

xuxinyi389 reviewed

View reviewed changes

paddlenlp/transformers/llama/modeling_auto_pp.py Outdated

+                          new_model.forward = forward0.__get__(new_model)
+                      else:
+                          new_model = _Pipeline_model_chunk(layer_lists[stage_idx * chunk_size : (stage_idx + 1) * chunk_size])
+                          def forward1(self, *args, **kwargs):

Collaborator

xuxinyi389 Jun 9, 2025

规范下命名，可以命名为 forward_with_decode

xuxinyi389 reviewed

View reviewed changes

paddlenlp/transformers/llama/modeling_auto_pp.py

+                  assert mode in ["VPP", "1F1B", "GPipe"]
+                  stages = manual_model_split(model, group.rank, group, mode, pp_degree)
+                  if mode == "VPP":
+                      schedule = ScheduleInterleaved1F1B(stages, n_microbatches = n_microbatches, loss_fn = loss_fn)

Collaborator

xuxinyi389 Jun 9, 2025

后续，Schdules 名称会和动手对齐。ScheduleGPipe应该要替换成ScheduleFThenB，ScheduleInterleaved1F1B 改成 ScheduleVPP


          style

e42e54c

xuxinyi389 reviewed

View reviewed changes

paddlenlp/transformers/llama/modeling_auto_pp.py


		return ret

		class _Pipeline_model_chunk(nn.Layer):

Collaborator

xuxinyi389 Jun 12, 2025

命名为 llama_chunk 吧

xuxinyi389 reviewed

View reviewed changes

paddlenlp/transformers/llama/modeling_auto_pp.py

+                  def __init__(self, layers):
+                      super(_Pipeline_model_chunk, self).__init__()
+                      self.layers = layers
+                  def forward(self, *args, **kwargs):

Collaborator

xuxinyi389 Jun 12, 2025

把下方注册forward逻辑的代码，在此处重写，使用if else 走不同分支，更容易让用户理解

Waynezee closed this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet