Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] 请在pipeline parallel上支持不同长度的input, label的计算。 #9

Open
1 task
Pryest opened this issue Nov 15, 2023 · 0 comments
Open
1 task

[Feature] 请在pipeline parallel上支持不同长度的input, label的计算。 #9

Pryest opened this issue Nov 15, 2023 · 0 comments
Assignees
@sunpengsdu
Labels
enhancement New feature or request

Comments

@Pryest
Copy link
Contributor

Pryest commented Nov 15, 2023

Describe the feature

在目前的情形下，pipeline parallel会在第一份数据进入时固定logits等的形状，但事实上，下游任务中很多时候最好能支持变长以节省算力，而不是进行过多的padding。

Will you implement it?

  • I would like to implement this feature and create a PR!
@Pryest Pryest added the enhancement New feature or request label Nov 15, 2023
@hellock hellock transferred this issue from InternLM/InternLM Jan 17, 2024
blankde pushed a commit to blankde/InternEvo that referenced this issue Jan 18, 2024
@huangting4201
Merge pull request InternLM#9 from yingtongxiong/feat/refactor-fstp-h…
d87d9f9 
…andler

feat(*) refactor fstp handler
@canghaiyunfan canghaiyunfan mentioned this issue Apr 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sunpengsdu sunpengsdu

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sunpengsdu @Pryest