Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fx]add gpt2 passes for pipeline performance test #1366

Merged
merged 36 commits into from
Jul 26, 2022

Conversation

YuliangLiu0306
Copy link
Contributor

This PR add some passes to walk around the bugs happened due to too flexible partition policy.
There are two main changes:

  1. In new split_module pass, if a node has non-tensor data and the args of node do NOT locate in same partition as node located in, it will be moved into its args partition to avoid non-tensor data communication. To avoid circular dependency, we will also move all ancetors of this node into same partition.
  2. In new split_with_split_nodes_pass, we will eliminate the unused placeholder nodes and output nodes to avoid unnecessary communication.

Only a part of changes in above pass will be added into the origin pass, and above pass will be deprecated in future.

YuliangLiu0306 and others added 30 commits April 13, 2022 17:30
@FrankLeeeee FrankLeeeee merged commit 5542816 into hpcaitech:main Jul 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants