Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Pipeline Executor #14

Merged
merged 25 commits into from Aug 20, 2021
Merged

[RFC] Pipeline Executor #14

merged 25 commits into from Aug 20, 2021

Conversation

huajsj
Copy link
Contributor

@huajsj huajsj commented Jul 31, 2021

Split relay graph into Subgraph then doing Subgraph Pipeline :RFC

In this RFC, we present a new framework for subgraph pipelining that first split relay graph into multiple subgraph/group operator s then doing subgraph/group operators scheduling

Discussion: https://discuss.tvm.apache.org/t/rfc-compute-graph-pipeline-with-new-subgraph-executor/9839

@huajsj huajsj changed the title pipeline compute RFC. Pipeline Compute Executor RFC. Jul 31, 2021
@comaniac comaniac added the status: need review RFC needs review label Aug 3, 2021
Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the RFC. My general opinions:

  • The whole picture of this RFC is not clear. If it only targets to a runtime executor, it is not a complete end-to-end solution. At high-level, you should present the ultimate goal, including graph partitioning and execution. Based on that, you could further break them down to several milestones so that we can track the progress while having a clear final picture.
  • Please do proof-reading in advance. There are too many spelling and grammar errors to read the proposal smoothly.

cc @tmoreau89 @areusch @jtuyls

rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
@comaniac comaniac added the status: need update RFC needs update based on feedback label Aug 3, 2021
huajsj and others added 12 commits August 4, 2021 18:59
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
@huajsj huajsj changed the title Pipeline Compute Executor RFC. [RFC]Pipeline Compute Executor. Aug 8, 2021
huajsj and others added 2 commits August 8, 2021 15:28
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
@huajsj huajsj force-pushed the main branch 2 times, most recently from 72b2c5d to e67db87 Compare August 9, 2021 19:41
@huajsj
Copy link
Contributor Author

huajsj commented Aug 9, 2021

@comaniac , @jtuyls , all review comments addressed, please review again.

rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
huajsj and others added 4 commits August 16, 2021 19:20
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Copy link
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM. It would be great if other folks can also take a look; otherwise I plan to merge this RFC after 3 days.

cc @areusch @jroesch @zhiics @junrushao1994 @jtuyls

rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
rfcs/0012-pipeline-executor.md Outdated Show resolved Hide resolved
@comaniac comaniac changed the title [RFC]Pipeline Compute Executor. [RFC] Pipeline Executor Aug 17, 2021
huajsj and others added 3 commits August 17, 2021 11:56
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
@comaniac
Copy link
Contributor

btw, please change the RFC file name to align the PR number (0014).

@huajsj
Copy link
Contributor Author

huajsj commented Aug 17, 2021

btw, please change the RFC file name to align the PR number (0014).

@comaniac, thanks for the follow up, the file name change already done.

@comaniac comaniac added status: accepted RFC is accepted and removed status: need review RFC needs review status: need update RFC needs update based on feedback labels Aug 20, 2021
@comaniac comaniac merged commit 2d57c28 into apache:main Aug 20, 2021
@comaniac
Copy link
Contributor

Due to no objection, this RFC is now merged. Thanks @huajsj

@HungYangChang
Copy link

Hello @huajsj I am super interested in your work!
As I have asked in discussion form, I am using Hikey 970 with 4 big and 4 little CPUs to do inference now. One of my interest is to split my BERT model into the different layers (sub-graph) and then assign to different cores.

I will definitely try to replicate your work in these two days and see what can I get. Thanks for your great work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: accepted RFC is accepted
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants