[Improvement](pipeline) Do parallel preparation for multiple fragments #34421
Conversation
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
TeamCity be ut coverage result: |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
TeamCity be ut coverage result: |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
TeamCity be ut coverage result: |
|
run buildall |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| } | ||
|
|
||
| Status PipelineFragmentContext::prepare(const doris::TPipelineFragmentParams& request) { | ||
| Status PipelineFragmentContext::prepare(const doris::TPipelineFragmentParams& request, |
There was a problem hiding this comment.
warning: function 'prepare' exceeds recommended size/complexity thresholds [readability-function-size]
^Additional context
be/src/pipeline/pipeline_fragment_context.cpp:238: 99 lines including whitespace and comments (threshold 80)
^|
|
||
| Status PipelineFragmentContext::_build_pipeline_tasks( | ||
| const doris::TPipelineFragmentParams& request) { | ||
| Status PipelineFragmentContext::_build_pipeline_tasks(const doris::TPipelineFragmentParams& request, |
There was a problem hiding this comment.
warning: function '_build_pipeline_tasks' exceeds recommended size/complexity thresholds [readability-function-size]
^Additional context
be/src/pipeline/pipeline_fragment_context.cpp:340: 177 lines including whitespace and comments (threshold 80)
^
|
run buildall |
TPC-H: Total hot run time: 41393 ms |
TPC-DS: Total hot run time: 186026 ms |
|
TeamCity be ut coverage result: |
|
run buildall |
|
TeamCity be ut coverage result: |
TPC-H: Total hot run time: 41987 ms |
TPC-DS: Total hot run time: 184668 ms |
|
run buildall |
| } | ||
|
|
||
| Status PipelineFragmentContext::prepare(const doris::TPipelineFragmentParams& request) { | ||
| Status PipelineFragmentContext::prepare(const doris::TPipelineFragmentParams& request, |
There was a problem hiding this comment.
warning: function 'prepare' exceeds recommended size/complexity thresholds [readability-function-size]
^Additional context
be/src/pipeline/pipeline_fragment_context.cpp:238: 88 lines including whitespace and comments (threshold 80)
^|
|
||
| Status PipelineFragmentContext::_build_pipeline_tasks( | ||
| const doris::TPipelineFragmentParams& request) { | ||
| Status PipelineFragmentContext::_build_pipeline_tasks(const doris::TPipelineFragmentParams& request, |
There was a problem hiding this comment.
warning: function '_build_pipeline_tasks' exceeds recommended size/complexity thresholds [readability-function-size]
^Additional context
be/src/pipeline/pipeline_fragment_context.cpp:329: 177 lines including whitespace and comments (threshold 80)
^
|
TeamCity be ut coverage result: |
|
|
||
| Status PipelineFragmentContext::_build_pipeline_tasks( | ||
| const doris::TPipelineFragmentParams& request) { | ||
| Status PipelineFragmentContext::_build_pipeline_tasks(const doris::TPipelineFragmentParams& request, |
There was a problem hiding this comment.
warning: function '_build_pipeline_tasks' exceeds recommended size/complexity thresholds [readability-function-size]
^Additional context
be/src/pipeline/pipeline_fragment_context.cpp:436: 177 lines including whitespace and comments (threshold 80)
^
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
| return pipeline; | ||
| } | ||
|
|
||
| Status PipelineFragmentContext::prepare(const doris::TPipelineFragmentParams& request, |
There was a problem hiding this comment.
warning: function 'prepare' exceeds recommended size/complexity thresholds [readability-function-size]
^Additional context
be/src/pipeline/pipeline_fragment_context.cpp:248: 88 lines including whitespace and comments (threshold 80)
^|
|
||
| Status PipelineFragmentContext::_build_pipeline_tasks( | ||
| const doris::TPipelineFragmentParams& request) { | ||
| Status PipelineFragmentContext::_build_pipeline_tasks(const doris::TPipelineFragmentParams& request, |
There was a problem hiding this comment.
warning: function '_build_pipeline_tasks' exceeds recommended size/complexity thresholds [readability-function-size]
^Additional context
be/src/pipeline/pipeline_fragment_context.cpp:446: 177 lines including whitespace and comments (threshold 80)
^
|
TeamCity be ut coverage result: |
|
run buildall |
TPC-H: Total hot run time: 41930 ms |
TPC-DS: Total hot run time: 187602 ms |
|
TeamCity be ut coverage result: |
2 participants
Proposed changes
Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...