-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[opt](performance) opt full sort performance #53132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
34de0b2 to
fbb4ff2
Compare
| public String forceSortAlgorithm = ""; | ||
|
|
||
| @VariableMgr.VarAttr(name = FULL_SORT_MAX_BUFFERED_BYTES, needForward = true) | ||
| public long fullSortMaxBufferedBytes = -1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
直接把这个值的默认值设置为64MB 就好了;不需要在be 上再有一个默认值是INITIAL_BUFFERED_BLOCK_BYTES。
另外,得check一下这个值是否合理,比如最小值是不是得是1MB?太小了似乎也没意义?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1、fe默认值64MB,加上范围检查[16MB~1GB];(DONE)
2、BE的默认值省不了,如果是升级过程中,BE还拿不到FE传过来的值,所以BE还需要有个默认值;
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
TPC-H: Total hot run time: 33411 ms |
TPC-DS: Total hot run time: 186141 ms |
ClickBench: Total hot run time: 30.28 s |
7b4935c to
ed8b856
Compare
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
TPC-H: Total hot run time: 33366 ms |
TPC-DS: Total hot run time: 188391 ms |
ClickBench: Total hot run time: 29.89 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
TPC-H: Total hot run time: 34484 ms |
TPC-DS: Total hot run time: 185192 ms |
ClickBench: Total hot run time: 30.18 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
What problem does this PR solve?
make full_sort_max_buffered_bytes configurable.
Use tpcds 10G to test:
Performance improve:
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)