Skip to content

[Improvement](sort) Accumulate blocks to do partial sort#12336

Merged
HappenLee merged 6 commits intoapache:masterfrom
Gabriel39:opt_ps
Sep 7, 2022
Merged

[Improvement](sort) Accumulate blocks to do partial sort#12336
HappenLee merged 6 commits intoapache:masterfrom
Gabriel39:opt_ps

Conversation

@Gabriel39
Copy link
Contributor

@Gabriel39 Gabriel39 commented Sep 5, 2022

Proposed changes

Issue Number: close #12335

test on ssb_flat 1g.

Case 1

select * from lineorder_flat order by LO_ORDERKEY,LO_ORDERDATE, LO_LINENUMBER, LO_CUSTKEY, LO_PARTKEY;
+---------------------------+------------+---------------+--------------------------------------+
| experiment | rows. | e2e duration | sort node profile |
+---------------------------+------------+---------------+--------------------------------------+
| baseline | 6001215 | 28.07 sec | Active: 26s256ms, non-child: 66.65% |
+---------------------------+------------+---------------+--------------------------------------+
| accumulated | 6001215 | 21.75 sec | Active: 20s180ms, non-child: 68.97% |
+---------------------------+------------+---------------+--------------------------------------+

Case 2

select LO_ORDERDATE, LO_ORDERKEY, LO_LINENUMBER, LO_CUSTKEY, LO_PARTKEY from lineorder_flat order by LO_ORDERKEY,LO_ORDERDATE, LO_LINENUMBER, LO_CUSTKEY, LO_PARTKEY;
+---------------------------+------------+---------------+--------------------------------------+
| experiment | rows. | e2e duration | sort node profile |
+---------------------------+------------+---------------+--------------------------------------+
| baseline | 6001215 | 3.51 sec | Active: 3s377ms, non-child: 76.81% |
+---------------------------+------------+---------------+--------------------------------------+
| accumulated | 6001215 | 3.16 sec | Active: 1s926ms, non-child: 61.99% |
+---------------------------+------------+---------------+--------------------------------------+

Problem summary

Describe your changes.

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@Gabriel39 Gabriel39 requested a review from HappenLee September 6, 2022 10:43
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 7, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2022

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2022

PR approved by anyone and no changes requested.

@HappenLee HappenLee merged commit 449d0c2 into apache:master Sep 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. area/vectorization kind/test reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] Accumulate blocks to do partial sort

2 participants