Skip to content

Conversation

@jackwener
Copy link
Member

@jackwener jackwener commented Jun 27, 2023

Proposed changes

Issue Number: close #xxx

Infer distinct from Distinct SetOperator, and put distinct above children to reduce data.

tpcds_sf100 q14:

before
100 rows in set (7.60 sec)

after
100 rows in set (6.80 sec)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@jackwener
Copy link
Member Author

run buildall

@jackwener jackwener force-pushed the infer_distinct branch 2 times, most recently from ce16ccc to f97e24c Compare June 27, 2023 07:37
@jackwener
Copy link
Member Author

run buildall

@hello-stephen
Copy link
Contributor

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.95 seconds
stream load tsv: 457 seconds loaded 74807831229 Bytes, about 156 MB/s
stream load json: 23 seconds loaded 2358488459 Bytes, about 97 MB/s
stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select: 68.7 seconds inserted 10000000 Rows, about 145K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230627092034_clickbench_pr_168621.html

@jackwener
Copy link
Member Author

run buildall

@jackwener
Copy link
Member Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 29, 2023
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@jackwener jackwener merged commit 9756ff1 into apache:master Jun 29, 2023
@jackwener jackwener deleted the infer_distinct branch June 29, 2023 14:04
@morrySnow morrySnow added the not-merge/2.0 do not merge into 2.0 branch label Jun 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. area/nereids not-merge/2.0 do not merge into 2.0 branch reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants