Skip to content

Conversation

@Mryange
Copy link
Contributor

@Mryange Mryange commented Dec 1, 2023

Proposed changes

Issue Number: close #xxx

                              OLAP_SCAN_OPERATOR  (id=4804):
                                    -  PlanInfo
                                          -  TABLE:  default_cluster:tpcds.item(item),  PREAGGREGATION:  ON
                                          -  runtime  filters:  RF003[in_or_bloom]  ->  i_item_sk
                                          -  partitions=1/1  (item),  tablets=12/12,  tabletList=11509,11511,11513  ...
                                          -  cardinality=102000,  avgRowSize=659.5244,  numNodes=1
                                          -  pushAggOp=NONE
                                    -  BlocksProduced:  12
                                    -  CloseTime:  avg  56.509us,  max  100.554us,  min  37.528us
                                    -  ExecTime:  avg  229.644us,  max  361.417us,  min  161.896us
                                    -  OpenTime:  avg  166.987us,  max  285.12us,  min  107.403us
                                    -  ProjectionTime:  avg  0ns,  max  0ns,  min  0ns
                                    -  RowsProduced:  27.667K  (27667)
                                    -  RuntimeFilterInfo:  
                                        -  filter  id  =  3  filtered:  74.333K  (74333)
                                        -  filter  id  =  3  input:  102.0K  (102000)



                                  OLAP_SCAN_OPERATOR  (id=4808):
                                        -  PlanInfo
                                              -  TABLE:  default_cluster:tpcds.catalog_sales(catalog_sales),  PREAGGREGATION:  ON
                                              -  runtime  filters:  RF000[in_or_bloom]  ->  cs_bill_cdemo_sk,  RF001[in_or_bloom]  ->  cs_sold_date_sk,  RF002[in_or_bloom]  ->  cs_promo_sk
                                              -  partitions=1/1  (catalog_sales),  tablets=32/32,  tabletList=11252,11254,11256  ...
                                              -  cardinality=14401261,  avgRowSize=345.8473,  numNodes=1
                                              -  pushAggOp=NONE
                                        -  BlocksProduced:  96
                                        -  CloseTime:  avg  75.43us,  max  203.133us,  min  15.793us
                                        -  ExecTime:  avg  1.115ms,  max  1.866ms,  min  906.200us
                                        -  OpenTime:  avg  1.15ms,  max  1.774ms,  min  831.607us
                                        -  ProjectionTime:  avg  0ns,  max  0ns,  min  0ns
                                        -  RowsProduced:  69.08K  (69080)
                                        -  RuntimeFilterInfo:  
                                            -  filter  id  =  -1  filtered:  11.4374M  (11437400)
                                            -  filter  id  =  -1  input:  14.271377M  (14271377)
                                            -  filter  id  =  0  filtered:  2.764897M  (2764897)
                                            -  filter  id  =  0  input:  2.833977M  (2833977)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@Mryange
Copy link
Contributor Author

Mryange commented Dec 1, 2023

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2023

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit eba64488012b140aed8b29d260bec1808bf703d5, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4875	4661	4665	4661
q2	355	163	156	156
q3	1528	1283	1269	1269
q4	1135	956	948	948
q5	3240	3225	3233	3225
q6	249	128	133	128
q7	1035	518	553	518
q8	2241	2256	2223	2223
q9	6983	6932	6954	6932
q10	3273	3345	3331	3331
q11	331	208	213	208
q12	357	219	220	219
q13	4637	4593	5215	4593
q14	246	218	215	215
q15	605	551	532	532
q16	453	406	394	394
q17	1030	640	596	596
q18	8461	8424	7192	7192
q19	2923	1528	1503	1503
q20	570	337	317	317
q21	3410	2948	2973	2948
q22	367	296	306	296
Total cold run time: 48304 ms
Total hot run time: 42404 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4626	4595	4596	4595
q2	307	191	230	191
q3	3723	3713	3711	3711
q4	2525	2507	2503	2503
q5	6091	6073	6075	6073
q6	244	123	128	123
q7	2600	1935	1984	1935
q8	3749	3682	3672	3672
q9	9477	9408	9461	9408
q10	3999	4078	4121	4078
q11	642	495	509	495
q12	803	617	656	617
q13	4357	3639	3631	3631
q14	279	242	265	242
q15	593	530	534	530
q16	535	495	485	485
q17	2127	2067	2112	2067
q18	9352	8981	8805	8805
q19	1816	1812	1775	1775
q20	2300	1999	1987	1987
q21	7261	6968	6937	6937
q22	612	546	572	546
Total cold run time: 68018 ms
Total hot run time: 64406 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.66 seconds
stream load tsv: 559 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17162898208 Bytes

Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2023

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Dec 1, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2023

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit 68525fc into apache:master Dec 1, 2023
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants