Skip to content

[opt](nereids) add distribute specification to plan shape check and add distribute hint tests#27537

Closed
LiBinfeng-01 wants to merge 1 commit intoapache:masterfrom
LiBinfeng-01:add_distribute_spec
Closed

[opt](nereids) add distribute specification to plan shape check and add distribute hint tests#27537
LiBinfeng-01 wants to merge 1 commit intoapache:masterfrom
LiBinfeng-01:add_distribute_spec

Conversation

@LiBinfeng-01
Copy link
Contributor

Proposed changes

add test of distribute hint like [shuffle] or [broadcast] added to control join distribute strategy

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@LiBinfeng-01
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit 1457d71dad128cd12b47d3d123e7936726674222, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4926	4646	4640	4640
q2	358	157	159	157
q3	2053	1942	1947	1942
q4	1407	1256	1218	1218
q5	3956	3988	4064	3988
q6	256	129	128	128
q7	1447	882	875	875
q8	2817	2819	2780	2780
q9	9950	9630	9802	9630
q10	3478	3500	3498	3498
q11	391	249	253	249
q12	442	299	298	298
q13	4585	3813	3824	3813
q14	314	297	287	287
q15	582	532	522	522
q16	659	574	582	574
q17	1142	1002	943	943
q18	7928	7534	7434	7434
q19	1691	1689	1698	1689
q20	557	308	312	308
q21	4455	3968	4046	3968
q22	469	374	394	374
Total cold run time: 53863 ms
Total hot run time: 49315 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4589	4567	4607	4567
q2	334	228	237	228
q3	4041	4020	4006	4006
q4	2726	2713	2719	2713
q5	9649	9626	9655	9626
q6	252	118	125	118
q7	3025	2465	2494	2465
q8	4496	4485	4493	4485
q9	13222	13044	13193	13044
q10	4084	4193	4173	4173
q11	782	642	652	642
q12	975	799	806	799
q13	4288	3616	3516	3516
q14	397	350	347	347
q15	578	525	511	511
q16	745	697	673	673
q17	3899	3888	3819	3819
q18	9687	9102	9261	9102
q19	1802	1769	1766	1766
q20	2407	2097	2060	2060
q21	8765	8568	8685	8568
q22	915	809	780	780
Total cold run time: 81658 ms
Total hot run time: 78008 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.04 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 30 seconds loaded 2358488459 Bytes, about 74 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17097697524 Bytes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments