Skip to content

[fix](Nereids) reorder join generate plan is not stable (#31539)#31570

Merged
morrySnow merged 1 commit intoapache:branch-2.0from
morrySnow:2.0_31539
Feb 29, 2024
Merged

[fix](Nereids) reorder join generate plan is not stable (#31539)#31570
morrySnow merged 1 commit intoapache:branch-2.0from
morrySnow:2.0_31539

Conversation

@morrySnow
Copy link
Contributor

pick from master #31539
commit id 59cdef9

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50134 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 88744f1930b255a63ab0afe9be48910e186b856f, data reload: false

------ Round 1 ----------------------------------
q1	17905	4423	4345	4345
q2	2039	148	141	141
q3	10460	1902	1902	1902
q4	10324	1268	1318	1268
q5	8603	3985	4092	3985
q6	226	120	123	120
q7	2075	1622	1630	1622
q8	9592	2727	2735	2727
q9	10638	10629	10430	10430
q10	8659	3518	3517	3517
q11	421	244	244	244
q12	467	293	306	293
q13	18371	3981	4061	3981
q14	358	323	325	323
q15	497	450	462	450
q16	682	602	585	585
q17	1151	975	988	975
q18	7310	6879	6884	6879
q19	1700	1589	1532	1532
q20	510	309	315	309
q21	4484	4148	4103	4103
q22	506	403	408	403
Total cold run time: 116978 ms
Total hot run time: 50134 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4328	4335	4341	4335
q2	322	225	215	215
q3	4180	4139	4189	4139
q4	2757	2755	2764	2755
q5	7249	7238	7276	7238
q6	236	117	118	117
q7	3252	2854	2782	2782
q8	4338	4509	4510	4509
q9	16955	16990	16930	16930
q10	4244	4279	4276	4276
q11	779	694	680	680
q12	1014	865	849	849
q13	6894	3726	3694	3694
q14	459	426	408	408
q15	490	447	456	447
q16	763	696	706	696
q17	3792	3861	3855	3855
q18	8834	8749	8815	8749
q19	1717	1693	1649	1649
q20	2382	2190	2070	2070
q21	8510	8671	8502	8502
q22	1068	976	993	976
Total cold run time: 84563 ms
Total hot run time: 79871 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 201942 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 88744f1930b255a63ab0afe9be48910e186b856f, data reload: false

query1	907	389	388	388
query2	6515	2388	2190	2190
query3	6918	204	208	204
query4	20818	17923	17858	17858
query5	19724	6493	6445	6445
query6	283	207	229	207
query7	4153	288	290	288
query8	247	265	218	218
query9	3087	2679	2610	2610
query10	404	288	324	288
query11	11286	10708	10811	10708
query12	122	76	73	73
query13	5571	629	632	629
query14	17622	13513	13582	13513
query15	362	228	233	228
query16	6544	259	256	256
query17	1938	1462	873	873
query18	2318	398	402	398
query19	208	137	147	137
query20	75	77	74	74
query21	184	93	92	92
query22	5429	5092	5100	5092
query23	32583	31660	31844	31660
query24	7049	6534	6421	6421
query25	523	422	408	408
query26	518	157	160	157
query27	1833	289	286	286
query28	6162	2234	2192	2192
query29	2801	2792	2824	2792
query30	241	158	164	158
query31	897	710	720	710
query32	65	59	62	59
query33	389	238	251	238
query34	844	455	469	455
query35	1114	905	909	905
query36	1475	1549	1624	1549
query37	86	62	57	57
query38	3165	2910	2926	2910
query39	1379	1309	1315	1309
query40	198	91	94	91
query41	38	32	36	32
query42	83	78	81	78
query43	716	616	622	616
query44	1145	696	703	696
query45	233	228	222	222
query46	1238	958	964	958
query47	1769	1622	1746	1622
query48	968	673	660	660
query49	614	377	373	373
query50	866	635	612	612
query51	4796	4738	4674	4674
query52	93	76	77	76
query53	436	322	313	313
query54	2614	2439	2476	2439
query55	89	78	78	78
query56	216	197	190	190
query57	1245	1096	1144	1096
query58	210	203	209	203
query59	3524	3419	3360	3360
query60	203	189	192	189
query61	83	86	86	86
query62	827	466	453	453
query63	465	319	327	319
query64	2235	1483	1432	1432
query65	3607	3535	3527	3527
query66	750	364	362	362
query67	16139	17560	15306	15306
query68	8581	675	679	675
query69	555	328	333	328
query70	2047	1659	1925	1659
query71	398	300	314	300
query72	6489	3384	3393	3384
query73	729	313	319	313
query74	6390	5879	5839	5839
query75	4996	3713	3722	3713
query76	5086	1164	1203	1164
query77	844	261	248	248
query78	12579	11751	11573	11573
query79	8862	630	645	630
query80	1720	389	394	389
query81	494	229	227	227
query82	1644	100	95	95
query83	160	132	128	128
query84	263	67	70	67
query85	849	285	278	278
query86	372	393	361	361
query87	3220	2961	3004	2961
query88	5157	2337	2325	2325
query89	363	291	283	283
query90	1929	198	200	198
query91	147	118	112	112
query92	56	51	51	51
query93	6271	616	589	589
query94	698	205	203	203
query95	1073	1070	1096	1070
query96	637	331	328	328
query97	6487	6376	6327	6327
query98	189	173	174	173
query99	2917	926	853	853
Total cold run time: 313441 ms
Total hot run time: 201942 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 88744f1930b255a63ab0afe9be48910e186b856f, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.25	0.05	0.04
query4	1.82	0.06	0.06
query5	0.53	0.51	0.51
query6	1.24	0.62	0.62
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.53	0.47	0.49
query10	0.55	0.52	0.53
query11	0.11	0.08	0.08
query12	0.11	0.09	0.09
query13	0.62	0.61	0.60
query14	0.78	0.78	0.78
query15	0.78	0.76	0.75
query16	0.36	0.37	0.36
query17	0.96	1.03	1.03
query18	0.21	0.28	0.24
query19	1.94	1.86	1.80
query20	0.02	0.02	0.02
query21	15.46	0.56	0.56
query22	2.59	2.02	1.11
query23	17.29	1.02	0.81
query24	6.16	0.73	1.07
query25	0.43	0.10	0.04
query26	0.66	0.14	0.14
query27	0.05	0.04	0.04
query28	7.31	0.77	0.74
query29	12.63	2.12	2.20
query30	0.58	0.60	0.52
query31	2.83	0.40	0.39
query32	3.41	0.50	0.50
query33	3.06	3.07	3.04
query34	15.30	4.80	4.80
query35	4.86	4.83	4.82
query36	1.07	1.02	1.03
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.13
query41	0.07	0.02	0.02
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 105.01 s
Total hot run time: 29.67 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 88744f1930b255a63ab0afe9be48910e186b856f with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@morrySnow morrySnow merged commit daeabc7 into apache:branch-2.0 Feb 29, 2024
@morrySnow morrySnow deleted the 2.0_31539 branch March 4, 2024 04:12
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants