Skip to content

[fix](bucket) Raise autobucket_min_buckets default from 1 to 3#63729

Open
deardeng wants to merge 3 commits into
apache:masterfrom
deardeng:change-auto-min
Open

[fix](bucket) Raise autobucket_min_buckets default from 1 to 3#63729
deardeng wants to merge 3 commits into
apache:masterfrom
deardeng:change-auto-min

Conversation

@deardeng
Copy link
Copy Markdown
Collaborator

Prevent the auto bucket algorithm from computing a bucket number that is too small. A minimum of 3 buckets provides a more reasonable lower bound for parallelism and data distribution under the auto bucket strategy.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@deardeng
Copy link
Copy Markdown
Collaborator Author

/review

@deardeng
Copy link
Copy Markdown
Collaborator Author

run buildall

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary: I found one blocking test-state issue. The code change itself is focused and uses the existing auto-bucket config path, but the related regression suite still restores the old default value after mutating this mutable FE config.

Critical checkpoint conclusions:

  • Goal/test: The goal is to raise the auto-bucket minimum default from 1 to 3. Existing tests do not safely cover the new default because one suite resets the config back to 1.
  • Scope: The code change is small and focused.
  • Concurrency/lifecycle: No new concurrency or special lifecycle concerns found.
  • Configuration: This changes a mutable user-visible FE config default; related tests/config reset logic must be updated.
  • Compatibility/storage/transactions: No storage format, transaction, or persistence compatibility issue found for this default-only change.
  • Parallel paths: Both create-table auto bucket and dynamic-partition auto bucket read Config.autobucket_min_buckets, so the changed default reaches both paths.
  • Test coverage/results: Existing regression coverage has stale reset behavior and can pollute later suites.
  • Observability/performance: No additional observability or performance issue found.
  • User focus: No additional user-provided review focus was supplied.

Comment thread fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31532 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c912a5d2b69acdfa52535ea87b551dff32e87c06, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17776	4092	4067	4067
q2	q3	10749	1399	802	802
q4	4686	478	346	346
q5	7538	2270	2151	2151
q6	240	174	138	138
q7	948	800	639	639
q8	9414	1727	1556	1556
q9	5155	4956	4905	4905
q10	6341	2193	1889	1889
q11	430	275	247	247
q12	634	426	295	295
q13	18105	3416	2799	2799
q14	263	258	240	240
q15	q16	813	778	709	709
q17	927	833	881	833
q18	7028	5739	5747	5739
q19	1294	1263	1071	1071
q20	619	453	286	286
q21	6293	2955	2511	2511
q22	465	502	309	309
Total cold run time: 99718 ms
Total hot run time: 31532 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4770	4776	4691	4691
q2	q3	4969	5406	4634	4634
q4	2118	2167	1392	1392
q5	5013	4622	4706	4622
q6	242	187	132	132
q7	1923	1723	1552	1552
q8	2411	2129	2076	2076
q9	7746	7397	7415	7397
q10	4760	4654	4206	4206
q11	537	391	374	374
q12	728	735	522	522
q13	2993	3362	2821	2821
q14	276	281	250	250
q15	q16	684	681	610	610
q17	1274	1247	1250	1247
q18	7329	6855	6701	6701
q19	1118	1109	1104	1104
q20	2251	2237	1947	1947
q21	5257	4601	4524	4524
q22	508	452	418	418
Total cold run time: 56907 ms
Total hot run time: 51220 ms

@deardeng
Copy link
Copy Markdown
Collaborator Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31300 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6ee264b2e1ecfaf573835b25cb8869410828c2f6, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17611	3959	3978	3959
q2	q3	10783	1377	809	809
q4	4680	474	339	339
q5	7719	2264	2134	2134
q6	376	174	139	139
q7	929	794	637	637
q8	9354	1669	1497	1497
q9	7138	5005	4993	4993
q10	6458	2280	1863	1863
q11	435	276	251	251
q12	695	442	297	297
q13	18174	3393	2781	2781
q14	270	258	232	232
q15	q16	815	766	707	707
q17	1003	939	887	887
q18	6696	5797	5458	5458
q19	1190	1416	1112	1112
q20	515	411	272	272
q21	5906	2766	2624	2624
q22	461	370	309	309
Total cold run time: 101208 ms
Total hot run time: 31300 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4775	5096	4969	4969
q2	q3	4870	5311	4765	4765
q4	2127	2203	1424	1424
q5	4874	4794	4713	4713
q6	231	182	135	135
q7	1848	1765	1577	1577
q8	2458	1972	1928	1928
q9	7482	7457	7469	7457
q10	4749	4755	4235	4235
q11	542	416	386	386
q12	716	735	532	532
q13	3008	3389	2744	2744
q14	285	276	245	245
q15	q16	680	704	612	612
q17	1268	1253	1239	1239
q18	7471	7009	6617	6617
q19	1148	1085	1072	1072
q20	2220	2220	1971	1971
q21	5298	4575	4414	4414
q22	509	469	392	392
Total cold run time: 56559 ms
Total hot run time: 51427 ms

Prevent the auto bucket algorithm from computing a bucket number that is
too small. A minimum of 3 buckets provides a more reasonable lower bound
for parallelism and data distribution under the auto bucket strategy.
@deardeng
Copy link
Copy Markdown
Collaborator Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 171890 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ee264b2e1ecfaf573835b25cb8869410828c2f6, data reload: false

query5	4340	659	505	505
query6	344	225	202	202
query7	4251	560	317	317
query8	330	241	234	234
query9	8814	4094	4026	4026
query10	455	377	303	303
query11	5816	2412	2228	2228
query12	183	134	130	130
query13	1305	639	442	442
query14	6095	5468	5162	5162
query14_1	4499	4529	4473	4473
query15	214	208	194	194
query16	1017	490	458	458
query17	1142	742	608	608
query18	2613	510	366	366
query19	224	210	171	171
query20	139	135	135	135
query21	219	145	123	123
query22	13665	13575	13285	13285
query23	17331	16478	16199	16199
query23_1	16224	16316	16302	16302
query24	7452	1759	1289	1289
query24_1	1342	1342	1325	1325
query25	584	503	441	441
query26	1346	319	177	177
query27	2707	576	356	356
query28	4495	1992	1986	1986
query29	1037	665	523	523
query30	321	240	204	204
query31	1126	1096	957	957
query32	92	78	78	78
query33	563	366	296	296
query34	1198	1122	649	649
query35	782	818	694	694
query36	1396	1401	1198	1198
query37	156	113	90	90
query38	3212	3136	3128	3128
query39	937	921	893	893
query39_1	895	876	856	856
query40	233	146	126	126
query41	66	62	63	62
query42	110	112	106	106
query43	329	330	301	301
query44	
query45	211	203	203	203
query46	1062	1211	759	759
query47	2333	2383	2305	2305
query48	413	434	297	297
query49	740	492	383	383
query50	1070	355	251	251
query51	4461	4461	4354	4354
query52	106	107	93	93
query53	258	276	212	212
query54	315	268	269	268
query55	95	96	86	86
query56	300	318	316	316
query57	1506	1471	1385	1385
query58	305	272	273	272
query59	1695	1713	1494	1494
query60	328	331	324	324
query61	156	158	157	157
query62	708	658	589	589
query63	247	205	211	205
query64	2402	801	637	637
query65	
query66	1690	482	360	360
query67	29741	29877	29671	29671
query68	
query69	445	352	305	305
query70	1028	1030	990	990
query71	305	290	266	266
query72	3107	2627	2388	2388
query73	858	773	443	443
query74	5146	5035	4757	4757
query75	2701	2632	2265	2265
query76	2267	1153	760	760
query77	398	406	335	335
query78	12476	12503	11764	11764
query79	1511	1010	767	767
query80	1355	531	450	450
query81	525	283	238	238
query82	984	157	120	120
query83	359	281	249	249
query84	269	135	114	114
query85	936	553	463	463
query86	458	366	300	300
query87	3440	3383	3277	3277
query88	3615	2718	2716	2716
query89	458	394	341	341
query90	1876	187	189	187
query91	179	184	140	140
query92	85	79	76	76
query93	1664	1505	873	873
query94	738	391	295	295
query95	669	480	351	351
query96	1053	784	319	319
query97	2733	2732	2569	2569
query98	238	227	227	227
query99	1163	1159	1028	1028
Total cold run time: 255662 ms
Total hot run time: 171890 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31500 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 77358f6f51ce557a135c7b9ba4e6e4cdf2e82cdf, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17757	4060	4059	4059
q2	q3	10778	1365	805	805
q4	4686	473	343	343
q5	7818	2237	2119	2119
q6	330	180	135	135
q7	956	803	634	634
q8	9357	1787	1806	1787
q9	7121	5059	4993	4993
q10	6448	2255	1864	1864
q11	436	279	248	248
q12	696	429	301	301
q13	18171	3414	2808	2808
q14	266	258	234	234
q15	q16	825	781	710	710
q17	936	969	947	947
q18	6999	5943	5505	5505
q19	1184	1207	1100	1100
q20	519	403	260	260
q21	5662	2608	2336	2336
q22	427	355	312	312
Total cold run time: 101372 ms
Total hot run time: 31500 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4344	4314	4332	4314
q2	q3	4502	4944	4381	4381
q4	2096	2177	1391	1391
q5	4429	4261	4919	4261
q6	264	205	150	150
q7	2007	1827	1577	1577
q8	2497	2123	2198	2123
q9	7973	8034	8036	8034
q10	4795	4718	4492	4492
q11	607	417	383	383
q12	743	769	532	532
q13	3253	3549	2956	2956
q14	298	305	276	276
q15	q16	699	750	628	628
q17	1370	1410	1287	1287
q18	7877	7417	6822	6822
q19	1118	1084	1130	1084
q20	2219	2217	1946	1946
q21	5265	4547	4445	4445
q22	518	464	455	455
Total cold run time: 56874 ms
Total hot run time: 51537 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 172030 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 77358f6f51ce557a135c7b9ba4e6e4cdf2e82cdf, data reload: false

query5	4312	653	519	519
query6	323	222	197	197
query7	4221	565	329	329
query8	327	234	219	219
query9	8805	3966	3954	3954
query10	457	349	317	317
query11	5792	2597	2281	2281
query12	186	125	122	122
query13	1251	592	418	418
query14	6048	5439	5109	5109
query14_1	4474	4417	4425	4417
query15	213	205	187	187
query16	1007	447	432	432
query17	952	720	582	582
query18	2470	489	350	350
query19	213	220	156	156
query20	137	130	126	126
query21	218	132	112	112
query22	13596	13638	13478	13478
query23	17318	16581	16190	16190
query23_1	16370	16269	16380	16269
query24	8241	1760	1307	1307
query24_1	1307	1370	1366	1366
query25	547	475	433	433
query26	1349	314	177	177
query27	4255	582	338	338
query28	4505	2042	2073	2042
query29	1021	652	530	530
query30	316	251	208	208
query31	1158	1098	989	989
query32	91	80	79	79
query33	568	365	309	309
query34	1166	1154	662	662
query35	792	815	713	713
query36	1395	1454	1277	1277
query37	155	106	93	93
query38	3240	3163	3068	3068
query39	935	927	897	897
query39_1	864	892	898	892
query40	240	152	128	128
query41	74	69	70	69
query42	117	107	109	107
query43	341	333	290	290
query44	
query45	218	205	207	205
query46	1084	1188	760	760
query47	2370	2378	2286	2286
query48	411	423	320	320
query49	657	514	408	408
query50	1014	350	249	249
query51	4337	4307	4293	4293
query52	107	106	96	96
query53	254	293	202	202
query54	332	295	291	291
query55	97	94	90	90
query56	326	334	313	313
query57	1436	1429	1332	1332
query58	314	280	279	279
query59	1559	1621	1433	1433
query60	328	343	326	326
query61	187	180	180	180
query62	702	656	588	588
query63	238	208	213	208
query64	2473	830	708	708
query65	
query66	1725	478	366	366
query67	29715	29676	29464	29464
query68	
query69	460	341	314	314
query70	1030	963	995	963
query71	311	272	260	260
query72	3136	2698	2372	2372
query73	865	762	419	419
query74	5121	4951	4808	4808
query75	2689	2605	2314	2314
query76	2275	1143	779	779
query77	404	418	326	326
query78	12423	12533	11828	11828
query79	1446	990	741	741
query80	734	550	450	450
query81	475	282	240	240
query82	1371	157	126	126
query83	347	294	257	257
query84	305	145	116	116
query85	890	551	463	463
query86	398	333	343	333
query87	3439	3387	3265	3265
query88	3586	2728	2695	2695
query89	453	396	364	364
query90	1850	190	194	190
query91	181	176	139	139
query92	84	73	75	73
query93	1485	1531	883	883
query94	591	351	317	317
query95	680	485	347	347
query96	1057	796	323	323
query97	2731	2739	2608	2608
query98	239	232	229	229
query99	1167	1159	1021	1021
Total cold run time: 256512 ms
Total hot run time: 172030 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31539 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 77358f6f51ce557a135c7b9ba4e6e4cdf2e82cdf, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17854	4037	4023	4023
q2	q3	10802	1393	801	801
q4	4684	476	348	348
q5	7531	2240	2057	2057
q6	245	174	140	140
q7	950	765	665	665
q8	9346	1672	1557	1557
q9	6649	4957	4947	4947
q10	6462	2244	1886	1886
q11	435	275	241	241
q12	694	425	293	293
q13	18222	3371	2781	2781
q14	270	258	236	236
q15	q16	824	768	712	712
q17	946	867	998	867
q18	6776	5713	5637	5637
q19	1229	1227	1194	1194
q20	526	408	280	280
q21	6047	2742	2569	2569
q22	457	389	305	305
Total cold run time: 100949 ms
Total hot run time: 31539 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4793	4923	4957	4923
q2	q3	4935	5265	4702	4702
q4	2129	2182	1408	1408
q5	4818	4885	4635	4635
q6	232	179	125	125
q7	1885	1743	1554	1554
q8	2397	1980	1913	1913
q9	7445	7448	7336	7336
q10	4764	4658	4206	4206
q11	570	406	347	347
q12	718	734	520	520
q13	3005	3387	2865	2865
q14	265	276	245	245
q15	q16	684	689	600	600
q17	1270	1242	1246	1242
q18	7286	6861	6957	6861
q19	1121	1089	1108	1089
q20	2223	2231	1932	1932
q21	5214	4552	4407	4407
q22	528	465	422	422
Total cold run time: 56282 ms
Total hot run time: 51332 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 171381 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 77358f6f51ce557a135c7b9ba4e6e4cdf2e82cdf, data reload: false

query5	4343	667	536	536
query6	331	229	199	199
query7	4223	597	306	306
query8	338	231	219	219
query9	8774	4013	3984	3984
query10	458	335	294	294
query11	5852	2391	2289	2289
query12	183	129	124	124
query13	1304	635	433	433
query14	6087	5461	5161	5161
query14_1	4457	4455	4397	4397
query15	209	208	183	183
query16	981	462	352	352
query17	960	713	584	584
query18	2433	475	351	351
query19	209	201	161	161
query20	135	136	130	130
query21	211	140	114	114
query22	13600	13624	13357	13357
query23	17368	16562	16234	16234
query23_1	16240	16368	16278	16278
query24	7421	1772	1338	1338
query24_1	1304	1323	1331	1323
query25	552	475	415	415
query26	1313	318	167	167
query27	2709	578	347	347
query28	4474	2003	2010	2003
query29	995	659	544	544
query30	294	237	195	195
query31	1126	1079	940	940
query32	87	75	74	74
query33	541	357	298	298
query34	1178	1188	638	638
query35	769	790	684	684
query36	1403	1396	1211	1211
query37	162	100	94	94
query38	3185	3103	3069	3069
query39	928	916	890	890
query39_1	886	881	901	881
query40	220	143	121	121
query41	66	65	62	62
query42	108	109	109	109
query43	330	339	295	295
query44	
query45	223	208	200	200
query46	1083	1226	748	748
query47	2338	2398	2276	2276
query48	409	433	304	304
query49	628	491	378	378
query50	994	362	257	257
query51	4352	4311	4315	4311
query52	109	106	94	94
query53	254	283	206	206
query54	327	273	256	256
query55	95	90	83	83
query56	305	297	299	297
query57	1427	1435	1351	1351
query58	312	278	276	276
query59	1689	1695	1531	1531
query60	329	319	306	306
query61	163	158	163	158
query62	699	670	604	604
query63	246	204	210	204
query64	2488	867	719	719
query65	
query66	1769	522	375	375
query67	30117	29856	29559	29559
query68	
query69	487	362	315	315
query70	1068	999	909	909
query71	305	282	274	274
query72	3236	2873	2446	2446
query73	829	753	416	416
query74	5070	5005	4774	4774
query75	2694	2592	2262	2262
query76	2273	1128	765	765
query77	411	408	333	333
query78	12334	12438	12109	12109
query79	1464	1048	773	773
query80	653	546	457	457
query81	463	284	239	239
query82	1336	162	127	127
query83	352	277	247	247
query84	302	147	110	110
query85	874	531	442	442
query86	405	357	321	321
query87	3435	3382	3221	3221
query88	3565	2751	2744	2744
query89	451	393	342	342
query90	1986	189	180	180
query91	180	169	146	146
query92	81	79	73	73
query93	1461	1528	904	904
query94	560	356	315	315
query95	686	382	435	382
query96	1050	791	364	364
query97	2731	2738	2615	2615
query98	236	223	235	223
query99	1172	1145	1034	1034
Total cold run time: 254281 ms
Total hot run time: 171381 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 0.00% (0/37) 🎉
Increment coverage report
Complete coverage report

@deardeng
Copy link
Copy Markdown
Collaborator Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31552 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c68dde0edb9c54eef261a5127ef4f8edbd50c75b, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	18005	4185	3956	3956
q2	q3	10771	1459	803	803
q4	4681	481	350	350
q5	7551	2309	2096	2096
q6	240	176	138	138
q7	914	791	663	663
q8	9436	1801	1563	1563
q9	5194	4969	4956	4956
q10	6371	2205	1895	1895
q11	441	271	243	243
q12	628	436	293	293
q13	18169	3390	2710	2710
q14	265	250	235	235
q15	q16	824	775	713	713
q17	950	917	936	917
q18	6940	5706	5573	5573
q19	1333	1377	1128	1128
q20	577	436	278	278
q21	6256	2820	2673	2673
q22	459	369	452	369
Total cold run time: 100005 ms
Total hot run time: 31552 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4782	4832	4724	4724
q2	q3	4972	5313	4651	4651
q4	2136	2245	1414	1414
q5	5026	4700	4727	4700
q6	234	178	128	128
q7	1979	1759	1559	1559
q8	2398	2095	2113	2095
q9	8008	7389	7430	7389
q10	4958	4659	4233	4233
q11	551	387	351	351
q12	742	735	519	519
q13	2996	3377	2790	2790
q14	283	279	255	255
q15	q16	689	697	621	621
q17	1265	1243	1261	1243
q18	7309	6955	7141	6955
q19	1155	1084	1098	1084
q20	2209	2219	1963	1963
q21	5272	4556	4435	4435
q22	536	469	405	405
Total cold run time: 57500 ms
Total hot run time: 51514 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 172798 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c68dde0edb9c54eef261a5127ef4f8edbd50c75b, data reload: false

query5	4323	664	527	527
query6	335	225	198	198
query7	4233	549	309	309
query8	340	242	221	221
query9	8810	4050	4042	4042
query10	475	345	301	301
query11	5664	2587	2218	2218
query12	185	130	124	124
query13	1311	641	450	450
query14	6110	5496	5189	5189
query14_1	4482	4456	4406	4406
query15	217	208	187	187
query16	1045	481	458	458
query17	1179	763	626	626
query18	2730	500	372	372
query19	230	214	169	169
query20	141	134	132	132
query21	225	142	121	121
query22	13573	13545	13433	13433
query23	17514	16586	16344	16344
query23_1	16478	16308	16344	16308
query24	7376	1786	1339	1339
query24_1	1317	1308	1322	1308
query25	607	504	448	448
query26	1319	345	181	181
query27	2678	556	359	359
query28	4435	2041	2022	2022
query29	1007	646	534	534
query30	309	252	203	203
query31	1119	1089	979	979
query32	93	80	79	79
query33	557	389	323	323
query34	1188	1142	700	700
query35	776	793	698	698
query36	1439	1448	1265	1265
query37	156	106	89	89
query38	3189	3163	3043	3043
query39	929	912	897	897
query39_1	890	895	878	878
query40	232	148	125	125
query41	65	62	63	62
query42	111	106	111	106
query43	329	332	289	289
query44	
query45	214	204	203	203
query46	1096	1253	746	746
query47	2359	2370	2259	2259
query48	409	443	298	298
query49	643	495	385	385
query50	1037	349	256	256
query51	4317	4344	4300	4300
query52	104	108	94	94
query53	244	287	199	199
query54	317	275	255	255
query55	93	91	84	84
query56	311	305	292	292
query57	1468	1427	1355	1355
query58	307	277	269	269
query59	1551	1594	1437	1437
query60	328	328	308	308
query61	164	160	163	160
query62	698	654	589	589
query63	253	209	209	209
query64	2366	828	647	647
query65	
query66	1655	495	367	367
query67	29721	29726	29574	29574
query68	
query69	483	336	310	310
query70	1025	1022	998	998
query71	301	272	262	262
query72	2946	2756	2457	2457
query73	859	739	434	434
query74	5086	4948	4783	4783
query75	2718	2620	2270	2270
query76	2323	1154	778	778
query77	398	422	340	340
query78	12404	12354	11982	11982
query79	1457	1078	731	731
query80	654	549	448	448
query81	463	282	243	243
query82	1401	166	122	122
query83	353	281	252	252
query84	257	146	113	113
query85	892	552	474	474
query86	419	336	315	315
query87	3368	3373	3250	3250
query88	3636	2767	2740	2740
query89	449	396	349	349
query90	1931	182	181	181
query91	182	169	144	144
query92	83	81	74	74
query93	1452	1450	863	863
query94	536	302	322	302
query95	689	474	349	349
query96	1073	806	362	362
query97	2775	2753	2608	2608
query98	235	231	232	231
query99	1189	1134	1046	1046
Total cold run time: 254317 ms
Total hot run time: 172798 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 0.00% (0/30) 🎉
Increment coverage report
Complete coverage report

@deardeng
Copy link
Copy Markdown
Collaborator Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31588 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 60a1dcd1680d4d1041262dbeea6b6058b57c30c4, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17655	4102	4023	4023
q2	q3	10840	1376	807	807
q4	4685	482	340	340
q5	7584	2260	2100	2100
q6	250	179	138	138
q7	943	784	651	651
q8	9368	1665	1558	1558
q9	5126	4950	4910	4910
q10	6383	2199	1833	1833
q11	425	277	253	253
q12	645	428	296	296
q13	18122	3371	2764	2764
q14	266	259	231	231
q15	q16	830	773	719	719
q17	970	899	875	875
q18	7166	5765	5737	5737
q19	2717	1194	1188	1188
q20	583	448	281	281
q21	6430	2871	2560	2560
q22	558	382	324	324
Total cold run time: 101546 ms
Total hot run time: 31588 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4900	4816	4878	4816
q2	q3	4903	5316	4699	4699
q4	2097	2297	1389	1389
q5	5069	4669	4637	4637
q6	247	196	143	143
q7	1896	1702	1600	1600
q8	2414	2153	2112	2112
q9	7771	7419	7444	7419
q10	4704	4653	4213	4213
q11	548	398	348	348
q12	734	732	534	534
q13	3073	3328	2803	2803
q14	274	277	261	261
q15	q16	672	693	602	602
q17	1285	1272	1270	1270
q18	7369	6849	6907	6849
q19	1117	1127	1112	1112
q20	2221	2221	1944	1944
q21	5276	4561	4421	4421
q22	517	450	417	417
Total cold run time: 57087 ms
Total hot run time: 51589 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 171707 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 60a1dcd1680d4d1041262dbeea6b6058b57c30c4, data reload: false

query5	4295	669	528	528
query6	342	229	204	204
query7	4367	557	329	329
query8	325	238	223	223
query9	8782	4050	4034	4034
query10	451	345	297	297
query11	5816	2417	2260	2260
query12	192	125	123	123
query13	1273	605	444	444
query14	6107	5518	5180	5180
query14_1	4565	4526	4425	4425
query15	212	206	183	183
query16	1023	444	433	433
query17	1122	714	573	573
query18	2690	482	355	355
query19	222	202	161	161
query20	137	133	128	128
query21	213	137	115	115
query22	13630	13723	13406	13406
query23	17206	16607	16214	16214
query23_1	16367	16241	16354	16241
query24	7443	1784	1330	1330
query24_1	1319	1318	1339	1318
query25	565	476	422	422
query26	1302	324	176	176
query27	2700	539	354	354
query28	4434	2025	2013	2013
query29	1031	644	489	489
query30	309	231	196	196
query31	1126	1082	958	958
query32	86	75	81	75
query33	555	356	305	305
query34	1184	1151	651	651
query35	767	793	687	687
query36	1384	1404	1276	1276
query37	158	96	93	93
query38	3242	3160	3076	3076
query39	925	924	892	892
query39_1	863	899	868	868
query40	230	149	127	127
query41	66	63	62	62
query42	112	111	112	111
query43	336	333	297	297
query44	
query45	219	201	206	201
query46	1113	1226	721	721
query47	2378	2442	2249	2249
query48	412	417	306	306
query49	641	497	390	390
query50	996	356	255	255
query51	4411	4405	4244	4244
query52	107	104	92	92
query53	259	284	205	205
query54	333	293	264	264
query55	96	92	87	87
query56	309	314	325	314
query57	1445	1434	1340	1340
query58	301	271	272	271
query59	1584	1644	1444	1444
query60	325	324	323	323
query61	185	183	184	183
query62	702	656	592	592
query63	247	210	210	210
query64	2495	865	700	700
query65	
query66	1719	500	390	390
query67	29896	29766	29559	29559
query68	
query69	514	360	320	320
query70	1102	1027	979	979
query71	313	275	274	274
query72	3236	2915	2454	2454
query73	876	766	450	450
query74	5117	5034	4790	4790
query75	2697	2599	2271	2271
query76	2330	1160	805	805
query77	403	408	341	341
query78	12502	12424	11821	11821
query79	1515	1044	770	770
query80	651	536	464	464
query81	456	284	242	242
query82	1386	164	120	120
query83	373	280	257	257
query84	251	141	114	114
query85	887	526	450	450
query86	410	317	344	317
query87	3463	3408	3233	3233
query88	3656	2730	2750	2730
query89	456	390	349	349
query90	1975	191	182	182
query91	178	169	172	169
query92	84	75	77	75
query93	1503	1422	841	841
query94	537	351	327	327
query95	672	489	366	366
query96	1032	783	336	336
query97	2729	2741	2634	2634
query98	239	228	232	228
query99	1186	1169	1036	1036
Total cold run time: 255415 ms
Total hot run time: 171707 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 0.00% (0/16) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants