Skip to content

[fix](cluster key) support cloud mode#40593

Merged
dataroaring merged 1 commit intoapache:masterfrom
mymeiyi:modify-cluster-key-index-2-cloud
Sep 20, 2024
Merged

[fix](cluster key) support cloud mode#40593
dataroaring merged 1 commit intoapache:masterfrom
mymeiyi:modify-cluster-key-index-2-cloud

Conversation

@mymeiyi
Copy link
Contributor

@mymeiyi mymeiyi commented Sep 10, 2024

cluster key support cloud mode

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@mymeiyi mymeiyi force-pushed the modify-cluster-key-index-2-cloud branch from d35dcab to f172952 Compare September 10, 2024 06:35
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Sep 10, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.83% (9393/25507)
Line Coverage: 28.23% (77489/274538)
Region Coverage: 27.62% (40000/144807)
Branch Coverage: 24.24% (20340/83896)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f172952673b0bc6a9de8b71d8af013dcede4022d_f172952673b0bc6a9de8b71d8af013dcede4022d/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 38474 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f172952673b0bc6a9de8b71d8af013dcede4022d, data reload: false

------ Round 1 ----------------------------------
q1	17614	4450	4298	4298
q2	2030	189	189	189
q3	11746	947	1079	947
q4	10523	723	745	723
q5	7757	2889	2884	2884
q6	234	142	141	141
q7	972	618	621	618
q8	9348	2128	2099	2099
q9	7158	6594	6644	6594
q10	7012	2286	2198	2198
q11	500	258	249	249
q12	400	225	224	224
q13	18689	3095	3088	3088
q14	286	245	264	245
q15	539	511	503	503
q16	528	439	423	423
q17	1003	735	721	721
q18	7427	6896	6990	6896
q19	1400	1121	1122	1121
q20	684	337	340	337
q21	4320	2957	2956	2956
q22	1127	1049	1020	1020
Total cold run time: 111297 ms
Total hot run time: 38474 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4402	4377	4357	4357
q2	382	272	275	272
q3	2926	2671	2650	2650
q4	1973	1659	1739	1659
q5	5804	5706	5879	5706
q6	232	138	135	135
q7	2236	1811	1801	1801
q8	3331	3441	3585	3441
q9	8922	8910	8929	8910
q10	3694	3424	3375	3375
q11	610	507	505	505
q12	821	670	668	668
q13	14597	3348	3302	3302
q14	338	298	300	298
q15	541	496	504	496
q16	529	505	506	505
q17	1852	1567	1561	1561
q18	8307	7966	7969	7966
q19	1758	1585	1465	1465
q20	2207	1974	1912	1912
q21	5956	5577	5269	5269
q22	1134	1066	1109	1066
Total cold run time: 72552 ms
Total hot run time: 57319 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199146 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f172952673b0bc6a9de8b71d8af013dcede4022d, data reload: false

query1	1290	904	909	904
query2	6370	1989	1972	1972
query3	10601	4122	3932	3932
query4	59747	28601	23283	23283
query5	5039	504	494	494
query6	401	194	167	167
query7	5636	298	294	294
query8	328	221	214	214
query9	7807	2509	2486	2486
query10	394	268	284	268
query11	17472	15098	15341	15098
query12	152	109	99	99
query13	1451	394	384	384
query14	10592	7438	7311	7311
query15	211	193	179	179
query16	6824	473	461	461
query17	1096	584	587	584
query18	1453	299	291	291
query19	202	166	163	163
query20	128	110	112	110
query21	214	107	107	107
query22	4662	4515	4642	4515
query23	34158	33557	33603	33557
query24	6049	2906	2906	2906
query25	486	384	385	384
query26	594	158	160	158
query27	1581	278	278	278
query28	3727	2102	2120	2102
query29	650	410	409	409
query30	238	154	162	154
query31	952	752	806	752
query32	67	56	54	54
query33	430	289	295	289
query34	879	475	491	475
query35	833	732	739	732
query36	1054	940	942	940
query37	141	89	90	89
query38	4058	3911	3940	3911
query39	1466	1417	1422	1417
query40	199	120	120	120
query41	51	50	48	48
query42	117	98	96	96
query43	529	473	472	472
query44	1105	760	766	760
query45	195	166	171	166
query46	1102	756	745	745
query47	1900	1794	1839	1794
query48	382	295	303	295
query49	794	451	491	451
query50	862	434	428	428
query51	7047	6988	6958	6958
query52	100	91	94	91
query53	253	194	184	184
query54	577	476	475	475
query55	78	78	78	78
query56	292	271	286	271
query57	1227	1124	1052	1052
query58	236	239	267	239
query59	3090	2837	2695	2695
query60	309	293	281	281
query61	126	122	121	121
query62	738	673	678	673
query63	221	193	185	185
query64	1470	763	738	738
query65	3227	3185	3169	3169
query66	666	345	352	345
query67	15913	15599	15443	15443
query68	1565	887	915	887
query69	431	342	341	341
query70	1220	1174	1219	1174
query71	385	369	359	359
query72	5102	3994	3638	3638
query73	608	583	604	583
query74	9078	8863	8980	8863
query75	3077	3027	3036	3027
query76	996	890	910	890
query77	544	406	409	406
query78	9390	9155	9285	9155
query79	970	885	863	863
query80	802	830	806	806
query81	461	258	260	258
query82	270	268	268	268
query83	192	187	193	187
query84	195	111	107	107
query85	624	433	381	381
query86	318	333	309	309
query87	4431	4459	4436	4436
query88	4440	4122	4255	4122
query89	384	369	372	369
query90	1481	330	319	319
query91	121	139	126	126
query92	79	81	77	77
query93	972	983	970	970
query94	542	375	402	375
query95	441	431	431	431
query96	478	500	522	500
query97	3145	3175	3097	3097
query98	236	230	225	225
query99	1347	1309	1301	1301
Total cold run time: 296564 ms
Total hot run time: 199146 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.51 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f172952673b0bc6a9de8b71d8af013dcede4022d, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.04	0.04
query3	0.23	0.06	0.06
query4	1.65	0.10	0.10
query5	0.50	0.50	0.50
query6	1.13	0.74	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.47	0.48
query10	0.56	0.55	0.53
query11	0.15	0.13	0.12
query12	0.16	0.13	0.12
query13	0.60	0.59	0.59
query14	1.38	1.44	1.46
query15	0.85	0.83	0.82
query16	0.38	0.36	0.36
query17	1.08	1.06	1.03
query18	0.22	0.21	0.21
query19	1.93	1.88	1.87
query20	0.02	0.01	0.02
query21	15.39	0.67	0.65
query22	4.00	6.31	2.71
query23	18.33	1.40	1.34
query24	2.11	0.22	0.21
query25	0.15	0.08	0.08
query26	0.28	0.19	0.17
query27	0.07	0.08	0.07
query28	13.28	1.02	1.00
query29	12.59	3.36	3.39
query30	0.25	0.05	0.05
query31	2.88	0.41	0.39
query32	3.24	0.50	0.48
query33	2.98	3.00	3.04
query34	16.97	4.42	4.38
query35	4.51	4.47	4.47
query36	0.66	0.50	0.47
query37	0.19	0.16	0.16
query38	0.17	0.15	0.14
query39	0.04	0.03	0.04
query40	0.15	0.13	0.13
query41	0.10	0.05	0.05
query42	0.07	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.02 s
Total hot run time: 32.51 s

@mymeiyi mymeiyi force-pushed the modify-cluster-key-index-2-cloud branch from f172952 to 7d2092b Compare September 18, 2024 01:56
@mymeiyi mymeiyi force-pushed the modify-cluster-key-index-2-cloud branch from 7d2092b to 6f7a9b7 Compare September 18, 2024 02:04
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Sep 18, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.33% (9576/25654)
Line Coverage: 28.70% (79116/275687)
Region Coverage: 28.19% (40977/145385)
Branch Coverage: 24.81% (20885/84182)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6f7a9b7ae44b1c382704924ab8e686e09435c965_6f7a9b7ae44b1c382704924ab8e686e09435c965/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41559 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6f7a9b7ae44b1c382704924ab8e686e09435c965, data reload: false

------ Round 1 ----------------------------------
q1	17587	8208	7290	7290
q2	2061	166	171	166
q3	10588	1103	1199	1103
q4	10546	741	676	676
q5	7748	3073	3075	3073
q6	238	156	149	149
q7	1011	615	606	606
q8	9428	2027	2061	2027
q9	6866	6385	6387	6385
q10	7056	2273	2303	2273
q11	446	253	253	253
q12	414	225	227	225
q13	17776	2994	2984	2984
q14	242	212	211	211
q15	580	522	519	519
q16	698	612	621	612
q17	977	813	796	796
q18	7287	6668	6763	6668
q19	1458	922	982	922
q20	564	292	283	283
q21	4331	3400	3320	3320
q22	1093	1018	1019	1018
Total cold run time: 108995 ms
Total hot run time: 41559 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7270	7270	7257	7257
q2	331	242	232	232
q3	3087	3061	2916	2916
q4	2010	1826	1841	1826
q5	5636	5654	5659	5654
q6	233	148	154	148
q7	2198	1797	1815	1797
q8	3310	3404	3427	3404
q9	8806	8953	8786	8786
q10	3492	3403	3491	3403
q11	599	483	481	481
q12	830	607	577	577
q13	8694	3162	3177	3162
q14	307	285	286	285
q15	578	540	513	513
q16	742	670	704	670
q17	1798	1592	1598	1592
q18	8312	7804	7736	7736
q19	1758	1741	1461	1461
q20	2121	1904	1915	1904
q21	5510	5309	5478	5309
q22	1109	1038	1068	1038
Total cold run time: 68731 ms
Total hot run time: 60151 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199592 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6f7a9b7ae44b1c382704924ab8e686e09435c965, data reload: false

query1	1278	880	880	880
query2	6408	2087	2018	2018
query3	10765	3937	3903	3903
query4	63824	27844	23860	23860
query5	4920	513	491	491
query6	409	193	176	176
query7	5422	327	301	301
query8	319	237	222	222
query9	8226	2653	2647	2647
query10	449	287	299	287
query11	16806	15311	15940	15311
query12	154	102	103	102
query13	1401	422	427	422
query14	10648	6866	7846	6866
query15	199	174	178	174
query16	6707	499	441	441
query17	1237	573	566	566
query18	1475	311	307	307
query19	222	146	161	146
query20	124	110	110	110
query21	216	105	104	104
query22	4787	4693	4652	4652
query23	34955	34335	33871	33871
query24	6181	2901	2902	2901
query25	492	395	396	395
query26	613	155	154	154
query27	1644	279	283	279
query28	4249	2450	2439	2439
query29	647	413	409	409
query30	231	156	157	156
query31	984	778	808	778
query32	78	53	55	53
query33	421	288	287	287
query34	905	492	490	490
query35	841	717	728	717
query36	1056	936	948	936
query37	149	86	86	86
query38	3974	3883	3974	3883
query39	1465	1425	1416	1416
query40	203	96	95	95
query41	48	46	47	46
query42	115	96	98	96
query43	526	490	459	459
query44	1160	818	793	793
query45	190	160	163	160
query46	1138	766	762	762
query47	1893	1777	1804	1777
query48	472	359	362	359
query49	678	411	423	411
query50	838	422	424	422
query51	7070	6950	6988	6950
query52	109	92	90	90
query53	257	187	185	185
query54	581	472	471	471
query55	81	79	79	79
query56	304	286	287	286
query57	1193	1093	1081	1081
query58	237	246	246	246
query59	3361	2976	3045	2976
query60	307	271	292	271
query61	126	121	122	121
query62	787	649	669	649
query63	228	196	200	196
query64	1376	747	717	717
query65	3258	3190	3181	3181
query66	635	315	314	314
query67	16071	15504	15393	15393
query68	1358	581	562	562
query69	436	303	305	303
query70	1198	1100	1178	1100
query71	324	281	283	281
query72	6143	4314	4135	4135
query73	799	322	323	322
query74	9225	9048	8959	8959
query75	3316	2656	2712	2656
query76	1379	1011	1005	1005
query77	447	299	310	299
query78	10469	9643	9283	9283
query79	1278	896	869	869
query80	789	584	578	578
query81	517	253	257	253
query82	374	232	239	232
query83	161	154	161	154
query84	236	100	106	100
query85	745	410	366	366
query86	325	328	329	328
query87	4444	4312	4278	4278
query88	4465	4043	4025	4025
query89	392	360	358	358
query90	1601	315	312	312
query91	160	159	167	159
query92	80	75	75	75
query93	941	896	907	896
query94	654	375	379	375
query95	451	411	415	411
query96	489	486	482	482
query97	3184	3118	3118	3118
query98	246	221	233	221
query99	1485	1275	1300	1275
Total cold run time: 306234 ms
Total hot run time: 199592 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.09 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6f7a9b7ae44b1c382704924ab8e686e09435c965, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.03	0.02
query3	0.23	0.07	0.06
query4	1.65	0.10	0.09
query5	0.53	0.51	0.52
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.55	0.49	0.48
query10	0.54	0.56	0.55
query11	0.14	0.11	0.10
query12	0.14	0.10	0.10
query13	0.61	0.59	0.59
query14	3.05	3.04	3.09
query15	0.90	0.82	0.83
query16	0.39	0.38	0.38
query17	1.07	1.03	1.06
query18	0.19	0.20	0.19
query19	1.91	1.86	2.06
query20	0.01	0.01	0.01
query21	15.36	0.61	0.60
query22	2.25	2.25	1.16
query23	17.63	0.94	0.78
query24	3.27	0.85	1.43
query25	0.27	0.14	0.08
query26	0.46	0.14	0.13
query27	0.05	0.04	0.04
query28	10.28	1.10	1.08
query29	12.59	3.25	3.30
query30	0.25	0.06	0.06
query31	2.87	0.39	0.38
query32	3.27	0.47	0.46
query33	2.96	3.03	3.04
query34	16.82	4.41	4.44
query35	4.42	4.38	4.41
query36	0.67	0.48	0.51
query37	0.08	0.05	0.06
query38	0.04	0.03	0.03
query39	0.04	0.02	0.02
query40	0.15	0.13	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.08 s
Total hot run time: 32.09 s

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Sep 18, 2024

run cloud_p0

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Sep 18, 2024

run feut

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Sep 18, 2024
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit f23a594 into apache:master Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants