[Fix](smooth-upgrade) fix unnecessary high version of smooth upgrade by zclllyybb · Pull Request #30283 · apache/doris

zclllyybb · 2024-01-23T12:42:28Z

Proposed changes

Issue Number: close #xxx

all the versions should be consistent with Doris' version advance. until now version 2.1 of Doris has not been released. So all changes of smooth-upgrade should and could safely use version number 3. If not so and if we import new function replacement, it will be vulnerable.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

zclllyybb · 2024-01-23T12:43:35Z

run buildall

github-actions · 2024-01-23T12:52:32Z

clang-tidy review says "All clean, LGTM! 👍"

HappenLee

LGTM

github-actions · 2024-01-23T13:16:15Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-01-23T13:16:18Z

PR approved by anyone and no changes requested.

doris-robot · 2024-01-23T13:31:41Z

TPC-H: Total hot run time: 39500 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc8308ffeedec193c4633e914196db496b6bddc7, data reload: false

------ Round 1 ----------------------------------
q1	17877	5452	5723	5452
q2	2045	140	138	138
q3	10808	1186	1224	1186
q4	10409	802	859	802
q5	7764	3206	3164	3164
q6	204	127	127	127
q7	874	506	495	495
q8	9224	1949	1961	1949
q9	7301	6433	6398	6398
q10	8275	3068	3080	3068
q11	411	226	204	204
q12	363	200	196	196
q13	18001	3411	3401	3401
q14	247	210	215	210
q15	553	512	507	507
q16	449	369	378	369
q17	982	580	544	544
q18	7417	7037	6836	6836
q19	1749	1349	1365	1349
q20	593	334	297	297
q21	2819	2513	2487	2487
q22	380	321	323	321
Total cold run time: 108745 ms
Total hot run time: 39500 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5595	5341	5257	5257
q2	332	220	214	214
q3	3344	3266	3279	3266
q4	2144	2038	2009	2009
q5	6045	5973	5906	5906
q6	208	124	121	121
q7	2315	1891	1852	1852
q8	3250	3387	3408	3387
q9	9052	8851	8940	8851
q10	3962	3855	3925	3855
q11	576	466	465	465
q12	812	605	628	605
q13	16930	3208	3193	3193
q14	287	253	267	253
q15	555	512	508	508
q16	528	486	482	482
q17	1871	1825	1888	1825
q18	9559	13489	9823	9823
q19	26012	1564	1558	1558
q20	4608	1948	1939	1939
q21	15142	5572	5425	5425
q22	1548	545	556	545
Total cold run time: 114675 ms
Total hot run time: 61339 ms

doris-robot · 2024-01-23T13:43:27Z

TPC-DS: Total hot run time: 187511 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc8308ffeedec193c4633e914196db496b6bddc7, data reload: false

query1	949	334	334	334
query2	6555	2019	1938	1938
query3	6711	202	198	198
query4	31705	22288	22288	22288
query5	4447	398	374	374
query6	251	165	161	161
query7	4601	290	263	263
query8	225	181	182	181
query9	8305	2576	2549	2549
query10	427	247	262	247
query11	16702	15595	15516	15516
query12	126	73	68	68
query13	1668	378	395	378
query14	10611	7032	7103	7032
query15	218	188	186	186
query16	5784	253	250	250
query17	951	472	485	472
query18	1791	257	261	257
query19	173	136	148	136
query20	74	77	71	71
query21	190	139	126	126
query22	4824	4514	4662	4514
query23	32319	31082	31016	31016
query24	5356	2933	2879	2879
query25	409	322	314	314
query26	774	147	148	147
query27	2256	298	289	289
query28	3758	1876	1836	1836
query29	918	646	638	638
query30	196	140	137	137
query31	929	755	752	752
query32	66	51	57	51
query33	415	219	216	216
query34	831	470	478	470
query35	884	850	818	818
query36	1293	1202	1235	1202
query37	100	62	59	59
query38	3369	3285	3204	3204
query39	1299	1279	1263	1263
query40	199	90	83	83
query41	39	36	34	34
query42	92	83	87	83
query43	509	487	465	465
query44	1081	693	687	687
query45	196	179	180	179
query46	1081	667	672	667
query47	1688	1511	1536	1511
query48	399	308	304	304
query49	631	299	303	299
query50	694	318	321	318
query51	5351	5355	5150	5150
query52	95	83	89	83
query53	318	260	254	254
query54	218	188	187	187
query55	87	77	78	77
query56	179	170	178	170
query57	971	919	960	919
query58	181	168	167	167
query59	2957	2929	2776	2776
query60	216	195	193	193
query61	83	82	82	82
query62	425	361	360	360
query63	286	267	266	266
query64	2536	1829	1802	1802
query65	3407	3345	3319	3319
query66	1165	318	342	318
query67	15684	15008	15180	15008
query68	12715	523	509	509
query69	622	316	317	316
query70	1656	1536	1525	1525
query71	10452	10205	10205	10205
query72	5096	2831	2846	2831
query73	3085	316	315	315
query74	6930	6454	6392	6392
query75	5473	2314	2374	2314
query76	6496	1069	1015	1015
query77	679	237	246	237
query78	10007	8641	8693	8641
query79	1024	511	493	493
query80	611	330	328	328
query81	470	212	208	208
query82	204	83	80	80
query83	143	122	122	122
query84	276	70	69	69
query85	1107	338	316	316
query86	387	389	361	361
query87	3539	3394	3321	3321
query88	3192	2220	2228	2220
query89	438	365	348	348
query90	2245	192	189	189
query91	164	121	125	121
query92	54	50	49	49
query93	1586	424	445	424
query94	1480	169	162	162
query95	509	443	453	443
query96	632	341	317	317
query97	4254	4170	4170	4170
query98	202	190	193	190
query99	1031	715	689	689
Total cold run time: 289987 ms
Total hot run time: 187511 ms

doris-robot · 2024-01-23T13:48:52Z

ClickBench: Total hot run time: 30.71 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc8308ffeedec193c4633e914196db496b6bddc7, data reload: false

query1	0.04	0.02	0.02
query2	0.06	0.02	0.02
query3	0.23	0.06	0.05
query4	1.69	0.08	0.08
query5	0.53	0.53	0.53
query6	1.30	0.64	0.64
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.55	0.50	0.49
query10	0.55	0.55	0.56
query11	0.12	0.09	0.08
query12	0.11	0.09	0.09
query13	0.61	0.61	0.60
query14	0.78	0.81	0.80
query15	0.79	0.78	0.79
query16	0.38	0.38	0.39
query17	1.04	0.98	1.02
query18	0.24	0.28	0.24
query19	1.82	1.80	1.77
query20	0.01	0.01	0.01
query21	15.42	0.58	0.56
query22	2.29	2.27	1.79
query23	17.09	0.80	0.82
query24	2.37	1.26	0.86
query25	0.37	0.28	0.11
query26	0.48	0.15	0.14
query27	0.08	0.04	0.05
query28	11.32	0.76	0.75
query29	12.55	3.18	3.15
query30	0.57	0.51	0.51
query31	2.78	0.34	0.35
query32	3.38	0.49	0.48
query33	3.22	3.23	3.23
query34	15.92	4.25	4.27
query35	4.42	4.28	4.26
query36	1.11	1.08	1.06
query37	0.07	0.05	0.05
query38	0.04	0.03	0.02
query39	0.02	0.01	0.02
query40	0.15	0.13	0.12
query41	0.06	0.01	0.02
query42	0.03	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.67 s
Total hot run time: 30.71 s

doris-robot · 2024-01-23T13:51:56Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit bc8308ffeedec193c4633e914196db496b6bddc7 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.4 seconds inserted 10000000 Rows, about 746K ops/s

zclllyybb · 2024-01-24T08:05:07Z

run buildall

github-actions · 2024-01-24T08:13:02Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-01-24T08:14:41Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-01-24T08:49:20Z

TPC-H: Total hot run time: 39338 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a1d73ca2f2c13d43d9e53edfe9a23f6a8573f81b, data reload: false

------ Round 1 ----------------------------------
q1	17651	5394	5272	5272
q2	2044	141	134	134
q3	10891	1176	1178	1176
q4	11491	739	790	739
q5	7867	3255	3213	3213
q6	201	125	125	125
q7	898	521	515	515
q8	9267	2012	1997	1997
q9	7943	6569	6545	6545
q10	8233	3061	3038	3038
q11	411	240	209	209
q12	362	186	186	186
q13	17990	3370	3355	3355
q14	250	205	212	205
q15	540	509	513	509
q16	448	400	375	375
q17	947	565	485	485
q18	7727	6928	6778	6778
q19	1594	1433	1447	1433
q20	569	307	303	303
q21	2927	2573	2436	2436
q22	351	324	310	310
Total cold run time: 110602 ms
Total hot run time: 39338 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5417	5292	5280	5280
q2	333	225	215	215
q3	3317	3202	3256	3202
q4	2146	2118	2013	2013
q5	6036	5927	5955	5927
q6	199	119	131	119
q7	2295	1888	1866	1866
q8	3210	3375	3389	3375
q9	9058	8940	8865	8865
q10	4016	3850	3874	3850
q11	564	463	467	463
q12	810	628	591	591
q13	16922	3171	3114	3114
q14	282	250	267	250
q15	549	501	506	501
q16	521	483	471	471
q17	2597	1718	1851	1718
q18	9433	16106	9657	9657
q19	20752	1590	1565	1565
q20	4622	1937	1930	1930
q21	14418	5339	5369	5339
q22	991	522	539	522
Total cold run time: 108488 ms
Total hot run time: 60833 ms

doris-robot · 2024-01-24T08:49:44Z

TeamCity be ut coverage result:
Function Coverage: 36.26% (8609/23741)
Line Coverage: 28.37% (70467/248390)
Region Coverage: 27.35% (36363/132962)
Branch Coverage: 24.15% (18642/77188)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a1d73ca2f2c13d43d9e53edfe9a23f6a8573f81b_a1d73ca2f2c13d43d9e53edfe9a23f6a8573f81b/report/index.html

doris-robot · 2024-01-24T09:01:23Z

TPC-DS: Total hot run time: 186894 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a1d73ca2f2c13d43d9e53edfe9a23f6a8573f81b, data reload: false

query1	951	333	332	332
query2	6562	2016	1938	1938
query3	6704	208	197	197
query4	32781	22384	22238	22238
query5	4445	449	368	368
query6	247	154	164	154
query7	4633	272	266	266
query8	224	183	193	183
query9	8379	2498	2487	2487
query10	418	229	216	216
query11	16828	15675	15470	15470
query12	116	74	71	71
query13	1664	377	376	376
query14	10515	6839	6886	6839
query15	214	182	183	182
query16	5785	259	252	252
query17	949	470	475	470
query18	1802	268	255	255
query19	182	126	131	126
query20	70	69	73	69
query21	186	137	121	121
query22	5152	4988	4970	4970
query23	31609	30713	30829	30713
query24	12594	2788	2839	2788
query25	561	309	320	309
query26	1823	149	144	144
query27	3228	282	285	282
query28	7108	1813	1789	1789
query29	2120	637	611	611
query30	284	137	136	136
query31	940	748	759	748
query32	83	55	51	51
query33	720	211	206	206
query34	1168	456	461	456
query35	899	753	771	753
query36	1221	1217	1202	1202
query37	99	58	61	58
query38	3398	3271	3288	3271
query39	1324	1282	1250	1250
query40	343	84	78	78
query41	37	35	34	34
query42	90	82	88	82
query43	526	468	479	468
query44	1085	682	680	680
query45	193	177	180	177
query46	1139	659	657	657
query47	1623	1572	1586	1572
query48	395	316	301	301
query49	1222	293	279	279
query50	751	316	311	311
query51	5328	5248	5198	5198
query52	94	72	74	72
query53	340	263	251	251
query54	249	183	185	183
query55	79	80	77	77
query56	185	169	172	169
query57	1012	940	960	940
query58	189	160	162	160
query59	2918	2603	2619	2603
query60	205	187	178	178
query61	80	86	81	81
query62	629	380	366	366
query63	279	251	262	251
query64	6109	1769	1751	1751
query65	3326	3284	3233	3233
query66	1379	320	313	313
query67	15883	15155	15406	15155
query68	10898	520	509	509
query69	590	305	303	303
query70	1644	1514	1474	1474
query71	10429	10203	10204	10203
query72	4754	2794	2795	2794
query73	2141	327	318	318
query74	7055	6472	6408	6408
query75	4691	2318	2269	2269
query76	6479	1112	1111	1111
query77	633	235	237	235
query78	9220	8963	8700	8700
query79	1299	501	490	490
query80	624	322	308	308
query81	444	206	201	201
query82	186	88	78	78
query83	146	120	121	120
query84	275	67	66	66
query85	1070	339	305	305
query86	391	354	354	354
query87	3546	3393	3401	3393
query88	3535	2212	2226	2212
query89	473	349	363	349
query90	1973	187	187	187
query91	154	123	140	123
query92	54	46	46	46
query93	3504	434	416	416
query94	1341	160	159	159
query95	509	465	451	451
query96	683	316	312	312
query97	4273	4114	4149	4114
query98	213	193	190	190
query99	1010	745	729	729
Total cold run time: 307911 ms
Total hot run time: 186894 ms

doris-robot · 2024-01-24T09:06:48Z

ClickBench: Total hot run time: 30.38 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a1d73ca2f2c13d43d9e53edfe9a23f6a8573f81b, data reload: false

query1	0.03	0.02	0.02
query2	0.05	0.02	0.02
query3	0.23	0.05	0.05
query4	1.71	0.07	0.06
query5	0.54	0.52	0.52
query6	1.28	0.63	0.64
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.54	0.50	0.48
query10	0.55	0.54	0.55
query11	0.12	0.09	0.08
query12	0.10	0.09	0.09
query13	0.62	0.61	0.59
query14	0.81	0.78	0.80
query15	0.78	0.78	0.77
query16	0.39	0.38	0.36
query17	1.00	1.02	1.01
query18	0.23	0.27	0.23
query19	1.85	1.80	1.81
query20	0.02	0.01	0.01
query21	15.41	0.57	0.57
query22	2.90	2.44	1.77
query23	17.40	0.76	0.86
query24	2.32	0.74	1.65
query25	0.48	0.26	0.15
query26	0.43	0.14	0.15
query27	0.05	0.04	0.05
query28	11.03	0.76	0.80
query29	12.49	3.10	3.15
query30	0.54	0.45	0.54
query31	2.79	0.34	0.34
query32	3.38	0.47	0.48
query33	3.20	3.29	3.21
query34	15.83	4.34	4.26
query35	4.36	4.22	4.29
query36	1.11	1.07	1.07
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.13	0.13
query41	0.07	0.01	0.01
query42	0.03	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 105.01 s
Total hot run time: 30.38 s

doris-robot · 2024-01-24T09:09:53Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit a1d73ca2f2c13d43d9e53edfe9a23f6a8573f81b with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       14.7 seconds inserted 10000000 Rows, about 680K ops/s

…30283) fix unnecessary high version of smooth upgrade

zclllyybb changed the title ~~[Fix](smooth-upgrade)~~ [Fix](smooth-upgrade) fix unnecessary high version of smooth upgrade Jan 23, 2024

HappenLee previously approved these changes Jan 23, 2024

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jan 23, 2024

zclllyybb dismissed HappenLee’s stale review via c3950ed January 24, 2024 08:03

zclllyybb added 2 commits January 24, 2024 16:04

fix

b22510a

fix ut

a1d73ca

zclllyybb force-pushed the fix_version branch from c3950ed to a1d73ca Compare January 24, 2024 08:04

zhangstar333 approved these changes Jan 24, 2024

View reviewed changes

BiteTheDDDDt approved these changes Jan 24, 2024

View reviewed changes

BiteTheDDDDt merged commit 71de5d8 into apache:master Jan 24, 2024

yiguolei pushed a commit that referenced this pull request Jan 25, 2024

[Fix](smooth-upgrade) fix unnecessary high version of smooth upgrade (#…

2e6a006

…30283) fix unnecessary high version of smooth upgrade

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

Conversation

zclllyybb commented Jan 23, 2024

Proposed changes

Further comments

Uh oh!

zclllyybb commented Jan 23, 2024

Uh oh!

github-actions bot commented Jan 23, 2024

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 23, 2024

Uh oh!

github-actions bot commented Jan 23, 2024

Uh oh!

doris-robot commented Jan 23, 2024

Uh oh!

doris-robot commented Jan 23, 2024

Uh oh!

doris-robot commented Jan 23, 2024

Uh oh!

doris-robot commented Jan 23, 2024

Uh oh!

zclllyybb commented Jan 24, 2024

Uh oh!

github-actions bot commented Jan 24, 2024

Uh oh!

github-actions bot commented Jan 24, 2024

Uh oh!

doris-robot commented Jan 24, 2024

Uh oh!

doris-robot commented Jan 24, 2024

Uh oh!

doris-robot commented Jan 24, 2024

Uh oh!

doris-robot commented Jan 24, 2024

Uh oh!

doris-robot commented Jan 24, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants