Skip to content

[fix]fix be core when migration tablet to other disk #37712#38726

Merged
xiaokang merged 1 commit intoapache:branch-2.0from
xiaokang:pick_37712_to_apache_branch-2.0
Aug 1, 2024
Merged

[fix]fix be core when migration tablet to other disk #37712#38726
xiaokang merged 1 commit intoapache:branch-2.0from
xiaokang:pick_37712_to_apache_branch-2.0

Conversation

@xiaokang
Copy link
Contributor

@xiaokang xiaokang commented Aug 1, 2024

cherry pick from #37712

@z404289981 @xiaokang
[fix]fix be core when migration tablet to other disk (apache#37712)
37ebab8 
![image](https://github.com/user-attachments/assets/01f67160-3ebe-41a1-ac79-a7173d14605c)
The asynchronous task reference captures a local variable

## Proposed changes

Issue Number: close apache#36809

<!--Describe your changes.-->

Co-authored-by: huronghui <huronghui@huronghuideMac-mini.local>
@xiaokang
Copy link
Contributor Author

xiaokang commented Aug 1, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Aug 1, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Aug 1, 2024

TPC-H: Total hot run time: 50504 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 37ebab8cd2fbb50ecfda889783cea91c3a834495, data reload: false

------ Round 1 ----------------------------------
q1	17559	4388	4346	4346
q2	2067	158	151	151
q3	10255	1931	1923	1923
q4	10108	1264	1327	1264
q5	8551	3999	3991	3991
q6	237	126	126	126
q7	2038	1610	1612	1610
q8	9993	2886	2933	2886
q9	11450	10580	10544	10544
q10	8682	3531	3507	3507
q11	436	239	258	239
q12	513	303	311	303
q13	18455	4025	4098	4025
q14	354	332	342	332
q15	517	454	459	454
q16	696	578	571	571
q17	1134	974	1001	974
q18	7324	6849	6890	6849
q19	1804	1668	1539	1539
q20	518	315	287	287
q21	4414	4175	4147	4147
q22	547	444	436	436
Total cold run time: 117652 ms
Total hot run time: 50504 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4354	4331	4331	4331
q2	324	225	225	225
q3	4161	4218	4192	4192
q4	2775	2768	2784	2768
q5	7197	7131	7121	7121
q6	246	124	123	123
q7	3263	2897	2922	2897
q8	4446	4504	4532	4504
q9	17520	17172	17153	17153
q10	4291	4287	4339	4287
q11	771	681	712	681
q12	1040	849	853	849
q13	5400	3761	3777	3761
q14	451	415	427	415
q15	504	464	456	456
q16	746	684	681	681
q17	3873	3932	3806	3806
q18	8899	8828	8921	8828
q19	1741	1735	1704	1704
q20	2410	2203	2129	2129
q21	8664	8585	8534	8534
q22	1095	997	966	966
Total cold run time: 84171 ms
Total hot run time: 80411 ms

@doris-robot
Copy link

doris-robot commented Aug 1, 2024

TeamCity be ut coverage result:
Function Coverage: 37.75% (8104/21468)
Line Coverage: 29.39% (66370/225819)
Region Coverage: 28.91% (34252/118487)
Branch Coverage: 24.80% (17603/70988)
Coverage Report: http://coverage.selectdb-in.cc/coverage/37ebab8cd2fbb50ecfda889783cea91c3a834495_37ebab8cd2fbb50ecfda889783cea91c3a834495/report/index.html

@doris-robot
Copy link

doris-robot commented Aug 1, 2024

TPC-DS: Total hot run time: 204401 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 37ebab8cd2fbb50ecfda889783cea91c3a834495, data reload: false

query1	961	426	379	379
query2	6543	2759	2481	2481
query3	6919	206	204	204
query4	20173	17995	17950	17950
query5	19758	6516	6529	6516
query6	287	221	243	221
query7	4164	298	307	298
query8	437	413	408	408
query9	3187	2747	2663	2663
query10	400	309	306	306
query11	11277	10725	10757	10725
query12	125	78	73	73
query13	5614	693	709	693
query14	17591	13339	13546	13339
query15	389	251	250	250
query16	6484	290	268	268
query17	1726	1472	867	867
query18	2322	427	419	419
query19	210	163	150	150
query20	76	79	83	79
query21	186	99	96	96
query22	5350	5050	5139	5050
query23	32556	31966	31969	31966
query24	7089	6521	6463	6463
query25	537	435	460	435
query26	529	165	163	163
query27	1867	293	296	293
query28	6237	2462	2403	2403
query29	3073	2786	2767	2767
query30	246	171	167	167
query31	916	715	743	715
query32	69	68	60	60
query33	416	262	267	262
query34	848	464	469	464
query35	1126	990	922	922
query36	1234	1123	1020	1020
query37	87	62	62	62
query38	3129	2959	2946	2946
query39	1394	1327	1346	1327
query40	206	98	92	92
query41	49	44	44	44
query42	91	82	81	81
query43	865	625	646	625
query44	1140	731	743	731
query45	248	237	234	234
query46	1220	971	971	971
query47	1768	1742	1684	1684
query48	1012	704	720	704
query49	634	388	399	388
query50	872	618	649	618
query51	4763	4665	4728	4665
query52	93	79	82	79
query53	458	320	317	317
query54	2679	2475	2461	2461
query55	90	83	74	74
query56	236	231	187	187
query57	1226	1122	1066	1066
query58	237	228	208	208
query59	4100	3927	3969	3927
query60	212	218	197	197
query61	103	108	102	102
query62	796	481	457	457
query63	492	353	345	345
query64	2611	1557	1541	1541
query65	3649	3533	3548	3533
query66	781	390	393	390
query67	17422	16079	15913	15913
query68	6634	685	654	654
query69	560	353	354	353
query70	1558	1294	1444	1294
query71	418	314	331	314
query72	6536	3551	3539	3539
query73	734	337	321	321
query74	6380	5902	5792	5792
query75	4643	3688	3675	3675
query76	4299	1165	1213	1165
query77	541	261	270	261
query78	12727	11821	11698	11698
query79	8150	656	668	656
query80	2613	411	434	411
query81	547	239	233	233
query82	1555	98	98	98
query83	197	137	140	137
query84	267	74	71	71
query85	1435	347	351	347
query86	359	304	300	300
query87	3230	3025	3077	3025
query88	5395	2348	2340	2340
query89	382	289	288	288
query90	1745	210	208	208
query91	176	158	139	139
query92	60	52	53	52
query93	4875	560	597	560
query94	893	213	212	212
query95	1123	1059	1040	1040
query96	649	326	329	326
query97	6578	6437	6412	6412
query98	188	180	175	175
query99	2982	801	892	801
Total cold run time: 312014 ms
Total hot run time: 204401 ms

@doris-robot
Copy link

doris-robot commented Aug 1, 2024

ClickBench: Total hot run time: 31.26 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 37ebab8cd2fbb50ecfda889783cea91c3a834495, data reload: false

query1	0.03	0.02	0.03
query2	0.07	0.03	0.03
query3	0.24	0.04	0.05
query4	1.77	0.07	0.06
query5	0.53	0.53	0.51
query6	1.23	0.62	0.61
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.51	0.50	0.50
query10	0.54	0.53	0.53
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.63	0.62	0.61
query14	0.77	0.78	0.80
query15	0.80	0.77	0.77
query16	0.37	0.36	0.38
query17	1.02	1.03	1.02
query18	0.23	0.25	0.23
query19	1.95	1.84	1.88
query20	0.01	0.01	0.01
query21	15.50	0.55	0.55
query22	1.99	3.41	2.10
query23	16.90	1.07	0.87
query24	5.28	0.89	1.47
query25	0.35	0.10	0.06
query26	0.65	0.15	0.16
query27	0.05	0.04	0.03
query28	7.40	0.77	0.78
query29	12.61	2.36	2.32
query30	0.57	0.54	0.52
query31	2.81	0.39	0.38
query32	3.38	0.49	0.50
query33	3.08	3.08	3.07
query34	15.24	4.80	4.84
query35	4.88	4.89	4.86
query36	1.08	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.14	0.15
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 103.13 s
Total hot run time: 31.26 s

@doris-robot
Copy link

doris-robot commented Aug 1, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 37ebab8cd2fbb50ecfda889783cea91c3a834495 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@xiaokang xiaokang merged commit 6a54e57 into apache:branch-2.0 Aug 1, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
@xiaokang
[fix]fix be core when migration tablet to other disk apache#37712 (ap…
3cf911d 
…ache#38726)

Co-authored-by: z404289981
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@xiaokang @doris-robot @z404289981