Skip to content

[fix](regression) fault injection may cause fd to be closed twice#30070

Merged
yiguolei merged 1 commit intoapache:masterfrom
liaoxin01:fix_inject
Jan 18, 2024
Merged

[fix](regression) fault injection may cause fd to be closed twice#30070
yiguolei merged 1 commit intoapache:masterfrom
liaoxin01:fix_inject

Conversation

@liaoxin01
Copy link
Contributor

@liaoxin01 liaoxin01 commented Jan 17, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@liaoxin01
Copy link
Contributor Author

liaoxin01 commented Jan 17, 2024

run buildall

dataroaring
dataroaring approved these changes Jan 17, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 17, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2024

PR approved by anyone and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2024

clang-tidy review says "All clean, LGTM! 👍"

sollhui
sollhui approved these changes Jan 17, 2024
View reviewed changes
Copy link
Contributor

@sollhui sollhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

JackDrogon
JackDrogon reviewed Jan 17, 2024
View reviewed changes
Copy link
Contributor

@JackDrogon JackDrogon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

doris-robot commented Jan 17, 2024

TPC-H: Total hot run time: 39045 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4d45aedddc3851109765090a9a4bdb38879557f5, data reload: false

------ Round 1 ----------------------------------
q1	18294	5375	5291	5291
q2	2380	153	140	140
q3	10865	1271	1222	1222
q4	10865	778	799	778
q5	7864	3160	3149	3149
q6	199	122	123	122
q7	874	508	491	491
q8	9236	1970	1945	1945
q9	7320	6386	6409	6386
q10	8204	3064	3055	3055
q11	429	222	214	214
q12	355	187	189	187
q13	18009	3354	3347	3347
q14	251	213	214	213
q15	575	504	499	499
q16	435	386	390	386
q17	932	602	494	494
q18	7546	6950	6766	6766
q19	1564	1362	1377	1362
q20	588	329	295	295
q21	2751	2478	2402	2402
q22	357	301	315	301
Total cold run time: 109893 ms
Total hot run time: 39045 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5489	5245	5339	5245
q2	325	221	210	210
q3	3300	3283	3313	3283
q4	2115	2072	2055	2055
q5	6103	5973	5884	5884
q6	198	118	115	115
q7	2371	1872	1910	1872
q8	3259	3375	3394	3375
q9	8888	8855	8838	8838
q10	3965	3909	3848	3848
q11	570	465	458	458
q12	825	668	655	655
q13	16936	3161	3135	3135
q14	285	263	261	261
q15	557	502	507	502
q16	527	454	471	454
q17	1884	1837	1845	1837
q18	9670	19353	9635	9635
q19	23282	1589	1532	1532
q20	4616	1951	1939	1939
q21	14493	5143	5428	5143
q22	1002	568	567	567
Total cold run time: 110660 ms
Total hot run time: 60843 ms

@doris-robot
Copy link

doris-robot commented Jan 17, 2024

TeamCity be ut coverage result:
Function Coverage: 36.66% (8648/23588)
Line Coverage: 28.73% (70623/245811)
Region Coverage: 27.63% (36442/131907)
Branch Coverage: 24.34% (18644/76594)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4d45aedddc3851109765090a9a4bdb38879557f5_4d45aedddc3851109765090a9a4bdb38879557f5/report/index.html

@doris-robot
Copy link

doris-robot commented Jan 17, 2024

TPC-DS: Total hot run time: 177448 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4d45aedddc3851109765090a9a4bdb38879557f5, data reload: false

query1	929	335	326	326
query2	6572	2013	1846	1846
query3	6698	206	202	202
query4	33024	22402	22190	22190
query5	6869	527	600	527
query6	256	170	169	169
query7	4610	265	258	258
query8	225	187	187	187
query9	8401	2517	2512	2512
query10	424	241	227	227
query11	16970	15532	15573	15532
query12	118	69	67	67
query13	1706	367	376	367
query14	10488	6874	6820	6820
query15	208	179	185	179
query16	4602	236	242	236
query17	979	474	468	468
query18	1780	260	254	254
query19	180	143	142	142
query20	73	72	68	68
query21	204	136	134	134
query22	4999	4891	4936	4891
query23	31765	30970	30683	30683
query24	12881	2823	2802	2802
query25	579	323	306	306
query26	1814	142	148	142
query27	3567	273	272	272
query28	7214	1821	1805	1805
query29	2170	642	620	620
query30	287	140	141	140
query31	941	752	744	744
query32	84	53	50	50
query33	724	211	207	207
query34	1260	457	460	457
query35	909	780	759	759
query36	1391	1220	1276	1220
query37	171	62	60	60
query38	3374	3249	3220	3220
query39	1325	1260	1274	1260
query40	346	92	84	84
query41	38	35	34	34
query42	91	77	83	77
query43	527	499	465	465
query44	1075	687	698	687
query45	198	189	179	179
query46	1077	659	656	656
query47	1670	1562	1576	1562
query48	424	320	317	317
query49	1213	293	285	285
query50	680	308	312	308
query51	5305	5213	5220	5213
query52	86	80	77	77
query53	327	254	267	254
query54	880	434	442	434
query55	89	73	79	73
query56	186	164	167	164
query57	991	926	945	926
query58	197	165	165	165
query59	3030	2661	2732	2661
query60	218	187	188	187
query61	88	86	88	86
query62	622	359	401	359
query63	293	254	252	252
query64	6121	1757	1757	1757
query65	3356	3267	3267	3267
query66	1434	318	322	318
query67	15558	15412	15442	15412
query68	11308	507	514	507
query69	620	297	297	297
query70	1723	1562	1480	1480
query71	583	215	214	214
query72	4982	2835	2845	2835
query73	2167	312	324	312
query74	7021	6505	6494	6494
query75	5070	2263	2235	2235
query76	6293	1082	1000	1000
query77	903	238	237	237
query78	9770	8726	8516	8516
query79	1012	512	497	497
query80	638	320	304	304
query81	448	203	214	203
query82	207	84	80	80
query83	158	120	120	120
query84	273	73	72	72
query85	1066	327	342	327
query86	395	391	368	368
query87	3510	3387	3394	3387
query88	3002	2252	2261	2252
query89	427	350	350	350
query90	2140	191	192	191
query91	158	128	136	128
query92	59	43	45	43
query93	1068	437	404	404
query94	1291	159	157	157
query95	500	448	451	448
query96	605	324	325	324
query97	4246	4147	4154	4147
query98	205	192	191	191
query99	1038	720	732	720
Total cold run time: 299777 ms
Total hot run time: 177448 ms

@doris-robot
Copy link

doris-robot commented Jan 17, 2024

ClickBench: Total hot run time: 30.31 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4d45aedddc3851109765090a9a4bdb38879557f5, data reload: false

query1	0.03	0.03	0.02
query2	0.06	0.02	0.02
query3	0.22	0.05	0.05
query4	1.67	0.06	0.07
query5	0.54	0.53	0.52
query6	1.26	0.63	0.63
query7	0.02	0.01	0.02
query8	0.04	0.02	0.02
query9	0.54	0.50	0.49
query10	0.57	0.56	0.55
query11	0.12	0.09	0.09
query12	0.10	0.09	0.09
query13	0.60	0.60	0.59
query14	0.78	0.78	0.78
query15	0.81	0.79	0.79
query16	0.38	0.35	0.37
query17	1.00	1.03	1.03
query18	0.24	0.24	0.23
query19	1.90	1.78	1.79
query20	0.01	0.01	0.01
query21	15.40	0.58	0.56
query22	2.46	2.77	1.54
query23	17.09	0.86	0.75
query24	2.58	1.15	0.85
query25	0.35	0.11	0.09
query26	0.60	0.14	0.13
query27	0.06	0.06	0.06
query28	11.77	0.81	0.76
query29	12.58	3.18	3.20
query30	0.56	0.49	0.47
query31	2.78	0.34	0.34
query32	3.34	0.48	0.48
query33	3.21	3.23	3.19
query34	15.81	4.31	4.23
query35	4.32	4.24	4.25
query36	1.11	1.08	1.08
query37	0.06	0.05	0.05
query38	0.04	0.03	0.02
query39	0.03	0.01	0.01
query40	0.16	0.13	0.13
query41	0.07	0.01	0.02
query42	0.03	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 105.33 s
Total hot run time: 30.31 s

@doris-robot
Copy link

doris-robot commented Jan 17, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4d45aedddc3851109765090a9a4bdb38879557f5 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       14.8 seconds inserted 10000000 Rows, about 675K ops/s

@yiguolei yiguolei merged commit 5a330a1 into apache:master Jan 18, 2024
yiguolei pushed a commit that referenced this pull request Jan 18, 2024
@liaoxin01 @Doris-Extras
[fix](regression) fault injection may cause fd to be closed twice (#3…
9714a3c 
…0070)
hello-stephen pushed a commit to hello-stephen/doris that referenced this pull request Jan 19, 2024
@liaoxin01
[fix](regression) fault injection may cause fd to be closed twice (ap…
e0e8af1 
…ache#30070)
seawinde pushed a commit to seawinde/doris that referenced this pull request Jan 22, 2024
@liaoxin01 @seawinde
[fix](regression) fault injection may cause fd to be closed twice (ap…
8003ff5 
…ache#30070)
@liaoxin01 liaoxin01 deleted the fix_inject branch February 6, 2024 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

+2 more reviewers

@JackDrogon JackDrogon JackDrogon left review comments

@sollhui sollhui sollhui approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.0-merged dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@liaoxin01 @doris-robot @JackDrogon @sollhui @dataroaring @xiaokang @yiguolei