Skip to content

[log](load) print reason when LoadPathMgr failed to delete file #38235

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Jul 23, 2024
Merged

[log](load) print reason when LoadPathMgr failed to delete file #38235

dataroaring merged 1 commit into from
Jul 23, 2024

Conversation

@kaijchen
Copy link
Member

@kaijchen kaijchen commented Jul 23, 2024

Proposed changes

print reason when LoadPathMgr failed to delete file

@doris-robot
Copy link

doris-robot commented Jul 23, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@kaijchen
Copy link
Member Author

kaijchen commented Jul 23, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jul 23, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Jul 23, 2024

TPC-H: Total hot run time: 40107 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8e27d7b383797ea417d1f672155831d7e7768b2a, data reload: false

------ Round 1 ----------------------------------
q1	17611	4468	4315	4315
q2	2019	191	194	191
q3	10441	1263	1197	1197
q4	10205	766	900	766
q5	7544	2692	2696	2692
q6	224	140	139	139
q7	964	603	601	601
q8	9355	2089	2093	2089
q9	10110	6593	6598	6593
q10	8913	3788	3796	3788
q11	451	241	233	233
q12	417	228	220	220
q13	17785	2988	3018	2988
q14	290	240	233	233
q15	515	484	493	484
q16	508	396	380	380
q17	978	699	661	661
q18	7984	7391	7424	7391
q19	1623	1455	1388	1388
q20	682	313	327	313
q21	4946	3170	3163	3163
q22	347	285	282	282
Total cold run time: 113912 ms
Total hot run time: 40107 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4349	4276	4278	4276
q2	390	280	261	261
q3	3044	2737	2698	2698
q4	1893	1624	1681	1624
q5	5306	5302	5318	5302
q6	224	130	134	130
q7	2146	1774	1676	1676
q8	3192	3350	3309	3309
q9	8446	8398	8403	8398
q10	3895	3732	3721	3721
q11	569	484	492	484
q12	780	601	618	601
q13	17619	2945	2992	2945
q14	307	280	278	278
q15	511	489	483	483
q16	500	414	433	414
q17	1781	1495	1478	1478
q18	7444	7368	7494	7368
q19	1653	1474	1571	1474
q20	2007	1824	1774	1774
q21	4966	4789	4747	4747
q22	572	498	503	498
Total cold run time: 71594 ms
Total hot run time: 53939 ms

liaoxin01
liaoxin01 approved these changes Jul 23, 2024
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 23, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jul 23, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 23, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Jul 23, 2024

TPC-DS: Total hot run time: 172291 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8e27d7b383797ea417d1f672155831d7e7768b2a, data reload: false

query1	903	372	371	371
query2	6437	1939	1839	1839
query3	6632	211	215	211
query4	23139	17507	17328	17328
query5	4196	485	472	472
query6	265	189	182	182
query7	4575	291	285	285
query8	259	189	186	186
query9	8488	2391	2380	2380
query10	443	292	276	276
query11	10650	9852	10206	9852
query12	140	88	82	82
query13	1644	377	367	367
query14	10228	7747	7575	7575
query15	207	175	163	163
query16	7753	463	477	463
query17	1323	564	519	519
query18	1939	274	285	274
query19	201	145	150	145
query20	86	86	85	85
query21	207	123	128	123
query22	4251	4000	3896	3896
query23	33849	33150	33295	33150
query24	11138	2985	2832	2832
query25	618	361	394	361
query26	1428	147	149	147
query27	2970	270	274	270
query28	7344	1973	1968	1968
query29	944	622	633	622
query30	288	147	151	147
query31	975	745	759	745
query32	91	53	54	53
query33	773	332	324	324
query34	898	469	494	469
query35	849	724	726	724
query36	1081	934	898	898
query37	146	76	82	76
query38	2853	2763	2720	2720
query39	854	818	800	800
query40	274	122	148	122
query41	48	46	45	45
query42	113	101	100	100
query43	500	490	466	466
query44	1159	710	714	710
query45	189	166	158	158
query46	1070	734	711	711
query47	1861	1742	1726	1726
query48	372	287	297	287
query49	1077	412	426	412
query50	802	397	396	396
query51	6801	6796	6677	6677
query52	105	94	92	92
query53	363	287	291	287
query54	873	439	445	439
query55	77	72	74	72
query56	286	278	274	274
query57	1161	1059	1094	1059
query58	262	244	272	244
query59	2830	2710	2640	2640
query60	340	278	280	278
query61	99	99	96	96
query62	837	648	639	639
query63	333	290	305	290
query64	10398	2192	1650	1650
query65	3191	3109	3111	3109
query66	1315	350	346	346
query67	15681	14925	15111	14925
query68	4592	523	534	523
query69	461	344	340	340
query70	1107	1172	1098	1098
query71	403	288	280	280
query72	7089	5568	5390	5390
query73	757	323	327	323
query74	6094	5735	5667	5667
query75	3414	2663	2669	2663
query76	2486	934	899	899
query77	468	306	308	306
query78	9547	8913	8952	8913
query79	2660	511	526	511
query80	1872	479	490	479
query81	611	220	222	220
query82	1071	138	149	138
query83	304	174	177	174
query84	267	86	86	86
query85	1361	333	384	333
query86	462	331	333	331
query87	3271	3052	3074	3052
query88	3441	2372	2393	2372
query89	488	383	391	383
query90	1801	193	190	190
query91	131	102	99	99
query92	69	47	49	47
query93	1851	497	491	491
query94	1258	285	293	285
query95	401	332	318	318
query96	601	275	268	268
query97	3206	3068	3025	3025
query98	218	203	198	198
query99	1603	1261	1259	1259
Total cold run time: 276847 ms
Total hot run time: 172291 ms

@doris-robot
Copy link

doris-robot commented Jul 23, 2024

ClickBench: Total hot run time: 30.6 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8e27d7b383797ea417d1f672155831d7e7768b2a, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.08	0.07
query5	0.49	0.49	0.48
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.49	0.49
query10	0.55	0.54	0.54
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.59	0.59
query14	0.77	0.78	0.77
query15	0.87	0.80	0.82
query16	0.35	0.37	0.36
query17	0.96	1.03	1.03
query18	0.23	0.22	0.22
query19	1.92	1.75	1.72
query20	0.01	0.01	0.01
query21	15.41	0.76	0.65
query22	4.58	7.33	1.84
query23	18.27	1.36	1.28
query24	2.09	0.24	0.24
query25	0.17	0.09	0.08
query26	0.30	0.22	0.22
query27	0.45	0.24	0.23
query28	13.23	1.02	0.99
query29	12.64	3.27	3.29
query30	0.26	0.06	0.05
query31	2.87	0.39	0.39
query32	3.25	0.47	0.46
query33	2.90	2.97	2.92
query34	16.94	4.33	4.36
query35	4.41	4.44	4.40
query36	0.67	0.46	0.49
query37	0.18	0.16	0.15
query38	0.15	0.15	0.15
query39	0.04	0.03	0.03
query40	0.15	0.12	0.12
query41	0.10	0.05	0.05
query42	0.05	0.04	0.05
query43	0.04	0.05	0.04
Total cold run time: 109.99 s
Total hot run time: 30.6 s

dataroaring
dataroaring approved these changes Jul 23, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit ffe7e41 into apache:master Jul 23, 2024
dataroaring pushed a commit that referenced this pull request Jul 24, 2024
@kaijchen @dataroaring
[log](load) print reason when LoadPathMgr failed to delete file (#38235)
31bc598 
print reason when LoadPathMgr failed to delete file
kaijchen added a commit to kaijchen/doris that referenced this pull request Jul 25, 2024
@kaijchen
[log](load) print reason when LoadPathMgr failed to delete file (apac…
e5e7781 
…he#38235)

print reason when LoadPathMgr failed to delete file
kaijchen added a commit to kaijchen/doris that referenced this pull request Jul 25, 2024
@kaijchen
[log](load) print reason when LoadPathMgr failed to delete file (apac…
7722d39 
…he#38235)

print reason when LoadPathMgr failed to delete file
This was referenced Jul 25, 2024
Merged
Merged
dataroaring pushed a commit that referenced this pull request Jul 25, 2024
@kaijchen
[log](load) print reason when LoadPathMgr failed to delete file (#38235…
54787f5 
…) (#38340)

backport #38235
dataroaring pushed a commit that referenced this pull request Jul 25, 2024
@kaijchen
[log](load) print reason when LoadPathMgr failed to delete file (#38235…
2bbd0aa 
…) (#38339)

backport #38235
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@kaijchen
[log](load) print reason when LoadPathMgr failed to delete file (apac…
8701989 
…he#38235) (apache#38339)

backport apache#38235
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.14-merged dev/2.1.6-merged dev/3.0.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kaijchen @doris-robot @liaoxin01 @dataroaring