Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](routine-load) fix routine load lock do not release #33275

Merged
merged 1 commit into from
Apr 6, 2024

Conversation

HHoflittlefish777
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@HHoflittlefish777
Copy link
Contributor Author

run buildall

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 5, 2024
Copy link
Contributor

github-actions bot commented Apr 5, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Apr 5, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 38438 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 62ddbbe6b1b53daf9983ad82996e4dcfc5f8db5f, data reload: false

------ Round 1 ----------------------------------
q1	17646	4038	4055	4038
q2	2004	186	191	186
q3	10470	1139	1296	1139
q4	10206	847	891	847
q5	7469	2974	2942	2942
q6	217	132	132	132
q7	1084	612	620	612
q8	9411	2004	2020	2004
q9	6797	6200	6142	6142
q10	8459	3490	3543	3490
q11	420	242	242	242
q12	393	214	208	208
q13	17770	2922	2900	2900
q14	266	228	243	228
q15	519	483	490	483
q16	518	384	379	379
q17	948	874	859	859
q18	7175	6408	6561	6408
q19	1589	1536	1527	1527
q20	563	313	310	310
q21	3512	3072	3067	3067
q22	362	295	304	295
Total cold run time: 107798 ms
Total hot run time: 38438 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4064	3989	4007	3989
q2	319	215	216	215
q3	2953	2945	2923	2923
q4	1852	1829	1803	1803
q5	5176	5186	5178	5178
q6	205	122	122	122
q7	2221	1793	1841	1793
q8	3189	3269	3265	3265
q9	8558	8497	8497	8497
q10	3716	3872	3953	3872
q11	552	468	445	445
q12	754	573	568	568
q13	16912	3099	3052	3052
q14	299	265	270	265
q15	543	510	496	496
q16	496	434	448	434
q17	1763	1733	1725	1725
q18	8146	7740	7709	7709
q19	1702	1680	1693	1680
q20	2036	1836	1894	1836
q21	5008	4949	5002	4949
q22	491	440	425	425
Total cold run time: 70955 ms
Total hot run time: 55241 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 181369 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 62ddbbe6b1b53daf9983ad82996e4dcfc5f8db5f, data reload: false

query1	1220	366	1113	366
query2	6315	2079	1966	1966
query3	6648	206	206	206
query4	24383	21513	21334	21334
query5	4186	404	403	403
query6	276	179	171	171
query7	4596	301	297	297
query8	228	174	171	171
query9	8487	2310	2336	2310
query10	450	240	260	240
query11	14945	14575	14762	14575
query12	133	91	89	89
query13	1649	377	378	377
query14	8421	6858	6865	6858
query15	212	169	172	169
query16	6821	277	268	268
query17	984	608	568	568
query18	1852	286	305	286
query19	197	157	160	157
query20	92	89	93	89
query21	191	134	128	128
query22	4944	4781	4910	4781
query23	33199	32716	32796	32716
query24	10699	3133	3131	3131
query25	693	402	422	402
query26	1604	171	169	169
query27	3169	381	381	381
query28	7215	1923	1949	1923
query29	1110	608	623	608
query30	306	167	179	167
query31	1004	757	739	739
query32	97	62	60	60
query33	637	255	246	246
query34	1133	510	511	510
query35	845	729	717	717
query36	997	864	872	864
query37	148	72	69	69
query38	3618	3619	3617	3617
query39	1662	1705	1603	1603
query40	192	136	131	131
query41	47	43	45	43
query42	117	104	100	100
query43	460	427	442	427
query44	1187	729	754	729
query45	279	281	275	275
query46	1091	800	782	782
query47	1956	1876	1848	1848
query48	384	307	310	307
query49	849	379	362	362
query50	837	414	435	414
query51	6899	6698	6805	6698
query52	98	102	92	92
query53	359	280	302	280
query54	258	236	233	233
query55	87	75	76	75
query56	241	224	227	224
query57	1275	1177	1182	1177
query58	236	216	221	216
query59	2656	2424	2540	2424
query60	250	232	235	232
query61	90	88	88	88
query62	666	461	448	448
query63	308	278	275	275
query64	4886	3159	3130	3130
query65	3043	3015	3000	3000
query66	928	318	315	315
query67	15384	14777	14711	14711
query68	9089	575	610	575
query69	559	299	297	297
query70	1387	1054	1100	1054
query71	454	274	276	274
query72	6440	2717	2535	2535
query73	1489	326	322	322
query74	6601	6342	6288	6288
query75	3491	2292	2304	2292
query76	5678	1124	1232	1124
query77	656	243	242	242
query78	10891	10194	10157	10157
query79	10353	522	525	522
query80	1552	425	416	416
query81	492	234	228	228
query82	382	98	91	91
query83	204	161	165	161
query84	263	85	87	85
query85	927	286	273	273
query86	352	306	284	284
query87	3763	3484	3512	3484
query88	3787	2367	2373	2367
query89	527	370	368	368
query90	1920	177	178	177
query91	129	100	101	100
query92	63	48	47	47
query93	6218	542	518	518
query94	1155	186	181	181
query95	429	313	309	309
query96	605	272	274	272
query97	2661	2503	2513	2503
query98	225	217	216	216
query99	1216	843	840	840
Total cold run time: 294387 ms
Total hot run time: 181369 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.42 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 62ddbbe6b1b53daf9983ad82996e4dcfc5f8db5f, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.05	0.04
query3	0.23	0.04	0.04
query4	1.68	0.06	0.07
query5	0.48	0.47	0.47
query6	1.14	0.66	0.66
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.50	0.51
query10	0.55	0.57	0.57
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.60	0.60
query14	0.79	0.78	0.79
query15	0.86	0.83	0.85
query16	0.35	0.35	0.36
query17	0.95	0.99	0.98
query18	0.25	0.23	0.26
query19	1.83	1.70	1.74
query20	0.01	0.01	0.01
query21	15.40	0.63	0.63
query22	3.85	6.69	1.26
query23	18.01	1.30	1.24
query24	1.48	0.19	0.22
query25	0.16	0.08	0.07
query26	0.27	0.16	0.16
query27	0.07	0.08	0.08
query28	13.79	0.96	0.95
query29	12.62	3.30	3.24
query30	0.28	0.06	0.05
query31	2.87	0.38	0.38
query32	3.28	0.48	0.48
query33	2.91	2.85	2.85
query34	15.50	4.35	4.33
query35	4.38	4.38	4.35
query36	0.67	0.46	0.46
query37	0.17	0.14	0.15
query38	0.16	0.14	0.14
query39	0.05	0.04	0.03
query40	0.17	0.14	0.15
query41	0.11	0.04	0.04
query42	0.06	0.05	0.06
query43	0.04	0.04	0.03
Total cold run time: 107.07 s
Total hot run time: 29.42 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 62ddbbe6b1b53daf9983ad82996e4dcfc5f8db5f with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       16.6 seconds inserted 10000000 Rows, about 602K ops/s

@dataroaring dataroaring merged commit 8429353 into apache:master Apr 6, 2024
29 of 32 checks passed
seawinde pushed a commit to seawinde/doris that referenced this pull request Apr 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.x reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants