Skip to content

Conversation

@w41ter
Copy link
Contributor

@w41ter w41ter commented Jun 20, 2024

Cherry-pick #36487

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Jun 20, 2024

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.86% (8107/21415)
Line Coverage: 29.53% (66354/224705)
Region Coverage: 29.00% (34199/117915)
Branch Coverage: 24.87% (17559/70598)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1090fd90dde7770999cd96805c1c4b50c303fba0_1090fd90dde7770999cd96805c1c4b50c303fba0/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49929 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1090fd90dde7770999cd96805c1c4b50c303fba0, data reload: false

------ Round 1 ----------------------------------
q1	17531	4311	4289	4289
q2	2068	159	148	148
q3	10263	1902	1892	1892
q4	10102	1252	1331	1252
q5	8481	3914	3901	3901
q6	237	125	127	125
q7	2022	1614	1608	1608
q8	9308	2751	2720	2720
q9	10771	10989	10249	10249
q10	8648	3517	3518	3517
q11	432	238	262	238
q12	480	317	310	310
q13	18344	4018	4045	4018
q14	346	332	341	332
q15	508	466	460	460
q16	670	564	576	564
q17	1136	965	986	965
q18	7282	6982	6867	6867
q19	1791	1665	1603	1603
q20	545	320	310	310
q21	4468	4172	4122	4122
q22	532	439	455	439
Total cold run time: 115965 ms
Total hot run time: 49929 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4313	4324	4305	4305
q2	326	227	229	227
q3	4167	4172	4180	4172
q4	2754	2753	2752	2752
q5	7200	7114	7149	7114
q6	237	132	126	126
q7	3336	2861	2807	2807
q8	4432	4467	4517	4467
q9	17054	16851	16762	16762
q10	4294	4289	4314	4289
q11	741	667	667	667
q12	1024	855	875	855
q13	6598	3743	3759	3743
q14	455	429	421	421
q15	505	462	454	454
q16	738	677	676	676
q17	3828	3878	3886	3878
q18	8973	8769	8858	8769
q19	1731	1708	1656	1656
q20	2380	2125	2138	2125
q21	8484	8564	8585	8564
q22	1071	1028	1011	1011
Total cold run time: 84641 ms
Total hot run time: 79840 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204628 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1090fd90dde7770999cd96805c1c4b50c303fba0, data reload: false

query1	931	426	391	391
query2	6539	2662	2640	2640
query3	6921	207	199	199
query4	20287	18067	17945	17945
query5	19744	6550	6551	6550
query6	306	223	244	223
query7	4156	312	323	312
query8	457	407	437	407
query9	3180	2733	2671	2671
query10	409	298	298	298
query11	11400	10731	10613	10613
query12	127	79	74	74
query13	5611	705	713	705
query14	18213	13284	13421	13284
query15	369	251	244	244
query16	6454	293	264	264
query17	1691	1462	885	885
query18	2293	416	420	416
query19	221	155	153	153
query20	81	80	80	80
query21	189	97	95	95
query22	5176	5177	5015	5015
query23	32793	31986	32191	31986
query24	6940	6583	6674	6583
query25	534	441	418	418
query26	530	176	165	165
query27	1858	308	303	303
query28	6081	2408	2359	2359
query29	2914	2761	2637	2637
query30	240	165	169	165
query31	918	737	762	737
query32	69	65	62	62
query33	385	267	268	267
query34	855	480	495	480
query35	1112	918	913	913
query36	1501	1185	1286	1185
query37	90	60	63	60
query38	3115	2920	2909	2909
query39	1357	1331	1336	1331
query40	209	93	97	93
query41	47	43	44	43
query42	86	91	86	86
query43	677	780	700	700
query44	1150	721	717	717
query45	251	240	238	238
query46	1229	983	993	983
query47	1821	1857	1814	1814
query48	1016	726	722	722
query49	628	366	381	366
query50	878	602	623	602
query51	4753	4706	4655	4655
query52	95	85	80	80
query53	452	315	333	315
query54	2715	2480	2494	2480
query55	99	77	88	77
query56	242	235	205	205
query57	1236	1162	1168	1162
query58	227	208	208	208
query59	4410	4046	3909	3909
query60	220	215	218	215
query61	99	95	106	95
query62	857	451	504	451
query63	493	343	347	343
query64	2533	1520	1459	1459
query65	3647	3557	3547	3547
query66	812	380	394	380
query67	15438	15967	16616	15967
query68	9130	647	653	647
query69	560	333	353	333
query70	1584	1459	1504	1459
query71	416	307	299	299
query72	6619	3497	3534	3497
query73	741	321	320	320
query74	6338	5857	5808	5808
query75	5274	3794	3699	3699
query76	5636	1151	1193	1151
query77	938	253	260	253
query78	12551	11764	11477	11477
query79	8460	641	651	641
query80	1071	405	414	405
query81	491	231	238	231
query82	843	101	99	99
query83	170	137	133	133
query84	256	73	70	70
query85	881	326	326	326
query86	332	299	303	299
query87	3230	2976	3046	2976
query88	4559	2333	2332	2332
query89	483	304	296	296
query90	1951	222	215	215
query91	176	137	150	137
query92	57	54	56	54
query93	6642	620	577	577
query94	707	214	210	210
query95	1104	1076	1048	1048
query96	664	328	326	326
query97	6398	6327	6441	6327
query98	198	172	173	172
query99	2955	910	875	875
Total cold run time: 313782 ms
Total hot run time: 204628 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1090fd90dde7770999cd96805c1c4b50c303fba0, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.04	0.05
query4	1.78	0.06	0.06
query5	0.54	0.52	0.52
query6	1.23	0.67	0.63
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.54	0.47	0.49
query10	0.54	0.55	0.54
query11	0.12	0.08	0.09
query12	0.12	0.10	0.09
query13	0.62	0.62	0.62
query14	0.80	0.78	0.81
query15	0.78	0.76	0.76
query16	0.35	0.39	0.37
query17	1.00	1.02	0.96
query18	0.23	0.24	0.28
query19	1.90	1.89	1.87
query20	0.01	0.00	0.01
query21	15.47	0.55	0.56
query22	2.05	1.93	1.63
query23	17.58	0.95	0.91
query24	7.05	0.42	1.62
query25	0.57	0.06	0.07
query26	0.99	0.17	0.16
query27	0.04	0.03	0.04
query28	6.58	0.75	0.71
query29	12.65	2.35	2.29
query30	0.54	0.52	0.53
query31	2.81	0.39	0.38
query32	3.37	0.50	0.49
query33	3.11	3.04	3.13
query34	15.28	4.77	4.79
query35	4.86	4.82	4.84
query36	1.07	1.01	1.01
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.15	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.01
Total cold run time: 105.35 s
Total hot run time: 30.13 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1090fd90dde7770999cd96805c1c4b50c303fba0 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@xiaokang xiaokang changed the title [chore](be) Improve ingesting binlog error checking [chore](be) Improve ingesting binlog error checking #36487 Jun 20, 2024
@xiaokang xiaokang merged commit 94da4c9 into apache:branch-2.0 Jun 20, 2024
@w41ter w41ter deleted the pick_36487 branch June 20, 2024 11:37
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants