Skip to content

[Improvementation](profile) add some profile on vcollect_iterator#32794

Merged
BiteTheDDDDt merged 1 commit intoapache:masterfrom
BiteTheDDDDt:dev_0325
Mar 26, 2024
Merged

[Improvementation](profile) add some profile on vcollect_iterator#32794
BiteTheDDDDt merged 1 commit intoapache:masterfrom
BiteTheDDDDt:dev_0325

Conversation

@BiteTheDDDDt
Copy link
Contributor

Proposed changes

add some profile on vcollect_iterator

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@BiteTheDDDDt
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38339 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1efef8b88239bdfe65f1a7b7eedf3f4fbe38fbad, data reload: false

------ Round 1 ----------------------------------
q1	17623	4480	4162	4162
q2	2112	157	151	151
q3	10796	1212	1220	1212
q4	10692	757	824	757
q5	7460	3123	3081	3081
q6	206	127	124	124
q7	1053	615	576	576
q8	9446	2023	2037	2023
q9	7362	6749	6703	6703
q10	8386	3409	3590	3409
q11	420	226	219	219
q12	369	204	192	192
q13	17812	2843	2861	2843
q14	238	204	201	201
q15	501	462	475	462
q16	471	369	374	369
q17	955	541	615	541
q18	7217	6506	6480	6480
q19	1822	1452	1515	1452
q20	558	258	248	248
q21	3619	2922	2832	2832
q22	353	302	311	302
Total cold run time: 109471 ms
Total hot run time: 38339 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4198	4076	4094	4076
q2	326	238	238	238
q3	3019	2822	2836	2822
q4	1853	1555	1600	1555
q5	5366	5399	5395	5395
q6	195	117	118	117
q7	2253	1881	1798	1798
q8	3187	3331	3311	3311
q9	8765	8711	8761	8711
q10	3825	3770	3796	3770
q11	554	450	452	450
q12	729	532	540	532
q13	16891	2862	2858	2858
q14	272	244	249	244
q15	501	447	451	447
q16	496	438	440	438
q17	1746	1498	1488	1488
q18	7511	7289	7158	7158
q19	1610	1494	1511	1494
q20	1906	1736	1707	1707
q21	4722	4663	4750	4663
q22	548	458	450	450
Total cold run time: 70473 ms
Total hot run time: 53722 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 181892 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1efef8b88239bdfe65f1a7b7eedf3f4fbe38fbad, data reload: false

query1	937	377	363	363
query2	6537	2000	1930	1930
query3	6706	208	204	204
query4	31729	21317	21299	21299
query5	4303	399	397	397
query6	273	181	170	170
query7	4621	303	296	296
query8	233	177	173	173
query9	9392	2291	2281	2281
query10	560	243	262	243
query11	17327	14315	14379	14315
query12	132	93	93	93
query13	1621	420	411	411
query14	10016	8061	7868	7868
query15	305	198	188	188
query16	8232	269	255	255
query17	2110	607	562	562
query18	2102	285	279	279
query19	352	151	163	151
query20	92	88	89	88
query21	204	132	128	128
query22	4905	4855	4756	4756
query23	33623	32682	32919	32682
query24	11882	2912	2867	2867
query25	669	387	404	387
query26	1773	157	162	157
query27	3056	352	347	347
query28	7637	1942	1899	1899
query29	1036	649	622	622
query30	308	152	159	152
query31	959	744	728	728
query32	97	58	58	58
query33	772	254	257	254
query34	1110	486	499	486
query35	851	624	610	610
query36	1031	865	907	865
query37	265	67	68	67
query38	3488	3417	3485	3417
query39	1616	1483	1420	1420
query40	293	118	113	113
query41	52	51	47	47
query42	106	96	101	96
query43	495	461	444	444
query44	1182	733	722	722
query45	280	261	261	261
query46	1115	723	716	716
query47	1917	1872	1854	1854
query48	445	350	373	350
query49	1248	346	338	338
query50	763	379	372	372
query51	6614	6625	6602	6602
query52	113	92	91	91
query53	348	277	281	277
query54	323	243	241	241
query55	86	83	78	78
query56	255	234	229	229
query57	1255	1138	1136	1136
query58	239	214	212	212
query59	2854	2882	2586	2586
query60	263	256	249	249
query61	116	117	118	117
query62	671	452	458	452
query63	307	278	280	278
query64	6404	4045	4048	4045
query65	3129	3031	3045	3031
query66	1423	360	364	360
query67	15279	14876	14724	14724
query68	6795	539	515	515
query69	605	382	377	377
query70	1206	1145	1147	1145
query71	487	263	265	263
query72	6637	2756	2544	2544
query73	736	317	313	313
query74	6822	6434	6315	6315
query75	3441	2212	2240	2212
query76	4649	910	914	910
query77	601	251	260	251
query78	10982	10101	10135	10101
query79	8078	532	529	529
query80	1492	382	375	375
query81	530	216	216	216
query82	1219	96	86	86
query83	208	143	144	143
query84	287	84	78	78
query85	1519	333	316	316
query86	478	309	310	309
query87	3846	3539	3546	3539
query88	4744	2275	2273	2273
query89	520	375	366	366
query90	2003	177	183	177
query91	168	136	140	136
query92	65	50	49	49
query93	6045	495	479	479
query94	1157	179	177	177
query95	439	337	348	337
query96	617	279	271	271
query97	2681	2516	2477	2477
query98	229	218	212	212
query99	1254	918	912	912
Total cold run time: 310977 ms
Total hot run time: 181892 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1efef8b88239bdfe65f1a7b7eedf3f4fbe38fbad with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       22.1 seconds inserted 10000000 Rows, about 452K ops/s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.25% (8731/24769)
Line Coverage: 27.05% (71495/264349)
Region Coverage: 26.28% (37096/141143)
Branch Coverage: 23.18% (18966/81838)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1efef8b88239bdfe65f1a7b7eedf3f4fbe38fbad_1efef8b88239bdfe65f1a7b7eedf3f4fbe38fbad/report/index.html

Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 26, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@BiteTheDDDDt BiteTheDDDDt merged commit b9d8cb3 into apache:master Mar 26, 2024
yiguolei pushed a commit that referenced this pull request Mar 26, 2024
@BiteTheDDDDt BiteTheDDDDt deleted the dev_0325 branch January 20, 2025 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants