Skip to content

[refactor](queryctx) move tg related code to task group#30829

Merged
yiguolei merged 13 commits intoapache:masterfrom
yiguolei:refactor2
Feb 6, 2024
Merged

[refactor](queryctx) move tg related code to task group#30829
yiguolei merged 13 commits intoapache:masterfrom
yiguolei:refactor2

Conversation

@yiguolei
Copy link
Contributor

@yiguolei yiguolei commented Feb 5, 2024

Proposed changes

  1. init query ctx memtracker in queryctx constructor
  2. set all task group related property during set taskgroup

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@yiguolei
Copy link
Contributor Author

yiguolei commented Feb 5, 2024

run buildall

1 similar comment
@yiguolei
Copy link
Contributor Author

yiguolei commented Feb 5, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37197 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4f03a27adf0ae9db61a0628933937cca7fe3513a, data reload: false

------ Round 1 ----------------------------------
q1	17655	4846	4496	4496
q2	2452	152	136	136
q3	10950	940	959	940
q4	4938	815	746	746
q5	8033	2816	3002	2816
q6	190	124	121	121
q7	1184	770	752	752
q8	9539	2077	2061	2061
q9	7601	6371	6362	6362
q10	8094	2453	2465	2453
q11	417	196	197	196
q12	793	276	290	276
q13	18022	3335	3260	3260
q14	275	253	273	253
q15	526	496	489	489
q16	471	419	417	417
q17	948	489	512	489
q18	6842	6024	5927	5927
q19	1565	1354	1336	1336
q20	606	369	326	326
q21	6741	3119	3052	3052
q22	808	307	293	293
Total cold run time: 108650 ms
Total hot run time: 37197 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4442	4441	4479	4441
q2	330	233	238	233
q3	3030	2884	2833	2833
q4	1840	1626	1616	1616
q5	5243	5274	5217	5217
q6	188	112	115	112
q7	2107	1755	1796	1755
q8	3104	3240	3311	3240
q9	8343	8321	8335	8321
q10	5913	3608	3612	3608
q11	530	457	465	457
q12	757	574	583	574
q13	5201	3137	3090	3090
q14	281	263	265	263
q15	536	491	497	491
q16	532	473	470	470
q17	1855	1636	1641	1636
q18	8090	7783	7542	7542
q19	10945	1534	1548	1534
q20	2155	1913	1892	1892
q21	4805	4604	4644	4604
q22	562	490	433	433
Total cold run time: 70789 ms
Total hot run time: 54362 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.15% (8625/23857)
Line Coverage: 28.21% (70538/250040)
Region Coverage: 27.22% (36410/133768)
Branch Coverage: 24.00% (18643/77686)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4f03a27adf0ae9db61a0628933937cca7fe3513a_4f03a27adf0ae9db61a0628933937cca7fe3513a/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 181999 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4f03a27adf0ae9db61a0628933937cca7fe3513a, data reload: false

query1	920	347	338	338
query2	6530	1946	1995	1946
query3	6729	212	202	202
query4	31692	21946	22149	21946
query5	4309	355	407	355
query6	255	161	155	155
query7	4597	285	274	274
query8	256	170	192	170
query9	9024	2313	2294	2294
query10	421	220	202	202
query11	17570	15276	15374	15276
query12	128	74	75	74
query13	1625	416	418	416
query14	9281	6873	7196	6873
query15	253	178	179	178
query16	8151	266	241	241
query17	2070	533	502	502
query18	2095	269	260	260
query19	359	147	133	133
query20	81	75	76	75
query21	199	129	130	129
query22	4870	4795	4759	4759
query23	31461	30078	30014	30014
query24	10752	2745	2728	2728
query25	580	337	340	337
query26	1462	148	142	142
query27	2932	298	300	298
query28	7387	1852	1830	1830
query29	919	624	609	609
query30	281	133	138	133
query31	906	704	700	700
query32	86	50	50	50
query33	727	226	215	215
query34	1097	447	463	447
query35	851	766	766	766
query36	1036	928	919	919
query37	106	56	59	56
query38	3258	3148	3073	3073
query39	1286	1260	1250	1250
query40	258	95	95	95
query41	36	35	35	35
query42	99	87	85	85
query43	528	490	477	477
query44	1077	694	703	694
query45	191	176	172	172
query46	1054	647	643	643
query47	1591	1532	1563	1532
query48	415	380	356	356
query49	1180	288	275	275
query50	754	386	370	370
query51	5276	5127	5148	5127
query52	111	81	84	81
query53	330	268	277	268
query54	268	211	220	211
query55	82	75	81	75
query56	216	196	202	196
query57	956	894	896	894
query58	205	178	172	172
query59	2440	2323	2340	2323
query60	233	204	210	204
query61	85	92	83	83
query62	657	348	349	348
query63	296	254	281	254
query64	6169	3582	3647	3582
query65	3245	3243	3210	3210
query66	1067	320	303	303
query67	14409	14325	14449	14325
query68	4061	533	525	525
query69	445	305	319	305
query70	1250	1231	1220	1220
query71	319	243	257	243
query72	6370	2855	2679	2679
query73	690	319	316	316
query74	6503	6272	6223	6223
query75	3004	2336	2369	2336
query76	2528	962	883	883
query77	441	231	226	226
query78	9143	8690	8542	8542
query79	2889	489	474	474
query80	2053	353	339	339
query81	534	196	198	196
query82	841	80	80	80
query83	262	126	127	126
query84	286	80	80	80
query85	2290	332	319	319
query86	501	292	304	292
query87	3393	3244	3295	3244
query88	3931	2321	2298	2298
query89	449	355	373	355
query90	2109	163	165	163
query91	147	121	138	121
query92	53	41	44	41
query93	3343	497	461	461
query94	1406	177	175	175
query95	8047	7809	7899	7809
query96	599	279	270	270
query97	4254	4119	4074	4074
query98	229	206	189	189
query99	1116	672	640	640
Total cold run time: 293256 ms
Total hot run time: 181999 ms

@yiguolei
Copy link
Contributor Author

yiguolei commented Feb 5, 2024

run buildall

@yiguolei
Copy link
Contributor Author

yiguolei commented Feb 5, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.17% (8627/23853)
Line Coverage: 28.22% (70563/250033)
Region Coverage: 27.22% (36417/133766)
Branch Coverage: 24.01% (18655/77682)
Coverage Report: http://coverage.selectdb-in.cc/coverage/58a597b12fbc09a67e138ef658ed6c76c49b51fd_58a597b12fbc09a67e138ef658ed6c76c49b51fd/report/index.html

@yiguolei
Copy link
Contributor Author

yiguolei commented Feb 5, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

}

void add_query(TUniqueId query_id) { _query_id_set.insert(query_id); }
void add_query(TUniqueId query_id) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'add_query' can be made static [readability-convert-member-functions-to-static]

Suggested change
void add_query(TUniqueId query_id) {
static void add_query(TUniqueId query_id) {

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.16% (8626/23853)
Line Coverage: 28.22% (70560/250050)
Region Coverage: 27.22% (36408/133771)
Branch Coverage: 24.01% (18653/77682)
Coverage Report: http://coverage.selectdb-in.cc/coverage/92c3ccb27e906822d7b098a15758699a22421eed_92c3ccb27e906822d7b098a15758699a22421eed/report/index.html

@yiguolei
Copy link
Contributor Author

yiguolei commented Feb 5, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.15% (8627/23866)
Line Coverage: 28.18% (70540/250309)
Region Coverage: 27.19% (36407/133906)
Branch Coverage: 23.98% (18646/77766)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f27a01261358f5f6d84a42a4c0830165ea60c463_f27a01261358f5f6d84a42a4c0830165ea60c463/report/index.html

@github-actions
Copy link
Contributor

github-actions bot commented Feb 5, 2024

PR approved by anyone and no changes requested.

return _query_id_set.size();
}

bool is_shutdown() { return _is_shutdown; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'd better drop task group's shutdown as well

@yiguolei
Copy link
Contributor Author

yiguolei commented Feb 5, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@doris-robot
Copy link

TPC-H: Total hot run time: 37352 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 037cfcd89cb1513a21542edbf69439e917acbdcc, data reload: false

------ Round 1 ----------------------------------
q1	17627	4971	4468	4468
q2	2604	146	139	139
q3	11489	955	952	952
q4	4771	832	712	712
q5	8084	2931	2850	2850
q6	188	128	123	123
q7	1187	764	749	749
q8	9565	2080	2041	2041
q9	7944	6376	6350	6350
q10	8111	2451	2487	2451
q11	421	217	209	209
q12	730	281	292	281
q13	18014	3298	3300	3298
q14	276	240	239	239
q15	528	505	490	490
q16	462	401	410	401
q17	930	593	552	552
q18	6820	6009	6027	6009
q19	1565	1377	1308	1308
q20	590	346	340	340
q21	7071	3150	3099	3099
q22	810	291	292	291
Total cold run time: 109787 ms
Total hot run time: 37352 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4402	4486	4425	4425
q2	327	235	237	235
q3	2945	2866	2833	2833
q4	1958	1709	1655	1655
q5	5191	5293	5254	5254
q6	194	113	118	113
q7	2121	1811	1811	1811
q8	3075	3228	3263	3228
q9	8418	8339	8308	8308
q10	5746	3561	3541	3541
q11	562	469	467	467
q12	767	575	579	575
q13	7469	3124	3140	3124
q14	272	249	273	249
q15	539	494	491	491
q16	520	477	455	455
q17	1854	1647	1699	1647
q18	8059	7870	7542	7542
q19	9399	1529	1532	1529
q20	2116	1916	1913	1913
q21	4832	4610	4647	4610
q22	519	475	447	447
Total cold run time: 71285 ms
Total hot run time: 54452 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174343 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 037cfcd89cb1513a21542edbf69439e917acbdcc, data reload: false

query1	953	347	336	336
query2	6536	1886	1849	1849
query3	6716	211	210	210
query4	31573	21937	22015	21937
query5	4270	369	360	360
query6	255	168	167	167
query7	4599	283	279	279
query8	249	180	177	177
query9	9053	2316	2305	2305
query10	415	233	209	209
query11	18418	15242	15218	15218
query12	130	76	77	76
query13	1620	415	419	415
query14	9123	6914	7024	6914
query15	236	186	179	179
query16	8144	260	250	250
query17	1904	535	500	500
query18	2123	282	261	261
query19	355	146	143	143
query20	85	78	76	76
query21	198	134	128	128
query22	4847	4676	4781	4676
query23	30949	30124	30053	30053
query24	10880	2737	2775	2737
query25	593	356	345	345
query26	1433	141	146	141
query27	2944	301	300	300
query28	7632	1844	1857	1844
query29	901	613	621	613
query30	277	131	141	131
query31	924	703	719	703
query32	92	52	49	49
query33	720	224	225	224
query34	1056	455	461	455
query35	841	783	747	747
query36	1006	900	927	900
query37	115	56	56	56
query38	3282	3138	3121	3121
query39	1313	1272	1247	1247
query40	263	95	95	95
query41	37	34	34	34
query42	103	93	87	87
query43	490	460	484	460
query44	1075	694	697	694
query45	193	186	171	171
query46	1050	650	643	643
query47	1564	1454	1548	1454
query48	416	349	349	349
query49	1147	284	284	284
query50	759	377	379	377
query51	5230	5201	5167	5167
query52	102	90	92	90
query53	342	274	263	263
query54	270	220	224	220
query55	80	76	79	76
query56	217	204	200	200
query57	979	861	883	861
query58	191	168	179	168
query59	2364	2390	2257	2257
query60	232	208	218	208
query61	85	86	89	86
query62	641	368	358	358
query63	300	270	274	270
query64	6101	3682	3529	3529
query65	3250	3209	3213	3209
query66	1066	322	307	307
query67	14363	14290	14046	14046
query68	4161	535	526	526
query69	455	320	335	320
query70	1263	1231	1263	1231
query71	330	244	249	244
query72	6022	2891	2666	2666
query73	712	313	320	313
query74	6687	6297	6321	6297
query75	3050	2351	2371	2351
query76	2544	922	910	910
query77	350	229	234	229
query78	9112	8865	8619	8619
query79	2943	482	490	482
query80	2048	356	340	340
query81	525	200	199	199
query82	837	83	78	78
query83	253	124	126	124
query84	280	83	82	82
query85	2169	342	325	325
query86	477	293	300	293
query87	3406	3318	3266	3266
query88	4066	2312	2299	2299
query89	453	359	340	340
query90	2023	165	164	164
query91	158	122	127	122
query92	59	42	42	42
query93	4200	487	476	476
query94	1288	175	175	175
query95	7991	7899	359	359
query96	604	269	270	269
query97	4233	4087	4134	4087
query98	223	212	205	205
query99	1089	693	616	616
Total cold run time: 293681 ms
Total hot run time: 174343 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.96 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 037cfcd89cb1513a21542edbf69439e917acbdcc, data reload: false

query1	0.04	0.04	0.03
query2	0.06	0.02	0.03
query3	0.22	0.06	0.06
query4	1.66	0.09	0.10
query5	0.54	0.52	0.52
query6	1.18	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.55	0.50	0.50
query10	0.56	0.54	0.55
query11	0.12	0.08	0.09
query12	0.10	0.09	0.09
query13	0.61	0.62	0.60
query14	0.79	0.79	0.79
query15	0.80	0.78	0.78
query16	0.38	0.39	0.39
query17	1.02	0.96	0.99
query18	0.23	0.22	0.27
query19	1.92	1.73	1.72
query20	0.01	0.01	0.01
query21	15.40	0.56	0.57
query22	2.80	2.12	1.21
query23	17.28	0.80	0.83
query24	2.66	1.12	0.53
query25	0.34	0.14	0.11
query26	0.53	0.14	0.14
query27	0.04	0.06	0.04
query28	11.87	0.84	0.82
query29	12.49	3.25	3.22
query30	0.72	0.55	0.58
query31	2.80	0.34	0.34
query32	3.36	0.49	0.48
query33	3.22	3.19	3.22
query34	15.73	4.36	4.31
query35	4.32	4.27	4.38
query36	1.12	1.05	1.05
query37	0.07	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.01	0.02
query40	0.17	0.12	0.12
query41	0.07	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.03	0.02
Total cold run time: 105.94 s
Total hot run time: 29.96 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.20% (8627/23834)
Line Coverage: 28.21% (70550/250102)
Region Coverage: 27.22% (36418/133794)
Branch Coverage: 24.00% (18650/77722)
Coverage Report: http://coverage.selectdb-in.cc/coverage/037cfcd89cb1513a21542edbf69439e917acbdcc_037cfcd89cb1513a21542edbf69439e917acbdcc/report/index.html

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 037cfcd89cb1513a21542edbf69439e917acbdcc with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.7 seconds inserted 10000000 Rows, about 729K ops/s

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 6, 2024
@wangbo
Copy link
Contributor

wangbo commented Feb 6, 2024

run buildall

1 similar comment
@yiguolei
Copy link
Contributor Author

yiguolei commented Feb 6, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37406 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6b9051f76023b7db089b34e7e36c4235c050cfda, data reload: false

------ Round 1 ----------------------------------
q1	17678	4856	4582	4582
q2	2046	141	138	138
q3	10682	924	919	919
q4	4674	802	783	783
q5	7696	2839	2858	2839
q6	185	122	122	122
q7	1196	758	759	758
q8	9332	1991	2006	1991
q9	7220	6357	6376	6357
q10	8114	2454	2419	2419
q11	409	222	212	212
q12	788	282	286	282
q13	18015	3341	3321	3321
q14	296	260	241	241
q15	530	483	497	483
q16	468	401	414	401
q17	931	523	518	518
q18	6815	5958	5980	5958
q19	1574	1476	1384	1384
q20	575	361	350	350
q21	6681	3125	3051	3051
q22	810	315	297	297
Total cold run time: 106715 ms
Total hot run time: 37406 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4483	4331	4411	4331
q2	330	237	238	237
q3	2996	2795	2828	2795
q4	1848	1604	1569	1569
q5	5247	5265	5229	5229
q6	191	114	115	114
q7	2122	1727	1800	1727
q8	3123	3238	3286	3238
q9	8373	8359	8334	8334
q10	5883	3627	3596	3596
q11	560	458	464	458
q12	749	558	562	558
q13	12454	3090	3087	3087
q14	288	245	276	245
q15	524	483	486	483
q16	518	467	477	467
q17	1817	1762	1715	1715
q18	7992	7704	7542	7542
q19	10012	1524	1559	1524
q20	2137	1897	1887	1887
q21	4796	4624	4439	4439
q22	537	468	440	440
Total cold run time: 76980 ms
Total hot run time: 54015 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173616 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6b9051f76023b7db089b34e7e36c4235c050cfda, data reload: false

query1	948	340	338	338
query2	6539	1804	1793	1793
query3	6694	213	209	209
query4	31731	22017	22100	22017
query5	4265	365	348	348
query6	252	164	162	162
query7	4592	275	272	272
query8	237	168	171	168
query9	9023	2281	2274	2274
query10	410	212	206	206
query11	18308	15397	15326	15326
query12	131	71	76	71
query13	1628	414	418	414
query14	9260	6959	6846	6846
query15	243	185	181	181
query16	8106	274	255	255
query17	1871	532	504	504
query18	2100	261	260	260
query19	362	139	142	139
query20	80	75	78	75
query21	197	126	135	126
query22	4615	4602	4679	4602
query23	30880	30139	30024	30024
query24	7581	2754	2792	2754
query25	560	341	336	336
query26	708	142	149	142
query27	2157	307	302	302
query28	5174	1862	1837	1837
query29	879	608	611	608
query30	281	132	136	132
query31	900	701	713	701
query32	91	53	50	50
query33	479	221	217	217
query34	827	449	466	449
query35	840	738	735	735
query36	1001	923	929	923
query37	96	59	58	58
query38	3229	3074	3126	3074
query39	1305	1261	1245	1245
query40	181	92	94	92
query41	36	36	34	34
query42	98	89	88	88
query43	530	463	498	463
query44	1044	687	702	687
query45	197	183	170	170
query46	1052	648	649	648
query47	1599	1469	1449	1449
query48	426	348	351	348
query49	1043	281	283	281
query50	766	370	373	370
query51	5311	5078	5098	5078
query52	92	92	78	78
query53	336	274	272	272
query54	244	210	218	210
query55	78	76	74	74
query56	216	200	200	200
query57	981	905	855	855
query58	198	176	181	176
query59	2293	2280	2272	2272
query60	234	215	215	215
query61	83	81	81	81
query62	589	347	346	346
query63	293	276	267	267
query64	4786	3480	3576	3480
query65	3256	3233	3208	3208
query66	805	306	303	303
query67	14403	14265	14000	14000
query68	4872	527	544	527
query69	479	320	334	320
query70	1240	1211	1259	1211
query71	327	249	245	245
query72	6025	2817	2640	2640
query73	690	326	325	325
query74	6584	6220	6205	6205
query75	3019	2362	2370	2362
query76	2945	954	983	954
query77	359	230	222	222
query78	9226	8586	8448	8448
query79	5376	480	484	480
query80	1824	340	334	334
query81	529	200	198	198
query82	1597	84	81	81
query83	251	121	122	121
query84	284	78	78	78
query85	1861	324	322	322
query86	481	283	293	283
query87	3352	3162	3193	3162
query88	4699	2317	2297	2297
query89	426	356	356	356
query90	1954	159	160	159
query91	149	120	142	120
query92	55	50	41	41
query93	5901	490	453	453
query94	1210	177	169	169
query95	7975	351	7959	351
query96	594	281	265	265
query97	4234	4107	4120	4107
query98	226	205	200	200
query99	1140	662	688	662
Total cold run time: 289634 ms
Total hot run time: 173616 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.27 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6b9051f76023b7db089b34e7e36c4235c050cfda, data reload: false

query1	0.03	0.04	0.03
query2	0.06	0.03	0.02
query3	0.23	0.06	0.06
query4	1.66	0.10	0.10
query5	0.54	0.52	0.51
query6	1.23	0.64	0.63
query7	0.01	0.02	0.01
query8	0.04	0.03	0.02
query9	0.55	0.51	0.50
query10	0.57	0.55	0.56
query11	0.12	0.08	0.09
query12	0.11	0.08	0.09
query13	0.60	0.60	0.62
query14	0.79	0.80	0.79
query15	0.82	0.77	0.76
query16	0.36	0.38	0.39
query17	0.99	1.04	1.03
query18	0.23	0.26	0.24
query19	1.91	1.76	1.80
query20	0.01	0.00	0.01
query21	15.40	0.57	0.58
query22	2.64	2.11	1.47
query23	17.58	0.81	0.69
query24	2.32	0.93	0.89
query25	0.30	0.26	0.04
query26	0.46	0.15	0.13
query27	0.05	0.04	0.05
query28	12.08	0.87	0.82
query29	12.49	3.10	3.16
query30	0.64	0.53	0.48
query31	2.79	0.34	0.34
query32	3.36	0.48	0.47
query33	3.23	3.24	3.22
query34	15.91	4.26	4.28
query35	4.29	4.28	4.31
query36	1.10	1.06	1.07
query37	0.07	0.05	0.06
query38	0.04	0.03	0.03
query39	0.02	0.02	0.01
query40	0.17	0.13	0.15
query41	0.07	0.01	0.02
query42	0.03	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 105.92 s
Total hot run time: 30.27 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 6b9051f76023b7db089b34e7e36c4235c050cfda with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.9 seconds inserted 10000000 Rows, about 719K ops/s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.20% (8629/23839)
Line Coverage: 28.21% (70570/250142)
Region Coverage: 27.22% (36422/133808)
Branch Coverage: 24.01% (18659/77726)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6b9051f76023b7db089b34e7e36c4235c050cfda_6b9051f76023b7db089b34e7e36c4235c050cfda/report/index.html

@yiguolei yiguolei merged commit 6bf0c0e into apache:master Feb 6, 2024
yiguolei added a commit that referenced this pull request Feb 6, 2024
init query ctx memtracker in queryctx constructor
set all task group related property during set taskgroup
yiguolei added a commit that referenced this pull request Feb 16, 2024
init query ctx memtracker in queryctx constructor
set all task group related property during set taskgroup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed workload-group

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants