Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve](complex type) delete fe config disable_nested_complex_type #36255

Merged
merged 5 commits into from
Jun 18, 2024

Conversation

amorynan
Copy link
Contributor

@amorynan amorynan commented Jun 13, 2024

Proposed changes

same with #36255

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added area/planner Issues or PRs related to the query planner kind/test labels Jun 13, 2024
@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50195 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e35f353c38cbe05a160cf72e746886e5e98d8a80, data reload: false

------ Round 1 ----------------------------------
q1	17731	4412	4357	4357
q2	2051	164	153	153
q3	10262	1902	1921	1902
q4	10788	1252	1345	1252
q5	8398	3926	3933	3926
q6	231	126	129	126
q7	2072	1612	1604	1604
q8	9531	2741	2731	2731
q9	10569	10471	10211	10211
q10	8758	3522	3587	3522
q11	425	244	237	237
q12	477	301	309	301
q13	18583	4068	4197	4068
q14	371	335	351	335
q15	514	469	471	469
q16	707	613	625	613
q17	1142	945	947	945
q18	7679	7226	6966	6966
q19	1756	1640	1661	1640
q20	1684	332	309	309
q21	4515	4192	4088	4088
q22	544	440	452	440
Total cold run time: 118788 ms
Total hot run time: 50195 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4334	4325	4278	4278
q2	320	226	222	222
q3	4182	4093	4125	4093
q4	2742	2736	2733	2733
q5	7158	7157	7026	7026
q6	235	123	117	117
q7	3222	2902	2804	2804
q8	4374	4480	4457	4457
q9	16757	16741	16622	16622
q10	4225	4256	4250	4250
q11	747	672	699	672
q12	1016	867	866	866
q13	6713	3719	3718	3718
q14	446	420	427	420
q15	510	466	456	456
q16	742	685	678	678
q17	3930	3869	3855	3855
q18	8832	8794	8801	8794
q19	1778	1686	1643	1643
q20	2405	2096	2116	2096
q21	8487	8555	8511	8511
q22	1062	988	947	947
Total cold run time: 84217 ms
Total hot run time: 79258 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202923 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e35f353c38cbe05a160cf72e746886e5e98d8a80, data reload: false

query1	926	424	383	383
query2	6520	2704	2636	2636
query3	6917	207	199	199
query4	20798	18006	17951	17951
query5	19722	6482	6519	6482
query6	298	220	230	220
query7	4163	306	312	306
query8	421	380	426	380
query9	3168	2690	2627	2627
query10	417	304	297	297
query11	11279	10765	10695	10695
query12	126	78	73	73
query13	5597	691	684	684
query14	18197	13467	13347	13347
query15	367	254	245	245
query16	6439	305	265	265
query17	1684	1446	887	887
query18	2307	420	414	414
query19	214	151	153	151
query20	82	79	84	79
query21	190	97	97	97
query22	5163	4934	4961	4934
query23	32525	31791	31926	31791
query24	6924	6476	6481	6476
query25	506	423	432	423
query26	511	167	162	162
query27	1771	302	297	297
query28	6223	2375	2322	2322
query29	2967	2759	2799	2759
query30	242	166	171	166
query31	908	708	732	708
query32	73	58	49	49
query33	398	263	260	260
query34	867	468	483	468
query35	1119	929	912	912
query36	1551	1092	1148	1092
query37	89	61	60	60
query38	3059	2917	2911	2911
query39	1366	1346	1326	1326
query40	205	96	93	93
query41	46	44	44	44
query42	89	87	79	79
query43	816	631	655	631
query44	1132	709	714	709
query45	256	234	239	234
query46	1225	958	978	958
query47	1859	1842	1797	1797
query48	1024	724	700	700
query49	611	372	367	367
query50	883	659	630	630
query51	4740	4646	4630	4630
query52	96	96	77	77
query53	447	324	315	315
query54	2609	2461	2486	2461
query55	91	72	83	72
query56	255	235	203	203
query57	1148	1116	1141	1116
query58	216	215	196	196
query59	4191	4074	4165	4074
query60	212	203	210	203
query61	98	94	97	94
query62	906	517	485	485
query63	494	349	352	349
query64	2517	1533	1505	1505
query65	3638	3541	3555	3541
query66	748	388	383	383
query67	15554	15071	14695	14695
query68	10456	640	646	640
query69	582	361	342	342
query70	2047	1381	1452	1381
query71	417	314	314	314
query72	6508	3501	3489	3489
query73	734	330	321	321
query74	6248	5816	5784	5784
query75	5489	3699	3696	3696
query76	6857	1134	1219	1134
query77	1189	266	249	249
query78	12706	12578	11649	11649
query79	11688	672	627	627
query80	693	395	405	395
query81	486	236	232	232
query82	689	105	98	98
query83	176	135	132	132
query84	266	73	72	72
query85	923	324	325	324
query86	313	295	281	281
query87	3257	3022	3079	3022
query88	5415	2312	2330	2312
query89	420	299	293	293
query90	2713	208	212	208
query91	183	137	142	137
query92	57	53	52	52
query93	6442	601	561	561
query94	1168	214	204	204
query95	1110	1045	1046	1045
query96	642	331	321	321
query97	6493	6446	6374	6374
query98	186	177	178	177
query99	2850	882	890	882
Total cold run time: 321603 ms
Total hot run time: 202923 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.09 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e35f353c38cbe05a160cf72e746886e5e98d8a80, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.03
query3	0.25	0.05	0.04
query4	1.78	0.06	0.06
query5	0.54	0.53	0.52
query6	1.25	0.62	0.62
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.53	0.48	0.51
query10	0.54	0.53	0.53
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.62	0.61	0.61
query14	0.78	0.79	0.79
query15	0.77	0.78	0.75
query16	0.35	0.37	0.37
query17	0.99	1.04	0.97
query18	0.24	0.23	0.25
query19	1.92	1.84	1.86
query20	0.02	0.01	0.01
query21	15.50	0.57	0.58
query22	1.98	2.46	2.04
query23	17.14	1.20	0.92
query24	3.65	3.83	1.89
query25	0.36	0.08	0.05
query26	0.75	0.16	0.15
query27	0.04	0.03	0.04
query28	5.86	0.76	0.70
query29	12.73	2.41	2.38
query30	0.61	0.53	0.53
query31	2.81	0.38	0.37
query32	3.39	0.49	0.49
query33	3.08	3.04	3.10
query34	15.24	4.80	4.80
query35	4.82	4.83	4.82
query36	1.06	0.99	1.02
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.03	0.01	0.02
query40	0.16	0.14	0.13
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 100.39 s
Total hot run time: 32.09 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit e35f353c38cbe05a160cf72e746886e5e98d8a80 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jun 16, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@xiaokang xiaokang changed the title [Improve](Config)Del disable complex nested type [Improve](complex type) delete config disable_nested_complex_type Jun 16, 2024
@xiaokang xiaokang changed the title [Improve](complex type) delete config disable_nested_complex_type [Improve](complex type) delete fe config disable_nested_complex_type Jun 16, 2024
@amorynan
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49480 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 57f173025909457fb3db29bdf9bccb8c6539eb21, data reload: false

------ Round 1 ----------------------------------
q1	18069	4342	4304	4304
q2	2073	151	145	145
q3	10280	1887	1896	1887
q4	10361	1222	1327	1222
q5	8892	3836	3888	3836
q6	233	124	150	124
q7	2024	1581	1581	1581
q8	9294	2710	2683	2683
q9	10597	10252	10348	10252
q10	8643	3489	3482	3482
q11	419	232	250	232
q12	474	303	299	299
q13	18352	3929	4039	3929
q14	346	324	321	321
q15	513	451	456	451
q16	679	574	569	569
q17	1121	952	962	952
q18	7486	6856	6888	6856
q19	1806	1619	1571	1571
q20	553	326	294	294
q21	4397	4120	4050	4050
q22	531	444	440	440
Total cold run time: 117143 ms
Total hot run time: 49480 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4289	4236	4242	4236
q2	328	226	222	222
q3	4128	4136	4154	4136
q4	2737	2751	2761	2751
q5	7160	7085	7025	7025
q6	235	118	115	115
q7	3170	2794	2889	2794
q8	4330	4447	4424	4424
q9	16789	16878	16656	16656
q10	4251	4248	4257	4248
q11	743	685	732	685
q12	1025	850	856	850
q13	4188	3748	3699	3699
q14	463	425	429	425
q15	507	467	459	459
q16	735	687	690	687
q17	3789	3826	3810	3810
q18	8892	8800	8634	8634
q19	1696	1703	1661	1661
q20	2368	2154	2096	2096
q21	8377	8527	8500	8500
q22	1070	992	982	982
Total cold run time: 81270 ms
Total hot run time: 79095 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204169 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 57f173025909457fb3db29bdf9bccb8c6539eb21, data reload: false

query1	917	416	376	376
query2	6540	2788	2713	2713
query3	6924	204	206	204
query4	20459	17985	17924	17924
query5	19727	6512	6492	6492
query6	291	220	235	220
query7	4159	301	309	301
query8	457	414	386	386
query9	3118	2654	2580	2580
query10	422	301	274	274
query11	11388	10739	10726	10726
query12	124	76	71	71
query13	5599	696	675	675
query14	17505	13200	13479	13200
query15	355	260	253	253
query16	6453	285	266	266
query17	1506	1512	901	901
query18	2245	419	415	415
query19	208	150	155	150
query20	72	78	78	78
query21	194	100	92	92
query22	5078	5025	4974	4974
query23	32527	31902	31868	31868
query24	6987	6615	6523	6523
query25	529	439	435	435
query26	539	161	160	160
query27	1846	295	297	295
query28	6036	2354	2313	2313
query29	2814	2858	2744	2744
query30	238	168	168	168
query31	926	748	740	740
query32	71	64	62	62
query33	411	251	241	241
query34	840	475	490	475
query35	1128	916	903	903
query36	1535	1349	1273	1273
query37	91	60	60	60
query38	3079	2902	2918	2902
query39	1379	1329	1318	1318
query40	207	100	95	95
query41	47	48	45	45
query42	85	81	83	81
query43	710	775	748	748
query44	1163	724	730	724
query45	248	233	241	233
query46	1215	975	969	969
query47	1803	1813	1700	1700
query48	1013	706	717	706
query49	618	379	366	366
query50	849	641	594	594
query51	4713	4578	4618	4578
query52	106	89	85	85
query53	439	336	318	318
query54	2650	2464	2476	2464
query55	89	88	83	83
query56	225	226	195	195
query57	1179	1161	1253	1161
query58	207	207	194	194
query59	4117	4167	4034	4034
query60	209	205	203	203
query61	100	96	117	96
query62	886	426	478	426
query63	487	337	349	337
query64	2514	1564	1498	1498
query65	3647	3557	3552	3552
query66	771	397	383	383
query67	15542	16795	15656	15656
query68	9985	638	625	625
query69	568	362	358	358
query70	1834	1621	1369	1369
query71	406	313	310	310
query72	6643	3512	3523	3512
query73	734	319	315	315
query74	6281	5844	5888	5844
query75	5279	3801	3748	3748
query76	6425	1157	1165	1157
query77	1079	253	250	250
query78	13037	12842	11747	11747
query79	14611	636	651	636
query80	955	402	416	402
query81	497	237	235	235
query82	598	104	92	92
query83	176	140	136	136
query84	263	72	73	72
query85	797	325	318	318
query86	331	289	287	287
query87	3266	3017	3009	3009
query88	5807	2296	2280	2280
query89	446	292	294	292
query90	2715	215	213	213
query91	169	138	140	138
query92	58	55	54	54
query93	7135	586	549	549
query94	1223	209	208	208
query95	1116	1040	1055	1040
query96	652	322	326	322
query97	6381	6415	6463	6415
query98	193	174	168	168
query99	2882	793	960	793
Total cold run time: 323098 ms
Total hot run time: 204169 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.29 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 57f173025909457fb3db29bdf9bccb8c6539eb21, data reload: false

query1	0.02	0.02	0.02
query2	0.08	0.02	0.03
query3	0.24	0.04	0.04
query4	1.80	0.06	0.07
query5	0.54	0.52	0.52
query6	1.23	0.62	0.61
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.48	0.49
query10	0.54	0.52	0.56
query11	0.12	0.08	0.09
query12	0.13	0.09	0.09
query13	0.61	0.62	0.61
query14	0.80	0.78	0.77
query15	0.77	0.76	0.75
query16	0.38	0.36	0.40
query17	1.02	1.03	1.00
query18	0.20	0.27	0.24
query19	1.94	1.87	1.86
query20	0.01	0.01	0.01
query21	15.47	0.54	0.53
query22	2.53	1.93	2.22
query23	17.21	0.99	0.90
query24	3.87	1.60	1.34
query25	0.35	0.13	0.04
query26	0.49	0.15	0.15
query27	0.05	0.05	0.03
query28	8.48	0.78	0.70
query29	12.68	2.33	2.33
query30	0.59	0.49	0.52
query31	2.81	0.38	0.37
query32	3.39	0.49	0.48
query33	3.09	3.09	3.05
query34	15.25	4.80	4.77
query35	4.88	4.85	4.85
query36	1.06	1.01	1.02
query37	0.05	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.07	0.02	0.02
query42	0.02	0.02	0.01
query43	0.03	0.02	0.01
Total cold run time: 103.58 s
Total hot run time: 31.29 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 57f173025909457fb3db29bdf9bccb8c6539eb21 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@xiaokang xiaokang merged commit c43a20f into apache:branch-2.0 Jun 18, 2024
23 of 24 checks passed
@xiaokang xiaokang mentioned this pull request Jun 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. area/planner Issues or PRs related to the query planner dev/2.0.12-merged kind/test reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants