Skip to content

[feat](Nereids): update struct info map when there is new expr#32119

Merged
morrySnow merged 1 commit intoapache:masterfrom
keanji-x:construct_mv_table_map
Mar 13, 2024
Merged

[feat](Nereids): update struct info map when there is new expr#32119
morrySnow merged 1 commit intoapache:masterfrom
keanji-x:construct_mv_table_map

Conversation

@keanji-x
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@keanji-x keanji-x force-pushed the construct_mv_table_map branch from 5b24068 to 20cd0e4 Compare March 12, 2024 08:31
@keanji-x
Copy link
Contributor Author

run buildall

@keanji-x keanji-x changed the title [feat]: update struct info map when there is new expr [feat](Nereids): update struct info map when there is new expr Mar 12, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 36629 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 20cd0e47c7fd211433d7b87405e1b98da96d28d2, data reload: false

------ Round 1 ----------------------------------
q1	17673	4138	4096	4096
q2	2016	152	153	152
q3	10562	900	887	887
q4	4654	684	707	684
q5	7585	2713	2765	2713
q6	181	123	130	123
q7	1168	800	800	800
q8	9477	1999	2034	1999
q9	7316	6464	6379	6379
q10	8148	2499	2501	2499
q11	422	226	222	222
q12	771	301	302	301
q13	17961	2989	3035	2989
q14	282	260	252	252
q15	498	465	443	443
q16	481	399	400	399
q17	932	546	545	545
q18	6575	5932	5882	5882
q19	1574	1384	1407	1384
q20	547	285	308	285
q21	7469	3296	3391	3296
q22	805	314	299	299
Total cold run time: 107097 ms
Total hot run time: 36629 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4079	4066	4045	4045
q2	326	234	235	234
q3	2989	2821	2845	2821
q4	1902	1561	1566	1561
q5	5179	5245	5252	5245
q6	206	122	116	116
q7	2202	1809	1837	1809
q8	3142	3255	3257	3255
q9	8511	8519	8599	8519
q10	6135	3655	3730	3655
q11	539	448	439	439
q12	739	556	534	534
q13	15591	2889	2886	2886
q14	283	248	251	248
q15	497	463	444	444
q16	440	426	415	415
q17	1708	1490	1464	1464
q18	7620	7337	7044	7044
q19	4870	1526	1568	1526
q20	1903	1719	1724	1719
q21	4770	4757	4836	4757
q22	499	471	477	471
Total cold run time: 74130 ms
Total hot run time: 53207 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187256 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 20cd0e47c7fd211433d7b87405e1b98da96d28d2, data reload: false

query1	931	357	341	341
query2	7660	2293	2141	2141
query3	6806	226	216	216
query4	32637	20943	20886	20886
query5	5952	426	432	426
query6	266	173	189	173
query7	4625	298	295	295
query8	221	173	176	173
query9	9735	2303	2285	2285
query10	445	241	251	241
query11	14812	14216	14261	14216
query12	136	90	86	86
query13	1628	428	426	426
query14	11099	10323	10317	10317
query15	261	204	195	195
query16	8154	258	258	258
query17	1990	548	535	535
query18	2101	290	277	277
query19	342	153	170	153
query20	98	82	85	82
query21	205	127	127	127
query22	4603	4466	4407	4407
query23	31724	31065	31247	31065
query24	11422	2841	2817	2817
query25	613	394	380	380
query26	1185	157	159	157
query27	2844	352	353	352
query28	7548	1886	1857	1857
query29	893	664	613	613
query30	307	149	147	147
query31	976	699	714	699
query32	105	57	58	57
query33	781	257	255	255
query34	1078	473	497	473
query35	831	614	606	606
query36	1036	887	902	887
query37	124	74	78	74
query38	3270	3137	3133	3133
query39	1397	1351	1403	1351
query40	209	121	135	121
query41	50	49	47	47
query42	101	100	126	100
query43	469	465	481	465
query44	1289	705	707	705
query45	266	250	249	249
query46	1106	707	710	707
query47	1633	1602	1603	1602
query48	443	358	361	358
query49	1106	344	334	334
query50	758	378	375	375
query51	6702	6671	6624	6624
query52	109	90	94	90
query53	347	279	280	279
query54	303	240	251	240
query55	85	81	78	78
query56	261	260	229	229
query57	1083	985	1008	985
query58	233	212	210	210
query59	2869	2581	2611	2581
query60	272	248	270	248
query61	111	112	110	110
query62	608	423	402	402
query63	303	284	284	284
query64	5516	3875	3936	3875
query65	3046	3056	3018	3018
query66	851	362	324	324
query67	14949	14154	14135	14135
query68	8721	542	547	542
query69	646	386	382	382
query70	1227	1221	1144	1144
query71	571	277	278	277
query72	6407	2671	2483	2483
query73	723	325	327	325
query74	6976	6537	6562	6537
query75	9971	7897	8014	7897
query76	5082	915	999	915
query77	781	255	259	255
query78	10343	9792	9680	9680
query79	12665	531	513	513
query80	1530	395	440	395
query81	499	209	207	207
query82	836	205	203	203
query83	199	153	153	153
query84	286	76	78	76
query85	1460	332	313	313
query86	427	299	286	286
query87	3414	3215	3181	3181
query88	5548	2352	2355	2352
query89	526	379	374	374
query90	2053	178	177	177
query91	163	148	132	132
query92	55	47	47	47
query93	7180	514	495	495
query94	1268	177	175	175
query95	424	341	326	326
query96	614	272	261	261
query97	4414	4248	4274	4248
query98	226	217	207	207
query99	1119	744	768	744
Total cold run time: 322273 ms
Total hot run time: 187256 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.54 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 20cd0e47c7fd211433d7b87405e1b98da96d28d2, data reload: false

query1	0.04	0.03	0.03
query2	0.05	0.03	0.03
query3	0.23	0.07	0.06
query4	1.68	0.10	0.10
query5	0.48	0.51	0.51
query6	1.18	0.63	0.61
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.48	0.49
query10	0.55	0.53	0.53
query11	0.11	0.09	0.10
query12	0.11	0.09	0.10
query13	0.57	0.58	0.57
query14	0.74	0.78	0.74
query15	0.81	0.77	0.77
query16	0.39	0.39	0.39
query17	0.95	1.01	1.05
query18	0.20	0.26	0.22
query19	1.90	1.77	1.65
query20	0.02	0.01	0.01
query21	15.41	0.56	0.55
query22	2.15	1.70	1.64
query23	17.45	0.86	0.69
query24	3.37	1.13	0.48
query25	0.06	0.06	0.03
query26	0.88	0.12	0.13
query27	0.03	0.02	0.03
query28	11.69	0.87	0.84
query29	12.60	3.27	3.36
query30	0.63	0.56	0.55
query31	2.80	0.34	0.34
query32	3.40	0.45	0.43
query33	2.75	2.85	2.80
query34	16.15	4.39	4.29
query35	4.32	4.27	4.32
query36	1.07	1.01	1.00
query37	0.08	0.05	0.05
query38	0.05	0.03	0.03
query39	0.02	0.02	0.02
query40	0.16	0.14	0.14
query41	0.07	0.03	0.01
query42	0.03	0.02	0.02
query43	0.02	0.03	0.02
Total cold run time: 105.84 s
Total hot run time: 29.54 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 20cd0e47c7fd211433d7b87405e1b98da96d28d2 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       16.3 seconds inserted 10000000 Rows, about 613K ops/s

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 13, 2024
@morrySnow morrySnow merged commit 5abeab4 into apache:master Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants