Skip to content

[chore](Nereids) enable nereids if update from doris 1.x#30369

Merged
morrySnow merged 1 commit intoapache:masterfrom
morrySnow:turn_on_nereids_if_update
Jan 26, 2024
Merged

[chore](Nereids) enable nereids if update from doris 1.x#30369
morrySnow merged 1 commit intoapache:masterfrom
morrySnow:turn_on_nereids_if_update

Conversation

@morrySnow
Copy link
Contributor

@morrySnow morrySnow commented Jan 25, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@morrySnow
Copy link
Contributor Author

morrySnow commented Jan 25, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Jan 25, 2024

TPC-H: Total hot run time: 38764 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8b7e1c59b8342a56a748df6467bff6afb0c7e493, data reload: false

------ Round 1 ----------------------------------
q1	18246	5335	5362	5335
q2	2798	141	137	137
q3	11240	1172	1134	1134
q4	10226	796	851	796
q5	7711	3139	3155	3139
q6	206	119	117	117
q7	865	488	475	475
q8	9234	1912	1965	1912
q9	7276	6362	6347	6347
q10	8199	3006	3059	3006
q11	420	206	208	206
q12	356	190	193	190
q13	17980	3389	3392	3389
q14	243	211	222	211
q15	530	510	487	487
q16	429	370	375	370
q17	937	551	515	515
q18	7473	7008	6742	6742
q19	1549	1394	1319	1319
q20	561	313	280	280
q21	2771	2348	2413	2348
q22	342	313	309	309
Total cold run time: 109592 ms
Total hot run time: 38764 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5480	5269	5304	5269
q2	327	218	213	213
q3	3333	3219	3178	3178
q4	2068	2013	2033	2013
q5	5998	5881	6042	5881
q6	193	118	117	117
q7	2314	1901	1865	1865
q8	3215	3364	3393	3364
q9	8878	8814	8776	8776
q10	3936	3872	3829	3829
q11	573	442	466	442
q12	828	649	628	628
q13	16927	3201	3180	3180
q14	279	268	262	262
q15	553	515	510	510
q16	505	459	466	459
q17	2587	1843	1834	1834
q18	9588	11599	9553	9553
q19	24542	1597	1533	1533
q20	4576	1936	1930	1930
q21	15411	5401	5311	5311
q22	2556	537	581	537
Total cold run time: 114667 ms
Total hot run time: 60684 ms

@doris-robot
Copy link

doris-robot commented Jan 25, 2024

TPC-DS: Total hot run time: 186521 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8b7e1c59b8342a56a748df6467bff6afb0c7e493, data reload: false

query1	936	335	333	333
query2	6545	1890	1939	1890
query3	6697	201	200	200
query4	32398	22202	22118	22118
query5	4444	374	436	374
query6	258	147	153	147
query7	4609	268	255	255
query8	226	179	182	179
query9	8264	2592	2575	2575
query10	418	225	233	225
query11	17105	15444	15565	15444
query12	126	69	62	62
query13	1683	372	372	372
query14	10503	7016	7022	7016
query15	219	176	184	176
query16	5790	259	248	248
query17	954	466	461	461
query18	1796	247	252	247
query19	174	129	128	128
query20	69	75	69	69
query21	188	138	136	136
query22	5000	4826	4830	4826
query23	31646	30871	30775	30775
query24	12648	2867	2798	2798
query25	556	309	310	309
query26	1819	138	140	138
query27	3232	283	279	279
query28	7347	1831	1827	1827
query29	2168	632	633	632
query30	282	131	140	131
query31	942	731	744	731
query32	86	56	53	53
query33	703	206	208	206
query34	1169	458	448	448
query35	873	742	765	742
query36	1381	1140	1143	1140
query37	171	57	56	56
query38	3353	3227	3261	3227
query39	1339	1274	1251	1251
query40	342	83	84	83
query41	37	35	34	34
query42	85	80	78	78
query43	521	468	488	468
query44	1120	687	693	687
query45	190	173	173	173
query46	1060	659	641	641
query47	1677	1566	1579	1566
query48	386	319	308	308
query49	1216	290	287	287
query50	671	300	312	300
query51	5237	5202	5211	5202
query52	97	75	88	75
query53	329	255	254	254
query54	242	182	182	182
query55	85	72	76	72
query56	166	169	165	165
query57	1003	926	938	926
query58	184	158	153	153
query59	2944	2565	2710	2565
query60	208	179	177	177
query61	84	81	81	81
query62	629	372	343	343
query63	278	252	253	252
query64	6166	1771	1764	1764
query65	3336	3276	3229	3229
query66	1434	320	318	318
query67	15994	15162	15496	15162
query68	10748	530	533	530
query69	576	300	291	291
query70	1646	1551	1477	1477
query71	10420	10196	10197	10196
query72	4757	2822	2798	2798
query73	2152	314	314	314
query74	7456	6497	6427	6427
query75	4437	2267	2322	2267
query76	6275	1051	1064	1051
query77	639	228	227	227
query78	9022	8849	8583	8583
query79	2224	492	500	492
query80	590	327	309	309
query81	446	205	204	204
query82	202	79	80	79
query83	139	115	113	113
query84	273	66	66	66
query85	1085	335	340	335
query86	387	391	391	391
query87	3490	3340	3378	3340
query88	3944	2206	2200	2200
query89	439	357	349	349
query90	1959	184	185	184
query91	149	122	128	122
query92	50	44	44	44
query93	4179	422	418	418
query94	1335	161	155	155
query95	504	469	447	447
query96	636	317	324	317
query97	4261	4154	4159	4154
query98	192	192	182	182
query99	1034	696	690	690
Total cold run time: 309484 ms
Total hot run time: 186521 ms

@doris-robot
Copy link

doris-robot commented Jan 25, 2024

ClickBench: Total hot run time: 30.52 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8b7e1c59b8342a56a748df6467bff6afb0c7e493, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.02	0.02
query3	0.22	0.05	0.05
query4	1.70	0.06	0.06
query5	0.54	0.52	0.52
query6	1.29	0.65	0.63
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.54	0.49	0.49
query10	0.56	0.56	0.55
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.62	0.61	0.60
query14	0.78	0.82	0.78
query15	0.80	0.77	0.79
query16	0.36	0.38	0.39
query17	0.99	0.99	1.02
query18	0.23	0.27	0.24
query19	1.86	1.72	1.81
query20	0.01	0.01	0.01
query21	15.42	0.57	0.58
query22	2.91	2.44	2.04
query23	17.20	0.88	0.74
query24	2.58	1.14	0.53
query25	0.39	0.22	0.21
query26	0.46	0.12	0.12
query27	0.06	0.05	0.05
query28	11.71	0.75	0.77
query29	12.52	3.14	3.23
query30	0.58	0.45	0.52
query31	2.77	0.34	0.35
query32	3.39	0.48	0.48
query33	3.25	3.26	3.25
query34	15.79	4.33	4.23
query35	4.31	4.25	4.27
query36	1.11	1.08	1.07
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.01	0.02
query42	0.02	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 105.75 s
Total hot run time: 30.52 s

@doris-robot
Copy link

doris-robot commented Jan 25, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 8b7e1c59b8342a56a748df6467bff6afb0c7e493 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       15.0 seconds inserted 10000000 Rows, about 666K ops/s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 26, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jan 26, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 26, 2024

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit 192dc9c into apache:master Jan 26, 2024
@morrySnow morrySnow deleted the turn_on_nereids_if_update branch January 26, 2024 04:05
yiguolei pushed a commit that referenced this pull request Jan 27, 2024
@morrySnow @Doris-Extras
[chore](Nereids) enable nereids if update from doris 1.x (#30369)
4bdc2cd
@wm1581066 wm1581066 added the usercase Important user case type label label Jan 31, 2024
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jan 31, 2024
@morrySnow
[chore](Nereids) enable nereids if update from doris 1.x (apache#30369)
3b9cf1e 
pick from master apache#30369
commit id 192dc9c
@morrySnow morrySnow mentioned this pull request Jan 31, 2024
Merged
morrySnow added a commit that referenced this pull request Jan 31, 2024
@morrySnow
[chore](Nereids) enable nereids if update from doris 1.x (#30369) (#3…
b761f76 
…0619)

pick from master #30369
commit id 192dc9c
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@morrySnow
[chore](Nereids) enable nereids if update from doris 1.x (apache#30369)…
877d894 
… (apache#30619)

pick from master apache#30369
commit id 192dc9c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

+1 more reviewer

@keanji-x keanji-x keanji-x approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.5-merged dev/3.0.0-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@morrySnow @doris-robot @keanji-x @starocean999 @wm1581066 @dataroaring