Skip to content

[fix](planner)nullable info of agg node's intermediate and output slots should be same #29315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
starocean999 merged 2 commits into from
Jan 2, 2024
Merged

[fix](planner)nullable info of agg node's intermediate and output slots should be same #29315

starocean999 merged 2 commits into from
Jan 2, 2024

Conversation

@starocean999
Copy link
Contributor

@starocean999 starocean999 commented Dec 29, 2023

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@starocean999
Copy link
Contributor Author

starocean999 commented Dec 29, 2023

run buildall

@starocean999
Copy link
Contributor Author

starocean999 commented Dec 29, 2023

run buildall

@doris-robot
Copy link

doris-robot commented Dec 29, 2023

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit c3ff7f739c45973f515f514f6c84ee7d2afbfd84, data reload: false

------ Round 1 ----------------------------------
q1	17627	5123	5061	5061
q2	2011	165	154	154
q3	10577	1056	1117	1056
q4	10170	777	764	764
q5	7823	2927	2901	2901
q6	213	140	140	140
q7	955	519	505	505
q8	9285	2012	2031	2012
q9	6877	6374	6333	6333
q10	8296	2962	3004	2962
q11	435	235	223	223
q12	396	238	244	238
q13	17989	3654	3610	3610
q14	248	215	217	215
q15	598	557	542	542
q16	492	402	393	393
q17	957	469	515	469
q18	7224	6714	6646	6646
q19	1609	1446	1355	1355
q20	684	354	346	346
q21	2815	2320	2386	2320
q22	380	339	330	330
Total cold run time: 107661 ms
Total hot run time: 38575 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5053	5131	5063	5063
q2	336	266	260	260
q3	3302	3260	3264	3260
q4	2116	2010	1967	1967
q5	5797	5797	5778	5778
q6	221	133	129	129
q7	2312	1916	1889	1889
q8	3387	3476	3434	3434
q9	8848	8906	8872	8872
q10	3958	3989	3831	3831
q11	591	470	505	470
q12	830	641	625	625
q13	7334	3221	3202	3202
q14	298	260	269	260
q15	604	574	559	559
q16	561	484	484	484
q17	1952	1764	1760	1760
q18	8680	8286	8260	8260
q19	1652	1586	1621	1586
q20	2216	1981	1956	1956
q21	5590	5304	5245	5245
q22	528	470	478	470
Total cold run time: 66166 ms
Total hot run time: 59360 ms

@doris-robot
Copy link

doris-robot commented Dec 29, 2023

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit c3ff7f739c45973f515f514f6c84ee7d2afbfd84, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5360	5163	5119	5119
q2	399	175	158	158
q3	1489	1171	1217	1171
q4	1104	851	830	830
q5	3144	3081	2984	2984
q6	228	136	138	136
q7	954	546	555	546
q8	2161	2204	2192	2192
q9	6718	6626	6651	6626
q10	3200	3125	3194	3125
q11	358	227	228	227
q12	383	249	245	245
q13	4415	3616	3630	3616
q14	267	216	217	216
q15	613	548	566	548
q16	447	394	402	394
q17	1050	532	446	446
q18	7084	6727	6763	6727
q19	1651	1599	1522	1522
q20	615	365	366	365
q21	2887	2516	2451	2451
q22	409	329	321	321
Total cold run time: 44936 ms
Total hot run time: 39965 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5108	5062	5043	5043
q2	338	246	253	246
q3	3369	3327	3320	3320
q4	2154	1998	2000	1998
q5	5945	5918	5928	5918
q6	236	132	134	132
q7	2403	1918	1930	1918
q8	3595	3653	3655	3653
q9	9041	9096	9016	9016
q10	3863	3907	3942	3907
q11	586	477	483	477
q12	832	638	636	636
q13	3874	3178	3197	3178
q14	328	297	265	265
q15	642	584	561	561
q16	571	510	499	499
q17	2008	1828	1815	1815
q18	8812	8302	8316	8302
q19	1764	1740	1725	1725
q20	2271	1987	1982	1982
q21	5713	5307	5342	5307
q22	560	473	519	473
Total cold run time: 64013 ms
Total hot run time: 60371 ms

@doris-robot
Copy link

doris-robot commented Dec 29, 2023

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit c3ff7f739c45973f515f514f6c84ee7d2afbfd84, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	920	364	344	344
query2	6413	2102	1901	1901
query3	6646	229	218	218
query4	27110	22516	22503	22503
query5	6107	555	586	555
query6	299	189	173	173
query7	4711	269	260	260
query8	233	206	197	197
query9	8099	2701	2705	2701
query10	465	273	251	251
query11	16313	15948	15619	15619
query12	144	81	76	76
query13	1670	322	322	322
query14	12841	7233	7246	7233
query15	250	194	194	194
query16	6347	301	294	294
query17	1745	513	518	513
query18	1927	292	265	265
query19	273	144	136	136
query20	85	78	82	78
query21	185	101	94	94
query22	5000	4564	4731	4564
query23	32075	31410	31289	31289
query24	10720	2891	2784	2784
query25	591	353	353	353
query26	1316	142	149	142
query27	2845	291	287	287
query28	6941	1971	1961	1961
query29	1103	402	419	402
query30	283	140	144	140
query31	982	788	770	770
query32	92	63	62	62
query33	711	275	279	275
query34	834	444	449	444
query35	900	764	777	764
query36	1385	1270	1261	1261
query37	114	90	89	89
query38	3437	3282	3253	3253
query39	1341	1301	1279	1279
query40	186	94	99	94
query41	38	35	34	34
query42	102	90	100	90
query43	563	501	513	501
query44	1101	754	772	754
query45	194	194	188	188
query46	1080	629	662	629
query47	1616	1583	1647	1583
query48	334	274	265	265
query49	1033	339	336	336
query50	822	350	387	350
query51	5466	5239	5293	5239
query52	96	91	94	91
query53	216	149	148	148
query54	1341	614	633	614
query55	108	90	90	90
query56	219	204	201	201
query57	993	969	942	942
query58	221	211	211	211
query59	2857	2624	2709	2624
query60	265	245	243	243
query61	86	86	86	86
query62	633	465	475	465
query63	170	148	156	148
query64	4968	1749	1737	1737
query65	3324	3251	3304	3251
query66	958	342	331	331
query67	15130	15155	15280	15155
query68	11739	527	514	514
query69	594	270	282	270
query70	1654	1460	1556	1460
query71	574	229	233	229
query72	5621	3630	3602	3602
query73	1557	360	321	321
query74	6934	6402	6433	6402
query75	5731	2276	2288	2276
query76	5330	1098	1125	1098
query77	877	294	296	294
query78	9055	8582	8525	8525
query79	6563	508	497	497
query80	3042	381	368	368
query81	507	208	210	208
query82	351	123	121	121
query83	304	142	142	142
query84	255	57	57	57
query85	2590	299	277	277
query86	387	366	394	366
query87	3563	3352	3364	3352
query88	3050	2448	2433	2433
query89	335	259	276	259
query90	1910	249	253	249
query91	123	95	93	93
query92	76	54	57	54
query93	2705	508	433	433
query94	887	218	214	214
query95	521	471	464	464
query96	649	335	332	332
query97	4240	4200	4154	4154
query98	218	194	189	189
query99	1118	848	851	848
Total cold run time: 300633 ms
Total hot run time: 180547 ms

@doris-robot
Copy link

doris-robot commented Dec 29, 2023

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.53 seconds
stream load tsv: 574 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.1 seconds inserted 10000000 Rows, about 355K ops/s
storage size: 17188225529 Bytes

@starocean999 starocean999 marked this pull request as ready for review January 2, 2024 02:36
@wm1581066 wm1581066 added dev/2.0.4 usercase Important user case type label labels Jan 2, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 2, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jan 2, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 2, 2024

PR approved by anyone and no changes requested.

@starocean999 starocean999 merged commit 4692a62 into apache:master Jan 2, 2024
@starocean999 starocean999 mentioned this pull request Jan 2, 2024
Merged
seawinde pushed a commit to seawinde/doris that referenced this pull request Jan 3, 2024
@starocean999 @seawinde
[fix](planner)nullable info of agg node's intermediate and output slo…
a3bc99d 
…ts should be same (apache#29315)
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
@starocean999 @HappenLee
[fix](planner)nullable info of agg node's intermediate and output slo…
baad452 
…ts should be same (apache#29315)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@englefly englefly englefly approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.4-merged dev/3.0.0-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@starocean999 @doris-robot @englefly @morrySnow @xiaokang @wm1581066 @dataroaring