Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement](nereids)optimized aggregate node's error msg #38122

Merged
merged 2 commits into from
Jul 23, 2024

Conversation

starocean999
Copy link
Contributor

Proposed changes

create table t1
(
    c1 bigint, 
    c2 bigint
)
DISTRIBUTED BY HASH(c1) BUCKETS 3
PROPERTIES ("replication_num" = "1");

sql
select * from t1 group by 1;

the error msg before:
ERROR 1105 (HY000): errCode = 2, detailMessage = Invalid call to toSlot on unbound object
after:
ERROR 1105 (HY000): errCode = 2, detailMessage = c2 not in aggregate's output

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39572 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d10a37fc3776c5438b8125d1f636fb231133916e, data reload: false

------ Round 1 ----------------------------------
q1	17630	4307	4310	4307
q2	2024	198	198	198
q3	10431	1210	1041	1041
q4	10178	728	751	728
q5	7535	2664	2606	2606
q6	224	139	135	135
q7	958	596	604	596
q8	9223	2042	2072	2042
q9	8723	6517	6489	6489
q10	8821	3860	3775	3775
q11	484	235	233	233
q12	442	221	226	221
q13	18993	3001	3014	3001
q14	275	239	234	234
q15	532	484	486	484
q16	495	373	371	371
q17	958	619	587	587
q18	8082	7466	7420	7420
q19	8248	1323	1349	1323
q20	666	317	326	317
q21	4982	3183	3944	3183
q22	357	284	281	281
Total cold run time: 120261 ms
Total hot run time: 39572 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4419	4272	4413	4272
q2	368	273	276	273
q3	3148	2916	2997	2916
q4	2011	1728	1743	1728
q5	5559	5567	5522	5522
q6	232	139	132	132
q7	2246	1835	1875	1835
q8	3339	3415	3438	3415
q9	8973	9110	9022	9022
q10	4203	3940	4072	3940
q11	644	538	517	517
q12	853	683	671	671
q13	17982	3156	3150	3150
q14	345	298	298	298
q15	542	522	498	498
q16	499	467	449	449
q17	1844	1582	1537	1537
q18	8205	7916	7849	7849
q19	1709	1621	1642	1621
q20	2134	1875	1875	1875
q21	9597	4748	4722	4722
q22	572	514	521	514
Total cold run time: 79424 ms
Total hot run time: 56756 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173277 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d10a37fc3776c5438b8125d1f636fb231133916e, data reload: false

query1	918	381	374	374
query2	6438	1933	1773	1773
query3	6654	212	216	212
query4	28173	17556	17209	17209
query5	3595	482	479	479
query6	270	165	165	165
query7	4572	288	282	282
query8	230	192	187	187
query9	8548	2478	2462	2462
query10	441	291	268	268
query11	10847	9992	10076	9992
query12	119	92	87	87
query13	1645	369	372	369
query14	9682	7534	7790	7534
query15	218	168	173	168
query16	7633	492	482	482
query17	1253	564	548	548
query18	1741	289	291	289
query19	198	154	157	154
query20	93	84	84	84
query21	213	134	128	128
query22	4301	3984	3935	3935
query23	34010	33680	33386	33386
query24	7008	2981	2884	2884
query25	563	398	399	398
query26	702	165	149	149
query27	1946	274	278	274
query28	3892	2101	2121	2101
query29	884	659	674	659
query30	242	155	153	153
query31	967	768	757	757
query32	95	55	54	54
query33	579	327	346	327
query34	874	496	508	496
query35	832	754	744	744
query36	1092	956	1018	956
query37	146	86	78	78
query38	2992	2904	2857	2857
query39	911	895	845	845
query40	207	115	119	115
query41	44	43	42	42
query42	123	97	97	97
query43	503	455	470	455
query44	1078	737	737	737
query45	197	160	160	160
query46	1072	728	717	717
query47	1863	1774	1769	1769
query48	363	287	292	287
query49	760	421	434	421
query50	786	384	398	384
query51	6879	6593	6666	6593
query52	116	90	95	90
query53	358	285	286	285
query54	557	454	450	450
query55	76	73	73	73
query56	281	273	270	270
query57	1146	1029	1062	1029
query58	243	252	247	247
query59	2843	2679	2561	2561
query60	293	273	279	273
query61	104	99	98	98
query62	761	644	680	644
query63	314	288	285	285
query64	9069	2221	1674	1674
query65	3150	3113	3107	3107
query66	768	338	330	330
query67	15462	14911	14700	14700
query68	4579	549	543	543
query69	639	444	355	355
query70	1200	1150	1134	1134
query71	454	283	273	273
query72	7354	5616	5634	5616
query73	760	326	319	319
query74	6081	5715	5676	5676
query75	3403	2722	2670	2670
query76	2517	972	949	949
query77	639	316	318	316
query78	11043	9715	8957	8957
query79	5210	529	522	522
query80	1169	475	470	470
query81	576	233	224	224
query82	1064	135	137	135
query83	324	173	172	172
query84	276	92	87	87
query85	1050	310	293	293
query86	470	297	295	295
query87	3279	3101	3089	3089
query88	4596	2376	2384	2376
query89	490	380	391	380
query90	1929	192	188	188
query91	132	104	103	103
query92	62	49	49	49
query93	5208	514	512	512
query94	1181	298	290	290
query95	401	308	329	308
query96	611	277	273	273
query97	3223	3047	3041	3041
query98	213	200	196	196
query99	1598	1274	1281	1274
Total cold run time: 276664 ms
Total hot run time: 173277 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.86 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d10a37fc3776c5438b8125d1f636fb231133916e, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.22	0.05	0.06
query4	1.66	0.09	0.08
query5	0.50	0.50	0.50
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.56	0.49	0.50
query10	0.55	0.54	0.54
query11	0.16	0.12	0.11
query12	0.15	0.12	0.13
query13	0.61	0.58	0.58
query14	0.77	0.77	0.78
query15	0.85	0.82	0.81
query16	0.36	0.37	0.35
query17	0.99	0.94	0.96
query18	0.23	0.22	0.21
query19	1.82	1.65	1.68
query20	0.01	0.01	0.01
query21	15.41	0.71	0.64
query22	4.05	6.90	2.33
query23	18.37	1.37	1.20
query24	2.10	0.23	0.22
query25	0.16	0.10	0.08
query26	0.29	0.21	0.21
query27	0.45	0.23	0.24
query28	13.30	1.01	1.01
query29	12.63	3.38	3.29
query30	0.25	0.06	0.06
query31	2.86	0.40	0.39
query32	3.27	0.48	0.47
query33	2.90	2.89	2.94
query34	17.08	4.35	4.37
query35	4.41	4.38	4.42
query36	0.67	0.48	0.47
query37	0.19	0.15	0.15
query38	0.16	0.14	0.15
query39	0.04	0.04	0.04
query40	0.16	0.12	0.12
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.72 s
Total hot run time: 30.86 s

@starocean999 starocean999 marked this pull request as ready for review July 22, 2024 03:03
@@ -622,6 +623,14 @@ private Plan bindAggregate(MatchingContext<LogicalAggregate<Plan>> ctx) {
SimpleExprAnalyzer aggOutputAnalyzer = buildSimpleExprAnalyzer(
agg, cascadesContext, agg.children(), true, true);
List<NamedExpression> boundAggOutput = aggOutputAnalyzer.analyzeToList(agg.getOutputExpressions());
if (boundAggOutput.size() == 1 && boundAggOutput.get(0) instanceof BoundStar) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be contains BoundStar? what about select *, c1 from t group by 1

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40350 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 752543310efdd5c516d146ebcac18d7802b94149, data reload: false

------ Round 1 ----------------------------------
q1	17957	4678	4466	4466
q2	2679	195	192	192
q3	11847	1208	1101	1101
q4	10559	828	817	817
q5	8342	2769	2702	2702
q6	223	144	143	143
q7	977	613	609	609
q8	9305	2086	2105	2086
q9	8923	6640	6592	6592
q10	8836	3809	3842	3809
q11	451	250	249	249
q12	407	231	233	231
q13	19073	3012	3027	3012
q14	286	238	240	238
q15	531	495	499	495
q16	505	393	395	393
q17	978	683	674	674
q18	8128	7443	7414	7414
q19	3931	1403	1260	1260
q20	691	324	355	324
q21	4987	3251	3959	3251
q22	344	292	295	292
Total cold run time: 119960 ms
Total hot run time: 40350 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4404	4234	4244	4234
q2	385	271	271	271
q3	2969	2789	2789	2789
q4	1949	1643	1593	1593
q5	5336	5338	5322	5322
q6	217	128	135	128
q7	2142	1740	1710	1710
q8	3188	3383	3324	3324
q9	8507	8435	8392	8392
q10	3874	3740	3758	3740
q11	579	502	486	486
q12	820	602	604	602
q13	17699	2991	2968	2968
q14	302	280	274	274
q15	518	473	468	468
q16	474	433	424	424
q17	1772	1500	1486	1486
q18	7882	7636	7447	7447
q19	1696	1548	1635	1548
q20	2007	1795	1761	1761
q21	5044	4885	4844	4844
q22	597	527	511	511
Total cold run time: 72361 ms
Total hot run time: 54322 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173623 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 752543310efdd5c516d146ebcac18d7802b94149, data reload: false

query1	921	373	365	365
query2	6437	1959	1834	1834
query3	6666	209	211	209
query4	23737	17769	17396	17396
query5	4260	484	499	484
query6	274	166	190	166
query7	4600	292	285	285
query8	246	190	196	190
query9	8429	2466	2444	2444
query10	456	301	276	276
query11	11051	10193	10180	10180
query12	131	81	81	81
query13	1641	367	364	364
query14	10256	7381	7546	7381
query15	218	169	173	169
query16	7802	480	472	472
query17	1387	554	541	541
query18	1963	290	300	290
query19	203	161	157	157
query20	87	86	83	83
query21	210	132	125	125
query22	4314	4150	3985	3985
query23	33811	33401	33294	33294
query24	12211	2834	2834	2834
query25	639	355	365	355
query26	1701	147	148	147
query27	2954	268	270	268
query28	7517	2043	2024	2024
query29	970	652	623	623
query30	300	147	156	147
query31	931	730	738	730
query32	88	52	58	52
query33	768	320	317	317
query34	951	477	485	477
query35	833	743	735	735
query36	1111	970	956	956
query37	262	78	81	78
query38	2851	2792	2730	2730
query39	866	802	804	802
query40	272	120	118	118
query41	51	46	45	45
query42	119	101	102	101
query43	505	482	445	445
query44	1237	736	736	736
query45	192	161	160	160
query46	1084	717	738	717
query47	1851	1792	1779	1779
query48	368	286	294	286
query49	1188	424	416	416
query50	770	390	391	390
query51	6874	6731	6821	6731
query52	105	88	91	88
query53	359	290	296	290
query54	954	447	469	447
query55	77	72	76	72
query56	287	265	266	265
query57	1153	1047	1040	1040
query58	255	239	254	239
query59	2894	2922	2482	2482
query60	297	275	282	275
query61	95	92	90	90
query62	814	642	649	642
query63	319	294	292	292
query64	10432	2244	1672	1672
query65	3184	3110	3098	3098
query66	1363	335	338	335
query67	15704	15047	15137	15047
query68	4538	537	547	537
query69	523	325	335	325
query70	1216	1158	1198	1158
query71	402	278	276	276
query72	7699	5171	5828	5171
query73	743	330	326	326
query74	6258	5620	5701	5620
query75	3469	2724	2702	2702
query76	3032	911	967	911
query77	515	319	306	306
query78	12774	9667	10374	9667
query79	3516	530	526	526
query80	1597	490	482	482
query81	576	221	239	221
query82	366	135	133	133
query83	291	167	166	166
query84	280	88	95	88
query85	729	316	303	303
query86	478	371	324	324
query87	3262	3091	3090	3090
query88	4265	2391	2382	2382
query89	474	408	396	396
query90	2027	197	197	197
query91	128	104	99	99
query92	59	53	52	52
query93	1990	515	511	511
query94	1254	283	281	281
query95	402	312	310	310
query96	598	272	271	271
query97	3200	3005	3085	3005
query98	212	200	205	200
query99	1607	1243	1288	1243
Total cold run time: 285336 ms
Total hot run time: 173623 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.18 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 752543310efdd5c516d146ebcac18d7802b94149, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.04	0.04
query4	1.69	0.08	0.07
query5	0.49	0.49	0.48
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.49	0.49
query10	0.52	0.55	0.54
query11	0.15	0.11	0.11
query12	0.15	0.13	0.12
query13	0.60	0.59	0.58
query14	0.75	0.78	0.76
query15	0.85	0.82	0.80
query16	0.34	0.35	0.36
query17	0.99	1.01	0.94
query18	0.22	0.21	0.22
query19	1.76	1.71	1.70
query20	0.02	0.01	0.01
query21	15.43	0.78	0.65
query22	4.84	6.89	2.40
query23	18.22	1.55	1.33
query24	2.09	0.23	0.22
query25	0.15	0.08	0.09
query26	0.29	0.21	0.21
query27	0.46	0.23	0.23
query28	13.33	1.02	1.00
query29	12.63	3.31	3.35
query30	0.26	0.07	0.06
query31	2.85	0.38	0.39
query32	3.28	0.49	0.47
query33	2.92	2.91	2.91
query34	17.25	4.42	4.40
query35	4.46	4.44	4.48
query36	0.66	0.46	0.47
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.15	0.12	0.13
query41	0.09	0.04	0.04
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.47 s
Total hot run time: 31.18 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 22, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@starocean999 starocean999 merged commit 33c878f into apache:master Jul 23, 2024
27 of 29 checks passed
dataroaring pushed a commit that referenced this pull request Jul 24, 2024
## Proposed changes

```
create table t1
(
    c1 bigint, 
    c2 bigint
)
DISTRIBUTED BY HASH(c1) BUCKETS 3
PROPERTIES ("replication_num" = "1");
```

sql
`select * from t1 group by 1;`

the error msg before:
ERROR 1105 (HY000): errCode = 2, detailMessage = Invalid call to toSlot
on unbound object
after:
ERROR 1105 (HY000): errCode = 2, detailMessage = c2 not in aggregate's
output



Issue Number: close #xxx

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants