Skip to content

branch-3.1: [improve](function) support any_value function with complex type #49419 #53429

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 3 commits into from
Jul 18, 2025
Merged

branch-3.1: [improve](function) support any_value function with complex type #49419 #53429

morrySnow merged 3 commits into from
Jul 18, 2025

Conversation

@zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Jul 17, 2025 •
edited by morrySnow
Loading

pick from #49419

@zhangstar333
[improve](function) support any_value function with complex type (apa…
5d1b526 
…che#49419)

Problem Summary:

support any_value function with complex type: array/map/struct
bitmap/hll/quantile/agg_state/json
@zhangstar333 zhangstar333 requested a review from morrySnow as a code owner July 17, 2025 03:56
@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhangstar333
Copy link
Contributor Author

zhangstar333 commented Jul 17, 2025

run buildall

@zhangstar333
Copy link
Contributor Author

zhangstar333 commented Jul 17, 2025

run buildall

@zhangstar333
Copy link
Contributor Author

zhangstar333 commented Jul 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

TPC-H: Total hot run time: 39740 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4f5b598986df53c0a225479cbaf3ed6b0adc753d, data reload: false

------ Round 1 ----------------------------------
q1	17643	6804	6644	6644
q2	2073	193	163	163
q3	10513	1124	1148	1124
q4	10221	750	753	750
q5	7775	2852	2789	2789
q6	217	135	133	133
q7	984	631	604	604
q8	9356	1923	1980	1923
q9	6591	6348	6378	6348
q10	7050	2263	2272	2263
q11	467	270	255	255
q12	448	209	210	209
q13	17777	2991	2987	2987
q14	235	212	216	212
q15	509	461	467	461
q16	493	384	368	368
q17	978	524	564	524
q18	7208	6695	6673	6673
q19	1308	1063	1048	1048
q20	478	197	199	197
q21	4003	3084	3190	3084
q22	1104	995	981	981
Total cold run time: 107431 ms
Total hot run time: 39740 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6717	6628	6618	6618
q2	328	229	237	229
q3	2908	2923	2992	2923
q4	2115	1962	1906	1906
q5	5770	5815	5791	5791
q6	212	128	127	127
q7	2276	1791	1796	1791
q8	3336	3470	3520	3470
q9	8805	8926	8991	8926
q10	3544	3515	3502	3502
q11	592	490	486	486
q12	820	598	615	598
q13	8518	3200	3138	3138
q14	294	263	277	263
q15	504	480	471	471
q16	491	431	418	418
q17	1859	1659	1593	1593
q18	8220	7816	7805	7805
q19	1663	1366	1611	1366
q20	2022	1819	1915	1819
q21	5192	4957	4907	4907
q22	1175	1036	1023	1023
Total cold run time: 67361 ms
Total hot run time: 59170 ms

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

BE UT Coverage Report

Increment line coverage 18.18% (16/88) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.22% (12542/27733)
Line Coverage 36.09% (111390/308671)
Region Coverage 35.16% (57620/163860)
Branch Coverage 32.34% (31315/96828)

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

TPC-DS: Total hot run time: 196477 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4f5b598986df53c0a225479cbaf3ed6b0adc753d, data reload: false

query1	1302	919	921	919
query2	6379	1867	1881	1867
query3	10790	4339	4298	4298
query4	32865	23414	23996	23414
query5	4511	456	452	452
query6	271	171	175	171
query7	3995	314	315	314
query8	280	221	228	221
query9	9545	2593	2595	2593
query10	497	264	257	257
query11	17986	15257	15556	15257
query12	174	102	102	102
query13	1561	425	426	425
query14	10318	6458	6537	6458
query15	228	178	198	178
query16	7649	485	459	459
query17	1561	564	602	564
query18	2034	309	305	305
query19	231	165	166	165
query20	126	122	111	111
query21	218	113	110	110
query22	4650	4475	4386	4386
query23	34800	34408	33889	33889
query24	11373	2938	2922	2922
query25	633	390	407	390
query26	1268	171	168	168
query27	2449	355	343	343
query28	7502	2167	2140	2140
query29	884	432	444	432
query30	254	167	167	167
query31	1009	832	839	832
query32	91	57	57	57
query33	755	311	311	311
query34	971	523	513	513
query35	900	733	765	733
query36	1112	972	979	972
query37	131	66	68	66
query38	4118	3916	3965	3916
query39	1527	1479	1458	1458
query40	217	102	101	101
query41	50	48	51	48
query42	120	112	112	112
query43	495	472	468	468
query44	1428	840	827	827
query45	193	174	174	174
query46	1178	757	742	742
query47	2010	1882	1878	1878
query48	434	346	346	346
query49	905	429	414	414
query50	854	435	438	435
query51	7527	7386	7262	7262
query52	108	91	98	91
query53	265	183	187	183
query54	1255	479	471	471
query55	80	80	76	76
query56	270	257	258	257
query57	1321	1224	1219	1219
query58	225	214	225	214
query59	3264	2995	2934	2934
query60	294	265	258	258
query61	113	120	126	120
query62	869	675	705	675
query63	231	190	192	190
query64	4152	654	703	654
query65	3385	3257	3325	3257
query66	811	317	336	317
query67	16295	15535	15618	15535
query68	4554	583	595	583
query69	444	271	272	271
query70	1194	1116	1129	1116
query71	330	278	269	269
query72	6472	4196	4175	4175
query73	746	365	373	365
query74	10171	9215	9365	9215
query75	3386	2670	2666	2666
query76	2617	1048	1149	1048
query77	381	273	317	273
query78	10526	9600	9613	9600
query79	1677	605	597	597
query80	927	420	417	417
query81	550	230	224	224
query82	504	94	86	86
query83	249	141	141	141
query84	234	83	83	83
query85	1529	304	289	289
query86	459	285	286	285
query87	4332	4322	4226	4226
query88	4107	2418	2383	2383
query89	411	290	307	290
query90	1983	188	185	185
query91	137	105	108	105
query92	66	52	53	52
query93	2373	557	553	553
query94	873	293	253	253
query95	367	256	268	256
query96	625	284	282	282
query97	3275	3119	3112	3112
query98	224	228	199	199
query99	1558	1286	1285	1285
Total cold run time: 302578 ms
Total hot run time: 196477 ms

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

ClickBench: Total hot run time: 29.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4f5b598986df53c0a225479cbaf3ed6b0adc753d, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.11	0.10
query5	0.53	0.53	0.52
query6	1.13	0.72	0.76
query7	0.03	0.02	0.01
query8	0.04	0.04	0.04
query9	0.56	0.49	0.51
query10	0.56	0.55	0.55
query11	0.14	0.11	0.10
query12	0.13	0.11	0.11
query13	0.61	0.61	0.60
query14	0.77	0.79	0.79
query15	0.85	0.85	0.82
query16	0.40	0.39	0.38
query17	1.01	1.06	1.06
query18	0.24	0.22	0.23
query19	1.86	1.86	1.88
query20	0.02	0.01	0.01
query21	15.40	0.61	0.59
query22	2.34	2.24	1.36
query23	17.08	0.82	0.96
query24	3.27	0.38	1.43
query25	0.29	0.13	0.10
query26	0.34	0.13	0.14
query27	0.07	0.04	0.04
query28	10.67	0.53	0.46
query29	12.56	3.22	3.20
query30	0.25	0.07	0.05
query31	2.85	0.38	0.40
query32	3.24	0.45	0.46
query33	2.97	3.01	3.05
query34	16.88	4.53	4.52
query35	4.53	4.50	4.49
query36	0.67	0.49	0.47
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.02	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.71 s
Total hot run time: 29.25 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 17, 2025

BE Regression && UT Coverage Report

Increment line coverage 89.77% (79/88) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.18% (20873/27400)
Line Coverage 69.56% (214549/308433)
Region Coverage 67.58% (128341/189906)
Branch Coverage 61.19% (66831/109222)

1 similar comment
@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 17, 2025

BE Regression && UT Coverage Report

Increment line coverage 89.77% (79/88) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.18% (20873/27400)
Line Coverage 69.56% (214549/308433)
Region Coverage 67.58% (128341/189906)
Branch Coverage 61.19% (66831/109222)

@morrySnow morrySnow changed the title branch31: [improve](function) support any_value function with complex type (#49419) branch-3.1: [improve](function) support any_value function with complex type #49419 Jul 18, 2025
@morrySnow morrySnow merged commit 34ec333 into apache:branch-3.1 Jul 18, 2025
23 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@zhangstar333 @hello-stephen @doris-robot @morrySnow