Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](Variant) handle scalar variant with none string root #37794

Merged
merged 2 commits into from
Jul 16, 2024
Merged

[Fix](Variant) handle scalar variant with none string root #37794

merged 2 commits into from
Jul 16, 2024

Conversation

eldenmoon
Copy link
Member

@eldenmoon eldenmoon commented Jul 15, 2024 •
edited
Loading

  1. If columns is ColumnObject(Int32), root cast to ColumnString will result crash
  2. variant with invalid json will be stored as raw string instead of throwing exception

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@eldenmoon
Copy link
Member Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40179 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a6fb9cd7c25ae8aa14a82e9cfc4ff753fe0a4267, data reload: false

------ Round 1 ----------------------------------
q1	17620	4390	4338	4338
q2	2018	195	192	192
q3	10434	1269	1059	1059
q4	10202	835	891	835
q5	7535	2697	2765	2697
q6	226	143	140	140
q7	976	603	613	603
q8	9233	2090	2106	2090
q9	8782	6616	6567	6567
q10	8851	3800	3749	3749
q11	451	241	247	241
q12	429	236	236	236
q13	18972	2986	2958	2958
q14	285	229	244	229
q15	525	479	473	473
q16	512	394	381	381
q17	977	619	693	619
q18	8090	7469	7422	7422
q19	7780	1503	1346	1346
q20	707	321	327	321
q21	5040	3397	3429	3397
q22	347	293	286	286
Total cold run time: 119992 ms
Total hot run time: 40179 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4458	4337	4456	4337
q2	377	256	256	256
q3	3142	3012	2953	2953
q4	2076	1730	1714	1714
q5	5688	5539	5649	5539
q6	240	137	140	137
q7	2276	1890	1867	1867
q8	3376	3470	3478	3470
q9	8844	9075	8918	8918
q10	4186	3992	4085	3992
q11	601	519	523	519
q12	846	671	675	671
q13	17461	3402	3390	3390
q14	343	294	342	294
q15	551	514	540	514
q16	537	477	460	460
q17	1878	1548	1515	1515
q18	8888	8188	7871	7871
q19	1830	1554	1667	1554
q20	2155	1957	1865	1865
q21	8042	4943	4976	4943
q22	587	508	541	508
Total cold run time: 78382 ms
Total hot run time: 57287 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174073 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a6fb9cd7c25ae8aa14a82e9cfc4ff753fe0a4267, data reload: false

query1	924	374	360	360
query2	6454	1955	1776	1776
query3	6646	205	212	205
query4	24963	17509	17559	17509
query5	3621	491	481	481
query6	265	168	158	158
query7	4577	289	283	283
query8	242	190	188	188
query9	8477	2351	2338	2338
query10	426	280	264	264
query11	12011	10088	10013	10013
query12	118	90	86	86
query13	1633	365	349	349
query14	9723	7848	7773	7773
query15	227	168	167	167
query16	7733	315	305	305
query17	1660	565	533	533
query18	1955	289	285	285
query19	205	156	165	156
query20	92	86	85	85
query21	204	130	127	127
query22	4336	4073	4109	4073
query23	34070	33820	33748	33748
query24	10896	3008	2968	2968
query25	607	427	416	416
query26	716	159	159	159
query27	2323	279	284	279
query28	6346	2042	2031	2031
query29	905	677	652	652
query30	259	155	155	155
query31	972	764	793	764
query32	91	57	62	57
query33	664	320	307	307
query34	916	520	517	517
query35	712	602	619	602
query36	1142	970	992	970
query37	150	85	92	85
query38	2968	2805	2781	2781
query39	904	815	818	815
query40	205	125	126	125
query41	48	49	49	49
query42	128	106	104	104
query43	509	483	461	461
query44	1131	741	734	734
query45	199	161	162	161
query46	1102	759	733	733
query47	1861	1789	1767	1767
query48	369	302	299	299
query49	857	436	423	423
query50	775	398	406	398
query51	6870	6904	6734	6734
query52	105	94	93	93
query53	362	293	304	293
query54	896	464	452	452
query55	80	76	76	76
query56	300	289	284	284
query57	1129	1034	1080	1034
query58	256	260	250	250
query59	2778	2641	2549	2549
query60	319	298	305	298
query61	119	115	114	114
query62	800	634	654	634
query63	339	298	294	294
query64	9428	2239	1690	1690
query65	3253	3096	3098	3096
query66	834	332	328	328
query67	15642	14953	14980	14953
query68	9174	550	561	550
query69	718	443	354	354
query70	1455	1097	1152	1097
query71	536	276	278	276
query72	9543	5481	5525	5481
query73	2231	328	325	325
query74	6329	5709	5660	5660
query75	6077	2676	2683	2676
query76	5261	946	1029	946
query77	758	300	303	300
query78	9813	9030	8993	8993
query79	10111	517	519	517
query80	875	478	475	475
query81	571	223	216	216
query82	278	128	131	128
query83	337	164	165	164
query84	279	83	86	83
query85	1080	311	299	299
query86	355	314	327	314
query87	3289	3075	3113	3075
query88	4440	2450	2451	2450
query89	521	385	393	385
query90	2081	190	189	189
query91	127	100	100	100
query92	58	48	48	48
query93	5886	500	495	495
query94	1387	216	218	216
query95	407	318	319	318
query96	605	277	271	271
query97	3185	3011	3026	3011
query98	214	237	188	188
query99	1573	1262	1227	1227
Total cold run time: 301381 ms
Total hot run time: 174073 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.19 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a6fb9cd7c25ae8aa14a82e9cfc4ff753fe0a4267, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.03
query3	0.22	0.04	0.05
query4	1.68	0.08	0.08
query5	0.51	0.49	0.49
query6	1.14	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.56	0.50	0.50
query10	0.54	0.54	0.55
query11	0.16	0.11	0.12
query12	0.14	0.12	0.13
query13	0.59	0.59	0.59
query14	0.77	0.76	0.78
query15	0.85	0.83	0.81
query16	0.35	0.38	0.37
query17	1.04	1.07	1.03
query18	0.22	0.21	0.21
query19	1.82	1.73	1.79
query20	0.01	0.02	0.02
query21	15.41	0.75	0.66
query22	3.47	6.82	3.25
query23	18.43	1.33	1.28
query24	2.12	0.24	0.23
query25	0.16	0.09	0.09
query26	0.29	0.20	0.20
query27	0.45	0.23	0.22
query28	13.30	1.04	1.00
query29	12.61	3.41	3.36
query30	0.25	0.07	0.05
query31	2.88	0.40	0.38
query32	3.27	0.48	0.47
query33	2.95	2.91	2.97
query34	17.12	4.38	4.38
query35	4.46	4.42	4.41
query36	0.66	0.47	0.48
query37	0.19	0.15	0.15
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.16	0.12	0.12
query41	0.09	0.04	0.05
query42	0.05	0.05	0.05
query43	0.05	0.04	0.05
Total cold run time: 109.37 s
Total hot run time: 32.19 s

@eldenmoon
[Fix](Variant) handle scalar variant with none string root
026dc72 
If columns is ColumnObject(Int32), root cast to ColumnString will result crash
@eldenmoon
Copy link
Member Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39977 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 026dc72200a6b0479d477b1af4eec685226ee6b4, data reload: false

------ Round 1 ----------------------------------
q1	18025	4496	4396	4396
q2	2492	193	188	188
q3	11512	1207	1123	1123
q4	10300	758	830	758
q5	7656	2923	2764	2764
q6	231	142	148	142
q7	983	632	605	605
q8	9342	2098	2075	2075
q9	8807	6549	6525	6525
q10	8725	3724	3778	3724
q11	447	241	237	237
q12	402	232	229	229
q13	17769	3039	2988	2988
q14	279	229	233	229
q15	537	470	488	470
q16	451	374	378	374
q17	964	635	694	635
q18	8041	7609	7355	7355
q19	7738	1448	1380	1380
q20	709	319	332	319
q21	5031	3264	3180	3180
q22	345	281	289	281
Total cold run time: 120786 ms
Total hot run time: 39977 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4454	4260	4284	4260
q2	383	269	267	267
q3	2973	2744	2725	2725
q4	1895	1590	1621	1590
q5	5260	5281	5293	5281
q6	222	134	135	134
q7	2135	1716	1712	1712
q8	3210	3335	3307	3307
q9	8390	8365	8349	8349
q10	3949	3619	3688	3619
q11	586	504	482	482
q12	778	617	609	609
q13	17417	2977	3009	2977
q14	299	269	267	267
q15	515	489	470	470
q16	464	415	442	415
q17	1771	1502	1474	1474
q18	7728	7538	7394	7394
q19	1714	1456	1538	1456
q20	2029	1809	1795	1795
q21	4927	4610	4815	4610
q22	575	510	490	490
Total cold run time: 71674 ms
Total hot run time: 53683 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173887 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 026dc72200a6b0479d477b1af4eec685226ee6b4, data reload: false

query1	916	378	362	362
query2	6522	1826	1847	1826
query3	6675	210	220	210
query4	21716	17504	17209	17209
query5	4204	500	472	472
query6	272	163	160	160
query7	4591	297	295	295
query8	252	205	192	192
query9	8528	2428	2420	2420
query10	436	288	276	276
query11	11665	10068	9885	9885
query12	149	89	80	80
query13	1640	374	373	373
query14	10278	7736	7697	7697
query15	270	170	166	166
query16	7827	338	326	326
query17	1782	558	534	534
query18	1407	287	280	280
query19	196	157	159	157
query20	87	82	86	82
query21	205	129	126	126
query22	4240	4042	3982	3982
query23	33837	33620	33548	33548
query24	11067	2936	2895	2895
query25	620	387	395	387
query26	1376	158	147	147
query27	2699	275	278	275
query28	7458	2049	2037	2037
query29	895	662	633	633
query30	291	150	156	150
query31	974	729	745	729
query32	101	59	61	59
query33	805	313	310	310
query34	903	473	485	473
query35	704	580	577	577
query36	1127	953	950	950
query37	144	80	85	80
query38	2875	2785	2715	2715
query39	879	804	828	804
query40	282	121	121	121
query41	49	49	46	46
query42	126	106	103	103
query43	516	481	467	467
query44	1214	754	731	731
query45	192	167	165	165
query46	1105	738	730	730
query47	1849	1774	1779	1774
query48	373	297	304	297
query49	1060	438	424	424
query50	791	409	418	409
query51	6865	6824	6809	6809
query52	102	90	92	90
query53	365	302	304	302
query54	973	455	453	453
query55	78	74	75	74
query56	307	285	287	285
query57	1109	1060	1061	1060
query58	262	248	405	248
query59	2735	2591	2715	2591
query60	314	272	268	268
query61	96	93	93	93
query62	818	650	656	650
query63	336	289	295	289
query64	9650	2241	1656	1656
query65	3156	3111	3100	3100
query66	764	329	356	329
query67	15703	15064	14895	14895
query68	8604	560	584	560
query69	750	435	352	352
query70	1395	1116	1139	1116
query71	503	279	271	271
query72	8778	5721	5979	5721
query73	2241	330	330	330
query74	6106	5660	5686	5660
query75	4869	2687	2707	2687
query76	5417	987	974	974
query77	778	309	304	304
query78	9908	9315	9972	9315
query79	10381	534	538	534
query80	973	479	470	470
query81	586	233	220	220
query82	278	133	131	131
query83	331	170	175	170
query84	281	90	91	90
query85	1091	305	292	292
query86	341	307	304	304
query87	3317	3148	3118	3118
query88	4419	2395	2374	2374
query89	516	373	382	373
query90	1804	197	197	197
query91	140	102	103	102
query92	67	51	49	49
query93	5769	525	529	525
query94	714	216	217	216
query95	404	321	318	318
query96	623	275	270	270
query97	3225	3030	3017	3017
query98	227	203	194	194
query99	1535	1295	1336	1295
Total cold run time: 298174 ms
Total hot run time: 173887 ms

@eldenmoon eldenmoon force-pushed the fix-var-mtmv branch 2 times, most recently from cbb46d7 to 026dc72 Compare July 15, 2024 16:28
@eldenmoon
Copy link
Member Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

2 similar comments
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39727 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 026dc72200a6b0479d477b1af4eec685226ee6b4, data reload: false

------ Round 1 ----------------------------------
q1	17636	4370	4285	4285
q2	2016	190	182	182
q3	10448	1235	1151	1151
q4	10182	813	795	795
q5	7502	2648	2632	2632
q6	222	141	139	139
q7	957	596	592	592
q8	9187	2066	2034	2034
q9	8583	6569	6518	6518
q10	8842	3738	3779	3738
q11	439	240	234	234
q12	469	232	234	232
q13	17759	2961	2953	2953
q14	265	240	238	238
q15	533	490	495	490
q16	502	375	374	374
q17	959	648	760	648
q18	7996	7424	7394	7394
q19	5066	1267	1350	1267
q20	665	336	331	331
q21	4991	3218	3238	3218
q22	353	293	282	282
Total cold run time: 115572 ms
Total hot run time: 39727 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4514	4398	4239	4239
q2	378	258	271	258
q3	3024	2770	2961	2770
q4	1976	1747	1690	1690
q5	5629	5563	5525	5525
q6	240	131	135	131
q7	2204	1832	1874	1832
q8	3223	3433	3359	3359
q9	8784	8764	8853	8764
q10	4113	3867	3810	3810
q11	575	498	494	494
q12	825	621	640	621
q13	16515	3262	3183	3183
q14	321	275	289	275
q15	551	501	510	501
q16	518	430	449	430
q17	1801	1562	1544	1544
q18	8128	7946	7893	7893
q19	1710	1505	1705	1505
q20	2205	1859	1872	1859
q21	5159	4939	4775	4775
q22	560	481	504	481
Total cold run time: 72953 ms
Total hot run time: 55939 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174899 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 026dc72200a6b0479d477b1af4eec685226ee6b4, data reload: false

query1	919	374	367	367
query2	6463	1840	1849	1840
query3	6653	207	212	207
query4	28354	17661	17313	17313
query5	3698	505	512	505
query6	259	168	171	168
query7	4590	301	289	289
query8	244	191	196	191
query9	8519	2421	2407	2407
query10	446	290	281	281
query11	12530	10011	10108	10011
query12	118	89	82	82
query13	1646	377	372	372
query14	10320	7854	7833	7833
query15	224	171	168	168
query16	7443	337	315	315
query17	1791	569	538	538
query18	1830	285	289	285
query19	214	152	154	152
query20	92	82	81	81
query21	210	126	124	124
query22	4227	3939	4021	3939
query23	33964	34517	34322	34322
query24	10635	2926	3006	2926
query25	548	420	420	420
query26	693	159	159	159
query27	2238	285	292	285
query28	5829	2107	2131	2107
query29	891	658	645	645
query30	259	156	163	156
query31	1000	760	775	760
query32	93	56	60	56
query33	682	346	318	318
query34	890	531	502	502
query35	677	603	611	603
query36	1136	992	995	992
query37	159	89	97	89
query38	3006	2895	2820	2820
query39	918	884	885	884
query40	219	132	131	131
query41	50	52	48	48
query42	123	106	103	103
query43	529	469	474	469
query44	1109	737	736	736
query45	198	166	168	166
query46	1081	756	723	723
query47	1847	1759	1802	1759
query48	372	291	306	291
query49	838	434	512	434
query50	786	400	388	388
query51	6832	6843	6832	6832
query52	104	95	96	95
query53	348	291	286	286
query54	892	456	453	453
query55	87	73	75	73
query56	282	265	279	265
query57	1121	1067	1065	1065
query58	261	250	252	250
query59	2794	2640	2648	2640
query60	321	285	298	285
query61	95	95	91	91
query62	809	663	656	656
query63	323	297	307	297
query64	9149	2186	1719	1719
query65	3178	3117	3118	3117
query66	752	329	324	324
query67	15612	14811	14775	14775
query68	8020	576	579	576
query69	758	451	353	353
query70	1196	1093	1095	1093
query71	521	290	283	283
query72	8317	5910	5533	5533
query73	836	327	327	327
query74	6174	5687	5693	5687
query75	4821	2725	2677	2677
query76	4628	930	876	876
query77	785	319	308	308
query78	9716	9026	10969	9026
query79	8135	538	537	537
query80	1061	489	485	485
query81	577	227	220	220
query82	627	143	136	136
query83	376	167	167	167
query84	276	85	86	85
query85	1304	312	303	303
query86	402	315	332	315
query87	3292	3079	3089	3079
query88	4538	2391	2376	2376
query89	500	388	400	388
query90	2014	190	194	190
query91	132	101	103	101
query92	61	50	49	49
query93	5045	562	520	520
query94	1312	215	218	215
query95	407	328	325	325
query96	606	274	268	268
query97	3204	2998	3032	2998
query98	239	212	200	200
query99	1570	1256	1220	1220
Total cold run time: 295970 ms
Total hot run time: 174899 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.75 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 026dc72200a6b0479d477b1af4eec685226ee6b4, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.07	0.07
query5	0.49	0.48	0.48
query6	1.14	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.05
query9	0.55	0.49	0.49
query10	0.55	0.54	0.57
query11	0.15	0.11	0.11
query12	0.15	0.13	0.13
query13	0.59	0.58	0.59
query14	0.77	0.78	0.77
query15	0.86	0.81	0.82
query16	0.37	0.36	0.34
query17	0.96	0.98	0.97
query18	0.23	0.21	0.21
query19	1.88	1.71	1.76
query20	0.01	0.01	0.00
query21	15.39	0.71	0.65
query22	4.46	6.46	2.99
query23	18.31	1.39	1.26
query24	2.11	0.23	0.22
query25	0.15	0.08	0.08
query26	0.29	0.21	0.21
query27	0.45	0.23	0.23
query28	13.31	1.06	1.03
query29	12.64	3.43	3.47
query30	0.25	0.06	0.05
query31	2.88	0.40	0.39
query32	3.25	0.48	0.48
query33	2.92	2.85	2.89
query34	17.07	4.37	4.34
query35	4.42	4.37	4.53
query36	0.66	0.50	0.46
query37	0.19	0.16	0.16
query38	0.16	0.15	0.16
query39	0.04	0.03	0.04
query40	0.15	0.13	0.12
query41	0.09	0.05	0.05
query42	0.05	0.05	0.05
query43	0.05	0.04	0.05
Total cold run time: 110.1 s
Total hot run time: 31.75 s

@eldenmoon
Copy link
Member Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40102 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 44e198d799b20d1a195bf2803b347d1e5290926c, data reload: false

------ Round 1 ----------------------------------
q1	17812	4522	4413	4413
q2	2258	200	189	189
q3	10441	1176	1032	1032
q4	10194	837	793	793
q5	7555	2718	2679	2679
q6	229	136	136	136
q7	985	604	605	604
q8	9227	2099	2117	2099
q9	8750	6575	6617	6575
q10	8819	3791	3834	3791
q11	454	240	235	235
q12	393	221	228	221
q13	17761	2979	3002	2979
q14	289	228	248	228
q15	532	474	487	474
q16	491	381	378	378
q17	993	696	732	696
q18	8250	7570	7376	7376
q19	6885	1440	1435	1435
q20	710	328	330	328
q21	5011	3159	3289	3159
q22	354	282	285	282
Total cold run time: 118393 ms
Total hot run time: 40102 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4344	4269	4256	4256
q2	381	263	256	256
q3	3003	2785	2733	2733
q4	1915	1646	1650	1646
q5	5295	5326	5309	5309
q6	217	131	132	131
q7	2181	1767	1743	1743
q8	3224	3361	3322	3322
q9	8434	8343	8376	8343
q10	3917	3722	3653	3653
q11	554	495	484	484
q12	801	629	610	610
q13	16250	2984	3000	2984
q14	308	272	275	272
q15	519	474	472	472
q16	499	411	417	411
q17	1793	1506	1464	1464
q18	7736	7426	7375	7375
q19	5094	1620	1492	1492
q20	1993	1777	1772	1772
q21	4819	4757	4711	4711
q22	575	486	495	486
Total cold run time: 73852 ms
Total hot run time: 53925 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173480 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 44e198d799b20d1a195bf2803b347d1e5290926c, data reload: false

query1	926	388	371	371
query2	6469	1836	1800	1800
query3	6647	211	221	211
query4	28239	17479	17526	17479
query5	4182	484	487	484
query6	272	170	156	156
query7	4595	300	297	297
query8	237	198	194	194
query9	8580	2434	2408	2408
query10	443	288	312	288
query11	10666	10172	10083	10083
query12	136	88	80	80
query13	1644	359	362	359
query14	10256	7600	7694	7600
query15	219	168	165	165
query16	7597	323	316	316
query17	1807	551	551	551
query18	1922	278	274	274
query19	209	166	150	150
query20	93	83	82	82
query21	207	133	123	123
query22	4211	3968	4027	3968
query23	33876	33202	33817	33202
query24	11950	3053	2850	2850
query25	608	372	379	372
query26	1586	151	151	151
query27	2926	281	276	276
query28	7698	2020	2016	2016
query29	925	659	678	659
query30	285	149	149	149
query31	960	717	759	717
query32	97	57	53	53
query33	763	301	302	301
query34	931	476	475	475
query35	669	572	556	556
query36	1147	972	987	972
query37	167	78	80	78
query38	2901	2750	2777	2750
query39	862	801	821	801
query40	280	122	123	122
query41	47	47	45	45
query42	118	101	102	101
query43	499	472	459	459
query44	1247	757	739	739
query45	199	163	160	160
query46	1087	743	715	715
query47	1843	1774	1786	1774
query48	357	294	288	288
query49	1171	415	414	414
query50	811	394	390	390
query51	6899	6792	6773	6773
query52	105	91	102	91
query53	356	293	291	291
query54	887	448	445	445
query55	74	78	77	77
query56	310	270	268	268
query57	1136	1109	1042	1042
query58	257	260	247	247
query59	2925	2624	2635	2624
query60	298	284	295	284
query61	103	95	96	95
query62	816	629	654	629
query63	319	301	285	285
query64	10530	2198	1637	1637
query65	3181	3091	3130	3091
query66	1254	331	336	331
query67	15413	14976	14858	14858
query68	5108	548	541	541
query69	601	431	344	344
query70	1133	1180	1168	1168
query71	445	285	275	275
query72	6865	5936	5959	5936
query73	789	327	326	326
query74	6082	5749	5731	5731
query75	3479	2671	2689	2671
query76	3523	1012	920	920
query77	726	314	300	300
query78	9567	9031	8954	8954
query79	3127	527	523	523
query80	2012	476	472	472
query81	571	225	222	222
query82	774	134	137	134
query83	250	171	171	171
query84	281	84	94	84
query85	1516	332	309	309
query86	472	332	328	328
query87	3267	3124	3190	3124
query88	4373	2390	2360	2360
query89	496	399	373	373
query90	1949	198	189	189
query91	133	102	102	102
query92	63	51	49	49
query93	4319	520	511	511
query94	1218	215	280	215
query95	397	311	317	311
query96	603	270	271	270
query97	3203	3074	3014	3014
query98	222	200	195	195
query99	1761	1266	1275	1266
Total cold run time: 289755 ms
Total hot run time: 173480 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.51 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 44e198d799b20d1a195bf2803b347d1e5290926c, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.07	0.07
query5	0.51	0.48	0.49
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.54	0.50	0.48
query10	0.54	0.55	0.54
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.58	0.58	0.58
query14	0.76	0.77	0.80
query15	0.85	0.83	0.82
query16	0.37	0.36	0.36
query17	1.04	1.01	1.00
query18	0.23	0.22	0.22
query19	1.74	1.73	1.70
query20	0.01	0.01	0.02
query21	15.40	0.72	0.64
query22	5.27	6.73	1.75
query23	18.26	1.39	1.24
query24	2.10	0.21	0.23
query25	0.14	0.09	0.08
query26	0.29	0.21	0.21
query27	0.46	0.23	0.23
query28	13.31	1.03	1.00
query29	12.64	3.33	3.32
query30	0.25	0.07	0.06
query31	2.88	0.38	0.39
query32	3.27	0.47	0.47
query33	2.91	2.89	2.95
query34	17.03	4.39	4.39
query35	4.48	4.49	4.41
query36	0.66	0.46	0.47
query37	0.18	0.17	0.16
query38	0.15	0.15	0.15
query39	0.04	0.03	0.04
query40	0.15	0.13	0.11
query41	0.09	0.06	0.06
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.81 s
Total hot run time: 30.51 s

xiaokang
xiaokang approved these changes Jul 16, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGMT

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 16, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@eldenmoon eldenmoon merged commit 77156cf into apache:master Jul 16, 2024
26 of 30 checks passed
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Jul 16, 2024
@eldenmoon
[Fix](Variant) handle scalar variant with none string root (apache#37794
532e14a 
)

1. If columns is ColumnObject(Int32), root cast to ColumnString will
result crash
2. variant with invalid json will be stored as raw string instead of
throwing exception
@eldenmoon eldenmoon mentioned this pull request Jul 16, 2024
Merged
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Jul 16, 2024
@eldenmoon
[Fix](Variant) handle scalar variant with none string root (apache#37794
92b5f88 
)

1. If columns is ColumnObject(Int32), root cast to ColumnString will
result crash
2. variant with invalid json will be stored as raw string instead of
throwing exception
@eldenmoon eldenmoon mentioned this pull request Jul 16, 2024
Merged
yiguolei pushed a commit that referenced this pull request Jul 16, 2024
@eldenmoon
[Pick](Variant) pick some fix (#37922)
b15ccdb 
#37674
#37839
#37883 
#37857 
#37794
xiaokang pushed a commit that referenced this pull request Jul 17, 2024
@eldenmoon
[Pick](Variant) pick some fix (#37926)
2994232 
#37674
#37839
#37883
#37857
#37794
#37640
#37557
#37217
#37121
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 17, 2024
@eldenmoon @seawinde
[Fix](Variant) handle scalar variant with none string root (apache#37794
12c45fa 
)

1. If columns is ColumnObject(Int32), root cast to ColumnString will
result crash
2. variant with invalid json will be stored as raw string instead of
throwing exception
@yiguolei yiguolei mentioned this pull request Jul 19, 2024
Open
1 task
@xiaokang xiaokang removed the doing label Jul 29, 2024
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Aug 15, 2024
@eldenmoon
[Fix](Row store) fix row store with invalid json string in variant type
036a1c3 
Previous we allow invalid text as variant in PR apache#37794 and store as string type.But in encoding rowstore we CHECK the json is valid and store as jsonb binary field.In this PR we support the invalid json encoding as row store
@eldenmoon eldenmoon mentioned this pull request Aug 15, 2024
Merged
eldenmoon added a commit that referenced this pull request Aug 15, 2024
@eldenmoon
[Fix](Row store) fix row store with invalid json string in variant ty…
1deab37 
…pe (#39394)

Previous we allow invalid text as variant in PR #37794 and store as
string type.But in encoding rowstore we CHECK the json is valid and
store as jsonb binary field.In this PR we support the invalid json
encoding as row store
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Aug 16, 2024
@eldenmoon
[Fix](Row store) fix row store with invalid json string in variant ty…
c512f85 
…pe (apache#39394)

Previous we allow invalid text as variant in PR apache#37794 and store as
string type.But in encoding rowstore we CHECK the json is valid and
store as jsonb binary field.In this PR we support the invalid json
encoding as row store
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Aug 16, 2024
@eldenmoon
[Fix](Row store) fix row store with invalid json string in variant ty…
5b3051b 
…pe (apache#39394)

Previous we allow invalid text as variant in PR apache#37794 and store as
string type.But in encoding rowstore we CHECK the json is valid and
store as jsonb binary field.In this PR we support the invalid json
encoding as row store
dataroaring pushed a commit that referenced this pull request Aug 17, 2024
@eldenmoon @dataroaring
[Fix](Row store) fix row store with invalid json string in variant ty…
3be198f 
…pe (#39394)

Previous we allow invalid text as variant in PR #37794 and store as
string type.But in encoding rowstore we CHECK the json is valid and
store as jsonb binary field.In this PR we support the invalid json
encoding as row store
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xiaokang xiaokang xiaokang approved these changes

@qidaye qidaye qidaye approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.0-merged kind/behavior-changed reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@eldenmoon @doris-robot @xiaokang @qidaye @yiguolei