Skip to content

branch-3.0: [fix](variant) fix schema change for variant from not null to null #46279 - #46346

Merged
eldenmoon merged 1 commit into
branch-3.0from
auto-pick-46279-branch-3.0
Jan 3, 2025
Merged

branch-3.0: [fix](variant) fix schema change for variant from not null to null #46279#46346
eldenmoon merged 1 commit into
branch-3.0from
auto-pick-46279-branch-3.0

Conversation

@github-actions

@github-actions github-actions Bot commented Jan 3, 2025

Copy link
Copy Markdown
Contributor

Cherry-picked from #46279

@eldenmoon Your Name
[fix](variant) fix schema change for variant from not null to null (#…
1e40dda 
…46279)

1. cast bettween not null and null in cast_column
2. hierarchical_data_reader should fill null map when not null -> null
in linked schema change
@hello-stephen

hello-stephen commented Jan 3, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Jan 3, 2025
@dataroaring dataroaring reopened this Jan 3, 2025
@hello-stephen

hello-stephen commented Jan 3, 2025

Copy link
Copy Markdown
Contributor

run buildall

@doris-robot

doris-robot commented Jan 3, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 40748 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1e40dda8c34e2460f19e52b13a331e234823e8bc, data reload: false

------ Round 1 ----------------------------------
q1	17580	7420	7290	7290
q2	2069	183	189	183
q3	10560	1100	1189	1100
q4	10569	690	745	690
q5	7751	3085	2868	2868
q6	237	153	151	151
q7	980	623	610	610
q8	9369	1978	2087	1978
q9	6635	6405	6420	6405
q10	7045	2260	2327	2260
q11	465	267	267	267
q12	402	221	218	218
q13	17814	3002	2971	2971
q14	243	218	207	207
q15	556	523	525	523
q16	711	594	609	594
q17	993	553	575	553
q18	7252	6713	6656	6656
q19	1449	1013	1105	1013
q20	483	207	203	203
q21	4277	3290	3022	3022
q22	1116	986	1017	986
Total cold run time: 108556 ms
Total hot run time: 40748 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7354	7198	7258	7198
q2	328	229	229	229
q3	2962	2957	2932	2932
q4	2065	1878	1852	1852
q5	5702	5729	5745	5729
q6	223	143	148	143
q7	2251	1822	1826	1822
q8	3326	3554	3475	3475
q9	8962	8934	8924	8924
q10	3574	3568	3531	3531
q11	601	491	514	491
q12	808	589	616	589
q13	9869	3158	3185	3158
q14	307	274	273	273
q15	557	531	521	521
q16	710	683	664	664
q17	1840	1630	1578	1578
q18	8235	7724	7727	7724
q19	1648	1625	1658	1625
q20	2124	1889	1915	1889
q21	5463	5323	5376	5323
q22	1122	1045	1043	1043
Total cold run time: 70031 ms
Total hot run time: 60713 ms

@doris-robot

doris-robot commented Jan 3, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 198011 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1e40dda8c34e2460f19e52b13a331e234823e8bc, data reload: false

query1	1304	939	901	901
query2	6245	2128	2090	2090
query3	10840	4300	4363	4300
query4	65729	31375	23586	23586
query5	4808	469	443	443
query6	410	167	163	163
query7	5560	315	311	311
query8	310	234	229	229
query9	8958	2704	2710	2704
query10	471	296	265	265
query11	16426	15173	16107	15173
query12	156	99	105	99
query13	1524	476	427	427
query14	10498	7789	7777	7777
query15	217	198	191	191
query16	7091	488	455	455
query17	1193	581	564	564
query18	1327	324	328	324
query19	206	155	155	155
query20	117	112	118	112
query21	69	43	46	43
query22	4603	4620	4520	4520
query23	34722	34045	34439	34045
query24	6135	2963	2982	2963
query25	540	429	431	429
query26	671	172	168	168
query27	1743	311	320	311
query28	4565	2538	2471	2471
query29	712	475	458	458
query30	243	173	169	169
query31	1039	798	804	798
query32	64	57	57	57
query33	402	306	281	281
query34	932	514	519	514
query35	849	767	762	762
query36	1091	965	965	965
query37	118	74	80	74
query38	4146	4104	4051	4051
query39	1515	1503	1465	1465
query40	144	82	87	82
query41	50	47	53	47
query42	117	103	102	102
query43	549	486	506	486
query44	1190	845	847	845
query45	188	177	174	174
query46	1138	734	748	734
query47	2019	1902	1906	1902
query48	491	388	393	388
query49	740	394	390	390
query50	869	430	426	426
query51	7385	7163	7001	7001
query52	99	92	88	88
query53	247	185	176	176
query54	559	448	454	448
query55	75	73	79	73
query56	250	237	267	237
query57	1239	1125	1105	1105
query58	209	200	205	200
query59	3181	3052	3171	3052
query60	276	252	247	247
query61	109	110	110	110
query62	778	671	653	653
query63	210	196	198	196
query64	1402	677	629	629
query65	3241	3237	3194	3194
query66	617	303	308	303
query67	15946	15540	15750	15540
query68	3928	589	561	561
query69	422	274	266	266
query70	1101	1102	1104	1102
query71	359	254	256	254
query72	6442	4074	4009	4009
query73	767	350	349	349
query74	9622	9021	9070	9021
query75	3336	2689	2660	2660
query76	1825	1038	903	903
query77	507	267	286	267
query78	10704	9643	9503	9503
query79	1167	616	597	597
query80	819	420	433	420
query81	524	244	237	237
query82	1368	121	118	118
query83	225	150	144	144
query84	290	84	77	77
query85	862	309	297	297
query86	319	303	298	298
query87	4488	4390	4226	4226
query88	3649	2401	2379	2379
query89	415	293	287	287
query90	1948	187	187	187
query91	181	145	146	145
query92	53	51	52	51
query93	1320	545	538	538
query94	743	293	299	293
query95	347	249	255	249
query96	606	279	290	279
query97	3348	3188	3198	3188
query98	210	199	196	196
query99	1769	1269	1280	1269
Total cold run time: 314531 ms
Total hot run time: 198011 ms

@doris-robot

doris-robot commented Jan 3, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 33.18 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1e40dda8c34e2460f19e52b13a331e234823e8bc, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.02
query3	0.23	0.07	0.06
query4	1.65	0.10	0.10
query5	0.50	0.50	0.52
query6	1.15	0.73	0.72
query7	0.03	0.02	0.02
query8	0.04	0.05	0.03
query9	0.56	0.50	0.50
query10	0.56	0.54	0.55
query11	0.14	0.10	0.10
query12	0.15	0.11	0.11
query13	0.60	0.60	0.60
query14	2.94	2.90	2.91
query15	0.91	0.83	0.81
query16	0.37	0.39	0.38
query17	1.03	1.00	1.04
query18	0.24	0.22	0.22
query19	1.92	1.86	1.96
query20	0.01	0.01	0.01
query21	15.35	0.58	0.58
query22	2.94	2.06	2.30
query23	17.06	0.92	0.68
query24	2.68	1.17	1.90
query25	0.25	0.18	0.14
query26	0.29	0.14	0.14
query27	0.05	0.04	0.04
query28	10.01	1.12	1.08
query29	12.53	3.20	3.14
query30	0.24	0.06	0.08
query31	2.85	0.39	0.38
query32	3.28	0.46	0.46
query33	2.99	3.03	3.06
query34	16.96	4.48	4.43
query35	4.54	4.51	4.50
query36	0.68	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.14
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 106.31 s
Total hot run time: 33.18 s

@eldenmoon
eldenmoon merged commit 30b7c94 into branch-3.0 Jan 3, 2025
@github-actions
github-actions Bot deleted the auto-pick-46279-branch-3.0 branch January 3, 2025 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hello-stephen @doris-robot @eldenmoon @dataroaring