Skip to content

[fix](schema-change) Fix schema change run into nullable check specified for MV#38806

Merged
dataroaring merged 1 commit intoapache:masterfrom
Hastyshell:fix-column-type-compare
Aug 4, 2024
Merged

[fix](schema-change) Fix schema change run into nullable check specified for MV#38806
dataroaring merged 1 commit intoapache:masterfrom
Hastyshell:fix-column-type-compare

Conversation

@Hastyshell
Copy link
Collaborator

@Hastyshell Hastyshell commented Aug 2, 2024

Proposed changes

Fix using != to compare ColumnType as reference, introduced by #32810
Add a minimal verify case.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@Hastyshell Hastyshell force-pushed the fix-column-type-compare branch from 38e8997 to de78c20 Compare August 2, 2024 13:42
@Hastyshell Hastyshell marked this pull request as ready for review August 2, 2024 13:43
@Hastyshell Hastyshell force-pushed the fix-column-type-compare branch from de78c20 to d0aefec Compare August 2, 2024 13:44
@Hastyshell
Copy link
Collaborator Author

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 41412 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d0aefec8936d5c423cc58d2100a42e31a8721901, data reload: false

------ Round 1 ----------------------------------
q1	17693	4061	4000	4000
q2	2017	195	196	195
q3	10452	1260	1342	1260
q4	10160	797	880	797
q5	7656	2894	2940	2894
q6	219	137	137	137
q7	1050	601	612	601
q8	9446	1913	1946	1913
q9	9848	6591	6570	6570
q10	8791	3832	3844	3832
q11	422	242	255	242
q12	437	221	234	221
q13	17779	2978	2974	2974
q14	267	241	248	241
q15	528	492	498	492
q16	528	379	379	379
q17	972	914	918	914
q18	7942	7262	7192	7192
q19	1616	1228	1210	1210
q20	565	327	335	327
q21	5289	4830	4730	4730
q22	349	294	291	291
Total cold run time: 114026 ms
Total hot run time: 41412 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4141	4010	4162	4010
q2	329	229	221	221
q3	3008	3039	3164	3039
q4	2003	2011	1935	1935
q5	5598	5466	5385	5385
q6	221	131	132	131
q7	2114	1775	1856	1775
q8	3303	3361	3320	3320
q9	8619	8642	8785	8642
q10	3921	4079	3865	3865
q11	555	453	458	453
q12	743	578	595	578
q13	16652	3122	3137	3122
q14	306	268	265	265
q15	540	480	496	480
q16	466	428	416	416
q17	1764	1743	1739	1739
q18	8128	7649	7651	7649
q19	1721	1710	1704	1704
q20	2164	1822	1835	1822
q21	5674	5512	5219	5219
q22	513	467	492	467
Total cold run time: 72483 ms
Total hot run time: 56237 ms

@Hastyshell Hastyshell force-pushed the fix-column-type-compare branch from d0aefec to ab7c2fe Compare August 2, 2024 14:11
@Hastyshell
Copy link
Collaborator Author

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 41259 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ab7c2febd7827e6976444d4eb8eccdbfbb587a50, data reload: false

------ Round 1 ----------------------------------
q1	17820	4213	4028	4028
q2	2017	211	195	195
q3	10489	1227	1256	1227
q4	10167	844	921	844
q5	9094	2959	2933	2933
q6	215	137	137	137
q7	1039	607	618	607
q8	9429	1822	1905	1822
q9	8445	6537	6596	6537
q10	8703	3826	3830	3826
q11	425	246	243	243
q12	407	228	237	228
q13	17753	2908	2964	2908
q14	264	242	245	242
q15	518	481	500	481
q16	496	412	394	394
q17	956	894	893	893
q18	8200	7237	7234	7234
q19	1384	1214	1204	1204
q20	572	308	323	308
q21	5247	4683	4807	4683
q22	342	291	285	285
Total cold run time: 113982 ms
Total hot run time: 41259 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4047	4049	4041	4041
q2	325	232	217	217
q3	2983	2951	2978	2951
q4	1876	1861	1847	1847
q5	5224	5207	5196	5196
q6	213	129	130	129
q7	2052	1707	1679	1679
q8	3156	3236	3230	3230
q9	8274	8272	8217	8217
q10	3719	3825	3813	3813
q11	547	453	445	445
q12	719	560	549	549
q13	13607	2923	2940	2923
q14	290	249	253	249
q15	517	488	474	474
q16	442	394	386	386
q17	1726	1702	1662	1662
q18	7776	7394	7247	7247
q19	1672	1661	1652	1652
q20	1964	1744	1746	1744
q21	5397	5189	5102	5102
q22	526	467	447	447
Total cold run time: 67052 ms
Total hot run time: 54200 ms

@Hastyshell Hastyshell removed the doing label Aug 2, 2024
@doris-robot
Copy link

TPC-DS: Total hot run time: 168675 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ab7c2febd7827e6976444d4eb8eccdbfbb587a50, data reload: false

query1	917	383	370	370
query2	6482	1636	1623	1623
query3	6668	211	225	211
query4	20462	17394	17463	17394
query5	4303	518	526	518
query6	288	175	187	175
query7	4597	324	295	295
query8	251	211	198	198
query9	8508	2353	2351	2351
query10	434	281	261	261
query11	10666	9951	10027	9951
query12	141	86	88	86
query13	1632	392	418	392
query14	9984	6709	7594	6709
query15	203	165	162	162
query16	7072	475	465	465
query17	982	609	580	580
query18	1901	288	288	288
query19	193	151	151	151
query20	94	86	87	86
query21	210	97	102	97
query22	4168	4060	3897	3897
query23	33665	33190	32999	32999
query24	10226	2997	3063	2997
query25	695	412	407	407
query26	1778	156	156	156
query27	2877	287	287	287
query28	6917	1960	1949	1949
query29	1310	452	441	441
query30	286	153	151	151
query31	977	759	777	759
query32	108	54	58	54
query33	707	313	327	313
query34	927	485	494	485
query35	844	720	746	720
query36	976	867	882	867
query37	288	80	80	80
query38	2878	2739	2776	2739
query39	862	801	807	801
query40	289	114	116	114
query41	49	47	49	47
query42	118	108	101	101
query43	473	411	415	411
query44	1205	747	754	747
query45	208	180	180	180
query46	1083	813	778	778
query47	1803	1725	1689	1689
query48	372	305	302	302
query49	1196	440	457	440
query50	915	447	454	447
query51	6832	6594	6644	6594
query52	102	89	89	89
query53	253	187	180	180
query54	628	467	460	460
query55	78	73	82	73
query56	285	282	268	268
query57	1147	1069	1060	1060
query58	293	268	301	268
query59	2715	2355	2427	2355
query60	307	289	291	289
query61	116	112	119	112
query62	930	666	675	666
query63	221	185	183	183
query64	5993	1921	1868	1868
query65	3163	3101	3093	3093
query66	1429	353	350	350
query67	15402	14695	14743	14695
query68	6945	583	599	583
query69	765	431	320	320
query70	1129	1031	1087	1031
query71	544	276	287	276
query72	8082	2687	2548	2548
query73	956	344	341	341
query74	6018	5667	5665	5665
query75	4314	2695	2738	2695
query76	5549	1280	1286	1280
query77	769	313	324	313
query78	9483	8955	8838	8838
query79	3796	548	536	536
query80	1276	518	510	510
query81	573	225	227	225
query82	787	134	131	131
query83	344	172	173	172
query84	272	88	79	79
query85	1531	314	298	298
query86	449	300	285	285
query87	3320	3117	3099	3099
query88	3871	2512	2519	2512
query89	471	283	284	283
query90	2081	195	198	195
query91	133	103	103	103
query92	66	50	52	50
query93	5029	608	622	608
query94	1001	291	303	291
query95	395	272	267	267
query96	620	288	286	286
query97	3250	3091	3071	3071
query98	216	198	191	191
query99	1627	1269	1295	1269
Total cold run time: 279744 ms
Total hot run time: 168675 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.19 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ab7c2febd7827e6976444d4eb8eccdbfbb587a50, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.03
query3	0.22	0.05	0.05
query4	1.67	0.07	0.07
query5	0.49	0.47	0.48
query6	1.14	0.71	0.71
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.56	0.51	0.51
query10	0.57	0.55	0.57
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.63	0.61	0.60
query14	0.77	0.79	0.80
query15	0.90	0.87	0.85
query16	0.36	0.34	0.35
query17	1.00	1.00	0.99
query18	0.22	0.21	0.21
query19	1.86	1.78	1.80
query20	0.01	0.01	0.00
query21	15.41	0.74	0.65
query22	3.79	5.52	2.52
query23	17.94	1.44	1.33
query24	2.32	0.23	0.22
query25	0.17	0.08	0.08
query26	0.31	0.21	0.22
query27	0.47	0.23	0.23
query28	13.16	1.00	0.97
query29	12.61	3.35	3.31
query30	0.26	0.06	0.05
query31	2.85	0.41	0.40
query32	3.24	0.50	0.48
query33	2.93	2.88	2.98
query34	15.46	4.26	4.26
query35	4.30	4.29	4.29
query36	0.67	0.48	0.47
query37	0.18	0.17	0.16
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.15	0.13	0.12
query41	0.11	0.04	0.06
query42	0.06	0.05	0.04
query43	0.05	0.04	0.03
Total cold run time: 107.5 s
Total hot run time: 31.19 s

@Hastyshell Hastyshell force-pushed the fix-column-type-compare branch from ab7c2fe to c1e973b Compare August 2, 2024 14:53
@Hastyshell
Copy link
Collaborator Author

run buildall

@github-actions github-actions bot added the doing label Aug 2, 2024
@Hastyshell Hastyshell removed the doing label Aug 2, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 4, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 4, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 41310 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c1e973b609f5ea6af3af2b6ef2527c49b5b93d90, data reload: false

------ Round 1 ----------------------------------
q1	17613	4044	4034	4034
q2	2026	200	198	198
q3	10459	1412	1351	1351
q4	10159	823	939	823
q5	7642	3000	2923	2923
q6	223	139	137	137
q7	1056	611	602	602
q8	9426	1831	1941	1831
q9	8440	6598	6573	6573
q10	8780	3848	3809	3809
q11	427	244	237	237
q12	404	221	222	221
q13	17770	2936	2911	2911
q14	269	242	246	242
q15	514	490	475	475
q16	513	378	376	376
q17	959	921	913	913
q18	7976	7337	7181	7181
q19	1441	1207	1207	1207
q20	546	324	327	324
q21	5321	4670	4807	4670
q22	348	272	274	272
Total cold run time: 112312 ms
Total hot run time: 41310 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4061	4011	4023	4011
q2	323	225	220	220
q3	3001	3052	3097	3052
q4	1984	2007	1968	1968
q5	5589	5500	5542	5500
q6	223	134	131	131
q7	2166	1783	1853	1783
q8	3331	3385	3367	3367
q9	8801	8713	8845	8713
q10	3949	4121	3909	3909
q11	559	473	456	456
q12	770	593	604	593
q13	16813	3159	3143	3143
q14	310	284	278	278
q15	522	494	487	487
q16	481	418	434	418
q17	1749	1743	1723	1723
q18	8399	8271	8279	8271
q19	3979	1786	1777	1777
q20	2166	1913	1880	1880
q21	5732	5495	5342	5342
q22	529	466	484	466
Total cold run time: 75437 ms
Total hot run time: 57488 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169198 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c1e973b609f5ea6af3af2b6ef2527c49b5b93d90, data reload: false

query1	916	365	366	365
query2	6477	1654	1666	1654
query3	6669	212	220	212
query4	19546	17441	17058	17058
query5	3652	526	521	521
query6	269	188	160	160
query7	4595	314	293	293
query8	253	199	185	185
query9	8507	2325	2300	2300
query10	443	274	264	264
query11	10576	10102	10012	10012
query12	123	91	88	88
query13	1634	402	371	371
query14	8544	7675	6751	6751
query15	213	161	164	161
query16	7107	474	481	474
query17	1161	557	535	535
query18	2117	283	296	283
query19	191	141	140	140
query20	91	85	88	85
query21	208	105	105	105
query22	4053	4277	3927	3927
query23	34046	33873	33760	33760
query24	10277	3058	3028	3028
query25	693	421	410	410
query26	1696	166	156	156
query27	3148	288	287	287
query28	7536	1980	1965	1965
query29	1265	448	468	448
query30	243	158	163	158
query31	964	785	773	773
query32	104	61	59	59
query33	694	342	349	342
query34	921	487	507	487
query35	875	782	775	775
query36	1038	879	889	879
query37	205	86	95	86
query38	2962	2780	2807	2780
query39	856	815	813	813
query40	276	115	117	115
query41	48	46	48	46
query42	119	105	98	98
query43	472	454	437	437
query44	1177	736	747	736
query45	214	179	193	179
query46	1101	801	773	773
query47	1790	1716	1715	1715
query48	408	304	305	304
query49	961	428	442	428
query50	885	438	433	433
query51	6793	6704	6597	6597
query52	98	91	91	91
query53	254	185	183	183
query54	643	470	464	464
query55	75	79	78	78
query56	288	262	258	258
query57	1121	1050	1040	1040
query58	284	276	274	274
query59	2650	2222	2396	2222
query60	297	289	282	282
query61	104	95	100	95
query62	876	642	669	642
query63	214	191	183	183
query64	5585	1915	1880	1880
query65	3268	3107	3112	3107
query66	1262	336	328	328
query67	15300	15018	14843	14843
query68	4417	563	570	563
query69	442	303	302	302
query70	1113	1009	1087	1009
query71	426	299	294	294
query72	7095	2677	2507	2507
query73	764	342	329	329
query74	6054	5681	5653	5653
query75	3421	2726	2775	2726
query76	2654	1218	1274	1218
query77	433	314	307	307
query78	9597	8901	8897	8897
query79	2409	523	537	523
query80	1146	516	510	510
query81	569	223	229	223
query82	1078	134	128	128
query83	255	175	171	171
query84	235	85	79	79
query85	1361	315	318	315
query86	469	296	288	288
query87	3262	3083	3091	3083
query88	3111	2488	2470	2470
query89	384	288	288	288
query90	1766	195	196	195
query91	129	105	105	105
query92	63	51	51	51
query93	2007	608	614	608
query94	899	300	306	300
query95	396	275	271	271
query96	633	281	283	281
query97	3186	3075	3072	3072
query98	217	201	194	194
query99	1652	1305	1257	1257
Total cold run time: 263074 ms
Total hot run time: 169198 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.59 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c1e973b609f5ea6af3af2b6ef2527c49b5b93d90, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.04
query4	1.66	0.08	0.08
query5	0.48	0.49	0.50
query6	1.12	0.72	0.71
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.56	0.51	0.52
query10	0.58	0.58	0.57
query11	0.15	0.12	0.12
query12	0.15	0.13	0.13
query13	0.61	0.61	0.61
query14	0.78	0.81	0.79
query15	0.90	0.85	0.86
query16	0.36	0.36	0.36
query17	0.99	1.03	0.99
query18	0.23	0.22	0.21
query19	1.92	1.75	1.72
query20	0.01	0.01	0.01
query21	15.40	0.77	0.68
query22	4.41	6.50	1.74
query23	17.73	1.36	1.34
query24	2.20	0.24	0.22
query25	0.19	0.08	0.08
query26	0.31	0.21	0.21
query27	0.46	0.22	0.23
query28	13.15	0.99	0.98
query29	12.55	3.32	3.27
query30	0.25	0.06	0.05
query31	2.86	0.41	0.40
query32	3.22	0.50	0.48
query33	2.89	2.99	2.97
query34	15.42	4.29	4.28
query35	4.28	4.30	4.28
query36	0.68	0.50	0.47
query37	0.19	0.16	0.16
query38	0.16	0.15	0.16
query39	0.04	0.03	0.03
query40	0.16	0.14	0.13
query41	0.10	0.05	0.04
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 107.67 s
Total hot run time: 30.59 s

@dataroaring dataroaring merged commit 69857b0 into apache:master Aug 4, 2024
@Hastyshell
Copy link
Collaborator Author

Hastyshell commented Aug 5, 2024

2.0 and 2.1 are correct and normal

dataroaring pushed a commit that referenced this pull request Aug 11, 2024
…ied for MV (#38806)

## Proposed changes

Fix using `!=` to compare ColumnType as reference, introduced by #32810 
Add a minimal verify case.
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
…ied for MV (#38806)

## Proposed changes

Fix using `!=` to compare ColumnType as reference, introduced by #32810 
Add a minimal verify case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.2-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants