[bugfix](hive)use originHiveKeys for hive partitionvalue for 2.0

[wuwenchi]

Proposed changes

mirror: #32664 #32768

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[wuwenchi]

run buildall

[wuwenchi]

run buildall

[wuwenchi]

run buildall

[doris-robot]

TPC-H: Total hot run time: 49788 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dc0c14c574573d4e6cba8dff5cff70b1147548f3, data reload: false

------ Round 1 ----------------------------------
q1	17582	4394	4338	4338
q2	2050	157	149	149
q3	10355	1895	1916	1895
q4	10406	1242	1334	1242
q5	8421	4463	3944	3944
q6	221	124	122	122
q7	1999	1638	1593	1593
q8	9478	2708	2706	2706
q9	13207	10479	10441	10441
q10	8596	3548	3481	3481
q11	418	246	245	245
q12	466	298	305	298
q13	18335	3973	4055	3973
q14	351	316	333	316
q15	506	454	447	447
q16	686	586	593	586
q17	1124	956	905	905
q18	7292	6861	6862	6861
q19	1698	1567	1492	1492
q20	529	316	324	316
q21	4460	4156	4040	4040
q22	501	398	400	398
Total cold run time: 118681 ms
Total hot run time: 49788 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4292	4279	4283	4279
q2	323	222	223	222
q3	4126	4142	4129	4129
q4	2757	2752	2744	2744
q5	7220	7127	7156	7127
q6	234	119	116	116
q7	3168	2814	2733	2733
q8	4351	4389	4431	4389
q9	17018	17126	16900	16900
q10	4180	4239	4238	4238
q11	756	679	669	669
q12	1018	853	831	831
q13	6810	3759	3753	3753
q14	453	431	416	416
q15	493	446	463	446
q16	744	705	698	698
q17	3755	3764	3833	3764
q18	8727	8602	8812	8602
q19	1721	1666	1649	1649
q20	2341	2134	2093	2093
q21	8504	8408	8555	8408
q22	1030	978	962	962
Total cold run time: 84021 ms
Total hot run time: 79168 ms

[doris-robot]

TPC-DS: Total hot run time: 201202 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dc0c14c574573d4e6cba8dff5cff70b1147548f3, data reload: false

query1	926	399	383	383
query2	6540	2033	2001	2001
query3	6922	199	198	198
query4	19917	18052	18057	18052
query5	19958	6524	6466	6466
query6	277	225	224	224
query7	4190	297	301	297
query8	250	238	274	238
query9	3122	2715	2646	2646
query10	403	280	302	280
query11	12176	11519	10867	10867
query12	120	75	69	69
query13	5573	655	622	622
query14	18249	13746	13216	13216
query15	368	234	236	234
query16	6566	281	280	280
query17	3061	2443	935	935
query18	2552	417	422	417
query19	205	156	146	146
query20	86	86	83	83
query21	196	92	91	91
query22	5856	5631	5316	5316
query23	32833	31759	31994	31759
query24	6894	6424	6592	6424
query25	516	417	420	417
query26	505	156	157	156
query27	1763	296	294	294
query28	6103	2250	2230	2230
query29	3010	2585	2656	2585
query30	243	159	162	159
query31	890	737	723	723
query32	61	47	60	47
query33	393	243	243	243
query34	844	468	483	468
query35	1126	874	919	874
query36	1235	1320	1187	1187
query37	91	62	57	57
query38	3072	2924	2899	2899
query39	1361	1296	1328	1296
query40	201	97	85	85
query41	36	34	32	32
query42	78	84	83	83
query43	642	589	567	567
query44	1169	720	726	720
query45	240	227	223	223
query46	1230	967	975	967
query47	2025	1715	1715	1715
query48	974	680	657	657
query49	612	362	367	362
query50	859	613	587	587
query51	4758	4594	4688	4594
query52	87	74	89	74
query53	435	313	317	313
query54	2641	2454	2487	2454
query55	85	82	81	81
query56	210	207	193	193
query57	1145	1128	1103	1103
query58	220	204	188	188
query59	3420	3304	3285	3285
query60	204	177	190	177
query61	80	79	82	79
query62	840	503	464	464
query63	469	335	344	335
query64	2327	1472	1270	1270
query65	3617	3536	3548	3536
query66	747	362	363	362
query67	15947	17536	15226	15226
query68	9165	646	687	646
query69	565	334	329	329
query70	1747	1574	1365	1365
query71	414	297	299	297
query72	6482	3430	3421	3421
query73	727	319	322	319
query74	6262	5834	5815	5815
query75	5348	3774	3701	3701
query76	5603	1160	1202	1160
query77	944	249	250	249
query78	12514	11754	11840	11754
query79	7245	621	643	621
query80	1002	394	382	382
query81	492	228	226	226
query82	1534	101	98	98
query83	163	135	127	127
query84	270	68	70	68
query85	845	276	271	271
query86	343	317	292	292
query87	3232	3019	2998	2998
query88	5104	2349	2346	2346
query89	354	288	286	286
query90	1991	204	188	188
query91	147	124	114	114
query92	52	51	47	47
query93	5120	560	587	560
query94	694	208	206	206
query95	1103	1042	1039	1039
query96	646	332	334	332
query97	6475	6327	6369	6327
query98	188	172	166	166
query99	2953	909	841	841
Total cold run time: 313475 ms
Total hot run time: 201202 ms

[doris-robot]

ClickBench: Total hot run time: 30.97 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dc0c14c574573d4e6cba8dff5cff70b1147548f3, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.02
query3	0.25	0.05	0.04
query4	1.81	0.07	0.06
query5	0.54	0.52	0.53
query6	1.23	0.61	0.60
query7	0.01	0.00	0.01
query8	0.04	0.03	0.02
query9	0.52	0.46	0.46
query10	0.54	0.53	0.54
query11	0.11	0.08	0.08
query12	0.12	0.10	0.09
query13	0.62	0.61	0.61
query14	0.79	0.78	0.77
query15	0.76	0.74	0.77
query16	0.37	0.39	0.36
query17	0.96	0.99	0.99
query18	0.24	0.24	0.25
query19	1.94	1.78	1.83
query20	0.01	0.01	0.00
query21	15.46	0.53	0.53
query22	2.06	2.20	1.79
query23	17.15	0.88	0.95
query24	4.49	1.77	1.28
query25	0.39	0.10	0.04
query26	0.60	0.16	0.16
query27	0.04	0.04	0.04
query28	7.81	0.77	0.72
query29	12.68	2.29	2.32
query30	0.58	0.54	0.52
query31	2.80	0.38	0.37
query32	3.40	0.49	0.50
query33	3.09	3.04	3.09
query34	15.26	4.81	4.79
query35	4.86	4.84	4.86
query36	1.04	1.02	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.01
Total cold run time: 103.03 s
Total hot run time: 30.97 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit dc0c14c574573d4e6cba8dff5cff70b1147548f3 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.6 seconds inserted 10000000 Rows, about 485K ops/s

[wuwenchi]

run buildall

[doris-robot]

TPC-H: Total hot run time: 49822 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aed47293f57daee3a1b71f6a7d3d13414963e421, data reload: false

------ Round 1 ----------------------------------
q1	18180	4426	4310	4310
q2	2021	147	139	139
q3	10363	1914	1944	1914
q4	10107	1250	1267	1250
q5	8475	3931	3951	3931
q6	231	122	123	122
q7	2056	1614	1594	1594
q8	9327	2705	2743	2705
q9	10873	10609	10418	10418
q10	8634	3527	3538	3527
q11	432	244	249	244
q12	467	293	307	293
q13	18360	3984	4041	3984
q14	343	317	318	317
q15	508	461	455	455
q16	688	595	593	593
q17	1123	951	960	951
q18	7257	6840	6912	6840
q19	1688	1548	1473	1473
q20	541	297	297	297
q21	4492	4129	4079	4079
q22	505	388	386	386
Total cold run time: 116671 ms
Total hot run time: 49822 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4323	4291	4302	4291
q2	321	227	217	217
q3	4173	4144	4171	4144
q4	2736	2734	2727	2727
q5	7310	7211	7176	7176
q6	233	118	116	116
q7	3268	2817	2842	2817
q8	4370	4484	4473	4473
q9	17071	16948	16894	16894
q10	4186	4224	4295	4224
q11	793	675	690	675
q12	1022	845	870	845
q13	6892	3740	3756	3740
q14	458	429	429	429
q15	502	447	446	446
q16	754	707	707	707
q17	3836	3843	3900	3843
q18	8835	8787	8874	8787
q19	1710	1705	1622	1622
q20	2395	2126	2086	2086
q21	8566	8436	8434	8434
q22	1032	996	919	919
Total cold run time: 84786 ms
Total hot run time: 79612 ms

[doris-robot]

TPC-DS: Total hot run time: 200778 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aed47293f57daee3a1b71f6a7d3d13414963e421, data reload: false

query1	908	387	379	379
query2	6531	2231	2268	2231
query3	6914	199	194	194
query4	20673	17907	18100	17907
query5	19724	6456	6534	6456
query6	284	210	227	210
query7	4152	309	300	300
query8	254	266	249	249
query9	3103	2697	2610	2610
query10	408	296	301	296
query11	11425	10674	10731	10674
query12	118	78	71	71
query13	5572	638	619	619
query14	18242	13776	13171	13171
query15	367	233	231	231
query16	6435	274	254	254
query17	1731	1443	872	872
query18	2315	397	413	397
query19	214	143	156	143
query20	76	76	70	70
query21	189	93	92	92
query22	5205	4978	5027	4978
query23	32435	32053	31769	31769
query24	6919	6580	6515	6515
query25	519	421	413	413
query26	524	158	155	155
query27	1903	295	289	289
query28	6030	2284	2246	2246
query29	2890	2696	2788	2696
query30	241	161	167	161
query31	920	746	728	728
query32	68	61	56	56
query33	382	260	255	255
query34	853	471	480	471
query35	1113	918	912	912
query36	1304	1012	1159	1012
query37	91	64	59	59
query38	3092	2958	2939	2939
query39	1388	1327	1339	1327
query40	195	93	90	90
query41	34	33	31	31
query42	88	91	89	89
query43	609	540	622	540
query44	1163	709	722	709
query45	244	224	225	224
query46	1247	962	972	962
query47	1982	1648	1714	1648
query48	982	682	658	658
query49	632	356	377	356
query50	860	614	602	602
query51	4716	4688	4668	4668
query52	90	72	77	72
query53	442	312	320	312
query54	2612	2625	2438	2438
query55	99	77	82	77
query56	194	213	199	199
query57	1128	1200	1116	1116
query58	203	204	187	187
query59	3421	3257	3253	3253
query60	222	184	184	184
query61	90	83	87	83
query62	879	553	496	496
query63	479	329	331	329
query64	2504	1470	1457	1457
query65	3612	3562	3522	3522
query66	742	367	357	357
query67	15663	16050	15049	15049
query68	8630	672	691	672
query69	596	334	328	328
query70	1605	1386	1366	1366
query71	414	303	301	301
query72	6492	3479	3449	3449
query73	726	318	314	314
query74	6248	5873	5877	5873
query75	5309	3689	3781	3689
query76	5269	1168	1215	1168
query77	884	245	240	240
query78	12359	11614	11764	11614
query79	7608	623	644	623
query80	1040	399	398	398
query81	488	229	236	229
query82	1405	91	98	91
query83	170	136	133	133
query84	258	70	67	67
query85	866	302	275	275
query86	326	292	312	292
query87	3239	3074	3019	3019
query88	5115	2360	2370	2360
query89	357	296	292	292
query90	1919	201	205	201
query91	155	125	113	113
query92	55	51	48	48
query93	4615	579	574	574
query94	717	206	204	204
query95	1105	1078	1056	1056
query96	651	340	328	328
query97	6460	6385	6292	6292
query98	190	173	180	173
query99	2964	881	881	881
Total cold run time: 308879 ms
Total hot run time: 200778 ms

[doris-robot]

ClickBench: Total hot run time: 30.45 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aed47293f57daee3a1b71f6a7d3d13414963e421, data reload: false

query1	0.02	0.02	0.02
query2	0.08	0.02	0.02
query3	0.24	0.04	0.05
query4	1.80	0.07	0.06
query5	0.52	0.53	0.51
query6	1.23	0.61	0.61
query7	0.01	0.01	0.01
query8	0.03	0.03	0.02
query9	0.52	0.49	0.48
query10	0.54	0.54	0.55
query11	0.12	0.09	0.09
query12	0.12	0.10	0.10
query13	0.61	0.62	0.62
query14	0.78	0.80	0.79
query15	0.77	0.76	0.77
query16	0.36	0.36	0.37
query17	1.01	1.01	0.97
query18	0.23	0.25	0.26
query19	1.93	1.87	1.88
query20	0.01	0.01	0.01
query21	15.46	0.58	0.56
query22	1.95	1.81	1.51
query23	16.91	0.99	0.94
query24	5.93	0.77	0.91
query25	0.38	0.10	0.04
query26	0.60	0.15	0.17
query27	0.04	0.03	0.03
query28	7.54	0.73	0.72
query29	12.64	2.22	2.24
query30	0.57	0.52	0.54
query31	2.81	0.39	0.38
query32	3.38	0.50	0.49
query33	3.08	3.08	3.09
query34	15.28	4.81	4.79
query35	4.86	4.85	4.85
query36	1.06	1.03	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.17	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.02	0.02
query43	0.02	0.02	0.01
Total cold run time: 103.8 s
Total hot run time: 30.45 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit aed47293f57daee3a1b71f6a7d3d13414963e421 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s