Skip to content

[fix](jdbc catalog) Fix the wrong method used in loadColumnNames of JdbcIdentifierMapping#32168

Merged
zy-kkk merged 1 commit intoapache:masterfrom
zy-kkk:fix_jdbc_col_npe
Mar 13, 2024
Merged

[fix](jdbc catalog) Fix the wrong method used in loadColumnNames of JdbcIdentifierMapping#32168
zy-kkk merged 1 commit intoapache:masterfrom
zy-kkk:fix_jdbc_col_npe

Conversation

@zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented Mar 13, 2024

Proposed changes

Issue Number: close #xxx

In the loadColumnNames method of JdbcIdentifierMapping, a method with the functionality to populate the localColumnToRemoteColumn map must be called

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented Mar 13, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@zy-kkk
Copy link
Member Author

zy-kkk commented Mar 13, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Mar 13, 2024

TPC-H: Total hot run time: 36860 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 57e972fc4a926532cc5749433f3319b7da42f3ef, data reload: false

------ Round 1 ----------------------------------
q1	17862	6226	4222	4222
q2	2632	159	144	144
q3	11697	940	935	935
q4	5154	774	728	728
q5	7805	2837	2843	2837
q6	186	125	128	125
q7	1227	857	845	845
q8	9719	2037	2045	2037
q9	7865	6464	6425	6425
q10	8103	2489	2510	2489
q11	432	224	220	220
q12	772	304	290	290
q13	17980	2997	2968	2968
q14	287	249	262	249
q15	504	455	449	449
q16	487	389	404	389
q17	952	540	585	540
q18	6707	5889	5826	5826
q19	1562	1402	1344	1344
q20	531	280	297	280
q21	7325	3305	3220	3220
q22	800	298	298	298
Total cold run time: 110589 ms
Total hot run time: 36860 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4073	4124	4072	4072
q2	321	223	236	223
q3	2993	2811	2847	2811
q4	1865	1566	1563	1563
q5	5179	5221	5205	5205
q6	208	115	122	115
q7	2247	1894	1846	1846
q8	3151	3263	3275	3263
q9	8509	8518	8528	8518
q10	6186	3641	3633	3633
q11	527	450	444	444
q12	728	542	567	542
q13	9271	2899	2886	2886
q14	274	262	259	259
q15	478	456	434	434
q16	447	402	406	402
q17	1722	1462	1456	1456
q18	7706	7301	7042	7042
q19	5764	1518	1531	1518
q20	1944	1708	1733	1708
q21	4890	4767	4786	4767
q22	552	486	461	461
Total cold run time: 69035 ms
Total hot run time: 53168 ms

@doris-robot
Copy link

doris-robot commented Mar 13, 2024

TPC-DS: Total hot run time: 186368 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 57e972fc4a926532cc5749433f3319b7da42f3ef, data reload: false

query1	919	368	345	345
query2	7391	1923	1956	1923
query3	6710	216	211	211
query4	31000	20620	20613	20613
query5	4338	400	394	394
query6	260	182	180	180
query7	4627	291	285	285
query8	232	181	172	172
query9	9944	2287	2276	2276
query10	455	229	256	229
query11	14575	14085	14055	14055
query12	131	85	89	85
query13	1627	427	438	427
query14	10507	10233	10482	10233
query15	259	213	201	201
query16	7958	268	262	262
query17	1942	581	549	549
query18	2049	291	297	291
query19	236	156	157	156
query20	92	91	86	86
query21	214	131	126	126
query22	4679	4496	4401	4401
query23	32004	31081	31232	31081
query24	11904	2806	2793	2793
query25	667	385	375	375
query26	1881	154	156	154
query27	3286	361	359	359
query28	8211	1884	1856	1856
query29	1076	663	619	619
query30	332	152	149	149
query31	966	730	714	714
query32	95	59	62	59
query33	782	258	255	255
query34	1198	474	491	474
query35	841	591	602	591
query36	955	889	857	857
query37	280	74	74	74
query38	3351	3142	3154	3142
query39	1387	1348	1380	1348
query40	305	107	112	107
query41	48	47	48	47
query42	101	96	97	96
query43	484	453	454	453
query44	1199	705	708	705
query45	265	265	252	252
query46	1055	695	674	674
query47	1663	1585	1599	1585
query48	432	358	349	349
query49	1234	327	340	327
query50	765	365	408	365
query51	6906	6746	6742	6742
query52	108	86	91	86
query53	338	271	275	271
query54	310	223	214	214
query55	83	79	77	77
query56	235	223	218	218
query57	1098	1005	1008	1005
query58	217	198	201	198
query59	2781	2509	2561	2509
query60	262	238	296	238
query61	96	91	91	91
query62	583	388	393	388
query63	294	266	270	266
query64	6716	3792	3804	3792
query65	3111	2980	3033	2980
query66	1374	311	313	311
query67	14926	14366	14263	14263
query68	6070	532	523	523
query69	600	374	383	374
query70	1289	1130	1189	1130
query71	506	275	277	275
query72	6459	2705	2646	2646
query73	708	322	316	316
query74	7665	6599	6623	6599
query75	9229	8003	8145	8003
query76	4248	910	937	910
query77	610	266	256	256
query78	10348	9546	9525	9525
query79	8333	516	530	516
query80	1882	410	397	397
query81	550	212	215	212
query82	1475	211	209	209
query83	320	150	155	150
query84	294	82	82	82
query85	1698	390	337	337
query86	485	291	299	291
query87	3432	3227	3185	3185
query88	5030	2308	2286	2286
query89	531	361	354	354
query90	1977	181	173	173
query91	173	130	127	127
query92	63	46	46	46
query93	6794	520	488	488
query94	1206	174	175	174
query95	416	319	320	319
query96	595	273	275	273
query97	4412	4246	4288	4246
query98	231	210	197	197
query99	1205	740	772	740
Total cold run time: 315184 ms
Total hot run time: 186368 ms

@doris-robot
Copy link

doris-robot commented Mar 13, 2024

ClickBench: Total hot run time: 29.77 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 57e972fc4a926532cc5749433f3319b7da42f3ef, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.22	0.06	0.06
query4	1.68	0.10	0.10
query5	0.50	0.51	0.50
query6	1.19	0.64	0.63
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.51	0.50
query10	0.54	0.56	0.55
query11	0.12	0.09	0.10
query12	0.11	0.09	0.10
query13	0.57	0.58	0.55
query14	0.72	0.74	0.76
query15	0.79	0.78	0.78
query16	0.40	0.40	0.37
query17	0.99	0.97	0.97
query18	0.20	0.23	0.22
query19	1.79	1.74	1.75
query20	0.01	0.01	0.01
query21	15.41	0.55	0.57
query22	2.84	1.82	1.52
query23	17.15	0.86	0.78
query24	3.19	0.68	1.29
query25	0.23	0.18	0.10
query26	0.42	0.11	0.12
query27	0.03	0.04	0.03
query28	10.94	0.89	0.84
query29	12.53	3.17	3.14
query30	0.72	0.62	0.63
query31	2.80	0.33	0.33
query32	3.41	0.43	0.44
query33	2.81	2.84	2.79
query34	16.09	4.30	4.26
query35	4.34	4.31	4.30
query36	1.08	0.99	1.01
query37	0.07	0.06	0.05
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.14	0.14
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 104.99 s
Total hot run time: 29.77 s

@doris-robot
Copy link

doris-robot commented Mar 13, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 57e972fc4a926532cc5749433f3319b7da42f3ef with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.8 seconds inserted 10000000 Rows, about 724K ops/s

morningman
morningman approved these changes Mar 13, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Mar 13, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 13, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Mar 13, 2024

PR approved by anyone and no changes requested.

@zy-kkk zy-kkk merged commit 7466023 into apache:master Mar 13, 2024
@zy-kkk zy-kkk deleted the fix_jdbc_col_npe branch March 13, 2024 13:26
yiguolei pushed a commit that referenced this pull request Mar 15, 2024
@zy-kkk @Doris-Extras
[fix](jdbc catalog) Fix the wrong method used in loadColumnNames of J…
141189d 
…dbcIdentifierMapping (#32168)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@JNSimba JNSimba JNSimba approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@zy-kkk @doris-robot @morningman @JNSimba