Skip to content

branch-3.0: [fix](jdbc catalog) Optimize JDBC Connection Closing to Ensure ProperResource Release#43105

Closed
github-actions[bot] wants to merge 1 commit intobranch-3.0from
auto-pick-43059-branch-3.0
Closed

branch-3.0: [fix](jdbc catalog) Optimize JDBC Connection Closing to Ensure ProperResource Release#43105
github-actions[bot] wants to merge 1 commit intobranch-3.0from
auto-pick-43059-branch-3.0

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Nov 1, 2024

PR Body: This pull request optimizes the JDBC connection and resource management in JdbcClient and related methods. The main changes are as follows:

  • Manual Resource Closing: Explicitly place resource acquisition and closing within a try-catch-finally structure. In the finally block, manually close resources like Connection, Statement, and ResultSet to ensure that resources are properly released even if an exception occurs.

  • Remove try-with-resources: All usages of try-with-resources have been removed, switching to manual resource management instead.

  • Improve Exception Handling: When closing resources, if an exception occurs, log a warning but do not throw a new exception. This avoids masking the original exception during the resource release phase.

  • Unified Resource Management: Ensure consistent resource acquisition and closing methods across all functions, improving code readability and maintainability.

Functions modified include but are not limited to:

  • executeStmt
  • getColumnsFromQuery
  • getDatabaseNameList
  • getJdbcColumnsInfo
  • processTable
  • getColumnsDataTypeUseQuery

These changes enhance the robustness of the code, prevent potential resource leaks, and ensure that all JDBC connections and resources are properly closed without using try-with-resources.
Cherry-picked from #43059

…Resource Release (#43059)

This pull request optimizes the JDBC connection and resource management
in `JdbcClient` and related methods. The main changes are as follows:

- **Manual Resource Closing**: Explicitly place resource acquisition and
closing within a `try-catch-finally` structure. In the `finally` block,
manually close resources like `Connection`, `Statement`, and `ResultSet`
to ensure that resources are properly released even if an exception
occurs.

- **Remove try-with-resources**: All usages of `try-with-resources` have
been removed, switching to manual resource management instead.

- **Improve Exception Handling**: When closing resources, if an
exception occurs, log a warning but do not throw a new exception. This
avoids masking the original exception during the resource release phase.

- **Unified Resource Management**: Ensure consistent resource
acquisition and closing methods across all functions, improving code
readability and maintainability.

Functions modified include but are not limited to:

- `executeStmt`
- `getColumnsFromQuery`
- `getDatabaseNameList`
- `getJdbcColumnsInfo`
- `processTable`
- `getColumnsDataTypeUseQuery`

These changes enhance the robustness of the code, prevent potential
resource leaks, and ensure that all JDBC connections and resources are
properly closed without using `try-with-resources`.
@github-actions
Copy link
Contributor Author

github-actions bot commented Nov 1, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40131 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ac677ec3378c333be44822e8bd27a7d911d1c23b, data reload: false

------ Round 1 ----------------------------------
q1	17573	7328	7254	7254
q2	2060	158	155	155
q3	10717	1047	1147	1047
q4	10548	734	735	734
q5	7741	2763	2684	2684
q6	237	144	141	141
q7	975	624	593	593
q8	9570	1929	1946	1929
q9	7799	6383	6408	6383
q10	6968	2266	2300	2266
q11	453	252	254	252
q12	402	209	202	202
q13	17782	2954	2927	2927
q14	234	213	217	213
q15	568	511	511	511
q16	664	595	594	594
q17	965	551	571	551
q18	7222	6490	6493	6490
q19	1586	1082	1015	1015
q20	474	191	188	188
q21	3853	3139	3025	3025
q22	1116	1014	977	977
Total cold run time: 109507 ms
Total hot run time: 40131 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7311	7215	7187	7187
q2	328	232	229	229
q3	2847	2835	2884	2835
q4	1963	1759	1786	1759
q5	5637	5658	5698	5658
q6	222	141	136	136
q7	2201	1724	1762	1724
q8	3300	3457	3510	3457
q9	8697	8800	8722	8722
q10	3526	3480	3475	3475
q11	588	516	498	498
q12	771	584	624	584
q13	16465	3138	3194	3138
q14	300	273	264	264
q15	566	555	512	512
q16	727	662	658	658
q17	1818	1600	1557	1557
q18	8240	7674	7399	7399
q19	7458	1546	1514	1514
q20	2098	1836	1774	1774
q21	5272	5177	5298	5177
q22	1181	1029	1006	1006
Total cold run time: 81516 ms
Total hot run time: 59263 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193043 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ac677ec3378c333be44822e8bd27a7d911d1c23b, data reload: false

query1	1236	931	900	900
query2	6229	2059	2000	2000
query3	10907	4015	3752	3752
query4	67578	26624	23505	23505
query5	5494	459	432	432
query6	441	165	162	162
query7	6202	314	306	306
query8	327	215	221	215
query9	9398	2627	2635	2627
query10	470	253	244	244
query11	17890	15207	15711	15207
query12	161	100	101	100
query13	1596	411	404	404
query14	10909	7520	6961	6961
query15	216	179	185	179
query16	6979	502	471	471
query17	1062	560	555	555
query18	1806	303	299	299
query19	195	151	151	151
query20	116	108	103	103
query21	203	100	110	100
query22	4494	4172	4293	4172
query23	34220	33434	33715	33434
query24	5742	2841	2798	2798
query25	502	400	405	400
query26	662	169	161	161
query27	1689	286	296	286
query28	4180	2512	2494	2494
query29	675	427	428	427
query30	233	162	157	157
query31	975	792	809	792
query32	66	58	62	58
query33	451	273	268	268
query34	891	488	487	487
query35	825	701	717	701
query36	1069	926	936	926
query37	111	76	74	74
query38	3952	3842	3893	3842
query39	1506	1442	1432	1432
query40	199	99	97	97
query41	51	49	50	49
query42	115	99	92	92
query43	509	484	483	483
query44	1113	774	774	774
query45	182	163	170	163
query46	1116	704	721	704
query47	1866	1795	1797	1795
query48	458	366	373	366
query49	749	397	393	393
query50	800	411	404	404
query51	7288	7058	7176	7058
query52	109	90	93	90
query53	255	180	178	178
query54	559	458	446	446
query55	75	74	74	74
query56	258	260	247	247
query57	1192	1099	1092	1092
query58	213	201	208	201
query59	3139	2942	2923	2923
query60	285	258	254	254
query61	123	118	142	118
query62	799	667	669	667
query63	207	180	180	180
query64	1432	616	589	589
query65	3256	3153	3139	3139
query66	709	288	308	288
query67	15622	15146	15300	15146
query68	4770	541	547	541
query69	409	251	250	250
query70	1106	1076	1145	1076
query71	415	258	250	250
query72	6350	3832	3873	3832
query73	750	337	332	332
query74	10147	8865	8855	8855
query75	3334	2595	2596	2595
query76	2119	1008	925	925
query77	500	257	255	255
query78	10782	9572	9395	9395
query79	11251	583	572	572
query80	2400	403	398	398
query81	569	235	239	235
query82	1181	115	114	114
query83	304	134	135	134
query84	285	74	75	74
query85	1950	285	272	272
query86	489	299	303	299
query87	4463	4128	4255	4128
query88	5363	2353	2372	2353
query89	547	293	278	278
query90	2077	180	174	174
query91	169	140	135	135
query92	58	45	45	45
query93	7215	527	529	527
query94	904	294	290	290
query95	333	252	240	240
query96	630	279	278	278
query97	3360	3083	3140	3083
query98	219	193	193	193
query99	1640	1323	1336	1323
Total cold run time: 340426 ms
Total hot run time: 193043 ms

@dataroaring dataroaring closed this Nov 7, 2024
@dataroaring dataroaring deleted the auto-pick-43059-branch-3.0 branch December 27, 2024 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants