Skip to content

[#10253] Optimize driver degisteration logic for JDBC catalog to avoid possible OOM problem#10255

Open
yuqi1129 wants to merge 4 commits intoapache:mainfrom
yuqi1129:issue_10253
Open

[#10253] Optimize driver degisteration logic for JDBC catalog to avoid possible OOM problem#10255
yuqi1129 wants to merge 4 commits intoapache:mainfrom
yuqi1129:issue_10253

Conversation

@yuqi1129
Copy link
Contributor

@yuqi1129 yuqi1129 commented Mar 5, 2026

What changes were proposed in this pull request?

This pull request refactors the JDBC driver deregistration logic across multiple catalog implementations to centralize and standardize the process. The main change is moving driver deregistration from individual catalog subclasses to the base JdbcCatalogOperations class, reducing code duplication and improving maintainability. Additionally, the pull request updates related tests and adapts the OceanBase catalog to use a more appropriate operations class.

Why are the changes needed?

To avoid resource leakage

Fix: #10253

Does this PR introduce any user-facing change?

N/A

How was this patch tested?

UTs

yuqi1129 and others added 2 commits March 5, 2026 20:46
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 5, 2026 13:04
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR centralizes JDBC driver deregistration in JdbcCatalogOperations.close() to reduce per-catalog duplication and help prevent driver/classloader leaks in long-running processes.

Changes:

  • Add jdbcUrl tracking and a getDriver() hook in JdbcCatalogOperations, and deregister the driver during close().
  • Update PostgreSQL/Hologres/ClickHouse/MySQL-protocol operations to rely on the base close() and provide driver resolution via getDriver().
  • Add unit tests covering driver deregistration behavior and the “ignore getDriver failure during close” path; update OceanBase to use MySQLProtocolCompatibleCatalogOperations.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
catalogs/catalog-jdbc-common/src/main/java/org/apache/gravitino/catalog/jdbc/JdbcCatalogOperations.java Centralizes driver lookup/deregistration in base close() using stored jdbcUrl.
catalogs/catalog-jdbc-common/src/main/java/org/apache/gravitino/catalog/jdbc/MySQLProtocolCompatibleCatalogOperations.java Removes per-class driver deregistration and supplies a MySQL driver via getDriver().
catalogs/catalog-jdbc-postgresql/src/main/java/org/apache/gravitino/catalog/postgresql/PostgreSQLCatalogOperations.java Removes per-class close() override and supplies driver via getDriver().
catalogs-contrib/catalog-jdbc-hologres/src/main/java/org/apache/gravitino/catalog/hologres/HologresCatalogOperations.java Removes per-class close() override and supplies driver via getDriver() (PostgreSQL driver).
catalogs-contrib/catalog-jdbc-clickhouse/src/main/java/org/apache/gravitino/catalog/clickhouse/operations/ClickHouseCatalogOperations.java Removes per-class close() override and supplies driver via getDriver().
catalogs/catalog-jdbc-common/src/test/java/org/apache/gravitino/catalog/jdbc/TestJdbcCatalogOperations.java Adds UT coverage for base close() driver deregistration and exception tolerance.
catalogs-contrib/catalog-jdbc-oceanbase/src/main/java/org/apache/gravitino/catalog/oceanbase/OceanBaseCatalog.java Switches OceanBase to MySQL-protocol operations implementation.
Comments suppressed due to low confidence (1)

catalogs-contrib/catalog-jdbc-oceanbase/src/main/java/org/apache/gravitino/catalog/oceanbase/OceanBaseCatalog.java:50

  • OceanBaseCatalog now uses MySQLProtocolCompatibleCatalogOperations, which enforces a MySQL driver major version >= 8 (via checkJDBCDriverVersion()) and runs MySQL-specific cleanup on close(). Please confirm OceanBase is intended to require MySQL Connector/J 8+; otherwise this can break deployments using older MySQL-compatible drivers (note jdbc-oceanbase.conf still suggests com.mysql.jdbc.Driver). If older drivers must remain supported, consider keeping JdbcCatalogOperations or overriding checkJDBCDriverVersion() for OceanBase.
  protected CatalogOperations newOps(Map<String, String> config) {
    return new MySQLProtocolCompatibleCatalogOperations(
        createExceptionConverter(),
        createJdbcTypeConverter(),
        createJdbcDatabaseOperations(),
        createJdbcTableOperations(),
        createJdbcColumnDefaultValueConverter());

@github-actions
Copy link

github-actions bot commented Mar 5, 2026

Code Coverage Report

Overall Project 64.81% -0.2% 🟢
Files changed 7.69% 🔴

Module Coverage
aliyun 1.73% 🔴
api 46.15% 🟢
authorization-common 85.96% 🟢
aws 1.1% 🔴
azure 2.6% 🔴
catalog-common 10.0% 🔴
catalog-fileset 80.02% 🟢
catalog-hive 80.98% 🟢
catalog-jdbc-clickhouse 78.25% -0.21% 🟢
catalog-jdbc-common 37.34% -9.81% 🔴
catalog-jdbc-doris 80.28% 🟢
catalog-jdbc-hologres 59.28% -0.82% 🟢
catalog-jdbc-mysql 79.23% 🟢
catalog-jdbc-oceanbase 78.38% -2.11% 🟢
catalog-jdbc-postgresql 82.05% -0.42% 🟢
catalog-jdbc-starrocks 78.27% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 45.07% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 87.15% 🟢
catalog-lakehouse-paimon 77.71% 🟢
catalog-model 77.72% 🟢
cli 44.51% 🟢
client-java 77.83% 🟢
common 49.23% 🟢
core 81.21% 🟢
filesystem-hadoop3 76.97% 🟢
flink 38.86% 🔴
flink-runtime 0.0% 🔴
gcp 14.2% 🔴
hadoop-common 10.39% 🔴
hive-metastore-common 45.82% 🟢
iceberg-common 50.21% 🟢
iceberg-rest-server 66.24% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 23.78% 🔴
lance-rest-server 57.84% 🟢
lineage 53.02% 🟢
optimizer 82.47% 🟢
optimizer-api 21.95% 🔴
server 85.74% 🟢
server-common 68.6% 🟢
spark 32.79% 🔴
spark-common 39.6% 🔴
trino-connector 31.62% 🔴
Files
Module File Coverage
catalog-jdbc-clickhouse ClickHouseCatalogOperations.java 0.0% 🔴
catalog-jdbc-common JdbcCatalogOperations.java 9.13% 🔴
MySQLProtocolCompatibleCatalogOperations.java 0.0% 🔴
catalog-jdbc-hologres HologresCatalogOperations.java 0.0% 🔴
catalog-jdbc-oceanbase OceanBaseCatalog.java 0.0% 🔴
catalog-jdbc-postgresql PostgreSQLCatalogOperations.java 0.0% 🔴

Copy link
Contributor

@jerryshao jerryshao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Excellent refactor. Centralizing driver deregistration in JdbcCatalogOperations.close() via the getDriver() hook eliminates the duplicated override pattern across all subclasses.

Positive

  • Fixes System.err.println in ClickHouseCatalogOperations (now properly logged via LOG.warn in the base class).
  • Fixes the typo "dumpy_address""dummy_address" in MySQLProtocolCompatibleCatalogOperations.
  • Correctly moves OceanBaseCatalog to use MySQLProtocolCompatibleCatalogOperations (MySQL-protocol-compatible).
  • The TestableJdbcCatalogOperations test pattern is clean and covers both the success and exception-swallowing paths.

Minor suggestion

MySQLProtocolCompatibleCatalogOperations.close() still contains the AbandonedConnectionCleanupThread.uncheckedShutdown() logic in a try/catch block — this is now separate from the driver deregistration (which moved to the base close()). This is fine architecturally, but a brief comment that the base close() handles driver deregistration would make the layering clearer for future readers.

LGTM. The minor suggestion above is optional.

@yuqi1129 yuqi1129 changed the title [#10253] Optimize driver degisteration logic for JDBC catalog [#10253] Optimize driver degisteration logic for JDBC catalog to avoid possible OOM problem Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] Optimize driver degisteration logic for all JDBC catalogs

3 participants