Skip to content

[Bug] JNI scanner leaks off-heap memory and fails queries when getNext throws an unchecked exception #66037

Description

@Baymine

Search before asking

  • I had searched in the issues and found no similar issues.

Version

master / 4.0

What's Wrong?

JNI scan threads are pooled and reused across queries, and a JNI exception is sticky per-thread: it stays pending until explicitly cleared. Two related problems arise:

  1. FE JniScanner.getNextBatchMeta() only catches IOException from getNext(). Scanner subclasses (and the libraries they call) routinely surface failures as unchecked RuntimeException/NullPointerException. When such an exception escapes:

    • releaseTable() is skipped, leaking the off-heap VectorTable;
    • a bare pending JNI exception is left on the pooled scan thread, later misattributed to an unrelated JNI call (e.g. getStatistics) and able to fail an otherwise healthy query.
  2. BE JniReader::_get_statistics / JniReader::close and JniTableReader::_get_statistics run JNI calls while a stale pending exception (left by an earlier call on the same pooled thread) may still be present — undefined behavior per the JNI spec. In close() the early RETURN_ERROR_IF_EXC aborts before the resource-release/close calls, leaking the Java scanner and the off-heap table.

What You Expected?

An unchecked exception from getNext() should always release the scanner table and be normalized to a checked exception, and BE best-effort statistics/close paths should not run under a stale pending exception or leak resources.

How to Reproduce?

Run a JNI-based scan (e.g. an external table via a JNI scanner) where getNext() throws an unchecked exception (a malformed record or a connector-library RuntimeException/NPE). The off-heap table is leaked and a subsequent unrelated JNI call on the reused scan thread can fail the query.

Anything Else?

Fix: FE catches Throwable, always calls releaseTable(), and normalizes non-IOException throwables to IOException; BE discards any residual pending exception up front (logging it) before touching the JVM again.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions