Skip to content

[fix](arrow-flight-sql) Close temporary VectorSchemaRoot in createPreparedStatement to fix FE direct memory leak#65311

Open
GJ100 wants to merge 1 commit into
apache:masterfrom
GJ100:fix-arrow-flight-createpreparedstatement-memleak
Open

[fix](arrow-flight-sql) Close temporary VectorSchemaRoot in createPreparedStatement to fix FE direct memory leak#65311
GJ100 wants to merge 1 commit into
apache:masterfrom
GJ100:fix-arrow-flight-createpreparedstatement-memleak

Conversation

@GJ100

@GJ100 GJ100 commented Jul 7, 2026

Copy link
Copy Markdown
### What problem does this PR solve?

Issue Number: close #65305

Problem Summary:

`DorisFlightSqlProducer#createPreparedStatement` creates two `VectorSchemaRoot`
instances (an empty one for the parameter schema, and one from
`FlightSqlChannel#createOneOneSchemaRoot(...)` for the result metadata),
extracts only their `Schema`, and never closes them.

`createOneOneSchemaRoot` allocates a `VarCharVector` from the channel
`RootAllocator` (off-heap, Netty pooled direct buffer). Since the returned
`VectorSchemaRoot` is never closed, its off-heap buffer is leaked on every
prepare call (effectively every Arrow Flight query, because ADBC prepares each
statement). The buffer cannot be reclaimed by GC (strongly referenced by the
allocator) nor by closing the client session (`FlightSqlChannel#close()` only
invalidates the result cache and does not close the allocator; the leaked root
is never stored in any session map).

Under continuous Arrow Flight query load this makes FE direct memory grow
monotonically until:
`java.lang.OutOfMemoryError: Cannot reserve ... bytes of direct buffer memory (Internal; Prepare)`

This PR wraps both temporary roots in try-with-resources so their off-heap
buffers are released after the `Schema` is extracted. `Schema` is an immutable
POJO and remains valid after the root is closed. Compare with
`getCatalogs`/`getSchemas`/`getTables` in the same producer, which already use
try-with-resources.

### Release note

Fix FE Arrow Flight SQL direct memory leak in createPreparedStatement.

### Check List (For Author)

- Test
    - [ ] Regression test
    - [ ] Manual test: run many distinct Arrow Flight queries and verify FE
          direct memory / io.netty.buffer.PoolSubpage stays bounded.

…paredStatement to fix FE direct memory leak

Signed-off-by: 柳吟风 <523684989@qq.com>
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants