Skip to content

[VL] Reuse byte[] buffers in shuffle read and broadcast serialization paths#11777

Merged
zhztheplayer merged 1 commit intoapache:mainfrom
acvictor:acvictor/reuseBuffers
Mar 20, 2026
Merged

[VL] Reuse byte[] buffers in shuffle read and broadcast serialization paths#11777
zhztheplayer merged 1 commit intoapache:mainfrom
acvictor:acvictor/reuseBuffers

Conversation

@acvictor
Copy link
Contributor

@acvictor acvictor commented Mar 17, 2026

What changes are proposed in this pull request?

This PR makes changes to reuse temporary byte[] buffers instead of allocating new ones on every call.

  • OnHeapJniByteInputStream.read(): Cache the buffer as a field; only reallocate when a larger size is requested
  • UnsafeByteArray: Share a single lazily-initialized transient byte[] field across all methods

How was this patch tested?

Existing UTs

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the VELOX label Mar 17, 2026
@acvictor acvictor marked this pull request as ready for review March 17, 2026 17:07
@acvictor
Copy link
Contributor Author

@zhztheplayer can you please review this change? Thanks!

Copy link
Member

@zhztheplayer zhztheplayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Do you have any benchmark number to share here?

@zhztheplayer zhztheplayer changed the title [GLUTEN-CORE] Reuse byte[] buffers in shuffle read and broadcast serialization paths [VL] Reuse byte[] buffers in shuffle read and broadcast serialization paths Mar 20, 2026
@zhztheplayer zhztheplayer merged commit 10d759f into apache:main Mar 20, 2026
64 of 65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants