Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java] Provide a utility to improve the performance of vector loading/unloading #23476

Closed
asfimport opened this issue Nov 15, 2019 · 0 comments
Closed

Comments

@asfimport
Copy link

Vector loading/unloading transforms a set of vectors to and from a set of buffers with meta data. It is heavily used in flight/IPC.

In the loading/unloading operations, only the number of type buffers are really needed. However, the current code logic gets a copy of the type buffers, which is not necessary.

In this issue, we provide a utility to get the number of type buffers, given an arrow type. It improves the performance because it removes the following overhead:

  1. creating type buffer objects unnecessarily.
  2. creating a list and copying list contents (in TypeLayout#getBufferTypes) for vector unloading.

Reporter: Liya Fan / @liyafan82
Assignee: Liya Fan / @liyafan82

PRs and other links:

Note: This issue was originally created as ARROW-7177. Please see the migration documentation for further details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants