Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java] Add microbenchmark code to Java #19815

Open
asfimport opened this issue Oct 11, 2018 · 3 comments
Open

[Java] Add microbenchmark code to Java #19815

asfimport opened this issue Oct 11, 2018 · 3 comments

Comments

@asfimport
Copy link

asfimport commented Oct 11, 2018

@animeshtrivedi has done some microbenchmarking with the Java API. Let's consider adding them to the codebase.

Reporter: Li Jin / @icexelloss

Related issues:

Note: This issue was originally created as ARROW-3496. Please see the migration documentation for further details.

@asfimport
Copy link
Author

Animesh Trivedi / @animeshtrivedi:
To give you an idea what I have so far (https://github.com/animeshtrivedi/benchmarking-arrow).) (its README is outdated). A standalone java program to : 

i) basic data generation template to generate data for integers, longs, binary column types (we can extend to include any arbitrary types and schema) 

ii) In-memory data buffers to hold the generated data in the memory (either on on or off heap buffers).  

iii) readers to consume the generated data using various APIs (calling get*(), or the holder API variant, or just writing your own readers from the direct byte buffers). 

The whole benchmark is multi-threaded and all 3 steps can be done in parallel. It is the last step usually what is benchmarked. Obviously the current code base has a whole lot more code for my own testing and understanding, but we can clean it up gradually. 

Where do we want to have this code? and how should a user run this? May be part of the default build process where benchmark is compiled as a separate jar (arrow-java-benchmarks-0.12.jar, something like this) 

@asfimport
Copy link
Author

Wes McKinney / @wesm:
Since we're planning to start a benchmark database, it would be good timing to start writing some Java benchmarks. Is anyone interested in this?

@asfimport
Copy link
Author

Todd Farmer / @toddfarmer:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant