-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-12640][SQL] Add simple benchmarking utility class and add Parquet scan benchmarks. #10589
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
move this into util package?
|
Test build #48738 has finished for PR 10589 at commit
|
|
@nongli You don't need to repeat the title in the description of the PR (or it get duplicated in commit message). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be SQL with ParquetReader
…uet scan benchmarks. We've run benchmarks ad hoc to measure the scanner performance. We will continue to invest in this and it makes sense to get these benchmarks into code. This adds a simple benchmarking utility to do this.
|
Test build #48876 has finished for PR 10589 at commit
|
|
Test build #2341 has finished for PR 10589 at commit
|
|
Test build #48885 has finished for PR 10589 at commit
|
|
I've merged this. Thanks! |
[SPARK-12640][SQL] Add simple benchmarking utility class and add Parquet scan benchmarks.
We've run benchmarks ad hoc to measure the scanner performance. We will continue to invest in this
and it makes sense to get these benchmarks into code. This adds a simple benchmarking utility to do
this.