Skip to content

Conversation

@nongli
Copy link
Contributor

@nongli nongli commented Jan 5, 2016

[SPARK-12640][SQL] Add simple benchmarking utility class and add Parquet scan benchmarks.

We've run benchmarks ad hoc to measure the scanner performance. We will continue to invest in this
and it makes sense to get these benchmarks into code. This adds a simple benchmarking utility to do
this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move this into util package?

@SparkQA
Copy link

SparkQA commented Jan 5, 2016

Test build #48738 has finished for PR 10589 at commit 22afd1f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class Benchmark(name: String, valuesPerIteration: Long, iters: Int = 5)
    • case class Case(name: String, fn: Int => Unit)
    • case class Result(avgMs: Double, avgRate: Double)

@davies
Copy link
Contributor

davies commented Jan 5, 2016

@nongli You don't need to repeat the title in the description of the PR (or it get duplicated in commit message).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be SQL with ParquetReader

nongli and others added 2 commits January 6, 2016 14:34
…uet scan benchmarks.

We've run benchmarks ad hoc to measure the scanner performance. We will continue to invest in this
and it makes sense to get these benchmarks into code. This adds a simple benchmarking utility to do
this.
@SparkQA
Copy link

SparkQA commented Jan 7, 2016

Test build #48876 has finished for PR 10589 at commit f231e9a.

  • This patch passes all tests.
  • This patch does not merge cleanly.
  • This patch adds the following public classes (experimental):
    • case class Case(name: String, fn: Int => Unit)
    • case class Result(avgMs: Double, avgRate: Double)

@SparkQA
Copy link

SparkQA commented Jan 7, 2016

Test build #2341 has finished for PR 10589 at commit 4f682c5.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class Case(name: String, fn: Int => Unit)
    • case class Result(avgMs: Double, avgRate: Double)

@SparkQA
Copy link

SparkQA commented Jan 7, 2016

Test build #48885 has finished for PR 10589 at commit 4f682c5.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class Case(name: String, fn: Int => Unit)
    • case class Result(avgMs: Double, avgRate: Double)

@rxin
Copy link
Contributor

rxin commented Jan 7, 2016

I've merged this. Thanks!

@asfgit asfgit closed this in a74d743 Jan 7, 2016
@nongli nongli deleted the spark-12640 branch January 7, 2016 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants