Skip to content
This repository was archived by the owner on Oct 16, 2024. It is now read-only.

Issue 157: Introduce JMH for micro benchmarking#158

Closed
hellostreaming wants to merge 2 commits intoapache:masterfrom
hellostreaming:tests
Closed

Issue 157: Introduce JMH for micro benchmarking#158
hellostreaming wants to merge 2 commits intoapache:masterfrom
hellostreaming:tests

Conversation

@hellostreaming
Copy link
Copy Markdown
Contributor

@hellostreaming hellostreaming commented Aug 30, 2017

Descriptions of the changes in this PR:

  • add a tests/jmh and tests/jmh-0.4 module for micro-benchmarking
  • tests/jmh is to benchmark latest master
  • tests/jmh-0.4 is to benchmark 0.4.0-incubating
  • add benchmark for compression

@hellostreaming
Copy link
Copy Markdown
Contributor Author

hellostreaming commented Aug 30, 2017

This is the pull request for #157

@sijie sijie added this to the 0.5.0 milestone Aug 30, 2017
@sijie sijie requested review from mgodave and sijie August 30, 2017 01:26
@sijie
Copy link
Copy Markdown
Member

sijie commented Aug 30, 2017

/cc @leighst

@hellostreaming
Copy link
Copy Markdown
Contributor Author

hellostreaming commented Aug 30, 2017 •
edited
Loading

The benchmark result for latest master:

Benchmark                                                                 (size)   Mode  Cnt       Score      Error   Units
CompressionBenchmark.testCompressLZ4                                          10  thrpt   50    2422.408 ±   44.161  ops/ms
CompressionBenchmark.testCompressLZ4                                         100  thrpt   50    1793.394 ±   27.825  ops/ms
CompressionBenchmark.testCompressLZ4                                        1000  thrpt   50     928.596 ±   14.488  ops/ms
CompressionBenchmark.testCompressLZ4                                       10000  thrpt   50     239.443 ±    4.752  ops/ms
CompressionBenchmark.testCompressNone                                         10  thrpt   50  207570.657 ± 4656.234  ops/ms
CompressionBenchmark.testCompressNone                                        100  thrpt   50   90762.739 ± 1753.916  ops/ms
CompressionBenchmark.testCompressNone                                       1000  thrpt   50   10532.599 ±  207.739  ops/ms
CompressionBenchmark.testCompressNone                                      10000  thrpt   50    1034.792 ±   20.067  ops/ms
CompressionBenchmark.testDecompressLZ4                                        10  thrpt   50    4265.879 ±   80.825  ops/ms
CompressionBenchmark.testDecompressLZ4                                       100  thrpt   50    3975.214 ±  130.441  ops/ms
CompressionBenchmark.testDecompressLZ4                                      1000  thrpt   50    1821.820 ±   37.923  ops/ms
CompressionBenchmark.testDecompressLZ4                                     10000  thrpt   50     298.692 ±    8.591  ops/ms
CompressionBenchmark.testDecompressNone                                       10  thrpt   50  202194.272 ± 4431.166  ops/ms
CompressionBenchmark.testDecompressNone                                      100  thrpt   50   80907.829 ± 3869.202  ops/ms
CompressionBenchmark.testDecompressNone                                     1000  thrpt   50   10041.016 ±  391.215  ops/ms
CompressionBenchmark.testDecompressNone                                    10000  thrpt   50    1016.756 ±   26.645  ops/ms

The jvm allocation rate is huge, even with compression none:

CompressionBenchmark.testCompressLZ4:·gc.alloc.rate                           10  thrpt   50     123.090 ±    2.254  MB/sec
CompressionBenchmark.testCompressLZ4:·gc.alloc.rate                          100  thrpt   50     291.617 ±    4.542  MB/sec
CompressionBenchmark.testCompressLZ4:·gc.alloc.rate                         1000  thrpt   50    1217.140 ±   19.148  MB/sec
CompressionBenchmark.testCompressLZ4:·gc.alloc.rate                        10000  thrpt   50    3061.999 ±   60.421  MB/sec
CompressionBenchmark.testCompressNone:·gc.alloc.rate                          10  thrpt   50    4219.536 ±   95.041  MB/sec
CompressionBenchmark.testCompressNone:·gc.alloc.rate                         100  thrpt   50    6918.226 ±  134.556  MB/sec
CompressionBenchmark.testCompressNone:·gc.alloc.rate                        1000  thrpt   50    6796.270 ±  133.545  MB/sec
CompressionBenchmark.testCompressNone:·gc.alloc.rate                       10000  thrpt   50    6581.972 ±  126.883  MB/sec
CompressionBenchmark.testDecompressLZ4:·gc.alloc.rate                         10  thrpt   50     238.471 ±    4.534  MB/sec
CompressionBenchmark.testDecompressLZ4:·gc.alloc.rate                        100  thrpt   50    1131.557 ±   37.133  MB/sec
CompressionBenchmark.testDecompressLZ4:·gc.alloc.rate                       1000  thrpt   50    4683.743 ±   97.727  MB/sec
CompressionBenchmark.testDecompressLZ4:·gc.alloc.rate                      10000  thrpt   50    7619.130 ±  218.422  MB/sec
CompressionBenchmark.testDecompressNone:·gc.alloc.rate                        10  thrpt   50    4108.263 ±   90.288  MB/sec
CompressionBenchmark.testDecompressNone:·gc.alloc.rate                       100  thrpt   50    6168.327 ±  295.150  MB/sec
CompressionBenchmark.testDecompressNone:·gc.alloc.rate                      1000  thrpt   50    6481.412 ±  252.577  MB/sec
CompressionBenchmark.testDecompressNone:·gc.alloc.rate                     10000  thrpt   50    6467.379 ±  169.089  MB/sec

@hellostreaming
Copy link
Copy Markdown
Contributor Author

hellostreaming commented Aug 30, 2017

pushed a new change to exclude generated files for findbugs:check

@hellostreaming
Copy link
Copy Markdown
Contributor Author

hellostreaming commented Aug 30, 2017

the test result from 0.4.0-incubating (similar result as current master, since there is any change to compression codec)

expected to see performance improvements at #151

Benchmark                                                                 (size)   Mode  Cnt       Score      Error   Units
CompressionBenchmark.testCompressLZ4                                          10  thrpt   50    2422.408 ±   44.161  ops/ms
CompressionBenchmark.testCompressLZ4                                         100  thrpt   50    1793.394 ±   27.825  ops/ms
CompressionBenchmark.testCompressLZ4                                        1000  thrpt   50     928.596 ±   14.488  ops/ms
CompressionBenchmark.testCompressLZ4                                       10000  thrpt   50     239.443 ±    4.752  ops/ms
CompressionBenchmark.testCompressNone                                         10  thrpt   50  207570.657 ± 4656.234  ops/ms
CompressionBenchmark.testCompressNone                                        100  thrpt   50   90762.739 ± 1753.916  ops/ms
CompressionBenchmark.testCompressNone                                       1000  thrpt   50   10532.599 ±  207.739  ops/ms
CompressionBenchmark.testCompressNone                                      10000  thrpt   50    1034.792 ±   20.067  ops/ms
CompressionBenchmark.testDecompressLZ4                                        10  thrpt   50    4265.879 ±   80.825  ops/ms
CompressionBenchmark.testDecompressLZ4                                       100  thrpt   50    3975.214 ±  130.441  ops/ms
CompressionBenchmark.testDecompressLZ4                                      1000  thrpt   50    1821.820 ±   37.923  ops/ms
CompressionBenchmark.testDecompressLZ4                                     10000  thrpt   50     298.692 ±    8.591  ops/ms
CompressionBenchmark.testDecompressNone                                       10  thrpt   50  202194.272 ± 4431.166  ops/ms
CompressionBenchmark.testDecompressNone                                      100  thrpt   50   80907.829 ± 3869.202  ops/ms
CompressionBenchmark.testDecompressNone                                     1000  thrpt   50   10041.016 ±  391.215  ops/ms
CompressionBenchmark.testDecompressNone                                    10000  thrpt   50    1016.756 ±   26.645  ops/ms

sijie
sijie approved these changes Aug 30, 2017
View reviewed changes
Copy link
Copy Markdown
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well done! @zhaijack

Look forward to seeing the results of #151

@sijie
Copy link
Copy Markdown
Member

sijie commented Aug 30, 2017

all CI job passed.

@sijie sijie closed this in 02058d6 Aug 30, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@mgodave mgodave Awaiting requested review from mgodave

1 more reviewer

@sijie sijie sijie approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@hellostreaming hellostreaming

Labels

area/build area/tests release/0.5.0 type/task

Projects

None yet

Milestone

0.5.0

Development

Successfully merging this pull request may close these issues.

3 participants

@hellostreaming @sijie @jiazhai