New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add ubenchmark for UTF-8 decoding #988

Merged
merged 1 commit into from Oct 19, 2017

Conversation

Projects
None yet
4 participants
@marschall
Contributor

marschall commented Oct 15, 2017

We have custom UTF-8 decoding code in UTF8Encoding for performance
reasons but no companion benchmarks to verify its performance.

@marschall

This comment has been minimized.

Contributor

marschall commented Oct 15, 2017

These are the results for Java 8u144

# Run complete. Total time: 00:08:25

Benchmark                                (length)  Mode  Cnt     Score    Error  Units
UTF8Decoding.decoder_byteBufferReuse            1  avgt   10    28.801 ±  0.227  ns/op
UTF8Decoding.decoder_byteBufferReuse            5  avgt   10   109.644 ±  0.749  ns/op
UTF8Decoding.decoder_byteBufferReuse           10  avgt   10   215.843 ±  1.773  ns/op
UTF8Decoding.decoder_byteBufferReuse           50  avgt   10  1012.366 ±  6.846  ns/op
UTF8Decoding.decoder_byteBufferReuse          100  avgt   10  2006.056 ±  6.929  ns/op
UTF8Decoding.encodingDecodeUTF8_current         1  avgt   10    40.362 ±  0.271  ns/op
UTF8Decoding.encodingDecodeUTF8_current         5  avgt   10    99.789 ±  0.941  ns/op
UTF8Decoding.encodingDecodeUTF8_current        10  avgt   10   184.204 ±  1.431  ns/op
UTF8Decoding.encodingDecodeUTF8_current        50  avgt   10   857.471 ±  7.029  ns/op
UTF8Decoding.encodingDecodeUTF8_current       100  avgt   10  1780.866 ± 16.221  ns/op
UTF8Decoding.string_charset                     1  avgt   10    49.943 ±  0.348  ns/op
UTF8Decoding.string_charset                     5  avgt   10   111.489 ±  0.963  ns/op
UTF8Decoding.string_charset                    10  avgt   10   205.734 ±  2.416  ns/op
UTF8Decoding.string_charset                    50  avgt   10   927.066 ± 13.315  ns/op
UTF8Decoding.string_charset                   100  avgt   10  1911.948 ± 18.644  ns/op
UTF8Decoding.string_string                      1  avgt   10    58.777 ±  0.754  ns/op
UTF8Decoding.string_string                      5  avgt   10   119.739 ±  0.878  ns/op
UTF8Decoding.string_string                     10  avgt   10   208.635 ±  1.956  ns/op
UTF8Decoding.string_string                     50  avgt   10   930.595 ±  8.034  ns/op
UTF8Decoding.string_string                    100  avgt   10  1914.516 ± 20.509  ns/op
UTF8Decoding.utilsDecodeUTF8_old                1  avgt   10    74.967 ±  0.777  ns/op
UTF8Decoding.utilsDecodeUTF8_old                5  avgt   10   150.848 ±  1.929  ns/op
UTF8Decoding.utilsDecodeUTF8_old               10  avgt   10   281.153 ±  2.460  ns/op
UTF8Decoding.utilsDecodeUTF8_old               50  avgt   10  1233.773 ±  8.020  ns/op
UTF8Decoding.utilsDecodeUTF8_old              100  avgt   10  2109.437 ± 18.573  ns/op

Java 9+181 (GA)

# Run complete. Total time: 00:08:28

Benchmark                                (length)  Mode  Cnt     Score    Error  Units
UTF8Decoding.decoder_byteBufferReuse            1  avgt   10    27.160 ±  0.476  ns/op
UTF8Decoding.decoder_byteBufferReuse            5  avgt   10   103.948 ±  0.850  ns/op
UTF8Decoding.decoder_byteBufferReuse           10  avgt   10   203.903 ±  1.728  ns/op
UTF8Decoding.decoder_byteBufferReuse           50  avgt   10   964.805 ± 16.728  ns/op
UTF8Decoding.decoder_byteBufferReuse          100  avgt   10  1898.242 ±  8.270  ns/op
UTF8Decoding.encodingDecodeUTF8_current         1  avgt   10    43.708 ±  0.961  ns/op
UTF8Decoding.encodingDecodeUTF8_current         5  avgt   10   107.695 ±  2.652  ns/op
UTF8Decoding.encodingDecodeUTF8_current        10  avgt   10   191.572 ±  4.234  ns/op
UTF8Decoding.encodingDecodeUTF8_current        50  avgt   10   802.963 ±  9.245  ns/op
UTF8Decoding.encodingDecodeUTF8_current       100  avgt   10  1637.738 ± 21.761  ns/op
UTF8Decoding.string_charset                     1  avgt   10    37.956 ±  1.062  ns/op
UTF8Decoding.string_charset                     5  avgt   10   105.437 ±  2.390  ns/op
UTF8Decoding.string_charset                    10  avgt   10   171.914 ±  4.240  ns/op
UTF8Decoding.string_charset                    50  avgt   10   792.914 ± 13.757  ns/op
UTF8Decoding.string_charset                   100  avgt   10  1584.547 ± 37.837  ns/op
UTF8Decoding.string_string                      1  avgt   10    43.095 ±  1.156  ns/op
UTF8Decoding.string_string                      5  avgt   10   106.535 ±  2.861  ns/op
UTF8Decoding.string_string                     10  avgt   10   174.194 ±  1.917  ns/op
UTF8Decoding.string_string                     50  avgt   10   785.949 ± 14.624  ns/op
UTF8Decoding.string_string                    100  avgt   10  1580.724 ± 30.834  ns/op
UTF8Decoding.utilsDecodeUTF8_old                1  avgt   10    63.446 ±  1.122  ns/op
UTF8Decoding.utilsDecodeUTF8_old                5  avgt   10   149.747 ±  2.553  ns/op
UTF8Decoding.utilsDecodeUTF8_old               10  avgt   10   268.353 ±  4.446  ns/op
UTF8Decoding.utilsDecodeUTF8_old               50  avgt   10  1193.707 ± 17.955  ns/op
UTF8Decoding.utilsDecodeUTF8_old              100  avgt   10  2317.537 ± 34.240  ns/op

As you can see Java 9 is now slightly faster than our implementation.

They were taken on an Intel Core i7-7700T and Ubuntu 17.04

@marschall marschall force-pushed the marschall:benchmark-utf8-decode branch from f2aea8c to b4da81a Oct 15, 2017

@codecov-io

This comment has been minimized.

codecov-io commented Oct 15, 2017

Codecov Report

Merging #988 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #988   +/-   ##
=========================================
  Coverage     65.93%   65.93%           
- Complexity     3560     3561    +1     
=========================================
  Files           166      166           
  Lines         15244    15244           
  Branches       2465     2465           
=========================================
  Hits          10051    10051           
- Misses         4022     4023    +1     
+ Partials       1171     1170    -1
@@ -0,0 +1,108 @@
/*

This comment has been minimized.

@davecramer

davecramer Oct 15, 2017

Member

@marschall can you change this to 2017, this is a new file

This comment has been minimized.

@marschall

marschall Oct 15, 2017

Contributor

sure

* Copyright (c) 2004, PostgreSQL Global Development Group
* See the LICENSE file in the project root for more information.
*/
// Copyright (c) 2004, Open Cloud Limited.

This comment has been minimized.

@davecramer

davecramer Oct 15, 2017

Member

remove Open Cloud. again new file.

This comment has been minimized.

@marschall

marschall Oct 15, 2017

Contributor

sure

test: add ubenchmark for UTF-8 decoding
We have custom UTF-8 decoding code in UTF8Encoding for performance
reasons but no companion benchmarks to verify its performance.

@marschall marschall force-pushed the marschall:benchmark-utf8-decode branch from b4da81a to 0044fad Oct 15, 2017

@Measurement(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS)
@Warmup(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS)
@State(Scope.Thread)
@Threads(1)

This comment has been minimized.

@jorsol

jorsol Oct 15, 2017

Contributor

@marschall could you please share benchmarks with Threads > 1?

This comment has been minimized.

@marschall

marschall Oct 15, 2017

Contributor

The current UTF-8 encoding benchmark uses one thread as well https://github.com/pgjdbc/pgjdbc/blob/master/ubenchmark/src/main/java/org/postgresql/benchmark/encoding/UTF8Encoding.java#L42 I used the same parameters.

Is there a special reason why you're interested in threads > 1? If so what is the value?

This comment has been minimized.

@jorsol

jorsol Oct 15, 2017

Contributor

Just for curiosity.

@davecramer davecramer merged commit 0d918c3 into pgjdbc:master Oct 19, 2017

2 checks passed

codecov/project 65.93% remains the same compared to 7df56f8
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@davecramer

This comment has been minimized.

Member

davecramer commented Oct 19, 2017

Seems jdk9 can't be installed for some reason?

rhavermans added a commit to bolcom/pgjdbc that referenced this pull request Jul 13, 2018

test: add ubenchmark for UTF-8 decoding (pgjdbc#988)
We have custom UTF-8 decoding code in UTF8Encoding for performance
reasons but no companion benchmarks to verify its performance.

rhavermans added a commit to bolcom/pgjdbc that referenced this pull request Jul 13, 2018

test: add ubenchmark for UTF-8 decoding (pgjdbc#988)
We have custom UTF-8 decoding code in UTF8Encoding for performance
reasons but no companion benchmarks to verify its performance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment