Utf 8 encoding optimizations #1444
Conversation
…compatibility test Adds a two parameter constructor to Encoding to allow sub classes to specify whether their known ASCII compatability so as to skip testing. The only usage of it is the UTF8Encoding which is changed to use the new constructor.
Also optimize for java 9+ byte[] backed strings
|
|
pgjdbc/src/main/java/org/postgresql/core/ByteOptimizedUTF8Encoding.java
Outdated
Show resolved
Hide resolved
address style issues
|
|
address style issues
|
|
Codecov Report
@@ Coverage Diff @@
## master #1444 +/- ##
===========================================
+ Coverage 68.88% 68.99% +0.1%
- Complexity 3949 4154 +205
===========================================
Files 181 186 +5
Lines 16485 17122 +637
Branches 2679 2810 +131
===========================================
+ Hits 11356 11813 +457
- Misses 3885 4027 +142
- Partials 1244 1282 +38 |
use existing JavaVersion enum to pick implementation add unit test string values more consistency between byte and char based implementations
fix backwards comparison add more strings to unit test
|
|
|
|
# Conflicts: # pgjdbc/src/main/java/org/postgresql/core/Encoding.java # pgjdbc/src/main/java/org/postgresql/core/OptimizedUTF8Encoder.java
|
|
Move to using new String(byte[], int, int, Charset) rather than custom decoding for jre newer than 1.8.
|
|
Back to custom utf-8 decoding for performance gains while validating
|
|
|
|
|
|
|
Is this the right fix for this issue and is it going to be merged? |
|
@bokken can you fix the checkstyle complaint? |
pgjdbc/src/test/java/org/postgresql/test/jdbc2/Jdbc2TestSuite.java
Outdated
Show resolved
Hide resolved
pgjdbc/src/main/java/org/postgresql/core/OptimizedUTF8Encoder.java
Outdated
Show resolved
Hide resolved
| if (size <= thresholdSize) { | ||
| decoderArray = new char[size]; |
|
It appears we have a somewhat flaky test that is failing. It also appears that our use of maven tool chains means that our tests are not testing what we think they are testing: $ java -Xmx32m -version https://travis-ci.org/pgjdbc/pgjdbc/jobs/637954721?utm_medium=notification&utm_source=github_status |
|
I knew about the flaky test. Was not aware of the toolchain issue. |
|
It looks like a lot of the tests are actually executing with jdk 8. Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) Strangely, another test, also targeting oraclejdk9, does not have an issue: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T03:58:13-04:00) One thing I observed as being different here is the one which actually uses jdk9 has the build property JDK=9. There other does not. |
OK, lets flag this as another issue to fix the tests instead of getting bogged down here. |
|
Looks like travis was successful. I think this is good to go @davecramer. |
|
Is there a plan to incorporate this pull request into the next release? I'm assuming the next one would be 42.3.0 (I found the release board here: https://github.com/pgjdbc/pgjdbc/projects/2)? |
|
well it's merged so yes, but not sure why it would be 42.3.0. this is not a breaking change |
|
@davecramer Sounds good. Is there a tentative timeline for the next patch release? |
|
it's currently at the top of my queue |
* fix: Correct typo weather to whether * misc: Change internal Encoding.testAsciiNumbers(...) to be static * perf: Enhance Encoding constructor to allow skipping of ASCII number compatibility test Adds a two parameter constructor to Encoding to allow sub classes to specify whether their known ASCII compatability so as to skip testing. The only usage of it is the UTF8Encoding which is changed to use the new constructor. * fix: limit size of char[] for utf-8 decoding Also optimize for java 9+ byte[] backed strings * fix: limit size of char[] for utf-8 decoding address style issues * fix: limit size of char[] for utf-8 decoding address style issues * fix: limit size of char[] for utf-8 decoding use existing JavaVersion enum to pick implementation add unit test string values more consistency between byte and char based implementations * fix: limit size of char[] for utf-8 decoding fix backwards comparison add more strings to unit test * fix: limit size of char[] for utf-8 decoding Move to using new String(byte[], int, int, Charset) rather than custom decoding for jre newer than 1.8. * fix: limit size of char[] for utf-8 decoding Back to custom utf-8 decoding for performance gains while validating * javadoc * put test back into test suite * avoid creating an unnecessary `char[]` when growing cached array Co-authored-by: Sehrope Sarkuni <sehrope@jackdb.com> Co-authored-by: Dave Cramer <davecramer@gmail.com>
All Submissions:
New Feature Submissions:
Changes to Existing Features:
Addresses #1431 by limiting the size of ‘char[]’ cached for reuse.
Also provides different implementation for java 9+ where ‘String’ implementation changes to be backed by ‘byte[]’.