-
Notifications
You must be signed in to change notification settings - Fork 712
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix vectorized toLowerCase and toUppercase intrinsics on Z #4688
Fix vectorized toLowerCase and toUppercase intrinsics on Z #4688
Conversation
7b8fa9f
to
a6db4fd
Compare
I've improved the tests by including an exhaustive list of tests for uppercasing and lowercasing. There were two issues:
Both issues will be fixed. There will be a subsequent commit fixing the issues, but before I add it I'd like to kick off sanity testing with the new tests and ensure the failures are caught on the farm. In addition to the problem observed in #4597 we have one additional character we're not converting properly (0xB5). Hopefully both failures show up on the builds. Jenkins test sanity xlinux,zlinux JDK8,JDK11 |
It seems the previous tests that were added were only testing certain buffer lengths against certain subsets of characters. While ok, these tests are not exhaustive and do not test the entire range of characters which can be uppercased or lowercased. In this commit we add exhaustive testing for both intrinsic APIs by pre-generating a table of the uppercase and lowercase mapping. We use this map to compare the result of calling the intrinsic. Signed-off-by: Filip Jeremic <fjeremic@ca.ibm.com>
As expected the tests failed as follows [1]:
The second failure is the one observed in #4597. I'll test my fix locally tomorrow morning and attach it as a new commit to this PR. |
In eclipse-openj9#4597 we identified cases on which `toUpperCase` fails on Linux on Z. This is due to some faulty login in the evaluator for these intrinsics on Z. Bugs seem to have been introduced in eclipse-openj9#2318 for certain characters. Namely the test in eclipse-openj9#4597 fails to convert the `\u00FF` character. The bug is obvious, since `toUpperCase` of such a character cannot be handled and the JIT evaluator determines the invalid character range by checking if the character is greater than 0xFF, but it should be checking if greater than or equal to. In addition the compressed String evaluator also fails on 0xB5 as determined by the unit tests. Both issues are fixed. Fixes: eclipse-openj9#4597 Signed-off-by: Filip Jeremic <fjeremic@ca.ibm.com>
a6db4fd
to
38f9961
Compare
And the fix is in. I've tested manually the tests pass but for good measure we'll test on the farm as well. Requesting reviews from one of @smlambert / @llxia for the test piece of the PR and one of @NigelYiboYu / @dhong44 for the Z codegen piece. Jenkins test sanity xlinux,zlinux JDK8,JDK11 |
@andrewcraik @0dvictor FYI we'll need to take a look at these x86 failures as my unit tests have caught what are likely bugs in the x86 |
per discussion the issues on x86 are that we need to truncate the return value of the helper to be 0 or 1 per recent spec updates, ensure we check the |
There are two functional issues with the String case conversion intrinsics on x86: 1. The `getSupportsInlineStringCaseConversion` API is not respected and hence there is no way to disable the specific intrinsic 2. The evaluator will return a non binary value for a truthful result while it should only return a binary value (0 or 1) We fix both issues here. Signed-off-by: Filip Jeremic <fjeremic@ca.ibm.com>
Jenkins test sanity xlinux,zlinux JDK8,JDK11 |
The edge case tests needed to store a byte into a `char[]`, but this has to be performed in an endian aware manner because the intrinsics act on bytes and we subsequently do an array comparison which acts on chars. We risk the high order byte of the char being different on little endian vs. big endian following the conversion which makes the tests faulty. The logical and with 0xFF00 was written with big endian in mind. Here we make it endian aware. Signed-off-by: Filip Jeremic <fjeremic@ca.ibm.com>
Jenkins test sanity xlinux,zlinux JDK8,JDK11 |
All issues have been fixed. This is ready for final review from committers. |
@andrewcraik, can you give your +1 if you're fine with the x86 changes please? |
… list (#1230) * PR to remove the UnicodeCasingTest from the problem list. Original issue has been patched for JDK11 eclipse-openj9/openj9#4688
… list (adoptium#1230) * PR to remove the UnicodeCasingTest from the problem list. Original issue has been patched for JDK11 eclipse-openj9/openj9#4688
… list (adoptium#1230) * PR to remove the UnicodeCasingTest from the problem list. Original issue has been patched for JDK11 eclipse-openj9/openj9#4688
* Un-excluded java/lang/String/UnicodeCasingTest.java from problem list * Updated issue links Unicode casing test deleted as [#4688](eclipse-openj9/openj9#4688) fixes the issue * Updated weblinks * Updated issue links
* Un-excluded java/lang/String/UnicodeCasingTest.java from problem list Unicode casing test deleted as [#4688](eclipse-openj9/openj9#4688) fixes the issue * Links to permanent failures
* Un-excluded java/lang/String/UnicodeCasingTest.java from problem list * Updated issue links Unicode casing test deleted as [#4688](eclipse-openj9/openj9#4688) fixes the issue
* Un-excluded java/lang/String/UnicodeCasingTest.java from problem list * Updated issue links Unicode casing test deleted as [#4688](eclipse-openj9/openj9#4688) fixes the issue * Updated weblinks * Updated issue links * Issue links updated * Updated problem list weblinks * Updated issue links * Ricochet.java passes java/lang/invoke/RicochetTest.java is now passing on nightly build's`11.0.5+6-201909111831` and upwards ``` openjdk version "11.0.5" 2019-10-15 OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.5+6-201909111831) Eclipse OpenJ9 VM AdoptOpenJDK (build master-d65742671, JRE 11 Linux amd64-64-Bit Compressed References 20190911_331 (JIT enabled, AOT enabled) OpenJ9 - d65742671 OMR - 7e9584ea JCL - 8057a0754d based on jdk-11.0.5+6) ``` * CustomizedLambdaForm has been fixed as per issue eclipse-openj9/openj9#7080 * perm excludes point to #1297
* Un-excluded java/lang/String/UnicodeCasingTest.java from problem list * Updated issue links Unicode casing test deleted as [#4688](eclipse-openj9/openj9#4688) fixes the issue * Ricochet.java passes java/lang/invoke/RicochetTest.java is now passing on nightly build's`11.0.5+6-201909111831` and upwards ``` * CustomizedLambdaForm has been fixed as per issue eclipse-openj9/openj9#7080 * perm excludes point to #1297 * Perm exclude SpecialInterfaceCall * JDK11 Prob list links * AutomaticModulesTest is now passing * Reinclude currently failing tests * Reincluded failing tests, PR#1359 already does this
In #4597 we identified cases on which
toUpperCase
fails on Linux onZ. This is due to some faulty logic in the evaluator for these
intrinsics on Z. Bugs seem to have been introduced in #2318 for certain
characters. Namely the test in #4597 fails to convert the
\u00FF
character. The bug is obvious, since
toUpperCase
of such a charactercannot be handled and the JIT evaluator determines the invalid
character range by checking if the character is greater than 0xFF, but
it should be checking if greater than or equal to.
In addition the compressed String evaluator also fails on 0xB5 as
determined by the unit tests. Both issues are fixed.
Fixes: #4597