New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
8272358: Some tests may fail when executed with other locales than the US #5098
Conversation
|
Webrevs
|
Any volunteers to take a look at the tests update? |
There are still 1500+ failures due to non-US locale after this patch. So my question is: does it make sense to fix only 4 of them? |
These are the only tests reported in Fr and Ru locales in t1/t2/t3. Are you sure that 1500 failures are caused by the tests issues and not a problem in JDK? |
If I test with make test TEST="tier1 tier2 tier3" with your patch
make test JTREG="VM_OPTIONS=-Duser.language=en -Duser.country=US" TEST="tier1 tier2 tier3" with your patch
|
If the failure disappeared if the "user.language=en" is used does not mean that this is not a JDK bug or some other bugs in the tests. What is the reason for the falure? |
An important reason is that the expected output doesn't match due to different languages.
E.g., Test runtime/classFileParserBug/TestEmptyBootstrapMethodsAttr.java with the locale:
|
This error means that your OpenJDK build contains the translation for your locale, and different tools use the resource bundles to translate different outputs. Looks like nobody validates that the translated output is correct for the various cases since the tests fail on non-US translated locales. I guess in the ideal world that tests should be updated to validate the output for each locale, or probably even better they should check that the output is correct for some specific "key" in the resource bundle. The current issue is different, it fixes the tests depended on some specific float format and has no dependency on the resource bundles. |
I still don't understand why the failures are different. But the fix doesn't work even with these 4 tests.
But if I run with make test JTREG="VM_OPTIONS=-Duser.language=en -Duser.country=US" TEST="...", all of them get passed. |
The difference is that if the test uses golden data for EN locale only while the tested tool like javac supports more than one locale for some specific output message then the test should check that specific localized message.
Look like you use one of ja or zh_CN locales since only for these locales there are translations, the test should take it into account. It is not strictly correct to run add "-Duser.language=en -Duser.country=US" to it to fix such issues. |
Thanks @mrserb for your clarification. Shall we also fix ToolBasicTest.java and ToolSimpleTest.java just as what is done for LambdaTranslationTest1.java and LambdaTranslationTest2.java? And do you test the fix on Linux? |
I do not want to add a global locale parameter to the places which may trigger loading of the different translations.
On Linux I have checked "JA" and "RU" locales, nothing changed for "JA" it uses the same float format as the US, so the LambdaTranslationTest1/LambdaTranslationTest2 passed before/after the fix and the ToolBasicTest/ToolSimpleTest failed in the "JA" locale, one of the subtests compare the expected output saved for US locale with the real output of the test. For RU locale the tests fail before the fix and passed after because there is no specific translation so the US text is used, and this PR fixes the problem in the float format. |
So this change doesn't fix the failure with Chinese and JA locales. |
Yes, I can rephrase it as "it fixes the locales which do not have translations". |
Then people with RU locale should look at this fix. |
@aivanov-jdk please take a look |
@mrserb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration! |
@aivanov-jdk Alexey do you have any comments or suggestions? |
Sorry for my delayed reply. I have never used Russian locale when working with JDK, if I remember correctly, I even experienced JDK build failures with Russian locale. To me, the change looks good: it makes the tests more stable, these updated tests should pass even if the current locale is different than the US. I believe it's a common practice, as it's mentioned in Testing the JDK:
However, I haven't tried running the tests under Russian or French locale yet. As for translations, I believe the tests which specifically verify translations should also set the locale explicitly and check the expected output. |
I can confirm that these fixes make these tests pass in Russian locale.
Two of the tests fail with timeout for me: ToolBasicTest.java
and ToolSimpleTest.java
; the other two LambdaTranslationTest{1,2}.java
fail without the fix and pass with the proposed fix applied. I'm inclined to trust that the first pair also passes successfully with the fix.
@mrserb This change now passes all automated pre-integration checks. After integration, the commit message for the final commit will be:
You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been no new commits pushed to the
|
/integrate |
Going to push as commit 29e552c.
Your commit was automatically rebased without conflicts. |
As mentioned in the bug report this issue was reported a few times, I have checked all tests in tier1/tier2/tier3 and found these four tests which fail because of non-US locale. The common issue is that the output depends on the locale(ex: 3.14 VS 3,14).
Progress
Issue
Reviewers
Reviewing
Using
git
Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/5098/head:pull/5098
$ git checkout pull/5098
Update a local copy of the PR:
$ git checkout pull/5098
$ git pull https://git.openjdk.java.net/jdk pull/5098/head
Using Skara CLI tools
Checkout this PR locally:
$ git pr checkout 5098
View PR using the GUI difftool:
$ git pr show -t 5098
Using diff file
Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/5098.diff