Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8275007: Java fails to start with null charset if LC_ALL is set to certain locales #6282

Closed
wants to merge 10 commits into from

Conversation

naotoj
Copy link
Member

@naotoj naotoj commented Nov 5, 2021

Please review the subject fix. In light of JEP400, Java runtime can/should start in UTF-8 charset if the underlying native encoding is not supported.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8275007: Java fails to start with null charset if LC_ALL is set to certain locales

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/6282/head:pull/6282
$ git checkout pull/6282

Update a local copy of the PR:
$ git checkout pull/6282
$ git pull https://git.openjdk.java.net/jdk pull/6282/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 6282

View PR using the GUI difftool:
$ git pr show -t 6282

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/6282.diff

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Nov 5, 2021

👋 Welcome back naoto! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr label Nov 5, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Nov 5, 2021

@naotoj The following labels will be automatically applied to this pull request:

  • core-libs
  • nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added nio core-libs labels Nov 5, 2021
@mlbridge
Copy link

@mlbridge mlbridge bot commented Nov 5, 2021

@openjdk
Copy link

@openjdk openjdk bot commented Nov 5, 2021

@naotoj This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8275007: Java fails to start with null charset if LC_ALL is set to certain locales

Reviewed-by: rriggs, iris, joehw, alanb

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 186 new commits pushed to the master branch:

  • b687664: 8277159: Fix java/nio/file/FileStore/Basic.java test by ignoring /run/user/* mount points
  • 8f5a8f7: 8264293: Create implementation for NSAccessibilityMenu protocol peer
  • 9f2f46e: 8275037: Test vmTestbase/nsk/sysdict/vm/stress/btree/btree011/btree011.java crashes with memory exhaustion on Windows
  • 2af9e59: 8276139: TestJpsHostName.java not reliable, better to expand HostIdentifierCreate.java test
  • e9934e1: 8277221: G1: Remove methods without implementations in G1CollectedHeap
  • 9aa30de: 8275317: AArch64: Support some type conversion vectorization in SLP
  • 08f65a5: 8277313: Validate header failed for test/jdk/java/net/httpclient/HeadTest.java
  • 23e5117: 8276559: (httpclient) Consider adding an HttpRequest.Builder.HEAD method to build a HEAD request.
  • a77d8dd: 8276787: Improve warning messages for -XX:+RecordDynamicDumpInfo
  • 8ed384c: 8276609: Document setting property jdk.serialFilter to an invalid value throws ExceptionInInitializerError
  • ... and 176 more: https://git.openjdk.java.net/jdk/compare/7115892f96a5a57ce9d37602038b787d19da5d81...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Nov 5, 2021
Copy link
Member

@JoeWang-Java JoeWang-Java left a comment

Do you need to add test?

@naotoj
Copy link
Member Author

@naotoj naotoj commented Nov 5, 2021

Do you need to add test?

I would if I could, but specifying the environment dependent settings in the test would be fragile and error-prone, so I did not add a test but marked the issue as noreg-hard.

@takiguc
Copy link

@takiguc takiguc commented Nov 6, 2021

@naotoj
I'm not reviewer.
Could you explain more detail, why JEP-400 needs to touch sun.jnu.encoding system property's fallback ?

@AlanBateman
Copy link
Contributor

@AlanBateman AlanBateman commented Nov 6, 2021

Could you explain more detail, why JEP-400 needs to touch sun.jnu.encoding system property's fallback ?

This is about running on unusual configurations where the native encoding (native.encoding and sun.jnu.encoding) doesn't name a charset in java.base. There has been several reports of NPE in such cases. Now that the default charset is UTF-8 it means there is a sane fallback so the VM can startup.

@takiguc
Copy link

@takiguc takiguc commented Nov 6, 2021

Thanks @AlanBateman .
In my understanding, sun.jnu.encoding property may be related file system access.
Java may not be access to appropriate file.
I mean if the file name has malformed character, it may be changed to "?".
In this case, unexpected file access may be happened.
8-bit pass-through may be better...

@AlanBateman
Copy link
Contributor

@AlanBateman AlanBateman commented Nov 6, 2021

In my understanding, sun.jnu.encoding property may be related file system access.
Java may not be access to appropriate file.

Yes, it's a JDK internal property with the charset name to use when decoding or encoding file names (not the file content). The issue in this PR is about what to do when starting in unsupported configurations. In the recent releases the JDK will typically NPE or fail with an exception. Since JEP 400 this no longer impacts the default charset because it is UTF-8. This moves the problem on to choosing a fallback for places in the JDK that use the value of native.encoding or sun.jnu.encoding. I think the only choices to either fail at startup or default to UTF-8 as proposed.

@naotoj
Copy link
Member Author

@naotoj naotoj commented Nov 6, 2021

Thanks, @AlanBateman for the explanation.
@takiguc, this is not part of JEP400, but really is a bug fix where the launcher fails to start with an NPE (on recent releases), which incorrectly called new String(byte[]) assuming it would default to iso-8859-1, by calling Charset.defaultCharset(). NPE is thrown because at that initPhase1, system property is yet to be initialized. The gist of the change is to replace it with new String(byte[], UTF_8) so that it does not call Charset.defaultCharset()`.

@takiguc
Copy link

@takiguc takiguc commented Nov 7, 2021

If this issue is not a part of JEP-400, I think UTF_8 may not be only solution for sun/nio/fs/Util.java.
Please explain why UTF_8 is better than ISO_8859_1.

@naotoj
Copy link
Member Author

@naotoj naotoj commented Nov 8, 2021

The same rationale as JEP 400, that UTF-8 is the de-facto encoding these days. This is for the exceptional case where JDK does not support the native encoding, choosing fallback as UTF-8 is not a bad choice IMO.

@AlanBateman
Copy link
Contributor

@AlanBateman AlanBateman commented Nov 8, 2021

If this issue is not a part of JEP-400, I think UTF_8 may not be only solution for sun/nio/fs/Util.java.
Please explain why UTF_8 is better than ISO_8859_1.

Just to add to Naoto's comment. When the JDK starts in unusual/unsupported configurations then it needs a default for the stdout/stderr print streams, for encoding/decoding files paths in java.io, and for encoding/decoding files paths in the java.nio.file implementation. It's important that the latter two are the same, having one be UTF-8 and the other be 8859-1 won't work.

There may be an argument that the JDK should emit a warning at startup.

@takiguc
Copy link

@takiguc takiguc commented Nov 8, 2021

Thanks @AlanBateman .
Yeah, I got a message from my co-worker.
He said unmappable conversion was still there even if ISO8859_1, and there was no big difference between ISO8859_1 and UTF_8.

I'm very sorry for your confusion...

Copy link

@mamadaliev mamadaliev left a comment

LGTM

@naotoj
Copy link
Member Author

@naotoj naotoj commented Nov 9, 2021

There may be an argument that the JDK should emit a warning at startup.

Updated to emit a warning, if sun.jnu.encoding is not supported.

@AlanBateman
Copy link
Contributor

@AlanBateman AlanBateman commented Nov 10, 2021

I tested your code on my CentOS7 x86_64. I saw following exception.

This doesn't surprise me. It would be be useful if you could do the same experiment with main line and JDK 17 too. There have been several reports of NPE when trying to startup in environments where the locale does not map to a charset that the JDK supports.

In your output I see "WARNING: The encoding of the underlying platform's file system is not supported by the JVM: PT154". Thereafter what you are seeing a recursive initialisation issue. To break this cycle we need jnuEncoding to be set before initialising the file system.

@naotoj
Copy link
Member Author

@naotoj naotoj commented Nov 10, 2021

@takiguc, thank you for trying out the fix and informing us of the issue. I've been using Oracle Linux 7.6 with the same locale setting as yours, but I've never seen that failure. Will need to recreate the error and investigate.

@takiguc
Copy link

@takiguc takiguc commented Nov 11, 2021

I don't have Oracle Linux...
I tried to find out unsupported locales by using following commands.
I used RHEL7.9.

$ cat CharsetIsSupported.java 
import java.nio.charset.Charset;

public class CharsetIsSupported {
    public static void main(String[] args) throws Exception {
        for(String s : args) {
            if (!Charset.isSupported(s))
                System.out.println(s);
        }
    }
}
$ locale -a | while read LOC; do env LANG=$LOC locale charmap; done | sort -u | xargs ~/jdk-17.0.1/bin/java CharsetIsSupported.java > encoding.lst
$ locale -a | while read LOC; do echo -e "$LOC\t\c"; env LANG=$LOC locale charmap; done | grep -f encoding.lst
cy_GB	ISO-8859-14
cy_GB.iso885914	ISO-8859-14
hy_AM.armscii8	ARMSCII-8
ka_GE	GEORGIAN-PS
ka_GE.georgianps	GEORGIAN-PS
kk_KZ	PT154
kk_KZ.pt154	PT154
lg_UG	ISO-8859-10
lg_UG.iso885910	ISO-8859-10
tg_TJ	KOI8-T
tg_TJ.koi8t	KOI8-T

I tried following command, "sun.jnu.encoding" system property was referred following source code.
We may need to care about these files.

$ find src -type f | xargs grep '"sun.jnu.encoding"'
src/java.base/share/classes/java/lang/System.java:        var jnuEncoding = props.getProperty("sun.jnu.encoding");
src/java.base/share/classes/java/lang/System.java:                    props.getProperty("sun.jnu.encoding"));
src/java.base/share/classes/jdk/internal/util/SystemProps.java:        put(props, "sun.jnu.encoding", raw.propDefault(Raw._sun_jnu_encoding_NDX));
src/java.base/share/classes/sun/launcher/LauncherHelper.java:    private static final String encprop = "sun.jnu.encoding";
src/java.base/share/classes/sun/nio/fs/Util.java:            GetPropertyAction.privilegedGetProperty("sun.jnu.encoding"),
src/java.base/share/native/libjava/jni_util.h:    NO_ENCODING_YET = 0,        /* "sun.jnu.encoding" not yet set */
src/java.desktop/share/classes/sun/font/SunFontManager.java:                    String sysEncoding = System.getProperty("sun.jnu.encoding");

@AlanBateman
Copy link
Contributor

@AlanBateman AlanBateman commented Nov 11, 2021

I don't have Oracle Linux... I tried to find out unsupported locales by using following commands. I used RHEL7.9.

In your previous post you used env LC_ALL=kk_KZ.pt154 java Hello.java. Would it be possible to repeat that with JDK 17 and the main-line (latest JDK 18 EA build is okay). I'm wondering if you see the NPE.

In any case, our next step here will be to have fallback saved during early startup so that file system and other places don't do the lookup. This seems the sanest way to avoid the recursive initialisation issue.

@takiguc
Copy link

@takiguc takiguc commented Nov 11, 2021

I could see following output on my CentOS7.

$ env LC_ALL=kk_KZ.pt154 ~/jdk-17.0.1/bin/java -showversion Hello.java 
Error occurred during initialization of VM
java.lang.NullPointerException
	at java.lang.System.getProperty(java.base/System.java:918)
	at sun.security.action.GetPropertyAction.privilegedGetProperty(java.base/GetPropertyAction.java:106)
	at java.nio.charset.Charset.defaultCharset(java.base/Charset.java:607)
	at java.lang.String.<init>(java.base/String.java:1412)
	at java.lang.String.<init>(java.base/String.java:1432)
	at jdk.internal.util.SystemProps$Raw.platformProperties(java.base/Native Method)
	at jdk.internal.util.SystemProps$Raw.<init>(java.base/SystemProps.java:234)
	at jdk.internal.util.SystemProps.initProperties(java.base/SystemProps.java:54)
	at java.lang.System.initPhase1(java.base/System.java:2089)

$ env LC_ALL=kk_KZ.pt154 ~/jdk-18-b22/bin/java -showversion Hello.java 
Error occurred during initialization of VM
java.lang.NullPointerException
	at java.lang.System.getProperty(java.base/System.java:929)
	at sun.security.action.GetPropertyAction.privilegedGetProperty(java.base/GetPropertyAction.java:106)
	at java.nio.charset.Charset.defaultCharset(java.base/Charset.java:646)
	at java.lang.String.<init>(java.base/String.java:1411)
	at java.lang.String.<init>(java.base/String.java:1431)
	at jdk.internal.util.SystemProps$Raw.platformProperties(java.base/Native Method)
	at jdk.internal.util.SystemProps$Raw.<init>(java.base/SystemProps.java:240)
	at jdk.internal.util.SystemProps.initProperties(java.base/SystemProps.java:54)
	at java.lang.System.initPhase1(java.base/System.java:2100)

@takiguc
Copy link

@takiguc takiguc commented Nov 11, 2021

I applied same kind of patch into jdk17u.
The situation was same.

$ env LC_ALL=kk_KZ.pt154 ./build/linux-x86_64-server-release/images/jdk/bin/java -showversion Hello.java 
WARNING: The encoding of the underlying platform's file system is not supported by the JVM: PT154
openjdk version "17.0.1-internal" 2021-10-19
OpenJDK Runtime Environment (build 17.0.1-internal+0-adhoc.jdktest.jdk17u)
OpenJDK 64-Bit Server VM (build 17.0.1-internal+0-adhoc.jdktest.jdk17u, mixed mode, sharing)
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.util.ServiceConfigurationError: java.nio.charset.spi.CharsetProvider: Unable to load sun.nio.cs.ext.ExtendedCharsets
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586)
...

@naotoj
Copy link
Member Author

@naotoj naotoj commented Nov 12, 2021

I reproduced those failures on Debian Linux. Corrected the issue by replacing unsupported jnu encoding with UTF-8 in System::initPhase1.

if (!jnuEncodingSupported) {
props.setProperty("sun.jnu.encoding", "UTF-8");
}

Copy link
Contributor

@AlanBateman AlanBateman Nov 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could replace the two fields with something like "notSupportedJnuEncoding" that is only set when not supported. That keeps the additional code in initPhase1 to a minimum.

Copy link
Member Author

@naotoj naotoj Nov 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced as suggested.

System.err.printf(
"WARNING: The encoding of the underlying platform's" +
" file system is not supported by the JVM: %s%n",
jnuEncoding);
Copy link
Contributor

@AlanBateman AlanBateman Nov 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can drop "by the JVM" from the warning.

Also just to point out that this is running in initPhase3. I'm pretty sure the use of formatters here will execute code that checks VM.isBooted. It might be better to just use string concatenation and avoid loading formatters here.

Copy link
Member Author

@naotoj naotoj Nov 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

// Check if sun.jnu.encoding is supporetd or not
var jnuEncoding = props.getProperty("sun.jnu.encoding");
if (jnuEncoding == null || !Charset.isSupported(jnuEncoding)) {
notSupportedJnuEncoding = jnuEncoding == null ? "null" : jnuEncoding;
Copy link
Contributor

@AlanBateman AlanBateman Nov 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are nearly there. When setting notSupportedJnuEncoding then we also need to set the sun.jnu.encoding property to "UTF-8", setting it in initPhase3 is too late because the file system may be used using initPhase2.

Copy link
Member Author

@naotoj naotoj Nov 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, I thought that was the case, but I mistook your suggestion as setting only notSupportedJnuEncoding in initPhase1. Reverted the fallback location into initPhase1().

Copy link
Contributor

@AlanBateman AlanBateman Nov 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, I thought that was the case, but I mistook your suggestion as setting only notSupportedJnuEncoding in initPhase1. Reverted the fallback location into initPhase1().

np, it's always tricky to touch code that executes early in the startup. I think we are good now.

@takiguc
Copy link

@takiguc takiguc commented Nov 15, 2021

@naotoj , sorry for bothering you again.
Still I got exception. It seems value "jnuEncoding" should be overwritten or set "fastEncoding" to "FAST_UTF_8" on jnu_util.c.
Could you check this part again ?

$ env LC_ALL=kk_KZ.pt154 ./build/linux-x86_64-server-release/images/jdk/bin/java Hello.java 
WARNING: The encoding of the underlying platform's file system is not supported: PT154
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.util.ServiceConfigurationError: java.nio.charset.spi.CharsetProvider: Provider sun.nio.cs.ext.ExtendedCharsets not found
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
	at java.base/java.util.ServiceLoader.loadProvider(ServiceLoader.java:875)
	at java.base/java.util.ServiceLoader$ModuleServicesLookupIterator.hasNext(ServiceLoader.java:1084)
	at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
	at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
	at java.base/java.nio.charset.Charset$ExtendedProviderHolder$1.run(Charset.java:428)
	at java.base/java.nio.charset.Charset$ExtendedProviderHolder$1.run(Charset.java:422)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
	at java.base/java.nio.charset.Charset$ExtendedProviderHolder.extendedProviders(Charset.java:422)
	at java.base/java.nio.charset.Charset$ExtendedProviderHolder.<clinit>(Charset.java:418)
	at java.base/java.nio.charset.Charset.lookupExtendedCharset(Charset.java:442)
	at java.base/java.nio.charset.Charset.lookup2(Charset.java:472)
	at java.base/java.nio.charset.Charset.lookup(Charset.java:460)
	at java.base/java.nio.charset.Charset.isSupported(Charset.java:501)
	at java.base/jdk.internal.loader.NativeLibraries.findBuiltinLib(Native Method)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:157)
	at java.base/jdk.internal.loader.NativeLibraries.findFromPaths(NativeLibraries.java:322)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:289)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:274)
	at java.base/jdk.internal.loader.BootLoader.loadLibrary(BootLoader.java:149)
	at java.base/sun.nio.fs.UnixNativeDispatcher.<clinit>(UnixNativeDispatcher.java:667)
	at java.base/sun.nio.fs.UnixFileSystem.<init>(UnixFileSystem.java:65)
	at java.base/sun.nio.fs.LinuxFileSystem.<init>(LinuxFileSystem.java:39)
	at java.base/sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:46)
	at java.base/sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:39)
	at java.base/sun.nio.fs.UnixFileSystemProvider.<init>(UnixFileSystemProvider.java:55)
	at java.base/sun.nio.fs.LinuxFileSystemProvider.<init>(LinuxFileSystemProvider.java:41)
	at java.base/sun.nio.fs.DefaultFileSystemProvider.<clinit>(DefaultFileSystemProvider.java:35)
	at java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:114)
	at java.base/java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:103)
	at java.base/java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:101)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
	at java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:101)
	at java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:94)
	at java.base/java.nio.file.FileSystems.getDefault(FileSystems.java:183)
	at java.base/java.nio.file.Path.of(Path.java:147)
	at java.base/java.nio.file.Paths.get(Paths.java:69)
	at java.base/jdk.internal.jimage.ImageReaderFactory.<clinit>(ImageReaderFactory.java:51)
	at java.base/jdk.internal.module.SystemModuleFinders$SystemImage.<clinit>(SystemModuleFinders.java:385)
	at java.base/jdk.internal.module.SystemModuleFinders$SystemModuleReader.findImageLocation(SystemModuleFinders.java:429)
	at java.base/jdk.internal.module.SystemModuleFinders$SystemModuleReader.read(SystemModuleFinders.java:483)
	at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:809)
	at java.base/jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(BuiltinClassLoader.java:741)
	at java.base/jdk.internal.loader.BuiltinClassLoader.findClass(BuiltinClassLoader.java:621)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:633)
	at java.base/java.lang.Class.forName(Class.java:577)
	at java.base/java.lang.Class.forName(Class.java:552)
	at java.base/sun.launcher.LauncherHelper.loadModuleMainClass(LauncherHelper.java:748)
	at java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:683)

@naotoj
Copy link
Member Author

@naotoj naotoj commented Nov 15, 2021

Fallback to UTF-8 in jni_uti.c::InitializeEncoding() as well, if the jnu encoding is not supported. @takiguc, would you try the changeset on your environment?

@takiguc
Copy link

@takiguc takiguc commented Nov 16, 2021

Hello @naotoj .
Test results are as follows:

$ env LC_ALL=kk_KZ.pt154 ./build/linux-x86_64-server-release/images/jdk/bin/java Hello.java
WARNING: The encoding of the underlying platform's file system is not supported: PT154
Hello
$ env LC_ALL=kk_KZ.pt154 ./build/linux-x86_64-server-release/images/jdk/bin/java -XshowSettings:properties Hello.java 2>&1 | grep encoding
WARNING: The encoding of the underlying platform's file system is not supported: PT154
    file.encoding = UTF-8
    native.encoding = PT154
    sun.io.unicode.encoding = UnicodeLittle
    sun.jnu.encoding = UTF-8
$ env LC_ALL=kk_KZ.pt154 ./build/linux-x86_64-server-release/images/jdk/bin/java -Dfile.encoding=COMPAT Hello.java
WARNING: The encoding of the underlying platform's file system is not supported: PT154
Hello
$ env LC_ALL=kk_KZ.pt154 ./build/linux-x86_64-server-release/images/jdk/bin/java -XshowSettings:properties -Dfile.encoding=COMPAT Hello.java 2>&1 | grep encoding
WARNING: The encoding of the underlying platform's file system is not supported: PT154
    file.encoding = PT154
    native.encoding = PT154
    sun.io.unicode.encoding = UnicodeLittle
    sun.jnu.encoding = UTF-8
$ ./build/linux-x86_64-server-release/images/jdk/bin/java Hello.java
Hello
$ ./build/linux-x86_64-server-release/images/jdk/bin/java -XshowSettings:properties Hello.java 2>&1 | grep encoding
    file.encoding = UTF-8
    native.encoding = UTF-8
    sun.io.unicode.encoding = UnicodeLittle
    sun.jnu.encoding = UTF-8

These were what I expected.
Many thanks.

private static final Charset jnuEncoding =
Charset.forName(
GetPropertyAction.privilegedGetProperty("sun.jnu.encoding"),
sun.nio.cs.UTF_8.INSTANCE);
Copy link
Contributor

@AlanBateman AlanBateman Nov 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Startup will ensure that sun.jnu.encoding has a value that is a supported encoding so I assume the the fallback in sun.nio.ch.Util is no longer needed.

Copy link
Member Author

@naotoj naotoj Nov 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted the fix.

return 0;
}
(*env)->GetByteArrayRegion(env, hab, 0, len, (jbyte *)result);
result[len] = 0; /* NULL-terminate */
Copy link
Contributor

@AlanBateman AlanBateman Nov 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change to newSizedStringJava is okay but I think the change to getStringBytes needs more eyes. Will it fall through to the code path sets it to FAST_UTF_8 during startup or does that happen later? If later then I assume there is a race with other threads that read jnuEncoding and potential for the JNI global ref to be freed from under their feet.

Copy link
Member Author

@naotoj naotoj Nov 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Removed the premature jnuEncoding initialization in InitializeEncoding().

Eliminated premature jnuEncoding initializaiton in jni_util.c::InitializeEncoding()
Copy link
Contributor

@AlanBateman AlanBateman left a comment

Thanks for the update, the latest version looks good.

@naotoj
Copy link
Member Author

@naotoj naotoj commented Nov 18, 2021

/integrate

@openjdk
Copy link

@openjdk openjdk bot commented Nov 18, 2021

Going to push as commit b8453eb.
Since your change was applied there have been 196 commits pushed to the master branch:

  • 231fb61: 8276970: Default charset for PrintWriter that wraps PrintStream
  • 29e552c: 8272358: Some tests may fail when executed with other locales than the US
  • ce4471f: 8277346: ProblemList 7 serviceability/sa tests on macosx-x64
  • 45a60db: 8277045: G1: Remove unnecessary set_concurrency call in G1ConcurrentMark::weak_refs_work
  • 6bb0462: 8277224: sun.security.pkcs.PKCS9Attributes.toString() throws NPE
  • d8c0280: 8277316: ciReplay: dump_replay_data is not thread-safe
  • 007ad7c: 8277303: Terminology mismatch between JLS17-3.9 and SE17's javax.lang.model.SourceVersion method specs
  • 8881f29: 8277310: ciReplay: @CPI MethodHandle references not resolved
  • 262d070: 8277246: Check for NonRepudiation as well when validating a TSA certificate
  • a907b2b: 8276177: nsk/jvmti/RedefineClasses/StressRedefineWithoutBytecodeCorruption failed with "assert(def_ik->is_being_redefined()) failed: should be being redefined to get here"
  • ... and 186 more: https://git.openjdk.java.net/jdk/compare/7115892f96a5a57ce9d37602038b787d19da5d81...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Nov 18, 2021
@openjdk openjdk bot added integrated and removed ready rfr labels Nov 18, 2021
@openjdk
Copy link

@openjdk openjdk bot commented Nov 18, 2021

@naotoj Pushed as commit b8453eb.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-libs integrated nio
7 participants