JEP 371 Hidden Classes #9328

pshipton · 2020-04-22T20:53:51Z

https://openjdk.java.net/jeps/371

This is delivered to OpenJDK now, affecting OpenJ9 builds of jdk15.

pshipton · 2020-04-22T20:56:31Z

@DanHeidinga fyi.
I'm attempting to stub enough to build in #9292

pshipton · 2020-04-23T00:34:09Z

@DanHeidinga stubbing isn't sufficient, hidden classes are used at runtime.

Exception in thread "main" java/lang/InternalError: java.lang.NullPointerException
	at java/lang/invoke/InnerClassLambdaMetafactory.spinInnerClass (java.base@9/InnerClassLambdaMetafactory.java:357)
	at java/lang/invoke/InnerClassLambdaMetafactory.buildCallSite (java.base@9/InnerClassLambdaMetafactory.java:209)
	at java/lang/invoke/LambdaMetafactory.metafactory (java.base@9/LambdaMetafactory.java:328)
	at java/lang/invoke/MethodHandle.invokeBsm (java.base@9/MethodHandle.java:1076)
	at java/lang/invoke/MethodHandle.resolveInvokeDynamic (java.base@9/MethodHandle.java:1200)
	at jdk/internal/module/SystemModuleFinders$1.find (java.base@9/SystemModuleFinders.java:215)
	at jdk/internal/module/ModuleBootstrap.boot (java.base@9/ModuleBootstrap.java:230)
	at java/lang/ClassLoader.initializeClassLoaders (java.base@9/ClassLoader.java:211)
	at java/lang/Thread.initialize (java.base@9/Thread.java:430)
	at java/lang/Thread.<init> (java.base@9/Thread.java:155)

hangshao0 · 2020-04-28T20:09:24Z

I can pick this up.

DanHeidinga · 2020-04-28T20:11:15Z

Thanks @hangshao0. As mentioned on slack, I think the first step is to try and stub these apis with Unsafe.defineAnonyousClass() so we can build 15 again. Then look at a proper implementation

hangshao0 · 2020-05-01T21:32:20Z

I plan to take the change in #9292 and start from there. But it seems that I cannot build with the latest code in branch open-staging from https://github.com/ibmruntimes/openj9-openjdk-jdk.

https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDKnext_x86-64_linux_Personal/425/console

error: warnings found and -Werror specified
...
....
16:32:06  CompileJavaModules.gmk:607: recipe for target '/home/jenkins/workspace/Build_JDKnext_x86-64_linux_Personal/build/linux-x86_64-server-release/jdk/modules/java.base/_the.java.base_batch' failed
16:32:06  make/Main.gmk:189: recipe for target 'java.base-java' failed
16:32:06  === End of repeated output ===

The message error: warnings found and -Werror specified is not there in the passing builds launched by other people, which I believe are on open branch.

I tried manually with --disable-warnings-as-errors, and it does not help. I noticed there are large number of make file changes this week into openj9-staging (e.g. ibmruntimes/openj9-openjdk-jdk@109f6a5, ibmruntimes/openj9-openjdk-jdk@6b2547e), might be related to the build issue.

pshipton · 2020-05-01T22:37:21Z

There are 100+ warnings like the following, which aren't there in the last working build. I assume the compiler has changed. To fix it you can move final class ThunkTable, abstract class Comparator, final class ThunkTuple, and anything that shows up, out from MethodHandle.java to their own source files.

[2020-05-01T20:22:05.583Z] /home/jenkins/workspace/Build_JDKnext_x86-64_linux_Personal/build/linux-x86_64-server-release/support/j9jcl_sources/java.base/share/classes/java/lang/invoke/InvokeExactHandle.java:68: warning: auxiliary class ThunkTable in /home/jenkins/workspace/Build_JDKnext_x86-64_linux_Personal/build/linux-x86_64-server-release/support/j9jcl_sources/java.base/share/classes/java/lang/invoke/MethodHandle.java should not be accessed from outside its own source file
[2020-05-01T20:22:05.583Z]      private static final ThunkTable _thunkTable = new ThunkTable();

Otherwise to get started you can build from sha fed4f3e. It's the level I was using that was working up to the Optimizing the exploded image step, and then gets the exception in #9328 (comment)

pshipton · 2020-05-01T23:19:39Z

Ok, it's not that simple. I moved those classes into their own files, but there are many other warnings, and error: warnings found and -Werror specified still occurs. I suspect we removed -Werror previously but this change is lost and we need to restore it.

pshipton · 2020-05-02T00:47:22Z

This is the new code

https://github.com/ibmruntimes/openj9-openjdk-jdk/blob/dd30cb1200bd9d5a11c8ea9efdfdf63756a3d4fc/make/common/JavaCompilation.gmk#L242-#L243

While we figure out the best way to override that, export JAVA_WARNINGS_ARE_ERRORS= gets past that problem. Then the next problem is

Running ddrgen to generate j9ddr.dat and superset.dat
Blob written to file: ../j9ddr.dat
Superset written to file: ../superset.dat
make[4]: Leaving directory '/home/pete/openj9-openjdk-jdk/build/linux-x86_64-server-release/vm/ddr'
Compiling 89 properties into resource bundles for java.desktop
Creating support/modules_libs/java.base/libverify.so from 1 file(s)
Creating support/modules_libs/java.base/libjava.so from 66 file(s)
Creating support/native/java.base/libfdlibm.a from 57 file(s)
Creating support/modules_libs/java.base/libzip.so from 5 file(s)
Creating support/modules_libs/java.base/libjimage.so from 6 file(s)
Creating support/modules_libs/java.base/libjli.so from 8 file(s)
Creating support/modules_libs/java.base/libnet.so from 21 file(s)
Creating support/modules_libs/java.base/libnio.so from 20 file(s)
Updating support/src.zip
Compiling 2781 files for java.desktop
/usr/bin/ld: cannot find -ljvm
collect2: error: ld returned 1 exit status

pshipton · 2020-05-02T01:32:56Z

Here is a fix for the -Werror problem ibmruntimes/openj9-openjdk-jdk#201

andrew-m-leonard · 2020-05-04T11:31:49Z

jdkhead merge job test build failing due to this, various errors:

03:39:39  /home/jenkins/workspace/Build_JDKnext_x86-64_linux_Personal/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java:45: error: static import only from classes and interfaces
03:39:39  import static java.lang.invoke.MethodHandles.Lookup.ClassOption.NESTMATE;
03:39:39  ^
03:39:39  /home/jenkins/workspace/Build_JDKnext_x86-64_linux_Personal/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java:46: error: cannot find symbol
03:39:39  import static java.lang.invoke.MethodHandles.Lookup.ClassOption.STRONG;
03:39:39                                                     ^
03:39:39    symbol:   class ClassOption
03:39:39    location: class Lookup

keithc-ca · 2020-05-07T15:50:39Z

@hangshao0 I cherry-picked the commit at the head of your JEP371_Test branch and merged it with #9292 and #9471 and was able to build the openj9-staging branch of jdknext:

$ ./images/jdk/bin/java -version

openjdk version "15-internal" 2020-09-15
OpenJDK Runtime Environment (build 15-internal+0-adhoc.keithc.jdk15)
Eclipse OpenJ9 VM (build h371b-6e8c206915a, JRE 15 Linux amd64-64-Bit Compressed References 20200507_000000 (JIT enabled, AOT enabled)
OpenJ9   - 6e8c206915a
OMR      - b22bd644869
JCL      - f33985b9f78 based on jdk-15+21)

Do you anticipate more work is required, or is it time for you to open a pull request?

hangshao0 · 2020-05-07T15:57:10Z

Yes, I am about the open the PR. It seems that your machine finished building faster than mine.

Add implementation of new Hidden class APIs to unblock the JDK15 builds. issue eclipse-openj9#9328 Signed-off-by: Hang Shao <hangshao@ca.ibm.com>

1. Add checks for class bytes for hidden classes in MethodHanles. 2. Do bytecode verification for hidden classes. 3. Do not allow record and enum to be defined as hidden classes. 4. Fix APIs for hidden classes in Class.java. Closes eclipse-openj9#9328 Signed-off-by: Hang Shao <hangshao@ca.ibm.com>

pshipton added JEP jdk15 labels Apr 22, 2020

pshipton added the blocker label Apr 23, 2020

pshipton added this to the Release 0.22 (Java 15) milestone Apr 23, 2020

DanHeidinga assigned hangshao0 Apr 28, 2020

pshipton mentioned this issue May 2, 2020

JAVAC_WARNINGS is replaced by JAVA_WARNINGS_ARE_ERRORS ibmruntimes/openj9-openjdk-jdk#201

Merged

keithc-ca mentioned this issue May 6, 2020

Update makefiles for recent openjdk changes ibmruntimes/openj9-openjdk-jdk#202

Merged

hangshao0 mentioned this issue May 7, 2020

Java 15 Use Unsafe.defineAnonyousClass() to define Hidden classes. #9485

Merged

DanHeidinga closed this as completed in #10471 Sep 3, 2020

hangshao0 mentioned this issue Sep 3, 2020

(v0.22.0) Hidden class implementation 4 #10517

Closed

hangshao0 mentioned this issue Sep 3, 2020

(v0.22.0) Hidden class implementation 4 #10520

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JEP 371 Hidden Classes #9328

JEP 371 Hidden Classes #9328

pshipton commented Apr 22, 2020

pshipton commented Apr 22, 2020

pshipton commented Apr 23, 2020 •

edited

hangshao0 commented Apr 28, 2020

DanHeidinga commented Apr 28, 2020

hangshao0 commented May 1, 2020 •

edited

pshipton commented May 1, 2020 •

edited

pshipton commented May 1, 2020

pshipton commented May 2, 2020

pshipton commented May 2, 2020

andrew-m-leonard commented May 4, 2020

keithc-ca commented May 7, 2020

hangshao0 commented May 7, 2020

JEP 371 Hidden Classes #9328

JEP 371 Hidden Classes #9328

Comments

pshipton commented Apr 22, 2020

pshipton commented Apr 22, 2020

pshipton commented Apr 23, 2020 • edited

hangshao0 commented Apr 28, 2020

DanHeidinga commented Apr 28, 2020

hangshao0 commented May 1, 2020 • edited

pshipton commented May 1, 2020 • edited

pshipton commented May 1, 2020

pshipton commented May 2, 2020

pshipton commented May 2, 2020

andrew-m-leonard commented May 4, 2020

keithc-ca commented May 7, 2020

hangshao0 commented May 7, 2020

pshipton commented Apr 23, 2020 •

edited

hangshao0 commented May 1, 2020 •

edited

pshipton commented May 1, 2020 •

edited