Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8247666: Support Lambda proxy classes in static CDS archive #364

Closed
wants to merge 14 commits into from

Conversation

calvinccheung
Copy link
Member

@calvinccheung calvinccheung commented Sep 25, 2020

Following up on archiving lambda proxy classes in dynamic CDS archive (JDK-8198698), this RFE
adds the functionality of archiving of lambda proxy classes in static CDS archive.

When the -XX:DumpLoadedClassList is enabled, the constant pool index related to LambdaMetafactory that are resolved during application execution will be included in the classlist.
The entry for a lambda proxy class in a class list will be of the following format:

@lambda-proxy: <classname> <cp index>

e.g.
@lambda-proxy: test/java/lang/invoke/MethodHandlesGeneralTest 233
@lambda-proxy: test/java/lang/invoke/MethodHandlesGeneralTest 355

When dumping a CDS archive using the -Xshare:dump and -XX:ExtraSharedClassListFile options, when the above @lambda-proxy entry is encountered while parsing the classlist, we will resolve the corresponding constant pool indices (233 and 355 in the above example). As a result, lambda proxy classes will be generated for the constant pool entries, and will be cached using a similar mechanism to JDK-8198698.

During dumping, there is check on the cp index and on the created BootstrapInfo using the cp index. VM will exit with an error message if the check has failed.

During runtime when looking up a lambda proxy class, the lookup will be perform on the static CDS archive and if not found, then lookup from the dynamic archive if one is specified.

(Only name change (IsDynamicDumpingEnabled -> IsCDSDumpingEnabled) is involved in the core-libs code.)

Testing: tiers 1,2,3,4.

Performance results (javac on HelloWorld on linux-x64):
Results of " perf stat -r 40 bin/javac -J-Xshare:on -J-XX:SharedArchiveFile=javac2.jsa Bench_HelloWorld.java "
   1:   2228016795  2067752708 (-160264087)      -----    377.760   349.110 (-28.650)      -----
   2:   2223051476  2063016483 (-160034993)      -----    374.580   350.620 (-23.960)      ----
   3:   2225908334  2067673847 (-158234487)      -----    375.220   350.990 (-24.230)      ----
   4:   2225835999  2064596883 (-161239116)      -----    374.670   349.840 (-24.830)      ----
   5:   2226005510  2061694332 (-164311178)      -----    373.512   351.120 (-22.392)      ----
   6:   2225574949  2062657482 (-162917467)      -----    374.710   348.380 (-26.330)      -----
   7:   2224702424  2064634122 (-160068302)      -----    373.670   349.510 (-24.160)      ----
   8:   2226662277  2066301134 (-160361143)      -----    375.350   349.790 (-25.560)      ----
   9:   2226761470  2063162795 (-163598675)      -----    374.260   351.290 (-22.970)      ----
  10:   2230149089  2066203307 (-163945782)      -----    374.760   350.620 (-24.140)      ----
============================================================
        2226266109  2064768307 (-161497801)      -----    374.848   350.126 (-24.722)      ----
instr delta =   -161497801    -7.2542%
time  delta =      -24.722 ms -6.5951%

Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8247666: Support Lambda proxy classes in static CDS archive

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/364/head:pull/364
$ git checkout pull/364

@calvinccheung
Copy link
Member Author

/label remove hotspot

@calvinccheung
Copy link
Member Author

/label add hotspot-runtime,core-libs

@calvinccheung calvinccheung marked this pull request as ready for review September 25, 2020 18:00
@bridgekeeper
Copy link

bridgekeeper bot commented Sep 25, 2020

👋 Welcome back ccheung! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Sep 25, 2020

@calvinccheung The hotspot label was not set.

@openjdk openjdk bot added hotspot-runtime hotspot-runtime-dev@openjdk.org core-libs core-libs-dev@openjdk.org labels Sep 25, 2020
@openjdk
Copy link

openjdk bot commented Sep 25, 2020

@calvinccheung
The hotspot-runtime label was successfully added.

The core-libs label was successfully added.

@openjdk openjdk bot added the rfr Pull request is ready for review label Sep 25, 2020
@mlbridge
Copy link

mlbridge bot commented Sep 25, 2020

Copy link
Member

@iklam iklam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed a small part of the changes. Here are my initial comments

src/hotspot/share/classfile/systemDictionaryShared.cpp Outdated Show resolved Hide resolved
src/hotspot/share/classfile/systemDictionaryShared.cpp Outdated Show resolved Hide resolved
src/hotspot/share/classfile/systemDictionaryShared.cpp Outdated Show resolved Hide resolved
Copy link
Member

@mlchung mlchung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lambda-proxy: test/java/lang/invoke/MethodHandlesGeneralTest 233
@lambda-proxy: test/java/lang/invoke/MethodHandlesGeneralTest 355

It seems very fragile to require listing the CP index to invokedynamic entries of a class file. Have you considered a simpler usage model without CP indices and default to all invokedynamic to LambdaMetaFactory?

*/
public static native boolean isDynamicDumpingEnabled(); // will return false for static dumping.
public static native boolean isCDSDumpingEnabled();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to rename CDS::isCDSDumpingEnabled to CDS:isDumpingEnabled as this method is a static method in CDS case and the word CDS in the method name is just noise.

JVM entry point JVM_IsCDSDumpingEnabled is good.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in webrev 03.

JVMWrapper("JVM_IsDynamicDumpingEnable");
return DynamicDumpSharedSpaces;
JVM_ENTRY(jboolean, JVM_IsCDSDumpingEnabled(JNIEnv* env))
JVMWrapper("JVM_IsCDSDumpingEnable");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: missing d s/Enable/Enabled/

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in webrev 03.

…a-proxy in the classlist:

@lambda-proxy: LambHello run ()Ljava/lang/Runnable; ()V 6 LambHello lambdabash ()V ()V
2. Removed BadCPIndex.java; added LambdaProxyClassList.java test.
3. Mandy's comments.
@calvinccheung
Copy link
Member Author

Hi Mandy,
Thanks for your review.
We've tried the approach of archiving all the lambda proxy classes of a given class but we feel that there will be too many unused lambda proxy classes in the archive and it will also increase the CDS archive size.
So we opted for a little more complicated approach by storing the symbolic encoding lambda proxy resolution in the classlist. During dumping, the same info will be retrieved from the constant pool entry and compared with the one from the classlist. An example of the entry in the classlist is:
@lambda-proxy: LambHello run ()Ljava/lang/Runnable; ()V 6 LambHello lambda$main$0 ()V ()V
With the above approach, the number of lambda proxy classes the default CDS archive is 67.
If we archive all the lambda proxy classes of a given class, the number is 279.
Webrev 03 contains the above change and also addresses your other comments.
Thanks!
Calvin

#include "classfile/resolutionErrors.hpp"
#include "classfile/symbolTable.hpp"
#include "classfile/systemDictionary.hpp"
#include "classfile/systemDictionaryShared.hpp"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all the new includes necessary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those new includes are not needed. I've removed them.

classlist_file->print_cr("");
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if two threads try call ArchiveUtils::log_to_classlist at the same time, the output may be interleaved.

classlist_file is a fileStream, which uses fwrite to write to the file. In theory, if you write the whole line at once, the output should be thread safe (at least on POSIX and Windows). But in your case, you would need to first get the whole line into a buffer, and then write it out at once.

I think it would be safer, and more convenient, to use a lock to ensure thready safety. Maybe we can convert all uses of classlist_file->print() to something like

class ClassListWriter {
    static fileStream* _classlist_file;
    MutexLocker locker;
public:
   outputStream* stream() {
       return _classlist_file;
   }
   static bool is_enabled() {
     return _classlist_file != NULL && _classlist_file->is_open();
   }
   ClassListWriter() :  locker(Thread::current(), ClassListFile_lock) {}

   static void init() {
     // classlist_file init code from ostrea.cpp
   }
};

// WAS if (DumpLoadedClassList != NULL && classlist_file->is_open()) {
if (ClassListWriter::is_enabled()) {
  ClassListWriter w;
  w->stream()->print("aaaa");
  w->stream()->print("bbbb");
  w->stream()->cr();
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the ClassListWriter.hpp and changed other classes which operate on the classlist.

@@ -4208,6 +4208,11 @@ void InstanceKlass::log_to_classlist(const ClassFileStream* stream) const {
bool skip = false;
if (is_shared()) {
assert(stream == NULL, "shared class with stream");
if (is_hidden()) {
// Not including archived lambda proxy class in the classlist.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's clearer to say // Don't include archived lambda proxy class in the classlist.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

*/
public static native boolean isDynamicDumpingEnabled(); // will return false for static dumping.
public static native boolean isDumpingEnabled();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will be more consistent if we use the same pattern as CDS::isDumpingClassList()

    private static final boolean isDumpingArchive;
    static {
        isDumpingClassList = isDumpingArchive0();
    }

    /**
      * Is the VM writing to a (static or dynamic) CDS archive.
      */
    public static boolean isDumpingArchive() {
        return isDumpingArchive;
    }

Then in LambdaProxyClassArchive.java, there's no need to keep a separate variable of dumpArchive. You can simply call CDS.isDumpingArchive(). The JIT compiler will automatically inline the call so it will be just as fast.

LambdaProxyClassArchive::sharingEnabled should also be rewritten to use this pattern.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the API's in CDS.java to be consistent with CDS.isDumpingClasslist.

@mlchung
Copy link
Member

mlchung commented Oct 14, 2020

@lambda-proxy: LambHello run ()Ljava/lang/Runnable; ()V 6 LambHello lambda$main$0 ()V ()V

Can you explain the format?

@calvinccheung
Copy link
Member Author

@lambda-proxy LambHello run ()Ljava/lang/Runnable; ()V REF_invokeStatic LambHello lambda$main$0 ()V ()V
means
@lambda-proxy <classname> <intf-method-name> <intf-method-sig> <bsm-arg0> <bsm-arg1a> <bsm-arg1b>  <bsm-arg1c>  <bsm-arg1d> <bsm-arg2>

It is a symbolic representation of a invoke dynamic constant pool entry.

public class LambHello {
    public static void main(String[] args) {
        doit(() -> {
            System.out.println("Hello from Lambda");
        });
    }
    static void doit(Runnable t) {
        t.run();
    }
}

An invoke dynamic constant pool of the above program is:
- 7 : InvokeDynamic : bootstrap_method_index=43 name_and_type_index=8 arguments={50, 51, 50}
Other constant pool entries related to the above are:

 -   8 : NameAndType : name_index=9 signature_index=10
 -   9 : Utf8 : 'run'
 -  10 : Utf8 : '()Ljava/lang/Runnable;'

 -  50 : MethodType : signature_index=6
 -  51 : MethodHandle : ref_kind=6 ref_index=52
 -  52 : Method : klass_index=12 name_and_type_index=53
 -  53 : NameAndType : name_index=39 signature_index=6

 -   6 : Utf8 : '()V'
 -  12 : Class : 'LambHello' {0x0000000800c10040}
 -  39 : Utf8 : 'lambda$main$0'

The info included in the class list are:

    <classname>                  = LambHello
    <intf-method-name>     = run
    <intf-method-sig>         = ()Ljava/lang/Runnable;
    <bsm-arg0>                   = ()V
    <bsm-arg1a>                  = REF_invokeStatic
    <bsm-arg1b>                  = LambHello
    <bsm-arg1c>                  = lambda$main$0
    <bsm-arg1d>                  = ()V
    <bsm-arg2>                   = ()V

1. Added the ClassListWriter class to make writing to a classlist file thread safe;
2. Changed API's in CDS.java to be more consistent with CDS.isDumpingClasslist.
@openjdk
Copy link

openjdk bot commented Oct 15, 2020

@calvinccheung this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout 8247666
git fetch https://git.openjdk.java.net/jdk master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

@openjdk openjdk bot added the merge-conflict Pull request has merge conflict with target branch label Oct 15, 2020
@openjdk openjdk bot removed the merge-conflict Pull request has merge conflict with target branch label Oct 15, 2020
2. Enclose some debug statements with DEBUG_ONLY.
@mlchung
Copy link
Member

mlchung commented Oct 16, 2020

Since the class list file is not intended for users to edit/modify but rather a configuration file given when -Xshare:dump is used, the format of @lambda-proxy entry is internal implementation details and so it's fine.

BTW: for the method handle reference kind, does the entry have the value (e.g. 6) or the name ("REF_invokeStatic")? I see both formats are described in your different replies.

@lambda-proxy: LambHello run ()Ljava/lang/Runnable; ()V 6 LambHello lambda$main$0 ()V ()V
or
@lambda-proxy LambHello run ()Ljava/lang/Runnable; ()V REF_invokeStatic LambHello lambda$main$0 ()V ()V

@calvinccheung
Copy link
Member Author

For the reference kind, the first format (with number '6') will be stored in the classlist.

Copy link
Member

@mlchung mlchung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the java.base changes and have no additional comment.

@openjdk
Copy link

openjdk bot commented Oct 19, 2020

@calvinccheung This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8247666: Support Lambda proxy classes in static CDS archive

Reviewed-by: iklam, mchung

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 37 new commits pushed to the master branch:

  • e2e11d3: 8254955: x86: MethodHandlesAdapterBlob is too big
  • 0b51016: 8253660: Need better error report when artifact resolution fails in AotCompiler.java
  • 52cb329: 8254862: lldb in devkit doesn't work
  • 60f63ec: 8254796: Cleanup pervasive unnecessary parameter
  • 953e472: 8254967: com.sun.net.HttpsServer spins on TLS session close
  • 1da28de: 8255009: delta apply fixes for JDK-8246774 and JDK-8253455, pushed too soon
  • a0382cd: 8253235: JFR.dump does not respect maxage parameter
  • cb7701b: 8253970: Build error: address argument to atomic builtin must be a pointer to integer or pointer ('volatile narrowOop *' invalid)
  • 4ffed32: 8254940: AArch64: Cleanup non-product thread members
  • cd66e0f: 8253877: gc/g1/TestGCLogMessages.java fails - missing "Evacuation failure" message
  • ... and 27 more: https://git.openjdk.java.net/jdk/compare/546620bb9eebeed6fb74867f878f666b0fcf89b2...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Oct 19, 2020
Copy link
Member

@iklam iklam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@calvinccheung
Copy link
Member Author

@iklam and @mlchung, thanks again for the review and discussions.

@calvinccheung
Copy link
Member Author

/integrate

@openjdk openjdk bot closed this Oct 19, 2020
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Oct 19, 2020
@openjdk
Copy link

openjdk bot commented Oct 19, 2020

@calvinccheung Since your change was applied there have been 37 commits pushed to the master branch:

  • e2e11d3: 8254955: x86: MethodHandlesAdapterBlob is too big
  • 0b51016: 8253660: Need better error report when artifact resolution fails in AotCompiler.java
  • 52cb329: 8254862: lldb in devkit doesn't work
  • 60f63ec: 8254796: Cleanup pervasive unnecessary parameter
  • 953e472: 8254967: com.sun.net.HttpsServer spins on TLS session close
  • 1da28de: 8255009: delta apply fixes for JDK-8246774 and JDK-8253455, pushed too soon
  • a0382cd: 8253235: JFR.dump does not respect maxage parameter
  • cb7701b: 8253970: Build error: address argument to atomic builtin must be a pointer to integer or pointer ('volatile narrowOop *' invalid)
  • 4ffed32: 8254940: AArch64: Cleanup non-product thread members
  • cd66e0f: 8253877: gc/g1/TestGCLogMessages.java fails - missing "Evacuation failure" message
  • ... and 27 more: https://git.openjdk.java.net/jdk/compare/546620bb9eebeed6fb74867f878f666b0fcf89b2...master

Your commit was automatically rebased without conflicts.

Pushed as commit 74ac77e.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@calvinccheung calvinccheung deleted the 8247666 branch October 19, 2020 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-libs core-libs-dev@openjdk.org hotspot-runtime hotspot-runtime-dev@openjdk.org integrated Pull request has been integrated
3 participants