-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
8247536: Support for pre-generated java.lang.invoke classes in CDS static archive #193
Conversation
👋 Welcome back minqi! A progress list of the required criteria for merging this PR into |
|
@yminqi The following labels will be automatically applied to this pull request: When this pull request is ready to be reviewed, an RFR email will be sent to the corresponding mailing lists. If you would like to change these labels, use the |
Webrevs
|
Mailing list message from Mandy Chung on build-dev: src/java.base/share/classes/java/lang/invoke/GenerateJLIClassesHelper.java generateHolderClasses should never return null and so line 371-377 can 379???????????? Object[] ret_array = new Object[size * 2]; Rename `ret_array` to retArray to follow the naming convention using camel case. src/java.base/share/classes/jdk/internal/misc/VM.java This should be cached in the caller who checks if -XX:+DumpLoadedClassList Since there are many CDS-related methods, I think it's time to introduce I suggest to add CDS:logTraceResolve(String line) method that 493 * Output to DumpLoadedClassList, format is simimar to LF_RESOLVE s/simimar/similar 494 * @see InvokerBytecodeGenerator @see is typically placed after @param. Should it say @see java.lang.invoke.GenerateJLIClassesHelper::traceLambdaForm Mandy On 9/15/20 12:15 PM, Yumin Qi wrote: |
// exclude the existing class from dump | ||
// find_class assert on SystemDictionary_lock or safepoint | ||
MutexLocker lock(SystemDictionary_lock); | ||
InstanceKlass* old = SystemDictionary::find_class(class_name, cld); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no need to call find_class
here, since it will return the same class as klass
on line 117.
@@ -1870,7 +1871,8 @@ InstanceKlass* SystemDictionary::find_class(Symbol* class_name, ClassLoaderData* | |||
VerifyDuringGC || | |||
VerifyBeforeExit || | |||
VerifyDuringStartup || | |||
VerifyAfterGC, "too expensive"); | |||
VerifyAfterGC || | |||
DumpSharedSpaces, "too expensive"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may not be needed if you remove the find_class() call from LambdaFormInvokers::regenerate_holder_classes?
} | ||
if (VM.isDumpLoadedClassListSetAndOpen) { | ||
VM.cdsTraceResolve(traceLF); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GenerateJLIClassesHelper shouldn't need to know why the trace is needed. Also, "cdsTraceResolve" is too generic.
I think it's better to have
if (TRACE_RESOLVE || VM.CDS_TRACE_JLINV_RESOLVE) {
...
VM.cdsTraceJLINVResolve(traceLF);
The acronym JLINV is used in methodHandles.cpp
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With CDS related code moved to CDS.java, I think we should keep TRACE_RESOLVE here. A new name like suggested by Mandy, logTraceResolve in CDS.java
* Check if DumpLoadedClassList is set and the file is open. | ||
*/ | ||
public static boolean isDumpLoadedClassListSetAndOpen; | ||
private static native boolean isDumpLoadedClassListSetAndOpen0(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest to rename to isDumpingLoadedClassList
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will change.
Will change access to 'private'
Will drop the check and add Objects.requireNonNull(traces).
Created a separate issue https://bugs.openjdk.java.net/browse/JDK-8253208 to move CDS method to CDS.java
Right, will change that to the suggestion.
|
@yminqi this pull request can not be integrated into git checkout jdk-8247536
git fetch https://git.openjdk.java.net/jdk master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push |
"@lambda-form-invoker [LF_RESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeNothing L7_L (success)", | ||
"@lambda-form-invoker [LF_RESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeNothing LL_I (success)"), | ||
"@lambda-form-invoker [LF_RESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeNothing L7_L (anyword)", | ||
"@lambda-form-invoker [LF_RESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeNothing LL_I anyword"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We shouldn't allow the classlist to contain arbitrary data. These two cases should generate an error.
@@ -48,6 +48,16 @@ | |||
|
|||
GrowableArray<char*>* LambdaFormInvokers::_lambdaform_lines = NULL; | |||
|
|||
// trim white spaces from front and end of string. | |||
char* trim(char* s) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this creates unnecessary dependency between the C code and the Java code. The C code assumes that the Java code has appended something like "(salvaged)" into the output, and tries to get rid of that in a non-obvious way. It's better to modify the Java code from
static void traceSpeciesType(String cn, Class<?> salvage) {
if (TRACE_RESOLVE || CDS.isDumpLoadedClassList()) {
String traceSP = SPECIES_RESOLVE + " " + cn + (salvage != null ? " (salvaged)" : " (generated)");
if (TRACE_RESOLVE) {
System.out.println(traceSP);
}
CDS.logTraceResolve(traceSP);
}
}
to
if (TRACE_RESOLVE || CDS.isDumpLoadedClassList()) {
String traceSP = SPECIES_RESOLVE + " " + cn;
if (TRACE_RESOLVE) {
System.out.println(traceSP + (salvage != null ? " (salvaged)" : " (generated)"));
}
CDS.logTraceResolve(traceSP);
}
MESSAGE_OK); | ||
// 5. The two lines with worng LF format | ||
dumpShouldPass( | ||
"TESTCASE 5: With incorrect LF format, the line will be ignored", | ||
appJar, classlist( | ||
"Hello", | ||
"@lambda-form-invoker [LF_XYRESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeStatic L7_L (success)", | ||
"@lambda-form-invoker [LF_XYRESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeStatic LL_I (success)"), | ||
"@lambda-form-invoker [LF_XYRESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeStatic L7_L (any)", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should not allow incorrect input. This should generate an error.
"@lambda-form-invoker [LF_RESOLVE] my.nonexist.package.MyNonExistClassName$holder invokeStatic L7_L (success)", | ||
"@lambda-form-invoker [LF_RESOLVE] my.nonexist.package.MyNonExistClassName$holder invokeStatic LL_I (success)"), | ||
"@lambda-form-invoker [LF_RESOLVE] my.nonexist.package.MyNonExistClassName$holder invokeStatic L7_L", | ||
"@lambda-form-invoker [LF_RESOLVE] my.nonexist.package.MyNonExistClassName$holder invokeStatic LL_I"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's dangerous to allow arbitrary class names here. InvokerBytecodeGenerator doesn't check the classname. This will make it possible to overwrite the contents of arbitrary classes. We should have a check here and allow only the specific holder classes that are supported.
@@ -26,6 +26,15 @@ | |||
package jdk.internal.misc; | |||
|
|||
public class CDS { | |||
// cache the result | |||
static private boolean isDumpLoadedClassList; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isDumpLoadedClassList
is not gramatically correct. Also the field should be final. How about:
static final private boolean isDumpingClassList = isDumpingClassList0();
public static boolean isDumpingClassList() {
return isDumpingClassList;
}
private static boolean isDumpingClassList0();
/** | ||
* log output to DumpLoadedClassList | ||
*/ | ||
public static void logTraceResolve(String line) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logTraceResolve
is too generic. How about CDS.logLambdaFormInvoker()
to match the @lambda-form-invoker
in the classlist file?
…ccording to the changes. The invoke name verififcation is not implemented since not all the holder class are processed, not all the functions of processed holder classes are added. For holder class with DirectMethodHandle in its name, only the name in the DMH_METHOD_TYPE_MAP keyset is added, ithe line with other names just gets skipped silently. This makes the verification on invoke names difficul, a name not in the keyset should not fail the test. Also add a boolean to cdsGenerateHolderClasses to indicate call path.
…p to CDS as generateLambdaFormHolderClasses. Added input verification function in CDS before class generation. Added more test scenarios. Removed trailing unused ending words for output of lambda form trace line in case of DumpLoadedClassList.
…sed variables after last change.
shortenSignature(basicTypeSignature(type)); | ||
if (TRACE_RESOLVE) { | ||
System.out.println(traceLF + (resolvedMember != null ? " (success)" : " (fail)")); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest not to change the existing code. Instead, have CDS::traceLambdaFormInvoker
to take individual parameters Class<?> holder, String name, String shortenSignature
(rather than the formatted string). Something like:
if (CDS.isDumpLoadedClassList()) {
CDS.traceLambdaFormInvoker(holder, name, shortenSignature(basicTypeSignature(type));
}
This also gives flexibility to CDS to decide on what format to write to the class list (like this case, you drop the text "success/fail")
In addition, the conditional check on CDS.isDumpLoadedClassList()
is hard to relate to why CDS traces these events. I see Ioi's comment on this method name too. I agree with Ioi that isDumpingClassList
makes more sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
if (TRACE_RESOLVE) { | ||
System.out.println(traceSP + (salvage != null ? " (salvaged)" : " (generated)")); | ||
} | ||
CDS.traceLambdaFormInvoker(traceSP); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest leaving the existing code unchanged. Instead, add the following:
if (CDS.isDumpingClassList()) {
CDS.traceSpeciesType(cn);
}
The above uses Ioi's suggested method name which reads better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
/** | ||
* check if -XX:+DumpLoadedClassList and given file is open | ||
*/ | ||
public static boolean isDumpLoadedClassList() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with Ioi's suggestion to rename this to isDumpingClassList
which describes what the VM is doing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
…he existing function to log lambda form invokers. Changed isDumpLoadedClassList to a reasonable name isDumpingClassList as read only in CDS.
@yminqi This change now passes all automated pre-integration checks. ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details. After integration, the commit message for the final commit will be:
You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been 61 new commits pushed to the
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details. ➡️ To integrate this PR with the above commit message to the |
…ity check on class_list_file is not NULL before check on it is open.
|
||
// return null for invalid input | ||
private static Stream<String> validateInputLines(String[] lines) { | ||
ArrayList<String> list = new ArrayList<String>(lines.length); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: this can use diamond operatior like this: new ArrayList<>(lines.length)
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will update.
private static Object[] generateLambdaFormHolderClasses(String[] lines) { | ||
Objects.requireNonNull(lines); | ||
try { | ||
Stream<String> lineStream = validateInputLines(lines); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems clearer to have validateInputLines
do validation only and convert this line into:
validateInputLines(lines);
Stream<String> lineStream = Arrays.stream(lines);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Somewhere in the testing framework, the line in ExtraClassList was added '\f' which needs trim, I did not dig into deep the root cause so here just return a new List with ending white spaces trimmed instead. I will file a bug for it and when it fixed, we can do this way.
}; | ||
return retArray; | ||
} catch (Exception e) { | ||
e.printStackTrace(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this a debugging statement? If CDS swallows the exception thrown, I think VM should emit the warning message and print the stack trace if appropriate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I want to print this in java since in VM it is a little complex than java. It just used to give the stack trace where the exception happened for debug purpose. Maybe I should put comment for it.
…access it. Changed function validateInputLines to isValidInputLines and return a boolean to indicate its valid. Added more comments for review concern.
…ong inputs. Fixed missed archive_mirror issue which could lead crash for archived heap iteration. After call generateLambdaFormHolderClasses, should check exception first. Added more comments why print out exception message and stacktrace in CDS.
passed mach5 tier1-4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I reviewed the files under src/java.base
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Latest version LGTM.
/integrate |
@yminqi Since your change was applied there have been 61 commits pushed to the
Your commit was automatically rebased without conflicts. Pushed as commit e4469d2. 💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored. |
This patch is reorganized after 8252725, which is separated from this patch to refactor jlink glugin code. The previous webrev with hg can be found at: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05. With 8252725 integrated, the regeneration of holder classes is simply to call the new added GenerateJLIClassesHelper.cdsGenerateHolderClasses function.
Tests: tier1-4
Progress
Testing
Issue
Reviewers
Download
$ git fetch https://git.openjdk.java.net/jdk pull/193/head:pull/193
$ git checkout pull/193