Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8165276: Spec states to invoke the premain method in an agent class if it's public but implementation differs #1694

Closed
wants to merge 12 commits into from

Conversation

sspitsyn
Copy link
Contributor

@sspitsyn sspitsyn commented Dec 8, 2020

This change have been already reviewed by Mandy, Sundar, Alan and David.
Please, see the jdk 15 review thread:
http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-June/031998.html

Now, the PR approval is needed.
The push was postponed because the CSR was not approved at that time (it is now):
https://bugs.openjdk.java.net/browse/JDK-8248189
Investigation of existing popular java agents was requested by Joe.

The java.lang.instrument spec:
https://docs.oracle.com/en/java/javase/15/docs/api/java.instrument/java/lang/instrument/package-summary.html

Summary:
The java.lang.instrument spec clearly states:
"The agent class must implement a public static premain method
similar in principle to the main application entry point."
Current implementation of sun/instrument/InstrumentationImpl.java
allows the premain method be non-public which violates the spec.
This fix aligns the implementation with the spec.

Testing:
A mach5 run of jdk_instrument tests is in progress.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8165276: Spec states to invoke the premain method in an agent class if it's public but implementation differs

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/1694/head:pull/1694
$ git checkout pull/1694

…ass if it's public but implementation differs
@bridgekeeper
Copy link

bridgekeeper bot commented Dec 8, 2020

👋 Welcome back sspitsyn! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Dec 8, 2020
@openjdk
Copy link

openjdk bot commented Dec 8, 2020

@sspitsyn The following labels will be automatically applied to this pull request:

  • core-libs
  • serviceability

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added serviceability serviceability-dev@openjdk.org core-libs core-libs-dev@openjdk.org labels Dec 8, 2020
@mlbridge
Copy link

mlbridge bot commented Dec 8, 2020

@AlanBateman
Copy link
Contributor

/label remove core-libs

@openjdk openjdk bot removed the core-libs core-libs-dev@openjdk.org label Dec 8, 2020
@openjdk
Copy link

openjdk bot commented Dec 8, 2020

@AlanBateman
The core-libs label was successfully removed.

@sspitsyn sspitsyn changed the title 8165276: Spec states that invoke the premain method in an agent cl… 8165276: Spec states that invoke the premain method in an agent class if it's public but implementation differs Dec 8, 2020
@sspitsyn sspitsyn changed the title 8165276: Spec states that invoke the premain method in an agent class if it's public but implementation differs 8165276: Spec states to invoke the premain method in an agent class if it's public but implementation differs Dec 8, 2020
if (!m.canAccess(null)) {
String msg = "method " + classname + "." + methodname + " must be declared public";
throw new IllegalAccessException(msg);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If canAccess fails then it means that javaAgentClass is not public or the premain method is not public. The invoke will fail but I agree eat explicit canAccess check means we get finer control on the exception message.

(I can't help feeling that we should do a bit more cleanup here and not use getDeclaredMethod or be concerned with inheritance. This is because the Premain-Class is meant to specify the agent class. Also can't have a public static premain in super class and a non-public static premain in a sub-class).

Copy link
Contributor Author

@sspitsyn sspitsyn Dec 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My gut feeling is that it should be possible to get rid of the getDeclaredMethod calls with the reasoning you provided above. The canAccess check is not really needed in such a case as the getMethod returns public methods only (is my understanding correct?). This would simplify the implementation. Two disadvantages of this approach are:

  • less fine control on the exception message: NoSuchMethodException instead of IllegalAccessException for non-public premain methods
  • some compatibility risk is involved (most of agents define premain method correctly, so the only java agents that define premain methods with unusual tricks are impacted)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the java agent class declares a non-public premain method but its superclass declares a public premain method, what should be the expected behavior? The proposed fix will choose the java agent class's non-public premain and therefore it will fail to load the agent class with IAE.

If we get rid of the getDeclaredMethod call and find premain using getMethod, it will choose the public premain method defined directly in the java agent class or indirectly in its superclass.

As Alan stated, Premain-Class attribute is meant to specify the agent class. Also specified in the package spec of `java.lang.instrument', the agent class must implement a public static premain method similar in principle to the main application entry point. Also under "Manifest Attributes" section,

   Premain-Class
        When an agent is specified at JVM launch time this attribute specifies the agent class. That is, the class containing the premain method. 

It expects that there is a public premain method declared in the agent class, not in its superclass.

So I think it's best to fix this as well in this PR by dropping line 469-495 (i.e. keep getDeclaredMethod case and not search for inherited premain). Throw if the premain method is not found or it is not accessible. This needs to update the CSR for behavioral change.

@plummercj
Copy link
Contributor

This change have been already reviewed by Mandy, Sundar, Alan and David.
Now, the PR approval is needed.

Can you provide a link to the discussion? I'm mostly curious if there was some discussion as to why Instrument purposefully allowed non-public premain methods:

508         // make it accessible so we can call it
509         // Note: The spec says the following:
510         //     The agent class must implement a public static premain method...
511         setAccessible(m, true);```

@AlanBateman
Copy link
Contributor

All the discussion is in the bug and CSR:
https://bugs.openjdk.java.net/browse/JDK-8165276
https://bugs.openjdk.java.net/browse/JDK-8248189
We messed up in JDK-5070281 (JDK 6) and it came to light in JDK 9 when auditing the use of setAccessible in the JDK.

@sspitsyn
Copy link
Contributor Author

sspitsyn commented Dec 8, 2020

Chris, I've added link to the jdk 15 review thread to the PR description.

@sspitsyn
Copy link
Contributor Author

Alan, Mandy and Chris,
Thank you for your comments!
I've committed and pushed an update which implements the suggestion from Mandy to get rid of inherited premain/agentmain methods. I also had to fix 3 tests (InheritAgent0100, InheritAgent1000 and InheritAgent1100) which started to fail with this update as they are relying on the inherited premain methods. I've refactored them to negative tests.

Copy link
Member

@mlchung mlchung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update @bug in the tests to include this bug ID.

All InheritAgentXXX tests are updated to have the InheritAgentXXX class public. However, I think you need to go through them individually for this behavioral change. The existing comments may not be applicable and please update them where appropriate. For example InheritAgent0100 previously verifies the invocation of the premain in its superclass. Does the modified test ensure that this fails to load? Per the comment in the new InheritAgent0100Super class, it expects the superclass' premain should be called.

// finally try the inherited 1-arg method
try {
m = javaAgentClass.getMethod(methodname,
new Class<?>[] { String.class });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments in line 429-443 need to be updated.

@sspitsyn
Copy link
Contributor Author

sspitsyn commented Dec 11, 2020

Mandy, thank you for comments.
You are right, the comments at 429-443 and tests need some updates.
I've updated the tests which were converted to be negative:
InheritAgent0100, InheritAgent1000 and InheritAgent1100
These are the only tests that needed update with new expectations.

… in tests with new expectations that premain methods in supers should non be called
@sspitsyn
Copy link
Contributor Author

I've updated the @bug in the tests to include this bug ID.
All the comments from Mandy have been addressed now.

if (!m.canAccess(null)) {
String msg = "method " + classname + "." + methodname + " must be declared public";
throw new IllegalAccessException(msg);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the updated patch looks much better.

If the agent class doesn't declare a premain then NoSuchMethodException will be thrown.

The only thing that I'm wondering about now is the exception message when the agent class is not public. If I read the changes correctly it means that IllegalAccessException will be thrown saying that .premain should be public. Is that correct? We might need to adjust to the exception message to make it clear, or put in an explicit then to ensure to test that the agent class is public.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an explicit check for a public premain method is better to disambiguate the cases and provide appropriate error messages.

if (!m.canAccess(null)) {
String msg = "method " + classname + "." + methodname + " must be declared public";
throw new IllegalAccessException(msg);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an explicit check for a public premain method is better to disambiguate the cases and provide appropriate error messages.

@sspitsyn
Copy link
Contributor Author

Alan, David and Many, thank you for the comments!
I'll prepare an update according to the recent requests from you.
One question is if I need to clone this PR to the JDK 16 fork:
https://github.com/openjdk/jdk16
It depends on our chances to finalize this before RDP2.
An alternate approach would be to continue reviewing this PR until all comment are resolved and then clone it to jdk16.

@mlbridge
Copy link

mlbridge bot commented Dec 15, 2020

Mailing list message from Mandy Chung on serviceability-dev:

On 12/14/20 2:47 PM, Serguei Spitsyn wrote:

On Mon, 14 Dec 2020 02:39:30 GMT, David Holmes <dholmes at openjdk.org> wrote:

Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision:

added 8165276 to @bug list of impacted tests
Changes requested by dholmes (Reviewer).
Alan, David and Many, thank you for the comments!
I'll prepare an update according to the recent requests from you.
One question is if I need to clone this PR to the JDK 16 fork:
https://github.com/openjdk/jdk16
It depends on our chances to finalize this before RDP2.
An alternate approach would be to continue reviewing this PR until all comment are resolved and then clone it to jdk16.

Since this already passes RDP1, this can be retarget to JDK 17 and
continue with this PR to openjdk/jdk master.? I see no urgency to fix
this in JDK 16.

Mandy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20201214/7b98bcbd/attachment.htm>

@sspitsyn
Copy link
Contributor Author

sspitsyn commented Dec 16, 2020

Mandy, thank you for the suggestion.
I'll retarget the bug and CSR to jdk 17 as nobody is objecting.

Also, wanted to make it clear about Exception messages that are provided for two different cases.

The test java/lang/instrument/NonPublicPremainAgent.java defines a non-public premain method:
static void premain(String agentArgs, Instrumentation inst) {

With the m.canAccess check the following message is provided (it looks right):
Exception in thread "main" java.lang.IllegalAccessException: method NonPublicPremainAgent.premain must be declared public

Without this check the message was (not sure, it is good enough):
Exception in thread "main" java.lang.IllegalAccessException: class sun.instrument.InstrumentationImpl (in module java.instrument) cannot access a member of class NonPublicPremainAgent with modifiers "static"

Also, I've added a new test java/lang/instrument/NonPublicAgent.java which defines a non-public agent class with a public premain method.

With the m.canAccess check the following message is provided (it does not look right):
Exception in thread "main" java.lang.IllegalAccessException: method NonPublicPremainAgent.premain must be declared public

Without this check the message is (it looks pretty confusing):
Exception in thread "main" java.lang.IllegalAccessException: class sun.instrument.InstrumentationImpl (in module java.instrument) cannot access a member of class NonPublicAgent with modifiers "public static"

So, it seems we also need an explicit check for agent class being public with a right message provided.
I've added the following check:

@@ -426,6 +426,12 @@ public class InstrumentationImpl implements Instrumentation {
         NoSuchMethodException firstExc = null;
         boolean twoArgAgent = false;
 
+        // reject non-public owner agent class of premain or agentmain method
+        if ((javaAgentClass.getModifiers() & java.lang.reflect.Modifier.PUBLIC) == 0) {
+            String msg = "agent class of method " + classname + "." +  methodname + " must be declared public";
+            throw new IllegalAccessException(msg);
+        }
+
         // The agent class must have a premain or agentmain method that
         // has 1 or 2 arguments. We check in the following order:
         //

With the check above the message is:
Exception in thread "main" java.lang.IllegalAccessException: agent class of method NonPublicAgent.premain must be declared public

Please, let me know what you think.

I've done some tests refactoring motivated by some private requests from Mandy. Will publish the update as soon as it is ready. Hopefully, today.

@dholmes-ora
Copy link
Member

The agent class doesn't have to be public it just has to be accessible.

The premain method should be queried for public modifier rather than just relying on a failed invocation request.

@sspitsyn
Copy link
Contributor Author

David, thank you for catching this. I'm probably missing something here.
If the agent class is not public then the m.canAccess(null) check is not passed and IAE is thrown with the message:
Exception in thread "main" java.lang.IllegalAccessException: method NonPublicAgent.premain must be declared public

But the NonPublicAgent.premain is declared public as below:

    public static void premain(String agentArgs, Instrumentation inst) {
        System.out.println("premain: NonPublicAgent was loaded");
    }

It seems, the IAE is thrown because the agent class is not public.
Does it mean the m.canAccess(null) check is not fully correct?

@sspitsyn
Copy link
Contributor Author

sspitsyn commented Dec 17, 2020

Mandy, I've pushed updates with changes:

  • NegativeAgentRunner checks if the exit value is non-zero
  • AgentJarBuilder is replaced with JavaAgentBuilder
  • removed unneeded qualified exports

So setAccessible should only be called if the agent class is in an unnamed module.
Now the agent class is always loaded as in the unnamed module.
I tried to test this with a modular jar file which has module-info.class in it.
In fact, I don't know if there are any options that would help to load the premain agent class in named module.
I guess, it has to be implemented as part of the enhancement https://bugs.openjdk.java.net/browse/JDK-6932391 .

The following patch can be committed if you like as it does not harm:

@@ -476,11 +476,13 @@ public class InstrumentationImpl implements Instrumentation {
             String msg = "method " + classname + "." +  methodname + " must be declared public";
             throw new IllegalAccessException(msg);
         }
-
-        if ((javaAgentClass.getModifiers() & java.lang.reflect.Modifier.PUBLIC) == 0) {
-            // If the java agent class is not public, make the premain/agentmain
-            // accessible, so we can call it.
+        if ((javaAgentClass.getModifiers() & java.lang.reflect.Modifier.PUBLIC) == 0 &&
+            !javaAgentClass.getModule().isNamed()) {
+            // If the java agent class is not public and is in unnamed module
+            // then make the premain/agentmain accessible, so we can call it.
             setAccessible(m, true);
         }

Copy link
Member

@mlchung mlchung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sspitsyn Thanks for the update. I think it's good to add the unnamed module check before calling setAccessible that becomes clear that this check only intends for unnamed module. Maybe make it clear in the comment something like this:

    // if the java agent class is in an unnamed module, the java agent class can be non-public.
    // suppress access check upon the invocation of the premain/agentmain method

Since the agent class can be non-public, it means that it's not necessary to modify the test agent class to public as you did in this patch.

I don't think @library /test is needed, isn't it? The test library classes are under test/lib.

// The agent class must implement a public static premain method...
setAccessible(m, true);
// reject non-public premain or agentmain method
if ((m.getModifiers() & java.lang.reflect.Modifier.PUBLIC) == 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative way: if (Modifier.isPublic(m.getModifiers()))

Similar can be applied for checking if the javaagentClass is public.

@sspitsyn
Copy link
Contributor Author

@mlchung Thank you for the suggestions. I've addressed them in the latest update.
The @library /test is needed only in the tests that use NegativeAgentRunner as this class is located in folder one level up.
I've removed the this line from the other tests.

Copy link
Member

@mlchung mlchung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks okay with minor comments on the tests. Nit: it will be good to update @summary in the InheritAgentxxxx tests that now fail to load the agent.

/*
* @test
* @bug 8165276
* @summary Test that agent with non-public premain method is rejected to load
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the comment is incorrect. This tests a non-public agent class with a public premain method. The agent is not rejected.

@@ -38,7 +38,7 @@
import java.io.*;
import asmlib.*;

class RetransformAgent {
public class RetransformAgent {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making RetransformAgent as public is not necessary. This fix does not change this test and so better to revert it.

@@ -23,7 +23,7 @@

import java.lang.instrument.Instrumentation;

class SimpleAgent {
public class SimpleAgent {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no other change in this test except making SImpleAgent as public which is not necessary. So better to revert it.

@openjdk
Copy link

openjdk bot commented Dec 21, 2020

@sspitsyn This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8165276: Spec states to invoke the premain method in an agent class if it's public but implementation differs

Reviewed-by: mchung, dholmes, alanb

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 30 new commits pushed to the master branch:

  • 535c292: 8260306: Do not include osThread.hpp in thread.hpp
  • 06348df: 8259776: Remove ParallelGC non-CAS oldgen allocation
  • 6c4c96f: 8258742: Move PtrQueue reset to PtrQueueSet subclasses
  • b53d5ca: 8260315: Typo "focul" instead of "focus" in FocusSpec.html
  • f624dba: 8240247: No longer need to wrap files with contentContainer
  • 5cdcce1: 8260307: Do not include method.hpp in frame.hpp
  • 6f2a394: Merge
  • 685c03d: 8259271: gc/parallel/TestDynShrinkHeap.java still fails "assert(covered_region.contains(new_memregion)) failed: new region is not in covered_region"
  • d90e06a: 8259775: [Vector API] Incorrect code-gen for VectorReinterpret operation
  • ede1bea: 8227695: assert(pss->trim_ticks().seconds() == 0.0) failed: Unexpected partial trimming during evacuation
  • ... and 20 more: https://git.openjdk.java.net/jdk/compare/d066f2b06c08d41e47cd7a4b1f047f40df1a5972...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Dec 21, 2020
@sspitsyn
Copy link
Contributor Author

@mlchung All suggestions are right and addressed in the latest update.

@@ -23,7 +23,7 @@

/**
* @test
* @bug 6274264 6274241 5070281
* @bug 6274264 6274241 5070281 8165276
Copy link
Member

@mlchung mlchung Dec 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The copyright header and @bug change should be reverted.

@@ -1,5 +1,5 @@
/*
* Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2016, 2020, Oracle and/or its affiliates. All rights reserved.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The copyright header change should be reverted.

@sspitsyn
Copy link
Contributor Author

@mlchung
Thank you for the catch! Fixed it.
Also, reverted the fix in the NativeMethodPrefixAgent.java.
Additionally, got rid of the 'inherited' term in the summary of InheritAgent*.java tests.

Copy link
Member

@mlchung mlchung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the change. Looks okay.

@sspitsyn
Copy link
Contributor Author

Thank you a lot, Mandy!
Sorry for missed changes and overlooked issues in tests.

Copy link
Contributor

@AlanBateman AlanBateman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for persevering on this one. The updated checks look good. At some point I think we re-examine the issue of allowing agents to be non-public, maybe emit a warning and eventually disallow it. If/when support is added for developing and deployed java agents as modules it might be the time to re-examine it.

@sspitsyn
Copy link
Contributor Author

Thank you for review, Alan!

@bridgekeeper
Copy link

bridgekeeper bot commented Jan 20, 2021

@sspitsyn This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@sspitsyn
Copy link
Contributor Author

This PR has been approved by Mandy and Alan.
I'm also waiting for approval from David and the CSR approval from Joe.

Copy link
Member

@dholmes-ora dholmes-ora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Serguei,

This seems good to go.

Thanks,
David

@sspitsyn
Copy link
Contributor Author

Thank you for review, David!

@sspitsyn
Copy link
Contributor Author

/integrate

@openjdk openjdk bot closed this Jan 26, 2021
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jan 26, 2021
@openjdk
Copy link

openjdk bot commented Jan 26, 2021

@sspitsyn Since your change was applied there have been 48 commits pushed to the master branch:

  • 9ea9323: 8254246: SymbolHashMapEntry wastes space
  • 982e42b: 8259726: Use of HashSet leads to undefined order in test output
  • d6fb9d7: 8255464: Cannot access ModuleTree in a CompilationUnitTree
  • 12ccd21: 8260289: Unable to customize module lists after change JDK-8258411
  • 73c78c8: 8260329: Update references to TAOCP to latest edition
  • 5b0b24b: 8260381: ProblemList com/sun/management/DiagnosticCommandMBean/DcmdMBeanTestCheckJni.java on Win with ZGC
  • 47c7dc7: 8258833: Cancel multi-part cipher operations in SunPKCS11 after failures
  • ef247ab: 8260308: Update LogCompilation junit to 4.13.1
  • d076977: 8260169: LogCompilation: Unexpected method mismatch
  • 6e03735: 8259845: Move placeholder implementation details to cpp file and add logging
  • ... and 38 more: https://git.openjdk.java.net/jdk/compare/d066f2b06c08d41e47cd7a4b1f047f40df1a5972...master

Your commit was automatically rebased without conflicts.

Pushed as commit c538cd8.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@sspitsyn sspitsyn deleted the premain branch June 8, 2021 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated Pull request has been integrated serviceability serviceability-dev@openjdk.org
Development

Successfully merging this pull request may close these issues.

5 participants