Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8257853: Remove dependencies on JNF's JNI utility functions in AWT and 2D code #1679

Closed
wants to merge 10 commits into from
Closed

Conversation

@prrace
Copy link
Contributor

@prrace prrace commented Dec 7, 2020

This defines some macros to support declaring and initialising statically allocated instances of jclass, jmethodID and jfieldID
and changes many existing uses of JNF macros/functions to use these instead.
Then calls to JNFCall* and JNFNewObject - etc are updated to directly call JNI methods
JNI exception checking macros are added as needed.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8257853: Remove dependencies on JNF's JNI utility functions in AWT and 2D code

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/1679/head:pull/1679
$ git checkout pull/1679

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Dec 7, 2020

👋 Welcome back prr! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

@openjdk openjdk bot commented Dec 7, 2020

@prrace The following label will be automatically applied to this pull request:

  • awt

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the awt label Dec 7, 2020
@openjdk openjdk bot added the rfr label Dec 9, 2020
@mlbridge
Copy link

@mlbridge mlbridge bot commented Dec 9, 2020

Copy link
Contributor

@prsadhuk prsadhuk left a comment

Wouldn't it have been nice to have a new file having same JNF macros so it will not result in changing any source code but just instead of mapping to apple provided JNF layer, we will map to our macro through JNF wrapper and link/unlink apple JNF lib based on version or so in makefile?

@VladimirKempik
Copy link

@VladimirKempik VladimirKempik commented Dec 10, 2020

This siunds good, especially considering JEP-391, where we have to link ( at build time) with custom made JNF.framework and then provide jdk with JNF.framework in JAVA_HOME/lib

The alternative is to integrate JNF (bsd3 license) into openjdk.

@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 10, 2020

Wouldn't it have been nice to have a new file having same JNF macros so it will not result in changing any source code but just instead of mapping to apple provided JNF layer, we will map to our macro through JNF wrapper and link/unlink apple JNF lib based on version or so in makefile?

No. Not really. We should not keep the JNF namespace. Nor is there any desitre to have that 1:1 mapping.
This is NOT a "copy of JNF". It is a conscious attempt to be rid of it and all the unecessary baggage.
So I am not interested in that approach even one little bit

@openjdk openjdk bot removed the rfr label Dec 12, 2020
@openjdk openjdk bot added the rfr label Dec 12, 2020
@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 12, 2020

I updated this fix with the additional changes for the accessibility files.
There's a little bit of clean up as some declarations were unused and also these files shared a single defintion of some classes across files which were mostly unncecessary. One class I made static and declared per-file which is cleaner,
Also the only A11Y reference in AWTView was moved into its only call site.
With these changes I think the desktop module is now 100% free of the JNFCall* pattern and the CACHE pattern
I re-ran our full automated headful jtreg test suite and everything passed. I also ran VoiceOver and ran through SwingSet2 without seeing any issues although I am not sure what to expect for this so it would be best if @azuev-java checked it out.

@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 12, 2020

Updated with some fixes for the A11Y support.
Adding some logging that will report any problems looking up a JNI method or field or class.

@azuev-java
Copy link
Member

@azuev-java azuev-java commented Dec 14, 2020

I have tested the a11y part and haven't found any functional degradation there.

Copy link
Member

@azuev-java azuev-java left a comment

Accessibility part is perfectly fine, no regressions.

@openjdk
Copy link

@openjdk openjdk bot commented Dec 14, 2020

@prrace This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8257853: Remove dependencies on JNF's JNI utility functions in AWT and 2D code

Reviewed-by: psadhukhan, kizune

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 263 new commits pushed to the master branch:

  • c50b464: 8258715: [JVMCI] separate JVMCI code install timers for CompileBroker and hosted compilations
  • 64644a1: 8253881: Hotspot/Serviceability Terminology Refresh
  • 6a78b2a: 8258645: Bring Jemmy 1.3.11 to JDK test base
  • 7f92d18: 8258553: Limit number of fields in instance to be considered for scalar replacement
  • adf0e23: 8257800: CompileCommand TypedMethodOptionMatcher::parse_method_pattern() may over consume
  • 06c24e1: 8256213: Remove os::split_reserved_memory
  • be41468: 8258696: Temporarily revert use of pattern match instanceof until docs-reference is fixed
  • a4f393c: 8258661: Inner class ResponseCacheEntry could be static
  • 3c48819: 8169086: DTLS tests fail intermittently with too much loops or timeout
  • 71ae05d: 8258061: Improve diagnostic information about errors during class redefinition
  • ... and 253 more: https://git.openjdk.java.net/jdk/compare/55f5542ca2104df91e14693534cc7b3c36e81953...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Dec 14, 2020
@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 15, 2020

anyone up for reviewing the rest of it ?

@@ -314,7 +335,7 @@ + (void)initialize
(*env)->DeleteLocalRef(env, jString);
}

sAccessibilityClass = JNFCallStaticObjectMethod(env, jm_getAccessibility, result); // AWT_THREADING Safe (known object)
sAccessibilityClass = (*env)->CallStaticObjectMethod(env, sjc_CAccessibility, jm_getAccessibility, result); // AWT_THREADING Safe (known object)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need CHECK_EXCEPTION(); after CallStaticObjectMethod here?

Copy link
Contributor Author

@prrace prrace Dec 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add it.

Copy link
Contributor

@prsadhuk prsadhuk left a comment

I believe we could add CHECK_EXCEPTION in few places...pointing those out...

@gerard-ziemski
Copy link

@gerard-ziemski gerard-ziemski commented Dec 18, 2020

There seems to be a pattern where we replace JNFCallObjectMethod() with a pair of CallObjectMethod() immediately followed by CHECK_EXCEPTION()

There are places where currently in this fix, this pattern is missing the CHECK_EXCEPTION()

Can we replace JNFCallObjectMethod() by a single function call (or macro) that wraps both CallObjectMethod() and CHECK_EXCEPTION() into one?

This would make this change more robust and make it easier to review it.

@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 18, 2020

There seems to be a pattern where we replace JNFCallObjectMethod() with a pair of CallObjectMethod() immediately followed by CHECK_EXCEPTION()

There are places where currently in this fix, this pattern is missing the CHECK_EXCEPTION()

Can we replace JNFCallObjectMethod() by a single function call (or macro) that wraps both CallObjectMethod() and CHECK_EXCEPTION() into one?

This would make this change more robust and make it easier to review it.

Its is a reasonable idea and I thought about it but on the whole I'd prefer not to do it and insted fix missing cases.
Tthere are cases where we are using these macros directly in a native method and then returning to Java.
In those case I want to propagate the exception and this suggestion would make that difficult if not impossible.
Remember that these aren't likely to do anything whether missing or not to make things better or cause
a regression because we should not be generating these exceptions anyway.
This is mostly a recommended precautionary pattern "just in case".
And nowhere else in the JDK uses macro for this - the CHECK_EXCEPTION and similar patterns are
used elsewhere. And code already added by Oracle engineers after the initial macOS port
are using the standard JNI calls.

For comparison I've been running JDK 15 with -Xcheck:jni and there were a number of places that generated
these warnings and I've resolved them here.

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.

Copy link
Member

@mrserb mrserb left a comment

We should take care of native exceptions, and probably use more objective-c than the macro.

@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 18, 2020

We should take care of native exceptions, and probably use more objective-c than the macro.

We will take care of native exceptions - the ENTER/EXIT macros which have yet to be updated will handle these if they occur.
This part of the changes is about JNI up-calls.

@prrace prrace closed this Dec 18, 2020
@prrace prrace reopened this Dec 18, 2020
@mrserb
Copy link
Member

@mrserb mrserb commented Dec 18, 2020

There seems to be a pattern where we replace JNFCallObjectMethod() with a pair of CallObjectMethod() immediately followed by CHECK_EXCEPTION()
There are places where currently in this fix, this pattern is missing the CHECK_EXCEPTION()
Can we replace JNFCallObjectMethod() by a single function call (or macro) that wraps both CallObjectMethod() and CHECK_EXCEPTION() into one?
This would make this change more robust and make it easier to review it.

Its is a reasonable idea and I thought about it but on the whole I'd prefer not to do it and insted fix missing cases.
Tthere are cases where we are using these macros directly in a native method and then returning to Java.
In those case I want to propagate the exception and this suggestion would make that difficult if not impossible.

I guess it should work already in the JNF case? The java exception should be wrapped by the NSException in the JNFCallObjectMethod/etc and unwrapped in the JNF_COCOA_ENTER before exists to the java method?

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.

I am not sure about that.

@mrserb
Copy link
Member

@mrserb mrserb commented Dec 18, 2020

We should take care of native exceptions, and probably use more objective-c than the macro.

We will take care of native exceptions - the ENTER/EXIT macros which have yet to be updated will handle these if they occur.
This part of the changes is about JNI up-calls.

No, you skipped all exceptions in the JNI up-calls, which I am talking about.

@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 18, 2020

We should take care of native exceptions, and probably use more objective-c than the macro.

We will take care of native exceptions - the ENTER/EXIT macros which have yet to be updated will handle these if they occur.
This part of the changes is about JNI up-calls.

No, you skipped all exceptions in the JNI up-calls, which I am talking about.

Not sure what you mean.

@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 18, 2020

There seems to be a pattern where we replace JNFCallObjectMethod() with a pair of CallObjectMethod() immediately followed by CHECK_EXCEPTION()
There are places where currently in this fix, this pattern is missing the CHECK_EXCEPTION()
Can we replace JNFCallObjectMethod() by a single function call (or macro) that wraps both CallObjectMethod() and CHECK_EXCEPTION() into one?
This would make this change more robust and make it easier to review it.

Its is a reasonable idea and I thought about it but on the whole I'd prefer not to do it and insted fix missing cases.
Tthere are cases where we are using these macros directly in a native method and then returning to Java.
In those case I want to propagate the exception and this suggestion would make that difficult if not impossible.

I guess it should work already in the JNF case? The java exception should be wrapped by the NSException in the JNFCallObjectMethod/etc and unwrapped in the JNF_COCOA_ENTER before exists to the java method?

That is the pattern I know is there today and I don't think is needed.
It can be done but I don't think it needed. I'd prefer to see if we ever have such a situation and deal with it "locally"
at the call site.

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.

I am not sure about that.

I am looking at JNI warnings - not just guessing.

@mrserb
Copy link
Member

@mrserb mrserb commented Dec 18, 2020

We should take care of native exceptions, and probably use more objective-c than the macro.

We will take care of native exceptions - the ENTER/EXIT macros which have yet to be updated will handle these if they occur.
This part of the changes is about JNI up-calls.

No, you skipped all exceptions in the JNI up-calls, which I am talking about.

Not sure what you mean.

Non of your macro raises exceptions like it was before. All of them just log and clear a java exception, while before such exceptions were transformed to the NSException and handled appropriately.

@mrserb
Copy link
Member

@mrserb mrserb commented Dec 18, 2020

I guess it should work already in the JNF case? The java exception should be wrapped by the NSException in the JNFCallObjectMethod/etc and unwrapped in the JNF_COCOA_ENTER before exists to the java method?

That is the pattern I know is there today and I don't think is needed.
It can be done but I don't think it needed. I'd prefer to see if we ever have such a situation and deal with it "locally"
at the call site.

No, it is quite useful, and it will be good to preserve it. It is possible to work "locally" when the call site if come from java, but not if we are in the infinite native loop.

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.
I am not sure about that.
I am looking at JNI warnings - not just guessing.

I am looking at a new code that is not as good as before.

@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 18, 2020

I guess it should work already in the JNF case? The java exception should be wrapped by the NSException in the JNFCallObjectMethod/etc and unwrapped in the JNF_COCOA_ENTER before exists to the java method?

That is the pattern I know is there today and I don't think is needed.
It can be done but I don't think it needed. I'd prefer to see if we ever have such a situation and deal with it "locally"
at the call site.

No, it is quite useful, and it will be good to preserve it. It is possible to work "locally" when the call site if come from java, but not if we are in the infinite native loop.

an NS exception on the appkit will terminate the app - if unhandled - and probably will even if handled
so I am not sure what is better about that

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.
I am not sure about that.
I am looking at JNI warnings - not just guessing.

I am looking at a new code that is not as good as before.

I guess we disagree. I think this is better and simpler.

@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 18, 2020

Not trying to prolong or expand this discussion but a 1:1 mapping to JNF to really minimise the review needs to account for

  1. We aren't going to call it "JNF"
  2. The lookup methods for members is quirky in that it is the same for methods and fields and I can only suppose that the signature is parsed to look for "(...)" to decide which it should be
  3. The calling of static methods (and look up of static field values) doesn't take the class as an argument so there's
    some complexity in there just to support that which I think un-needed
  4. We'd need a bunch more macros to handle all the same "Call" cases as JNF has and that seems un-neccessary too.
  5. There are already a substantial number of cases where we don't use JNF
  6. So long as we have them in place the CHECK calls are simpler and if we ever reallly wanted to we could have a variant of it that does the odd Java->NS->Java exception think. Or just change the behaviour of the current one. But I stand by what I wrote that I don't think this is needed. This is code which doesn't get exercised - it is more like InternalError than IOException handling.

@mlbridge
Copy link

@mlbridge mlbridge bot commented Dec 19, 2020

Mailing list message from Alan Snyder on awt-dev:

I use JNF in my own (non-JDK) code, and I assume others do also.

Will the macOS JNI support functions and macros you want to use in the JDK be available for use in non-JDK code?

@mlbridge
Copy link

@mlbridge mlbridge bot commented Dec 19, 2020

Mailing list message from Philip Race on awt-dev:

Not something I'd considered as a goal.
I suppose its part of OpenJDK which means it will be there for anyone to
use under the GPL2+CP ...
but there's no thought of delivering it like we deliver jni.h with the
binary JDK distribution.

-phil.

On 12/18/20, 4:23 PM, Alan Snyder wrote:

I use JNF in my own (non-JDK) code, and I assume others do also.

Will the macOS JNI support functions and macros you want to use in the JDK be available for use in non-JDK code?

@mlbridge
Copy link

@mlbridge mlbridge bot commented Dec 19, 2020

Mailing list message from Alan Snyder on awt-dev:

If it?s better than JNF, as you suggest, perhaps it should be released and supported.

Alan

On Dec 18, 2020, at 4:28 PM, Philip Race <philip.race at oracle.com> wrote:

Not something I'd considered as a goal.
I suppose its part of OpenJDK which means it will be there for anyone to use under the GPL2+CP ...
but there's no thought of delivering it like we deliver jni.h with the binary JDK distribution.

-phil.

On 12/18/20, 4:23 PM, Alan Snyder wrote:

I use JNF in my own (non-JDK) code, and I assume others do also.

Will the macOS JNI support functions and macros you want to use in the JDK be available for use in non-JDK code?

@mlbridge
Copy link

@mlbridge mlbridge bot commented Dec 19, 2020

Mailing list message from Philip Race on awt-dev:

Ah .. not my call .. complicated.

-phil.

On 12/18/20, 4:40 PM, Alan Snyder wrote:

If it?s better than JNF, as you suggest, perhaps it should be released and supported.

Alan

On Dec 18, 2020, at 4:28 PM, Philip Race<philip.race at oracle.com> wrote:

Not something I'd considered as a goal.
I suppose its part of OpenJDK which means it will be there for anyone to use under the GPL2+CP ...
but there's no thought of delivering it like we deliver jni.h with the binary JDK distribution.

-phil.

On 12/18/20, 4:23 PM, Alan Snyder wrote:

I use JNF in my own (non-JDK) code, and I assume others do also.

Will the macOS JNI support functions and macros you want to use in the JDK be available for use in non-JDK code?

Copy link
Contributor

@prsadhuk prsadhuk left a comment

looks good to me.

@prrace
Copy link
Contributor Author

@prrace prrace commented Dec 19, 2020

/integrate

@openjdk openjdk bot closed this Dec 19, 2020
@openjdk openjdk bot added integrated and removed ready rfr labels Dec 19, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Dec 19, 2020

@prrace Since your change was applied there have been 266 commits pushed to the master branch:

  • 580af49: 8258006: Replaces while cycles with iterator with enhanced for in java.desktop
  • dc7f01f: 8257584: [macos] NullPointerException originating from LWCToolkit.java
  • c7c53d0: 8258554: javax/swing/JTable/4235420/bug4235420.java fails in GTK L&F
  • c50b464: 8258715: [JVMCI] separate JVMCI code install timers for CompileBroker and hosted compilations
  • 64644a1: 8253881: Hotspot/Serviceability Terminology Refresh
  • 6a78b2a: 8258645: Bring Jemmy 1.3.11 to JDK test base
  • 7f92d18: 8258553: Limit number of fields in instance to be considered for scalar replacement
  • adf0e23: 8257800: CompileCommand TypedMethodOptionMatcher::parse_method_pattern() may over consume
  • 06c24e1: 8256213: Remove os::split_reserved_memory
  • be41468: 8258696: Temporarily revert use of pattern match instanceof until docs-reference is fixed
  • ... and 256 more: https://git.openjdk.java.net/jdk/compare/55f5542ca2104df91e14693534cc7b3c36e81953...master

Your commit was automatically rebased without conflicts.

Pushed as commit fa50877.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@mrserb
Copy link
Member

@mrserb mrserb commented Dec 20, 2020

It is a mistake that this fix was integrated, actually without proper approval of the AWT side. It would be good to roll back it one way or another.

@mrserb
Copy link
Member

@mrserb mrserb commented Dec 20, 2020

I guess we disagree. I think this is better and simpler.

It is just horrible usage of macros in the objective-c code.

@prrace prrace deleted the jnf2jni_1 branch Feb 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
6 participants