8257853: Remove dependencies on JNF's JNI utility functions in AWT and 2D code #1679

prrace · 2020-12-07T21:24:57Z

This defines some macros to support declaring and initialising statically allocated instances of jclass, jmethodID and jfieldID
and changes many existing uses of JNF macros/functions to use these instead.
Then calls to JNFCall* and JNFNewObject - etc are updated to directly call JNI methods
JNI exception checking macros are added as needed.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8257853: Remove dependencies on JNF's JNI utility functions in AWT and 2D code

Reviewers

Prasanta Sadhukhan (@prsadhuk - Reviewer)
Alexander Zuev (@azuev-java - Reviewer) ⚠️ Review applies to b79f8bc

Download

$ git fetch https://git.openjdk.java.net/jdk pull/1679/head:pull/1679
$ git checkout pull/1679

…d 2D code

bridgekeeper · 2020-12-07T21:25:46Z

👋 Welcome back prr! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2020-12-07T21:26:32Z

@prrace The following label will be automatically applied to this pull request:

awt

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

…d 2D code

mlbridge · 2020-12-09T20:38:45Z

Webrevs

prsadhuk

Wouldn't it have been nice to have a new file having same JNF macros so it will not result in changing any source code but just instead of mapping to apple provided JNF layer, we will map to our macro through JNF wrapper and link/unlink apple JNF lib based on version or so in makefile?

VladimirKempik · 2020-12-10T09:57:39Z

This siunds good, especially considering JEP-391, where we have to link ( at build time) with custom made JNF.framework and then provide jdk with JNF.framework in JAVA_HOME/lib

The alternative is to integrate JNF (bsd3 license) into openjdk.

prrace · 2020-12-10T20:55:49Z

Wouldn't it have been nice to have a new file having same JNF macros so it will not result in changing any source code but just instead of mapping to apple provided JNF layer, we will map to our macro through JNF wrapper and link/unlink apple JNF lib based on version or so in makefile?

No. Not really. We should not keep the JNF namespace. Nor is there any desitre to have that 1:1 mapping.
This is NOT a "copy of JNF". It is a conscious attempt to be rid of it and all the unecessary baggage.
So I am not interested in that approach even one little bit

…d 2D code

prrace · 2020-12-12T18:59:39Z

I updated this fix with the additional changes for the accessibility files.
There's a little bit of clean up as some declarations were unused and also these files shared a single defintion of some classes across files which were mostly unncecessary. One class I made static and declared per-file which is cleaner,
Also the only A11Y reference in AWTView was moved into its only call site.
With these changes I think the desktop module is now 100% free of the JNFCall* pattern and the CACHE pattern
I re-ran our full automated headful jtreg test suite and everything passed. I also ran VoiceOver and ran through SwingSet2 without seeing any issues although I am not sure what to expect for this so it would be best if @azuev-java checked it out.

…d 2D code

prrace · 2020-12-12T22:28:52Z

Updated with some fixes for the A11Y support.
Adding some logging that will report any problems looking up a JNI method or field or class.

src/java.desktop/macosx/native/libawt_lwawt/awt/AWTView.m

azuev-java · 2020-12-14T11:45:46Z

I have tested the a11y part and haven't found any functional degradation there.

…d 2D code

azuev-java

Accessibility part is perfectly fine, no regressions.

openjdk · 2020-12-14T22:06:50Z

@prrace This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8257853: Remove dependencies on JNF's JNI utility functions in AWT and 2D code

Reviewed-by: psadhukhan, kizune

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 263 new commits pushed to the master branch:

c50b464: 8258715: [JVMCI] separate JVMCI code install timers for CompileBroker and hosted compilations
64644a1: 8253881: Hotspot/Serviceability Terminology Refresh
6a78b2a: 8258645: Bring Jemmy 1.3.11 to JDK test base
7f92d18: 8258553: Limit number of fields in instance to be considered for scalar replacement
adf0e23: 8257800: CompileCommand TypedMethodOptionMatcher::parse_method_pattern() may over consume
06c24e1: 8256213: Remove os::split_reserved_memory
be41468: 8258696: Temporarily revert use of pattern match instanceof until docs-reference is fixed
a4f393c: 8258661: Inner class ResponseCacheEntry could be static
3c48819: 8169086: DTLS tests fail intermittently with too much loops or timeout
71ae05d: 8258061: Improve diagnostic information about errors during class redefinition
... and 253 more: https://git.openjdk.java.net/jdk/compare/55f5542ca2104df91e14693534cc7b3c36e81953...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

prrace · 2020-12-15T17:42:18Z

anyone up for reviewing the rest of it ?

src/java.desktop/macosx/native/libawt_lwawt/awt/ApplicationDelegate.m

src/java.desktop/macosx/native/libawt_lwawt/awt/AWTView.m

src/java.desktop/macosx/native/libawt_lwawt/awt/ApplicationDelegate.m

src/java.desktop/macosx/native/libosxui/ScreenMenu.m

gerard-ziemski · 2020-12-17T17:55:19Z

src/java.desktop/macosx/native/libawt_lwawt/awt/JavaComponentAccessibility.m

@@ -314,7 +335,7 @@ + (void)initialize
            (*env)->DeleteLocalRef(env, jString);
        }

-        sAccessibilityClass = JNFCallStaticObjectMethod(env, jm_getAccessibility, result); // AWT_THREADING Safe (known object)
+        sAccessibilityClass = (*env)->CallStaticObjectMethod(env, sjc_CAccessibility, jm_getAccessibility, result); // AWT_THREADING Safe (known object)


Do we need CHECK_EXCEPTION(); after CallStaticObjectMethod here?

I will add it.

…d 2D code

prsadhuk

I believe we could add CHECK_EXCEPTION in few places...pointing those out...

src/java.desktop/macosx/native/libosxui/ScreenMenu.m

src/java.desktop/macosx/native/libawt_lwawt/awt/CDragSource.m

src/java.desktop/macosx/native/libawt_lwawt/awt/CDropTarget.m

src/java.desktop/macosx/native/libawt_lwawt/awt/CPrinterJob.m

src/java.desktop/macosx/native/libawt_lwawt/awt/CTrayIcon.m

src/java.desktop/macosx/native/libawt_lwawt/awt/PrinterView.m

gerard-ziemski · 2020-12-18T16:01:17Z

There seems to be a pattern where we replace JNFCallObjectMethod() with a pair of CallObjectMethod() immediately followed by CHECK_EXCEPTION()

There are places where currently in this fix, this pattern is missing the CHECK_EXCEPTION()

Can we replace JNFCallObjectMethod() by a single function call (or macro) that wraps both CallObjectMethod() and CHECK_EXCEPTION() into one?

This would make this change more robust and make it easier to review it.

…d 2D code

prrace · 2020-12-18T19:23:54Z

There seems to be a pattern where we replace JNFCallObjectMethod() with a pair of CallObjectMethod() immediately followed by CHECK_EXCEPTION()

There are places where currently in this fix, this pattern is missing the CHECK_EXCEPTION()

Can we replace JNFCallObjectMethod() by a single function call (or macro) that wraps both CallObjectMethod() and CHECK_EXCEPTION() into one?

This would make this change more robust and make it easier to review it.

Its is a reasonable idea and I thought about it but on the whole I'd prefer not to do it and insted fix missing cases.
Tthere are cases where we are using these macros directly in a native method and then returning to Java.
In those case I want to propagate the exception and this suggestion would make that difficult if not impossible.
Remember that these aren't likely to do anything whether missing or not to make things better or cause
a regression because we should not be generating these exceptions anyway.
This is mostly a recommended precautionary pattern "just in case".
And nowhere else in the JDK uses macro for this - the CHECK_EXCEPTION and similar patterns are
used elsewhere. And code already added by Oracle engineers after the initial macOS port
are using the standard JNI calls.

For comparison I've been running JDK 15 with -Xcheck:jni and there were a number of places that generated
these warnings and I've resolved them here.

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.

mrserb

We should take care of native exceptions, and probably use more objective-c than the macro.

prrace · 2020-12-18T21:52:01Z

We should take care of native exceptions, and probably use more objective-c than the macro.

We will take care of native exceptions - the ENTER/EXIT macros which have yet to be updated will handle these if they occur.
This part of the changes is about JNI up-calls.

mrserb · 2020-12-18T21:57:23Z

There seems to be a pattern where we replace JNFCallObjectMethod() with a pair of CallObjectMethod() immediately followed by CHECK_EXCEPTION()
There are places where currently in this fix, this pattern is missing the CHECK_EXCEPTION()
Can we replace JNFCallObjectMethod() by a single function call (or macro) that wraps both CallObjectMethod() and CHECK_EXCEPTION() into one?
This would make this change more robust and make it easier to review it.

Its is a reasonable idea and I thought about it but on the whole I'd prefer not to do it and insted fix missing cases.
Tthere are cases where we are using these macros directly in a native method and then returning to Java.
In those case I want to propagate the exception and this suggestion would make that difficult if not impossible.

I guess it should work already in the JNF case? The java exception should be wrapped by the NSException in the JNFCallObjectMethod/etc and unwrapped in the JNF_COCOA_ENTER before exists to the java method?

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.

I am not sure about that.

mrserb · 2020-12-18T22:07:12Z

We should take care of native exceptions, and probably use more objective-c than the macro.

We will take care of native exceptions - the ENTER/EXIT macros which have yet to be updated will handle these if they occur.
This part of the changes is about JNI up-calls.

No, you skipped all exceptions in the JNI up-calls, which I am talking about.

prrace · 2020-12-18T22:11:56Z

We should take care of native exceptions, and probably use more objective-c than the macro.

We will take care of native exceptions - the ENTER/EXIT macros which have yet to be updated will handle these if they occur.
This part of the changes is about JNI up-calls.

No, you skipped all exceptions in the JNI up-calls, which I am talking about.

Not sure what you mean.

prrace · 2020-12-18T22:14:52Z

There seems to be a pattern where we replace JNFCallObjectMethod() with a pair of CallObjectMethod() immediately followed by CHECK_EXCEPTION()
There are places where currently in this fix, this pattern is missing the CHECK_EXCEPTION()
Can we replace JNFCallObjectMethod() by a single function call (or macro) that wraps both CallObjectMethod() and CHECK_EXCEPTION() into one?
This would make this change more robust and make it easier to review it.

Its is a reasonable idea and I thought about it but on the whole I'd prefer not to do it and insted fix missing cases.
Tthere are cases where we are using these macros directly in a native method and then returning to Java.
In those case I want to propagate the exception and this suggestion would make that difficult if not impossible.

I guess it should work already in the JNF case? The java exception should be wrapped by the NSException in the JNFCallObjectMethod/etc and unwrapped in the JNF_COCOA_ENTER before exists to the java method?

That is the pattern I know is there today and I don't think is needed.
It can be done but I don't think it needed. I'd prefer to see if we ever have such a situation and deal with it "locally"
at the call site.

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.

I am not sure about that.

I am looking at JNI warnings - not just guessing.

mrserb · 2020-12-18T22:19:11Z

We should take care of native exceptions, and probably use more objective-c than the macro.

We will take care of native exceptions - the ENTER/EXIT macros which have yet to be updated will handle these if they occur.
This part of the changes is about JNI up-calls.

No, you skipped all exceptions in the JNI up-calls, which I am talking about.

Not sure what you mean.

Non of your macro raises exceptions like it was before. All of them just log and clear a java exception, while before such exceptions were transformed to the NSException and handled appropriately.

mrserb · 2020-12-18T22:22:33Z

I guess it should work already in the JNF case? The java exception should be wrapped by the NSException in the JNFCallObjectMethod/etc and unwrapped in the JNF_COCOA_ENTER before exists to the java method?

That is the pattern I know is there today and I don't think is needed.
It can be done but I don't think it needed. I'd prefer to see if we ever have such a situation and deal with it "locally"
at the call site.

No, it is quite useful, and it will be good to preserve it. It is possible to work "locally" when the call site if come from java, but not if we are in the infinite native loop.

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.
I am not sure about that.
I am looking at JNI warnings - not just guessing.

I am looking at a new code that is not as good as before.

prrace · 2020-12-18T22:26:45Z

I guess it should work already in the JNF case? The java exception should be wrapped by the NSException in the JNFCallObjectMethod/etc and unwrapped in the JNF_COCOA_ENTER before exists to the java method?

That is the pattern I know is there today and I don't think is needed.
It can be done but I don't think it needed. I'd prefer to see if we ever have such a situation and deal with it "locally"
at the call site.

No, it is quite useful, and it will be good to preserve it. It is possible to work "locally" when the call site if come from java, but not if we are in the infinite native loop.

an NS exception on the appkit will terminate the app - if unhandled - and probably will even if handled
so I am not sure what is better about that

So I believe we may end up more (theoretically) robust with my changes than we were before.
Perhaps they weren't a result of a JNF* call but still it seems improved.
I am not sure about that.
I am looking at JNI warnings - not just guessing.

I am looking at a new code that is not as good as before.

I guess we disagree. I think this is better and simpler.

prrace · 2020-12-18T23:46:05Z

Not trying to prolong or expand this discussion but a 1:1 mapping to JNF to really minimise the review needs to account for

We aren't going to call it "JNF"
The lookup methods for members is quirky in that it is the same for methods and fields and I can only suppose that the signature is parsed to look for "(...)" to decide which it should be
The calling of static methods (and look up of static field values) doesn't take the class as an argument so there's
some complexity in there just to support that which I think un-needed
We'd need a bunch more macros to handle all the same "Call" cases as JNF has and that seems un-neccessary too.
There are already a substantial number of cases where we don't use JNF
So long as we have them in place the CHECK calls are simpler and if we ever reallly wanted to we could have a variant of it that does the odd Java->NS->Java exception think. Or just change the behaviour of the current one. But I stand by what I wrote that I don't think this is needed. This is code which doesn't get exercised - it is more like InternalError than IOException handling.

mlbridge · 2020-12-19T00:24:58Z

Mailing list message from Alan Snyder on awt-dev:

I use JNF in my own (non-JDK) code, and I assume others do also.

Will the macOS JNI support functions and macros you want to use in the JDK be available for use in non-JDK code?

mlbridge · 2020-12-19T00:28:58Z

Mailing list message from Philip Race on awt-dev:

Not something I'd considered as a goal.
I suppose its part of OpenJDK which means it will be there for anyone to
use under the GPL2+CP ...
but there's no thought of delivering it like we deliver jni.h with the
binary JDK distribution.

-phil.

On 12/18/20, 4:23 PM, Alan Snyder wrote:

I use JNF in my own (non-JDK) code, and I assume others do also.

Will the macOS JNI support functions and macros you want to use in the JDK be available for use in non-JDK code?

mlbridge · 2020-12-19T00:41:56Z

Mailing list message from Alan Snyder on awt-dev:

If it?s better than JNF, as you suggest, perhaps it should be released and supported.

Alan

On Dec 18, 2020, at 4:28 PM, Philip Race <philip.race at oracle.com> wrote:

Not something I'd considered as a goal.
I suppose its part of OpenJDK which means it will be there for anyone to use under the GPL2+CP ...
but there's no thought of delivering it like we deliver jni.h with the binary JDK distribution.

-phil.

On 12/18/20, 4:23 PM, Alan Snyder wrote:

I use JNF in my own (non-JDK) code, and I assume others do also.

Will the macOS JNI support functions and macros you want to use in the JDK be available for use in non-JDK code?

mlbridge · 2020-12-19T00:53:56Z

Mailing list message from Philip Race on awt-dev:

Ah .. not my call .. complicated.

-phil.

On 12/18/20, 4:40 PM, Alan Snyder wrote:

If it?s better than JNF, as you suggest, perhaps it should be released and supported.

Alan

On Dec 18, 2020, at 4:28 PM, Philip Race<philip.race at oracle.com> wrote:

Not something I'd considered as a goal.
I suppose its part of OpenJDK which means it will be there for anyone to use under the GPL2+CP ...
but there's no thought of delivering it like we deliver jni.h with the binary JDK distribution.

-phil.

On 12/18/20, 4:23 PM, Alan Snyder wrote:

I use JNF in my own (non-JDK) code, and I assume others do also.

Will the macOS JNI support functions and macros you want to use in the JDK be available for use in non-JDK code?

prsadhuk

looks good to me.

prrace · 2020-12-19T23:34:26Z

/integrate

openjdk · 2020-12-19T23:35:27Z

@prrace Since your change was applied there have been 266 commits pushed to the master branch:

580af49: 8258006: Replaces while cycles with iterator with enhanced for in java.desktop
dc7f01f: 8257584: [macos] NullPointerException originating from LWCToolkit.java
c7c53d0: 8258554: javax/swing/JTable/4235420/bug4235420.java fails in GTK L&F
c50b464: 8258715: [JVMCI] separate JVMCI code install timers for CompileBroker and hosted compilations
64644a1: 8253881: Hotspot/Serviceability Terminology Refresh
6a78b2a: 8258645: Bring Jemmy 1.3.11 to JDK test base
7f92d18: 8258553: Limit number of fields in instance to be considered for scalar replacement
adf0e23: 8257800: CompileCommand TypedMethodOptionMatcher::parse_method_pattern() may over consume
06c24e1: 8256213: Remove os::split_reserved_memory
be41468: 8258696: Temporarily revert use of pattern match instanceof until docs-reference is fixed
... and 256 more: https://git.openjdk.java.net/jdk/compare/55f5542ca2104df91e14693534cc7b3c36e81953...master

Your commit was automatically rebased without conflicts.

Pushed as commit fa50877.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

mrserb · 2020-12-20T12:20:08Z

It is a mistake that this fix was integrated, actually without proper approval of the AWT side. It would be good to roll back it one way or another.

mrserb · 2020-12-20T12:24:46Z

I guess we disagree. I think this is better and simpler.

It is just horrible usage of macros in the objective-c code.

8257853: Remove dependencies on JNF's JNI utility functions in AWT an…

f3e17ce

…d 2D code

openjdk bot added the awt client-libs-dev@openjdk.org label Dec 7, 2020

8257853: Remove dependencies on JNF's JNI utility functions in AWT an…

2124623

…d 2D code

openjdk bot added the rfr Pull request is ready for review label Dec 9, 2020

8257853: Remove dependencies on JNF's JNI utility functions in AWT an…

e850b1e

…d 2D code

prsadhuk reviewed Dec 10, 2020

View reviewed changes

prrace added 2 commits December 10, 2020 13:09

src/java.desktop/macosx/native/libawt_lwawt/font/AWTStrike.m

839f8b7

8257853: Remove dependencies on JNF's JNI utility functions in AWT an…

7a084f9

…d 2D code

openjdk bot removed the rfr Pull request is ready for review label Dec 12, 2020

8257853: Remove dependencies on JNF's JNI utility functions in AWT an…

2ccaa75

…d 2D code

openjdk bot added the rfr Pull request is ready for review label Dec 12, 2020

8257853: Remove dependencies on JNF's JNI utility functions in AWT an…

97b9a91

…d 2D code

azuev-java reviewed Dec 14, 2020

View reviewed changes

src/java.desktop/macosx/native/libawt_lwawt/awt/AWTView.m Show resolved Hide resolved

8257853: Remove dependencies on JNF's JNI utility functions in AWT an…

b79f8bc

…d 2D code

azuev-java approved these changes Dec 14, 2020

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Dec 14, 2020

prsadhuk reviewed Dec 17, 2020

View reviewed changes

src/java.desktop/macosx/native/libawt_lwawt/awt/ApplicationDelegate.m Show resolved Hide resolved

prsadhuk reviewed Dec 17, 2020

View reviewed changes

src/java.desktop/macosx/native/libawt_lwawt/awt/AWTView.m Show resolved Hide resolved

src/java.desktop/macosx/native/libawt_lwawt/awt/ApplicationDelegate.m Show resolved Hide resolved

src/java.desktop/macosx/native/libosxui/ScreenMenu.m Show resolved Hide resolved

gerard-ziemski reviewed Dec 17, 2020

View reviewed changes

8257853: Remove dependencies on JNF's JNI utility functions in AWT an…

346cf1a

…d 2D code

prsadhuk reviewed Dec 18, 2020

View reviewed changes

8257853: Remove dependencies on JNF's JNI utility functions in AWT an…

d9e09b3

…d 2D code

mrserb suggested changes Dec 18, 2020

View reviewed changes

prrace closed this Dec 18, 2020

prrace reopened this Dec 18, 2020

prsadhuk approved these changes Dec 19, 2020

View reviewed changes

openjdk bot closed this Dec 19, 2020

openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Dec 19, 2020

prrace deleted the jnf2jni_1 branch February 10, 2021 19:37

8257853: Remove dependencies on JNF's JNI utility functions in AWT and 2D code #1679

8257853: Remove dependencies on JNF's JNI utility functions in AWT and 2D code #1679

Conversation

prrace commented Dec 7, 2020 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Download

bridgekeeper bot commented Dec 7, 2020

openjdk bot commented Dec 7, 2020

mlbridge bot commented Dec 9, 2020 • edited Loading

Webrevs

prsadhuk left a comment

Choose a reason for hiding this comment

VladimirKempik commented Dec 10, 2020

prrace commented Dec 10, 2020

prrace commented Dec 12, 2020

prrace commented Dec 12, 2020 • edited Loading

azuev-java commented Dec 14, 2020

azuev-java left a comment

Choose a reason for hiding this comment

openjdk bot commented Dec 14, 2020 • edited Loading

prrace commented Dec 15, 2020

gerard-ziemski Dec 17, 2020

Choose a reason for hiding this comment

prrace Dec 17, 2020

Choose a reason for hiding this comment

prsadhuk left a comment

Choose a reason for hiding this comment

gerard-ziemski commented Dec 18, 2020

prrace commented Dec 18, 2020

mrserb left a comment

Choose a reason for hiding this comment

prrace commented Dec 18, 2020

mrserb commented Dec 18, 2020

mrserb commented Dec 18, 2020

prrace commented Dec 18, 2020

prrace commented Dec 18, 2020

mrserb commented Dec 18, 2020

mrserb commented Dec 18, 2020

prrace commented Dec 18, 2020

prrace commented Dec 18, 2020

mlbridge bot commented Dec 19, 2020

mlbridge bot commented Dec 19, 2020

mlbridge bot commented Dec 19, 2020

mlbridge bot commented Dec 19, 2020

prsadhuk left a comment

Choose a reason for hiding this comment

prrace commented Dec 19, 2020

openjdk bot commented Dec 19, 2020

mrserb commented Dec 20, 2020

mrserb commented Dec 20, 2020

prrace commented Dec 7, 2020 •

edited by openjdk bot

Loading

mlbridge bot commented Dec 9, 2020 •

edited

Loading

prrace commented Dec 12, 2020 •

edited

Loading

openjdk bot commented Dec 14, 2020 •

edited

Loading