JDK-8293659: Improve UnsatisfiedLinkError error message to include dlopen error details #10286

MBaesken · 2022-09-15T11:54:41Z

When trying to load a x64 lib on macOS aarch64 one got previously this detailed message before JDK-8275703:

java.lang.UnsatisfiedLinkError: /testing/jco3/macOsx64/libsapjco3.dylib: dlopen(/testing/jco3/macOsx64/libsapjco3.dylib, 1): no suitable image found. Did find:
/testing/jco3/macOsx64/libsapjco3.dylib: mach-o, but wrong architecture
/testing/jco3/macOsx64/libsapjco3.dylib: mach-o, but wrong architecture [in thread "main"]

After JDK-8275703, the error message does not include the details:
java.lang.UnsatisfiedLinkError: Can't load library: /testing/jco3/macOsx64/libsapjco3.dylib [in thread "main"]

The error details are useful to help diagnosing user's error like mismatched target architecture.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8293659: Improve UnsatisfiedLinkError error message to include dlopen error details

Reviewers

Mandy Chung (@mlchung - Reviewer)
Lutz Schmidt (@RealLucy - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/10286/head:pull/10286
$ git checkout pull/10286

Update a local copy of the PR:
$ git checkout pull/10286
$ git pull https://git.openjdk.org/jdk pull/10286/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 10286

View PR using the GUI difftool:
$ git pr show -t 10286

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/10286.diff

bridgekeeper · 2022-09-15T11:55:21Z

👋 Welcome back mbaesken! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2022-09-15T11:57:39Z

@MBaesken The following label will be automatically applied to this pull request:

core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2022-09-15T12:00:18Z

Webrevs

01: Full - Incremental (8750d0f0)
00: Full (79834e6b)

mlchung · 2022-09-15T17:43:13Z

src/java.base/share/classes/jdk/internal/loader/NativeLibraries.java

+
+            // If the file exists but fails to load, UnsatisfiedLinkException thrown by the VM
+            // will include the error message from dlopen to provide diagnostic information
+            return fileExists;


Can drop the fileExists variable and simply return AccessController.doPrivileged(....);

Otherwise, looks good.

mlchung · 2022-09-15T17:56:09Z

Adding the background and details for this review:

The throwException parameter in JVM_LoadLibrary was added to support dynamic linker cache. dlopen may succeed when the given path does not exist but in the dynamic linker cache. System::loadLibrary on macOS that supports dynamic linker cache calls JVM_LoadLibrary with throwException == false and so VM doesn't throw UnsatisfiedLinkException. Instead the libraries code throws UnsatisfiedLinkException if it fails to load the library.

For this issue, a library to be loaded exists. A simple fix is to call JVM_LoadLibrary with throwException = true (if file exists) such that the VM throws UnsatisfiedLinkException with the detailed error message.

MBaesken · 2022-09-16T07:22:26Z

Hi Mandy, thanks for the comment and the suggestion above. I adjusted throwExceptionIfFail.

mlchung

Looks good to me. Thanks for taking this.

openjdk · 2022-09-16T16:45:33Z

@MBaesken This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8293659: Improve UnsatisfiedLinkError error message to include dlopen error details

Reviewed-by: mchung, lucy

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 73 new commits pushed to the master branch:

844a95b: 8292892: Javadoc index descriptions are not deterministic
8d1dd6a: 8294076: Improve ant detection in idea.sh
4e7cb15: 8293480: IGV: Update Bytecode and ControlFlow Component immediately when opening a new graph
8ecdaa6: 8294000: Filler array klass should be in jdk/vm/internal, not in java/vm/internal
379f309: 8287217: C2: PhaseCCP: remove not visited nodes, prevent type inconsistency
12e3510: 8293798: Fix test bugs due to incompatibility with -XX:+AlwaysIncrementalInline
cb72f80: 8293978: Duplicate simple loop back-edge will crash the vm
cddd6de: 8279941: sun/security/pkcs11/Signature/TestDSAKeyLength.java fails when NSS version detection fails
21008ca: 8285383: vmTestbase/nsk/jvmti/scenarios/hotswap/HS204/hs204t001/hs204t001.java failed with "exit code: 96"
3b438a6: 8294067: [macOS] javax/swing/JComboBox/6559152/bug6559152.java Cannot select an item from popup with the ENTER key.
... and 63 more: https://git.openjdk.org/jdk/compare/8f3bbe950fb5a3d9f6cae122209df01df0f342f0...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

tstuefe · 2022-09-17T07:00:09Z

Hi @mlchung,

I tried to understand this, and your fix for JDK-8275703 (#6127). What I don't understand is why we suppressed the exception in JVM_LoadLibrary in the first place.

So, in Big Sur dlopen could succeed if the library is not present in the file system. Therefore we have to omit the file exists check before attempting to dlopen. That makes sense.

Then we attempt to load the library, dive into JVM_LoadLibrary->os::dll_load(), and do a dlopen(), which could succeed or fail. If it succeeds, all is well. If it fails, os::dll_load() just returns dlerror() to JVM_LoadLibrary, which packs the error string into an exception and throws it.

But why did we have to prevent the exception in JVM_LoadLibrary at that point? As it is now, we don't get error details on systems with a dynamic loader cache, since we throw away the dlerror output?

MBaesken · 2022-09-20T07:32:55Z

Hi Thomas, I think previously (before supporting the macOS dl-cache) we did not even attempt to call JVM_LoadLibrary when the file was not present. I asked myself the same thing you mentioned , Mandy had this remark on the topic :
"System::loadLibrary attempts to load library found from "java.library.path" (and also from the system library path). For each path given in "java.library.path", it constructs the filepath of the library and checks if that exists. If exists, it loads the library using dlopen. In that case, throw exception by VM makes sense. If the filepath of that library does not exist, it could be in the dynamic cache or it could be just non-existent - at this point, we can't throw exception because we can't tell until all paths in "java.library.path" are searched. "

mlchung · 2022-09-20T17:13:41Z

If you load a library via System::load, it only loads the given file path which must be an absolute path name. In this case, it only calls JVM_LoadLibrary once and also it will use dlopen's error message if it fails to load it.

If loading a library via System::loadLibrary, it will search through the library paths and system library paths and UnsatisfiedLinkException will be thrown if either (1) no file exists and not present in the dynamic cache or (2) a named library exists but fail to load.

The case when JVM_LoadLibrary is called without exception is to allow the search through the library path (e.g. dir1:dir2) that libFoo does not exist in dir1 and dir2 but dir2/libFoo is present in the dynamic cache.

RealLucy

Changes look good to me.

MBaesken · 2022-09-21T11:31:29Z

/integrate

openjdk · 2022-09-21T11:32:34Z

Going to push as commit da4fdfb.
Since your change was applied there have been 75 commits pushed to the master branch:

cd1cdcd: 8293116: Incremental JDK build could be sped up
e9401e6: 8293364: IGV: Refactor Action in EditorTopComponent and fix minor bugs
844a95b: 8292892: Javadoc index descriptions are not deterministic
8d1dd6a: 8294076: Improve ant detection in idea.sh
4e7cb15: 8293480: IGV: Update Bytecode and ControlFlow Component immediately when opening a new graph
8ecdaa6: 8294000: Filler array klass should be in jdk/vm/internal, not in java/vm/internal
379f309: 8287217: C2: PhaseCCP: remove not visited nodes, prevent type inconsistency
12e3510: 8293798: Fix test bugs due to incompatibility with -XX:+AlwaysIncrementalInline
cb72f80: 8293978: Duplicate simple loop back-edge will crash the vm
cddd6de: 8279941: sun/security/pkcs11/Signature/TestDSAKeyLength.java fails when NSS version detection fails
... and 65 more: https://git.openjdk.org/jdk/compare/8f3bbe950fb5a3d9f6cae122209df01df0f342f0...master

Your commit was automatically rebased without conflicts.

openjdk · 2022-09-21T11:32:50Z

@MBaesken Pushed as commit da4fdfb.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

JDK-8293659

79834e6

openjdk bot added the rfr Pull request is ready for review label Sep 15, 2022

openjdk bot added the core-libs core-libs-dev@openjdk.org label Sep 15, 2022

mlchung reviewed Sep 15, 2022

View reviewed changes

Adjust throwExceptionIfFail

8750d0f

MBaesken requested a review from mlchung September 16, 2022 07:22

mlchung approved these changes Sep 16, 2022

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Sep 16, 2022

RealLucy approved these changes Sep 21, 2022

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Sep 21, 2022

openjdk bot closed this Sep 21, 2022

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Sep 21, 2022

JDK-8293659: Improve UnsatisfiedLinkError error message to include dlopen error details #10286

JDK-8293659: Improve UnsatisfiedLinkError error message to include dlopen error details #10286

MBaesken commented Sep 15, 2022 •

edited by openjdk bot

bridgekeeper bot commented Sep 15, 2022

openjdk bot commented Sep 15, 2022

mlbridge bot commented Sep 15, 2022 •

edited

mlchung Sep 15, 2022

mlchung commented Sep 15, 2022 •

edited

MBaesken commented Sep 16, 2022 •

edited

mlchung left a comment

openjdk bot commented Sep 16, 2022 •

edited

tstuefe commented Sep 17, 2022

MBaesken commented Sep 20, 2022 •

edited

mlchung commented Sep 20, 2022

RealLucy left a comment

MBaesken commented Sep 21, 2022

openjdk bot commented Sep 21, 2022

openjdk bot commented Sep 21, 2022

JDK-8293659: Improve UnsatisfiedLinkError error message to include dlopen error details #10286

JDK-8293659: Improve UnsatisfiedLinkError error message to include dlopen error details #10286

Conversation

MBaesken commented Sep 15, 2022 • edited by openjdk bot

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented Sep 15, 2022

openjdk bot commented Sep 15, 2022

mlbridge bot commented Sep 15, 2022 • edited

Webrevs

mlchung Sep 15, 2022

Choose a reason for hiding this comment

mlchung commented Sep 15, 2022 • edited

MBaesken commented Sep 16, 2022 • edited

mlchung left a comment

Choose a reason for hiding this comment

openjdk bot commented Sep 16, 2022 • edited

tstuefe commented Sep 17, 2022

MBaesken commented Sep 20, 2022 • edited

mlchung commented Sep 20, 2022

RealLucy left a comment

Choose a reason for hiding this comment

MBaesken commented Sep 21, 2022

openjdk bot commented Sep 21, 2022

openjdk bot commented Sep 21, 2022

MBaesken commented Sep 15, 2022 •

edited by openjdk bot

mlbridge bot commented Sep 15, 2022 •

edited

mlchung commented Sep 15, 2022 •

edited

MBaesken commented Sep 16, 2022 •

edited

openjdk bot commented Sep 16, 2022 •

edited

MBaesken commented Sep 20, 2022 •

edited