Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mongo: Class Not Found jnr.unixsocket.UnixSocket when running in Native #15434

Closed
Sgitario opened this issue Mar 3, 2021 · 29 comments
Closed
Labels

Comments

@Sgitario
Copy link
Contributor

Sgitario commented Mar 3, 2021

Describe the bug
We have a project that uses a lot of Quarkus extensions at the same time. One of these extensions is Mongo Panache and seems to be an incompatibility because when compiling the project in JVM, it works, but not in Native with the following error:

[002-quarkus-all-extensions-1.0.0-SNAPSHOT-runner:24]    classlist:  45.993,33 ms,  1,96 GB
[002-quarkus-all-extensions-1.0.0-SNAPSHOT-runner:24]        setup:   3.249,06 ms,  1,96 GB
Fatal error:java.lang.NoClassDefFoundError
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
	at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:488)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:370)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:529)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:119)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:561)
Caused by: java.lang.NoClassDefFoundError: jnr/unixsocket/UnixSocket
	at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
	at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
	at java.base/java.lang.Class.getDeclaredMethods(Class.java:2309)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleDeletedClass(AnnotationSubstitutionProcessor.java:512)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:305)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:265)
	at com.oracle.svm.hosted.NativeImageGenerator.createDeclarativeSubstitutionProcessor(NativeImageGenerator.java:929)
	at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:863)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:561)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:476)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.ClassNotFoundException: jnr.unixsocket.UnixSocket
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 16 more

Note that when using only Mongo Panache, it works fine. I could not identify what other extension is causing Mongo to fail with this missing class.

Expected behavior
The project should work fine regardless the number of Quarkus extensions we're using.

Actual behavior
It fails to compile, see exception above.

To Reproduce
1- git clone https://github.com/Sgitario/beefy-scenarios
2- git checkout reproducer_15434
3- cd beefy-scenarios/002-quarkus-all-extensions
4- mvn clean verify -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-native-image:21.0.0-java11 -Dnative -Dquarkus.native.container-build=true (it needs to have built quarkus master locally)

Environment (please complete the following information):

  • Quarkus version or git rev: 999-SNAPSHOT (1.12.1.Final)
@Sgitario Sgitario added the kind/bug Something isn't working label Mar 3, 2021
@quarkus-bot
Copy link

quarkus-bot bot commented Mar 3, 2021

/cc @evanchooly, @loicmathieu

@loicmathieu
Copy link
Contributor

It's very difficult to try to understand what's going on as the provided stacktrace didn'g give any stack inside Quarkus nor MongoDB.

Can you try to locate which extension have the com.github:jnr-unixsocket dependency ?

Can you also try to remove the quarkus-mongodb-panache extension and only include quarkus-mongodb-client ? I suspect the issue is from the MongoDB client extension as it is the one that provides native image support.

Lastly, you can try to add the missing class and see if it fixes the issue, if yes, we can add it from the MongoDB client extension.

You can use an empty class annotated with `@RegisterForReflection(targets={jnr.unixsocket.UnixSocket.class}) this would do the trick.

@Sgitario
Copy link
Contributor Author

Sgitario commented Mar 3, 2021

It's very difficult to try to understand what's going on as the provided stacktrace didn'g give any stack inside Quarkus nor MongoDB.

Can you try to locate which extension have the com.github:jnr-unixsocket dependency ?

Can you also try to remove the quarkus-mongodb-panache extension and only include quarkus-mongodb-client ? I suspect the issue is from the MongoDB client extension as it is the one that provides native image support.

Lastly, you can try to add the missing class and see if it fixes the issue, if yes, we can add it from the MongoDB client extension.

You can use an empty class annotated with `@RegisterForReflection(targets={jnr.unixsocket.UnixSocket.class}) this would do the trick.

  • No extensions bring the com.github:jnr-unixsocket dependency
  • I confirm it happens even when only including quarkus-mongodb-client
  • Adding the dependency com.github:jnr-unixsocket helped.

I didn't try your last suggestions, but if I understood correctly, are you going then add this dependency as part of the MongoDB client extension as a fix?

@loicmathieu
Copy link
Contributor

This library is used only in the io.quarkus:quarkus-extension-processor dependency as a transitive dependency on the asccidoctorj test dependency.

I have no idea why it causes a native build issue, but this has something to do with the Quarkus extension processor.

@loicmathieu
Copy link
Contributor

@aloubyansky the Quarkus extension processor have a transitive test dependency on com.github:jnr-unixsocket and under some circumstances it causes issue on native build.
Do you have any idea how it is possible that this extension ends somewhere on the classpath during a native build?

@aloubyansky
Copy link
Member

It appears it's not a new issue. See oracle/graal#1239
The dependency comes from the monodb itself. If you look into https://repo1.maven.org/maven2/org/mongodb/mongodb-driver-core/4.0.1/mongodb-driver-core-4.0.1.pom you'll see

    <dependency>
      <groupId>com.github.jnr</groupId>
      <artifactId>jnr-unixsocket</artifactId>
      <version>0.18</version>
      <scope>runtime</scope>
      <optional>true</optional>
    </dependency>

And if you look into com.mongodb.internal.connection.UnixSocketChannelStream which is referenced from io.quarkus.mongodb.runtime.graal.MongoClientSubstitutions, you'll see

import jnr.unixsocket.UnixSocketAddress;
import jnr.unixsocket.UnixSocketChannel;

@aloubyansky
Copy link
Member

It looks like --allow-incomplete-class-path should help as well.

@Sgitario
Copy link
Contributor Author

Sgitario commented Mar 4, 2021

--allow-incomplete-class-path

So, are you suggesting that we should use --allow-incomplete-class-path as a workaround or that the jnr dependency should be part of Mongo Client?

@loicmathieu
Copy link
Contributor

@Sgitario as a workaround you can include the jnr library or add the --allow-incomplete-class-path flag to native.

I will work on a substitution to fix the issue, I don't want to always include this dependency.

@loicmathieu
Copy link
Contributor

Trying to build the native application locally I have the following stacktrace:

Fatal error:java.lang.NoClassDefFoundError
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
	at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:488)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:370)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:529)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:119)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:561)
Caused by: java.lang.NoClassDefFoundError: com/sun/jna/platform/win32/WinNT$HANDLE
	at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
	at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
	at java.base/java.lang.Class.getDeclaredMethods(Class.java:2309)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleDeletedClass(AnnotationSubstitutionProcessor.java:512)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:305)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:265)
	at com.oracle.svm.hosted.NativeImageGenerator.createDeclarativeSubstitutionProcessor(NativeImageGenerator.java:929)
	at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:863)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:561)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:476)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.ClassNotFoundException: com.sun.jna.platform.win32.WinNT$HANDLE
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 16 more

@Sgitario
Copy link
Contributor Author

Sgitario commented Mar 4, 2021

Trying to build the native application locally I have the following stacktrace:

Fatal error:java.lang.NoClassDefFoundError
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
	at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:488)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:370)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:529)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:119)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:561)
Caused by: java.lang.NoClassDefFoundError: com/sun/jna/platform/win32/WinNT$HANDLE
	at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
	at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
	at java.base/java.lang.Class.getDeclaredMethods(Class.java:2309)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleDeletedClass(AnnotationSubstitutionProcessor.java:512)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:305)
	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:265)
	at com.oracle.svm.hosted.NativeImageGenerator.createDeclarativeSubstitutionProcessor(NativeImageGenerator.java:929)
	at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:863)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:561)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:476)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.ClassNotFoundException: com.sun.jna.platform.win32.WinNT$HANDLE
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 16 more

I also faced the same exception and realised that there is an existing issue for this: #6327

@loicmathieu
Copy link
Contributor

@aloubyansky the class com.mongodb.internal.connection.UnixSocketChannelStreamis deleted from the MongoDB extension so we should not have any occurence of JNR classes detected by native image.

@TargetClass(UnixSocketChannelStream.class)
@Delete
final class UnixSocketChannelStreamSubstitution {

}

Something else happens ...
Maybe it's an extension dependency issue, we should check with the PR that fix some extension dependency issue, maybe it's resolved there. WDYT @aloubyansky ?
If this PR is merged, we can test with the snapshot version.

@aloubyansky
Copy link
Member

Which PR are you talking about @loicmathieu ?

@loicmathieu
Copy link
Contributor

This one: #14609
If it could help, I can try to build it locally and use it. The issue reproducer uses the snapshot version of Quarkus so it's easy to setup.

@aloubyansky
Copy link
Member

It doesn't look like it's affecting the mongodb client extension though.

@loicmathieu
Copy link
Contributor

No but the issue is not with the mongodb client extension alone but when it is integrated with other extensions

@aloubyansky
Copy link
Member

Are you saying not every Quarkus application with the mongodb client extension will fail in this way?

@loicmathieu
Copy link
Contributor

No, in fact it's the only one I'm aware of, and this particular one is using almost all exiting Quarkus applicaiton. So the issue may only occurs when some other extension is used with Quarkus.

@aloubyansky
Copy link
Member

Oh, it's that tricky...

@aloubyansky
Copy link
Member

I confirm it happens even when only including quarkus-mongodb-client

I read it as the mongodb-client was the only extension included

@loicmathieu
Copy link
Contributor

I read it as the mongodb-client was the only extension included

Oh no, the context was that both mongodb-client and mongodb-panache was included so I ask to test with only the client to see if it only occurs with it.

@aloubyansky
Copy link
Member

I was trying to build it with my locally installed GraalVM and got a different error

Error: Classes that should be initialized at run time got initialized during image building:
 com.arjuna.ats.internal.arjuna.utils.SocketProcessId the class was requested to be initialized at run time (from feature io.quarkus.runner.AutoFeature.beforeAnalysis). com.arjuna.ats.internal.arjuna.utils.SocketProcessId has been initialized without the native-image initialization instrumentation and the stack trace can't be tracked. Try avoiding to initialize the class that caused initialization of com.arjuna.ats.internal.arjuna.utils.SocketProcessId

Is that a known one?

@aloubyansky
Copy link
Member

Could you please try adding --trace-class-initialization=jnr.unixsocket.UnixSocket and see whether there is more info, because I keep getting a different error.

@loicmathieu
Copy link
Contributor

@aloubyansky try it, no more info.

@zakkak
Copy link
Contributor

zakkak commented Apr 2, 2021

This looks like the same issue with #16139
Please see #16139 (comment) for the culprit.

@loicmathieu
Copy link
Contributor

@Sgitario can you check if removing the AWS extensions resolves the issue ?
If yes, it's the same one as #16139, the issue is upstream on the AWS SDK

@Sgitario
Copy link
Contributor Author

Sgitario commented Apr 6, 2021

@Sgitario can you check if removing the AWS extensions resolves the issue ?
If yes, it's the same one as #16139, the issue is upstream on the AWS SDK

Yep, removing all the AWS extensions fixes this issue. However, I think this issue should remain opened and once the another issue related to AWS is solved, I can try again with Mongo.

@loicmathieu
Copy link
Contributor

@Sgitario apparently the fix on AWS SDK has been integrated in Quarkus 1.13.5.Final.
Can you try again and close the issue if it fixes it ?

Sgitario added a commit to Sgitario/beefy-scenarios that referenced this issue Jun 9, 2021
The fix was done on AWS SDK which has been integrated in Quarkus 1.13.5.Final and later.

quarkusio/quarkus#15434
@Sgitario
Copy link
Contributor Author

Sgitario commented Jun 9, 2021

I've confirmed that this is indeed fixed. Thanks a lot!

@Sgitario Sgitario closed this as completed Jun 9, 2021
Sgitario added a commit to Sgitario/beefy-scenarios that referenced this issue Jun 9, 2021
The fix was done on AWS SDK which has been integrated in Quarkus 1.13.5.Final and later.

quarkusio/quarkus#15434
jsmrcka pushed a commit to quarkus-qe/beefy-scenarios that referenced this issue Jun 16, 2021
The fix was done on AWS SDK which has been integrated in Quarkus 1.13.5.Final and later.

quarkusio/quarkus#15434
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants