Using netty 4.1.36.Final for MQTT Connection in a native-image with graalVM 19.1.1 dos not work. #1572

ahelmbr · 2019-08-05T20:22:43Z

For my MQTT client application - using netty 4.1.36.Final
I tried to build a native-image with graalVM 19.1.1
The compilation is successful, the image will be build.

But running the application and trying an MQTT connect throws the following error:

Caused by: java.lang.IllegalArgumentException: Can't find '[toLeakAwareBuffer]' in io.netty.buffer.AbstractByteBufAllocator
        at io.netty.util.ResourceLeakDetector.addExclusions(ResourceLeakDetector.java:574)
        at io.netty.buffer.AbstractByteBufAllocator.<clinit>(AbstractByteBufAllocator.java:36)
        at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350)
        at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270)

My
native-image.properties are:

ImageName=mqtt
Args=-H:ReflectionConfigurationResources=${.}/reflection-config.json \
  -H:Class=com.hmq.cli.Mqtt \
  -H:+ReportExceptionStackTraces \
  -H:+ReportUnsupportedElementsAtRuntime \
  -H:IncludeResourceBundles=jline.console.completer.CandidateListCompletionHandler \
  -H:+JNI \
  --report-unsupported-elements-at-runtime \
  --allow-incomplete-classpath \
  --enable-all-security-services \
  -Dio.netty.noUnsafe=true \
  -Dio.netty.leakDetection.level=DISABLED \

Pom.xml dependencies for netty and substrateVM

        <dependency>
            <groupId>com.oracle.substratevm</groupId>
            <artifactId>svm</artifactId>
            <version>19.1.1</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-handler</artifactId>
            <version>4.1.36.Final</version>
        </dependency>

        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-transport-native-epoll</artifactId>
            <classifier>linux-x86_64</classifier>
            <version>4.1.36.Final</version>
        </dependency>

        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-codec-http</artifactId>
            <version>4.1.36.Final</version>
        </dependency>

reflection-config.json - snippet

  {
    "name": "io.netty.channel.socket.nio.NioSocketChannel",
    "methods": [
      { "name": "<init>", "parameterTypes": [] }
    ]
  },
  {
    "name": "io.netty.channel.socket.nio.NioServerSocketChannel",
    "methods": [
      { "name": "<init>", "parameterTypes": [] }
    ]
  }

Any hint what I have to add to reflection-config or to native image properties that netty is working?
Thanks.

The text was updated successfully, but these errors were encountered:

bobmcwhirter · 2019-08-06T19:49:36Z

I don't know the exact answer, but from

https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/ResourceLeakDetector.java#L564-L575

It appears that Netty is attempting to iterate all declared methods of io.netty.buffer.AbstractByteBufAllocator and is expecting to find a method named toLeakAwareBuffer(...) which does normally exist:

https://github.com/netty/netty/blob/4.1/buffer/src/main/java/io/netty/buffer/AbstractByteBufAllocator.java#L39-L59

But since dead-code elimination occurs with Substrate, that method, if un-used, is removed from the reflection data and appears to simply not exist.

If you add that combination of class (AbstractByteBufAllocator) and method (toLeakAwareBuffer(...), probably both variants would be safest), I suspect this error might stop occurring in your application.

bobmcwhirter · 2019-08-06T19:50:43Z

to be fair, even if that method is actually used, it won't be reflectable via getDeclaredMethods() unless you add it to the reflection config.

There is an outstanding PR, not yet merged, which might solve this also, but it has other more exciting issues at the moment. Until/unless that happens, adding the above solution might help.

ahelmbr · 2019-08-08T15:18:14Z

Now 'Im getting a little bit forward and running into the next error

Caused by: java.lang.ClassCastException: java.lang.Class cannot be cast to org.jctools.queues.LinkedQueueNode
        at org.jctools.queues.MpscLinkedQueue8.xchgProducerNode(MpscLinkedQueue8.java:28)
        at org.jctools.queues.MpscLinkedQueue.<init>(MpscLinkedQueue.java:57)
        at org.jctools.queues.MpscLinkedQueue8.<init>(MpscLinkedQueue8.java:23)
        at org.jctools.queues.MpscLinkedQueue.newMpscLinkedQueue(MpscLinkedQueue.java:45)
        at com.my.client.internal.mqtt.handler.subscribe.MqttSubscriptionHandler.<init>(MqttSubscriptionHandler.java:71)

I added further substitution


@TargetClass(io.netty.buffer.AbstractByteBufAllocator.class)
final class Target_io_netty_buffer_AbstractByteBufAllocator {
    @Substitute
    protected static ByteBuf toLeakAwareBuffer(ByteBuf buf) {
        return buf;
    }
}

@TargetClass(className = "org.jctools.util.UnsafeRefArrayAccess")
final class Target_org_jctools_util_UnsafeRefArrayAccess {
    @Alias
    @RecomputeFieldValue(kind = Kind.ArrayIndexShift, declClass = Object[].class) //
    public static int REF_ELEMENT_SHIFT;
}

@TargetClass(className = "org.jctools.util.UnsafeAccess")
final class Target_org_jctools_util_UnsafeAccess {
    @Substitute
    protected static long fieldOffset(Class clz, String fieldName) {
        return 0L;
    }
}

and in reflection.json

... {
    "name": "io.netty.buffer.AbstractByteBufAllocator",
    "allDeclaredConstructors" : true,
    "allPublicConstructors" : true,
    "allDeclaredMethods" : true,
    "allPublicMethods" : true
  }, {
    "name": "org.jctools.util.UnsafeAccess",
    "allDeclaredConstructors" : true,
    "allPublicConstructors" : true,
    "allDeclaredMethods" : true,
    "allPublicMethods" : true
  }

the native-image props - i override the deprecated settings from netty


ImageName=mqtt
Args=-H:ReflectionConfigurationResources=${.}/reflection-config.json \
  -H:Class=com.hmq.cli.Mqtt \
  -H:+ReportExceptionStackTraces \
  -H:+ReportUnsupportedElementsAtRuntime \
  -H:IncludeResourceBundles=jline.console.completer.CandidateListCompletionHandler \
  -H:+JNI \
  --report-unsupported-elements-at-runtime \
  --rerun-class-initialization-at-runtime=io.netty.handler.codec.http2.Http2CodecUtil\
  --initialize-at-run-time=io.netty.handler.codec.http2.DefaultHttp2FrameWriter\
  --initialize-at-run-time=io.netty.handler.codec.http.HttpObjectEncoder,io.netty.handler.codec.http.websocketx.WebSocket00FrameEncoder\
  --initialize-at-build-time=com.google.protobuf \
  --allow-incomplete-classpath \
  --enable-all-security-services \
  -Dio.netty.leakDetection.level=DISABLED \
  -Dio.netty.noUnsafe=true \

So I can build the image without any warnings
But get the Stacktrace mentioned above.
I guess if I try to fix this somehow weird - I stumble into the next thing - maybe a segmentation fault - so is there any chance to use it for my MQTT (TCP) connection use case ?

cstancu · 2019-08-26T20:47:58Z

@ahelmbr can you share code that can be used to replicate the reported error? I would recommend using the native-image-agent to automatically detect the reflectively accessed elements. Looking at your substitutions I think just

@TargetClass(className = "org.jctools.util.UnsafeRefArrayAccess")
final class Target_org_jctools_util_UnsafeRefArrayAccess {
    @Alias
    @RecomputeFieldValue(kind = Kind.ArrayIndexShift, declClass = Object[].class) //
    public static int REF_ELEMENT_SHIFT;
}

is required. We integrated the same in Netty. A similar integration could be done in JCTools. I think

@TargetClass(io.netty.buffer.AbstractByteBufAllocator.class)
final class Target_io_netty_buffer_AbstractByteBufAllocator {
    @Substitute
    protected static ByteBuf toLeakAwareBuffer(ByteBuf buf) {
        return buf;
    }
}

may break some internal netty assumptions and I don't think it should be needed.

I also think that you should remove:

@TargetClass(className = "org.jctools.util.UnsafeAccess")
final class Target_org_jctools_util_UnsafeAccess {
    @Substitute
    protected static long fieldOffset(Class clz, String fieldName) {
        return 0L;
    }
}

jramirez-isc · 2020-06-04T21:03:16Z

Closing this due to inactivity. Please reopen if the issue still persists.

deen13 · 2020-10-30T08:58:25Z

I'm currently working on a mqtt load generation tool which I would love to distribute as GraalVM Native Image but I'm stuck with some reflection problems.

Exception in thread "main" java.lang.ExceptionInInitializerError
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:291)
	at java.lang.Class.ensureInitialized(DynamicHub.java:515)
	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:236)
	at java.lang.Class.ensureInitialized(DynamicHub.java:515)
	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:236)
	at java.lang.Class.ensureInitialized(DynamicHub.java:515)
	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:236)
	at java.lang.Class.ensureInitialized(DynamicHub.java:515)
	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:236)
	at java.lang.Class.ensureInitialized(DynamicHub.java:515)
	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:236)
	at com.hivemq.client.internal.mqtt.handler.publish.outgoing.MqttOutgoingQosHandler.<init>(MqttOutgoingQosHandler.java:92)
	at com.hivemq.client.internal.mqtt.handler.publish.outgoing.MqttOutgoingQosHandler_Factory.newInstance(MqttOutgoingQosHandler_Factory.java:34)
	at com.hivemq.client.internal.mqtt.handler.publish.outgoing.MqttOutgoingQosHandler_Factory.get(MqttOutgoingQosHandler_Factory.java:25)
	at com.hivemq.client.internal.mqtt.handler.publish.outgoing.MqttOutgoingQosHandler_Factory.get(MqttOutgoingQosHandler_Factory.java:8)
	at dagger.internal.DoubleCheck.get(DoubleCheck.java:47)
	at com.hivemq.client.internal.mqtt.handler.MqttSession_Factory.get(MqttSession_Factory.java:35)
	at com.hivemq.client.internal.mqtt.handler.MqttSession_Factory.get(MqttSession_Factory.java:10)
	at dagger.internal.DoubleCheck.get(DoubleCheck.java:47)
	at com.hivemq.client.internal.mqtt.handler.connect.MqttConnectHandler_Factory.get(MqttConnectHandler_Factory.java:43)
	at com.hivemq.client.internal.mqtt.handler.connect.MqttConnectHandler_Factory.get(MqttConnectHandler_Factory.java:11)
	at dagger.internal.DoubleCheck.get(DoubleCheck.java:47)
	at com.hivemq.client.internal.mqtt.handler.MqttChannelInitializer_Factory.get(MqttChannelInitializer_Factory.java:60)
	at com.hivemq.client.internal.mqtt.handler.MqttChannelInitializer_Factory.get(MqttChannelInitializer_Factory.java:17)
	at dagger.internal.DoubleCheck.get(DoubleCheck.java:47)
	at com.hivemq.client.internal.mqtt.ioc.DaggerSingletonComponent$ClientComponentImpl$ConnectionComponentImpl.bootstrap(DaggerSingletonComponent.java:440)
	at com.hivemq.client.internal.mqtt.handler.connect.MqttConnAckSingle.connect(MqttConnAckSingle.java:86)
	at com.hivemq.client.internal.mqtt.handler.connect.MqttConnAckSingle.subscribeActual(MqttConnAckSingle.java:68)
	at io.reactivex.Single.subscribe(Single.java:3666)
	at io.reactivex.Single.blockingGet(Single.java:2869)
	at com.hivemq.client.internal.mqtt.MqttBlockingClient.connect(MqttBlockingClient.java:99)
	at com.hivemq.client.internal.mqtt.mqtt3.Mqtt3BlockingClientView.connect(Mqtt3BlockingClientView.java:77)
	at com.hivemq.client.internal.mqtt.mqtt3.Mqtt3BlockingClientView.connect(Mqtt3BlockingClientView.java:70)
	at de.smartsquare.kortance.mqtt.Mqtt3ClientWrapper.connect(Mqtt3ClientWrapper.kt:8)
	at de.smartsquare.kortance.scenarios.spike.SpikeCommand.call(SpikeCommand.kt:35)
	at de.smartsquare.kortance.scenarios.spike.SpikeCommand.call(SpikeCommand.kt:11)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1933)
	at picocli.CommandLine.access$1200(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2326)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2291)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159)
	at picocli.CommandLine.execute(CommandLine.java:2058)
	at de.smartsquare.kortance.KortanceKt.main(Kortance.kt:26)
Caused by: java.lang.RuntimeException: java.lang.NoSuchFieldException: consumerIndex
	at org.jctools.util.UnsafeAccess.fieldOffset(UnsafeAccess.java:91)
	at org.jctools.queues.BaseSpscLinkedArrayQueueConsumerField.<clinit>(BaseSpscLinkedArrayQueue.java:47)
	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
	... 43 more
Caused by: java.lang.NoSuchFieldException: consumerIndex
	at java.lang.Class.getDeclaredField(DynamicHub.java:2070)
	at org.jctools.util.UnsafeAccess.fieldOffset(UnsafeAccess.java:87)
	... 46 more

Steps to reproduce:

git clone https://github.com/SmartsquareGmbH/kortance
cd kortance
git checkout develop
./gradlew nativeImage
/build/graal/kortance spike https://test.mosquitto.org/ 1883

@jramirez-isc I hope that this fits this issue.

cstancu self-assigned this Aug 6, 2019

cstancu added the native-image label Aug 6, 2019

ahelmbr mentioned this issue Aug 7, 2019

Refactor HiveMQ-MQTT-Client library to be usable with GraalVM hivemq/mqtt-cli#11

Closed

ahelmbr changed the title ~~Using netty 4.1.36.Final in a native-image with graalVM 19.1.1 dos not work.~~ Using netty 4.1.36.Final for MQTT Connection in a native-image with graalVM 19.1.1 dos not work. Aug 8, 2019

cstancu assigned jramirez-isc and unassigned cstancu Aug 26, 2019

jramirez-isc closed this as completed Jun 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using netty 4.1.36.Final for MQTT Connection in a native-image with graalVM 19.1.1 dos not work. #1572

Using netty 4.1.36.Final for MQTT Connection in a native-image with graalVM 19.1.1 dos not work. #1572

ahelmbr commented Aug 5, 2019 •

edited

bobmcwhirter commented Aug 6, 2019

bobmcwhirter commented Aug 6, 2019 •

edited

ahelmbr commented Aug 8, 2019 •

edited

cstancu commented Aug 26, 2019

jramirez-isc commented Jun 4, 2020

deen13 commented Oct 30, 2020 •

edited

Using netty 4.1.36.Final for MQTT Connection in a native-image with graalVM 19.1.1 dos not work. #1572

Using netty 4.1.36.Final for MQTT Connection in a native-image with graalVM 19.1.1 dos not work. #1572

Comments

ahelmbr commented Aug 5, 2019 • edited

bobmcwhirter commented Aug 6, 2019

bobmcwhirter commented Aug 6, 2019 • edited

ahelmbr commented Aug 8, 2019 • edited

cstancu commented Aug 26, 2019

jramirez-isc commented Jun 4, 2020

deen13 commented Oct 30, 2020 • edited

ahelmbr commented Aug 5, 2019 •

edited

bobmcwhirter commented Aug 6, 2019 •

edited

ahelmbr commented Aug 8, 2019 •

edited

deen13 commented Oct 30, 2020 •

edited