Incredible use of RAM even on small example/rest-json

[hanzo2001]

GraalVM 19.3.1
OpenJDK 11.0.5

I have recently started exploring Quarkus for compiling some Camel projects down to native but I am having some serious RAM issues.
I have mistakenly tried some of my own projects before testing the examples provided here.

However, now that I have tried a simple camel-quarkus example as described in https://camel.apache.org/camel-quarkus/latest/first-steps.html, I can reliably say that there is an enormous use of RAM even for miniature projects.

Is this a known issue?
Is there a way to keep the use of RAM under control?
Is it because of my JDK version? (nobody should be starting projects on Java 8)
Am I doing something wrong?
How can I help?

I tried the camel-quarkus/example/rest-json project from the link. These are the compilation details:

~/lib/oracle/graalvm-ce-java11-19.3.1/bin/native-image -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar rest-json-0.0.1-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http,https --enable-all-security-services -H:NativeLinkerOption=-no-pie -H:+JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace rest-json-0.0.1-runner
[rest-json-0.0.1-runner:4045]    classlist:  10,359.71 ms
[rest-json-0.0.1-runner:4045]        (cap):   2,456.32 ms
[rest-json-0.0.1-runner:4045]        setup:   5,250.59 ms
09:57:45,180 INFO  [org.jbo.threads] JBoss Threads version 3.0.0.Final
[rest-json-0.0.1-runner:4045]   (typeflow):  77,624.60 ms
[rest-json-0.0.1-runner:4045]    (objects):  42,548.84 ms
[rest-json-0.0.1-runner:4045]   (features):   1,372.48 ms
[rest-json-0.0.1-runner:4045]     analysis: 124,972.96 ms
[rest-json-0.0.1-runner:4045]     (clinit):   1,294.10 ms
[rest-json-0.0.1-runner:4045]     universe:   4,580.92 ms
[rest-json-0.0.1-runner:4045]      (parse):  11,093.54 ms
[rest-json-0.0.1-runner:4045]     (inline):  11,022.70 ms
[rest-json-0.0.1-runner:4045]    (compile):  68,941.43 ms
[rest-json-0.0.1-runner:4045]      compile:  93,748.62 ms
[rest-json-0.0.1-runner:4045]        image:   5,082.08 ms
[rest-json-0.0.1-runner:4045]        write:   1,204.12 ms
[rest-json-0.0.1-runner:4045]      [total]: 245,833.79 ms

I have ~16GB RAM with 14GB available for compilation (-2GB on a fresh restart).

The example took 4min to complete and consumed almost the remaining 14GB RAM

This miniature has only 2 dependencies (managed by a camel parent):

    <parent>
        <groupId>org.apache.camel.quarkus</groupId>
        <artifactId>camel-quarkus-bom</artifactId>
        <version>1.0.0-M3</version>
    </parent>
    <dependencies>
        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-platform-http</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-jackson</artifactId>
        </dependency>
    </dependencies>

[ppalaga]

Sounds like you do not like the amount of RAM spent by the native image compilation. If that's really the problem:

• yes, it is known. The more RAM the faster the native compilation.
• You can set a limit passing -Dnative-image.xmx=Ng where N is the number of gigabytes the native image compilation can use.

[oscerd]

By the way you should not use a plain camel component like camel-jackson in Quarkus

[hanzo2001]

By the way you should not use a plain camel component like camel-jackson in Quarkus

I didn't notice that! I used the example available from this repository and did not give a second look. I double checked and the current commit for master has it this way. I will change it to the adequate dependency and report back if there are any changes.

[hanzo2001]

* yes, it is known. The more RAM the faster the native compilation.

Ok, cool to know. I was having issues with other projects when my system ran out of RAM and started swapping like crazy which made everything grind to a halt

* You can set a limit passing `-Dnative-image.xmx=Ng` where N is the number of gigabytes the native image compilation can use.

Thank you. I will test this now

[hanzo2001]

I swapped the camel-jackson for camel-quarkus-jackson and the results are as follows:

[rest-json-0.0.1-runner:7655]    classlist:   8,194.51 ms
11:19:40,433 INFO  [com.sun.xml.bin.v2.run.ref.opt.AccessorInjector] The optimized code generation is disabled
[rest-json-0.0.1-runner:7655]        (cap):     903.63 ms
[rest-json-0.0.1-runner:7655]        setup:   3,103.63 ms
11:19:47,626 INFO  [org.jbo.threads] JBoss Threads version 3.0.0.Final
[rest-json-0.0.1-runner:7655]   (typeflow): 231,698.99 ms
[rest-json-0.0.1-runner:7655]    (objects): 100,311.45 ms
[rest-json-0.0.1-runner:7655]   (features):   3,706.09 ms
[rest-json-0.0.1-runner:7655]     analysis: 343,944.08 ms
[rest-json-0.0.1-runner:7655]     (clinit):   2,825.57 ms
[rest-json-0.0.1-runner:7655]     universe:   8,777.28 ms
[rest-json-0.0.1-runner:7655]      (parse):  18,308.30 ms
[rest-json-0.0.1-runner:7655]     (inline):  17,601.61 ms
[rest-json-0.0.1-runner:7655]    (compile):  97,148.54 ms
[rest-json-0.0.1-runner:7655]      compile: 137,709.46 ms
[rest-json-0.0.1-runner:7655]        image:   7,721.02 ms
[rest-json-0.0.1-runner:7655]        write:   3,682.59 ms
[rest-json-0.0.1-runner:7655]      [total]: 514,278.63 ms

Now it took more than twice as much time but I believe that increase might be due to the OS swapping pages into the harddrive (1.2GB). This did not occur in the previous test.

I then tested with the memory limiting parameter -Dnative-image.xmx=8g

[rest-json-0.0.1-runner:8017]    classlist:  10,580.43 ms
11:31:57,468 INFO  [com.sun.xml.bin.v2.run.ref.opt.AccessorInjector] The optimized code generation is disabled
[rest-json-0.0.1-runner:8017]        (cap):   2,433.17 ms
[rest-json-0.0.1-runner:8017]        setup:   5,180.72 ms
11:32:06,626 INFO  [org.jbo.threads] JBoss Threads version 3.0.0.Final
[rest-json-0.0.1-runner:8017]   (typeflow): 454,400.69 ms
[rest-json-0.0.1-runner:8017]    (objects): 115,100.61 ms
[rest-json-0.0.1-runner:8017]   (features):   3,107.69 ms
[rest-json-0.0.1-runner:8017]     analysis: 580,229.01 ms
[rest-json-0.0.1-runner:8017]     (clinit):   2,489.84 ms
[rest-json-0.0.1-runner:8017]     universe:   8,472.11 ms
[rest-json-0.0.1-runner:8017]      (parse):  19,024.18 ms
[rest-json-0.0.1-runner:8017]     (inline):  15,650.93 ms
[rest-json-0.0.1-runner:8017]    (compile):  93,101.09 ms
[rest-json-0.0.1-runner:8017]      compile: 131,778.66 ms
[rest-json-0.0.1-runner:8017]        image:   7,701.51 ms
[rest-json-0.0.1-runner:8017]        write:   1,025.10 ms
[rest-json-0.0.1-runner:8017]      [total]: 746,234.41 ms

Now it takes more than triple the amount of time but it does not freeze up my system entirely (CPU 100% all the time but I think this is a good thing).

Thank you for helping me out. I believe this to be a non-issue so I will close it.

I'll try to push an update fixing the example so that it uses a quarkus-based dependency.