Out of memory error while building GraalVM native images #37686

KD23243 · 2022-09-08T04:53:28Z

Description

Getting the following error while building a native image for the mentioned code.

Exception in thread "native-image pid watcher" java.lang.OutOfMemoryError: GC overhead limit exceeded

Stack Traces

=== Image generator watchdog detected no activity. This can be a sign of a deadlock during image building. Dumping all stack traces. Current time: Wed Sep 07 16:22:26 IST 2022
"main" Id=1 in RUNNABLE
    at java.base@11.0.15/java.lang.Thread.yield(Native Method)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.awaitQuiescence(ForkJoinPool.java:2990)
    at app//com.oracle.graal.pointsto.util.CompletionExecutor.complete(CompletionExecutor.java:243)
    at app//com.oracle.graal.pointsto.PointsToAnalysis.doTypeflow(PointsToAnalysis.java:688)
    at app//com.oracle.graal.pointsto.PointsToAnalysis.finish(PointsToAnalysis.java:676)
    at app//com.oracle.graal.pointsto.PointsToAnalysis.runAnalysis(PointsToAnalysis.java:732)
    at app//com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:723)
    at app//com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:558)
    at app//com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:515)
    at app//com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:407)
    at app//com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:585)
    at app//com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:128)
    at app//com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:615)

"Reference Handler" Id=2 in RUNNABLE
    at java.base@11.0.15/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
    at java.base@11.0.15/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
    at java.base@11.0.15/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)

"Finalizer" Id=3 in WAITING on lock=java.lang.ref.ReferenceQueue$Lock@62ad7040
    at java.base@11.0.15/java.lang.Object.wait(Native Method)
    at java.base@11.0.15/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
    at java.base@11.0.15/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
    at java.base@11.0.15/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)

"Signal Dispatcher" Id=4 in RUNNABLE

"JVMCI-native CompilerThread0" Id=6 in RUNNABLE

"Common-Cleaner" Id=9 in TIMED_WAITING on lock=java.lang.ref.ReferenceQueue$Lock@27aff271
    at java.base@11.0.15/java.lang.Object.wait(Native Method)
    at java.base@11.0.15/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
    at java.base@11.0.15/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
    at java.base@11.0.15/java.lang.Thread.run(Thread.java:829)
    at java.base@11.0.15/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161)

"native-image pid watcher" Id=13 in TIMED_WAITING on lock=java.util.TaskQueue@6763e12f
    at java.base@11.0.15/java.lang.Object.wait(Native Method)
    at java.base@11.0.15/java.util.TimerThread.mainLoop(Timer.java:553)
    at java.base@11.0.15/java.util.TimerThread.run(Timer.java:506)

"ForkJoinPool.commonPool-worker-13" Id=28 in WAITING on lock=java.util.concurrent.ForkJoinPool@292ae0eb
    at java.base@11.0.15/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.15/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

"Libgraal MBean Registration" Id=30 in TIMED_WAITING
    at java.base@11.0.15/java.lang.Thread.sleep(Native Method)
    at platform/jdk.internal.vm.compiler.management/org.graalvm.compiler.hotspot.management.Factory.run(Factory.java:111)

"Thread-2" Id=31 in RUNNABLE
    at java.management@11.0.15/sun.management.ThreadImpl.dumpThreads0(Native Method)
    at java.management@11.0.15/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:521)
    at java.management@11.0.15/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:509)
    at app//com.oracle.svm.hosted.DeadlockWatchdog.threadDump(DeadlockWatchdog.java:113)
    at app//com.oracle.svm.hosted.DeadlockWatchdog.watchdogThread(DeadlockWatchdog.java:80)
    at app//com.oracle.svm.hosted.DeadlockWatchdog$$Lambda$160/0x00000007c0bcb840.run(Unknown Source)
    at java.base@11.0.15/java.lang.Thread.run(Thread.java:829)

"JFR Recorder Thread" Id=33 in RUNNABLE

"ForkJoinPool-4-worker-7" Id=44 in WAITING on lock=java.util.concurrent.ForkJoinPool@16c21521
    at java.base@11.0.15/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.15/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

"Timer-0" Id=50 in TIMED_WAITING on lock=java.util.TaskQueue@1fe458c9
    at java.base@11.0.15/java.lang.Object.wait(Native Method)
    at java.base@11.0.15/java.util.TimerThread.mainLoop(Timer.java:553)
    at java.base@11.0.15/java.util.TimerThread.run(Timer.java:506)

"ForkJoinPool-2-worker-3" Id=52 in WAITING on lock=java.util.concurrent.ForkJoinPool@47800d3e
    at java.base@11.0.15/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.15/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

"ForkJoinPool-2-worker-5" Id=53 in WAITING on lock=java.util.concurrent.ForkJoinPool@47800d3e
    at java.base@11.0.15/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.15/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

"ForkJoinPool-2-worker-9" Id=55 in WAITING on lock=java.util.concurrent.ForkJoinPool@47800d3e
    at java.base@11.0.15/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.15/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

"ForkJoinPool-2-worker-11" Id=56 in WAITING on lock=java.util.concurrent.ForkJoinPool@47800d3e
    at java.base@11.0.15/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.15/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

"ForkJoinPool-2-worker-15" Id=58 in WAITING on lock=java.util.concurrent.ForkJoinPool@47800d3e
    at java.base@11.0.15/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.15/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

"ForkJoinPool-2-worker-7" Id=54 in RUNNABLE
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.graph.Node.removeThisFromUsages(Node.java:1231)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.graph.Node.updateUsages(Node.java:647)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.graph.Node.replaceFirstInput(Node.java:1186)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.GraphUtil.killCFGInner(GraphUtil.java:139)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.nodes.util.GraphUtil.killCFG(GraphUtil.java:291)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance$Tool.deleteBranch(CanonicalizerPhase.java:647)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.nodes.IfNode.simplify(IfNode.java:294)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.tryCanonicalize(CanonicalizerPhase.java:476)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.processNode(CanonicalizerPhase.java:385)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.processWorkSet(CanonicalizerPhase.java:362)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase$Instance.run(CanonicalizerPhase.java:320)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:236)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase.applyIncremental(CanonicalizerPhase.java:251)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.CanonicalizerPhase.applyIncremental(CanonicalizerPhase.java:247)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.IterativeConditionalEliminationPhase.run(IterativeConditionalEliminationPhase.java:60)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.common.IterativeConditionalEliminationPhase.run(IterativeConditionalEliminationPhase.java:34)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:236)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:160)
    at app//com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:194)
    at app//com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:357)
    at app//com.oracle.graal.pointsto.flow.MethodTypeFlow.createTypeFlow(MethodTypeFlow.java:301)
      - locked com.oracle.graal.pointsto.flow.MethodTypeFlow@3f92e678
    at app//com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureTypeFlowCreated(MethodTypeFlow.java:290)
    at app//com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:111)
    at app//com.oracle.graal.pointsto.DefaultAnalysisPolicy$DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultAnalysisPolicy.java:252)
    at app//com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:552)
    at app//com.oracle.graal.pointsto.PointsToAnalysis$1.run(PointsToAnalysis.java:631)
    at app//com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:193)
    at app//com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:177)
    at app//com.oracle.graal.pointsto.util.CompletionExecutor$$Lambda$557/0x00000007c2391040.run(Unknown Source)
    at java.base@11.0.15/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
    at java.base@11.0.15/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

"ForkJoinPool-2-worker-13" Id=57 in WAITING on lock=java.util.concurrent.ForkJoinPool@47800d3e
    at java.base@11.0.15/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.15/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

"ForkJoinPool-2-worker-1" Id=59 in WAITING on lock=java.util.concurrent.ForkJoinPool@47800d3e
    at java.base@11.0.15/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@11.0.15/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    at java.base@11.0.15/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at java.base@11.0.15/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)


=== Memory statistics (in MB):
=== Used heap size: 6372
=== Free heap size: 1949
=== Maximum heap size: 6372
=== Image generator watchdog is aborting image generation. To configure the watchdog, use the options -H:DeadlockWatchdogInterval=10 and -H:+DeadlockWatchdogExitOnTimeout

Steps to Reproduce

Ballerina code

import ballerina/http;
import ballerina/log;
import ballerinax/prometheus as _;
import ballerinax/jaeger as _;

service /hello on new http:Listener(9090) {
    
    resource function get sayHello(http:Caller caller, http:Request req) returns error? {
        log:printInfo("This is a test Info log");
        log:printError("This is a test Error log");
        http:Response res = new;
        res.setPayload("Hello, World!");
        check caller->respond(res);
    }
    
}

Build command

native-image -J-Xmx7g -jar <jar-file-path>.jar \
  --no-fallback \
  --enable-url-protocols=http,https \
  --initialize-at-build-time=org.slf4j \
  --initialize-at-run-time=io.netty.handler.ssl.BouncyCastleAlpnSslUtils \
  --initialize-at-run-time=io.netty.handler.codec.http2 \
  --initialize-at-run-time=io.netty.handler.codec.compression.ZstdOptions \
  --initialize-at-run-time=io.netty.handler.ssl.OpenSsl \
  --initialize-at-run-time=io.netty.handler.ssl.OpenSslPrivateKeyMethod \
  --initialize-at-run-time=io.netty.handler.ssl.ReferenceCountedOpenSslEngine  \
  --initialize-at-run-time=io.netty.internal.tcnative \
  --initialize-at-run-time=io.netty.handler.ssl.OpenSslAsyncPrivateKeyMethod \
  --initialize-at-run-time=io.netty.handler.codec.compression.BrotliDecoder \
  -H:ResourceConfigurationFiles=test/target/bin/config/resource-config.json \
  -H:ReflectionConfigurationFiles=test/target/bin/config/reflect-config.json\
  -H:JNIConfigurationFiles=test/target/bin/config/jni-config.json \
  -H:DynamicProxyConfigurationFiles=test/target/bin/config/proxy-config.json \
  -H:SerializationConfigurationFiles=test/target/bin/config/serialization-config.json \
  -H:MaxDuplicationFactor=25.0 \
  -H:+AllowVMInspection \
  -H:Name=<native-image-name>

Affected Version(s)

No response

OS, DB, other environment details and versions

Ballerina 2201.1.2 (Swan Lake Update 1)

GraalVM CE 22.1.0

Labels

jBallerina

Related issue(s) (optional)

No response

Suggested label(s) (optional)

Area/GraalVM

Suggested assignee(s) (optional)

No response

The text was updated successfully, but these errors were encountered:

KD23243 · 2022-09-22T05:15:41Z

Getting the deadlock with just prometheus but it works with just jaeger.

KD23243 added the Type/Bug label Sep 8, 2022

github-actions bot added the Team/jBallerina All the issues related to BIR, JVM backend code generation and runtime label Sep 8, 2022

KD23243 added Area/GraalVM Graal vm native image generation related issues. Type/Task and removed Type/Bug labels Sep 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Out of memory error while building GraalVM native images #37686

Out of memory error while building GraalVM native images #37686

KD23243 commented Sep 8, 2022

KD23243 commented Sep 22, 2022

Out of memory error while building GraalVM native images #37686

Out of memory error while building GraalVM native images #37686

Comments

KD23243 commented Sep 8, 2022

Description

Steps to Reproduce

Affected Version(s)

OS, DB, other environment details and versions

Labels

Related issue(s) (optional)

Suggested label(s) (optional)

Suggested assignee(s) (optional)

KD23243 commented Sep 22, 2022