Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core dumped when running Micronaut native image with Jaeger and Zipkin #1025

Closed
ilopmar opened this issue Mar 1, 2019 · 2 comments

Comments

2 participants
@ilopmar
Copy link

commented Mar 1, 2019

As a workaround for #1024 I've added the missing dependencies (which are Zipkin and I don't really need it because I'm using Jaeger).

To reproduce the issue:

  • git clone https://github.com/ilopmar/micronaut-jaeger-graal
  • cd micronaut-jaeger-graal
  • git checkout workaround
  • Use GraalRC12. I've also tried with graal master branch (rc14-dev) and the error is the same.
  • ./build-native-image.sh

Now creating the native image works but when I run it I get a core dumped:

$ ./mn-jaeger-graal 
17:11:24.872 [main] DEBUG i.j.t.i.senders.ThriftSenderFactory - Using the UDP Sender to send spans to the agent.
[ [ SubstrateSegfaultHandler caught signal 11 ] ]

General Purpose Register Set Values: 

  RAX 0000000000000001
  RBX 00007ffec4f3c360
  RCX 0000000000000000
  RDX 000000000000001c
  RBP 0000000000000001
  RSI 00007ffec4f3c368
  RDI 0000000000000000
  RSP 00007ffec4f3c378
  R8  0000000000000000
  R9  00007fddb7da8f50
  R10 00007fddb8645730
  R11 0000000000000000
  R12 0000000000000000
  R13 0000000000000008
  R14 000055908e430000
  R15 0000559090849260
  EFL 0000000000010206
  RIP 000055908dc27bd2
  

JavaFrameAnchor dump:

  No anchors
  
TopFrame info:

  Lookup TotalFrameSize in CodeInfoTable:
  SourceTotalFrameSize 112
  
VMThreads info:

  VMThread 00007fddb0000b20  STATUS_IN_NATIVE  java.lang.Thread@0x7fddb7b01a98
  VMThread 0000559090849260  STATUS_IN_JAVA (safepoints disabled)  java.lang.Thread@0x55908f5091b0
  
VM Thread State for current thread 0000559090849260:

  0 (8 bytes): com.oracle.svm.jni.JNIThreadLocalEnvironment.jniFunctions = (bytes) 
    0000559090849260: 0000000000000000
  
  8 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.pinnedTLAB = (bytes) 
    0000559090849268: 0000000000000000 0000000000000000
    0000559090849278: 0000000000000000 0000000000000000
    
  
  40 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.regularTLAB = (bytes) 
    0000559090849288: 00007fddb5d00000 00007fddb5e00000
    0000559090849298: 00007fddb5d81b50 0000000000000000
    
  
  72 (8 bytes): com.oracle.svm.core.genscavenge.PinnedAllocatorImpl.openPinnedAllocator = (Object) null
  80 (8 bytes): com.oracle.svm.core.heap.NoAllocationVerifier.openVerifiers = (Object) null
  88 (8 bytes): com.oracle.svm.core.jdk.IdentityHashCodeSupport.hashCodeGeneratorTL = (Object) java.util.SplittableRandom  00007fddb7b526d8
  96 (8 bytes): com.oracle.svm.core.snippets.SnippetRuntime.currentException = (Object) null
  104 (8 bytes): com.oracle.svm.core.thread.JavaThreads.currentThread = (Object) java.lang.Thread  000055908f5091b0
  112 (8 bytes): com.oracle.svm.core.thread.ThreadingSupportImpl.activeTimer = (Object) null
  120 (8 bytes): com.oracle.svm.jni.JNIThreadLocalHandles.handles = (Object) null
  128 (8 bytes): com.oracle.svm.jni.JNIThreadLocalPendingException.pendingException = (Object) null
  136 (8 bytes): com.oracle.svm.jni.JNIThreadLocalPinnedObjects.pinnedObjectsListHead = (Object) null
  144 (8 bytes): com.oracle.svm.jni.JNIThreadOwnedMonitors.ownedMonitors = (Object) null
  152 (8 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.freeList = (Word) 0  0000000000000000
  160 (8 bytes): com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.stackBoundaryTL = (Word) 140732194394112  00007ffec4749000
  168 (8 bytes): com.oracle.svm.core.stack.JavaFrameAnchors.lastAnchor = (Word) 0  0000000000000000
  176 (8 bytes): com.oracle.svm.core.thread.VMThreads.IsolateTL = (Word) 94079350407168  000055908e430000
  184 (8 bytes): com.oracle.svm.core.thread.VMThreads.OSThreadIdTL = (Word) 140590262374976  00007fddb8a5d240
  192 (8 bytes): com.oracle.svm.core.thread.VMThreads.nextTL = (Word) 0  0000000000000000
  200 (4 bytes): com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.yellowZoneStateTL = (int) 1  00000001
  204 (4 bytes): com.oracle.svm.core.thread.Safepoint.safepointRequested = (int) -671923  fff5bf4d
  208 (4 bytes): com.oracle.svm.core.thread.Safepoint.safepointRequestedValueBeforeSafepoint = (int) 0  00000000
  212 (4 bytes): com.oracle.svm.core.thread.ThreadingSupportImpl.currentPauseDepth = (int) 0  00000000
  216 (4 bytes): com.oracle.svm.core.thread.VMOperationControl.isLockOwner = (int) 0  00000000
  220 (4 bytes): com.oracle.svm.core.thread.VMThreads$StatusSupport.safepointsDisabledTL = (int) 1  00000001
  224 (4 bytes): com.oracle.svm.core.thread.VMThreads$StatusSupport.statusTL = (int) 1  00000001
  
VMOperation dump:

  No VMOperation in progress
  
Dump Counters:

  
Raw Stacktrace:

  00007ffec4f3c378: 0100007fffff0000 0000559000000000
  00007ffec4f3c388: 000055908ee835d8 00002a4d27951770
  00007ffec4f3c398: 000000008dc2065f 00007fddb5d81770
  00007ffec4f3c3a8: 000055908dc0cfc4 00001aaf8f50a450
  00007ffec4f3c3b8: 00007fddb5d81930 00007fddb5d81718
  00007ffec4f3c3c8: 000055908dc0be26 00007fddb5d81930
  00007ffec4f3c3d8: 000055908e430000 000055908ee835d8
  00007ffec4f3c3e8: 00007fddb5d81810 00007fddb5d81718
  00007ffec4f3c3f8: 000055908d197ebd 00007fddb5d815b8
  00007ffec4f3c408: 00007fddb5d81810 00007fddb5d81718
  00007ffec4f3c418: 00001aaf8ee839b8 000055908ee835d8
  00007ffec4f3c428: 000055908d19dd9f 000055908ee664d8
  00007ffec4f3c438: 000055908e430000 000055908e430000
  00007ffec4f3c448: 000055908ee83928 00001aaf8ee66400
  00007ffec4f3c458: 000055908ee835d8 00007fddb5d71710
  00007ffec4f3c468: 000055908d19d30d 00007fddb5d6a1c8
  00007ffec4f3c478: 000055908e430000 00000000000a4196
  00007ffec4f3c488: 000000000046ca97 00002a4d2793a190
  00007ffec4f3c498: 00007fddb5d709c0 000055908ee83370
  00007ffec4f3c4a8: 00007fddb5d70868 00007fddb5d6a228
  00007ffec4f3c4b8: 00007fddb5d71710 00002a4d2793a1c8
  00007ffec4f3c4c8: 00007fddb5d6a1c8 00007fddb5d3d170
  00007ffec4f3c4d8: 000055908d1933ea 0000000000000035
  00007ffec4f3c4e8: 00007fddb5d69e28 000055908f50a3e8
  00007ffec4f3c4f8: 00007fddb5d6a190 00007fddb5d3d170
  00007ffec4f3c508: 000055908d193964 00007fddb5d6a180
  00007ffec4f3c518: 0000000000000000 000055908f5292e0
  00007ffec4f3c528: 000055908db83004 00007fddb5d69c50
  00007ffec4f3c538: 00007fddb5d69ec8 000000008f1bce48
  00007ffec4f3c548: 000055908ee66400 00007fddb5d709c0
  00007ffec4f3c558: 000055908ee664c0 00007fddb5d6a268
  00007ffec4f3c568: 000055908f50a3a0 000055908ee83370
  00007ffec4f3c578: 00007fddb5d3d170 00007fddb5fd9c88
  00007ffec4f3c588: 000055908d1731d9 00007fddb5d69d28
  00007ffec4f3c598: 000055908d176266 000055908f1bcd48
  00007ffec4f3c5a8: 00000000b5fd9bc8 00007fddb5d6a180
  00007ffec4f3c5b8: 00007fddb5d69f70 00007fddb5d3d1b8
  00007ffec4f3c5c8: 000055908d175969 00007fddb5d5fb98
  00007ffec4f3c5d8: 0000000000e89a68 00007fddb5d1c0b8
  00007ffec4f3c5e8: 00000000b5d56e20 000055908e430000
  00007ffec4f3c5f8: 00007fddb5d69f70 00007fddb5d1c0b8
  00007ffec4f3c608: 000055908d83e5ed 00007fddb5d5a3d8
  00007ffec4f3c618: 000000008e430000 00007fddb7b027e0
  00007ffec4f3c628: 00007fddb5fd9bc8 00007fddb5d4fa70
  00007ffec4f3c638: 000055908d833254 00007fddb5d5fac8
  00007ffec4f3c648: 000055908d30ebce 00007fddb5d4fa70
  00007ffec4f3c658: 000055908f2b9a68 00007fddb7b027e0
  00007ffec4f3c668: 00007fddb5fd9bc8 00007fddb5d59398
  00007ffec4f3c678: 000055908d3146c7 00002a4d2792f8f0
  00007ffec4f3c688: 000000058ee89ef0 000055908f5091b0
  00007ffec4f3c698: 000000018ee89ef0 00007fddb7b9ba68
  00007ffec4f3c6a8: 000055908df2d9fc 000055908ee89ef0
  00007ffec4f3c6b8: 000055908d3c9a59 00000000b5d57630
  00007ffec4f3c6c8: 000055908df27359 00007fddb5f94f00
  00007ffec4f3c6d8: 000000018d68a82c 00007fddb7b9ba68
  00007ffec4f3c6e8: 000055908f22ca68 00007fddb5fd9bc8
  00007ffec4f3c6f8: 000055908e430000 000055908e430000
  00007ffec4f3c708: 000055908e430000 00007fddb5d59398
  00007ffec4f3c718: 00007fddb5fd9bc8 00007fddb7b027e0
  00007ffec4f3c728: 000055908d312850 00002a4d27927630
  00007ffec4f3c738: 000055908ee89ef0 00007fddb5d5a2d8
  00007ffec4f3c748: 00007fddb7b028e0 000055908e430000
  00007ffec4f3c758: 000055908f1bcf48 00007fddb5d59398
  00007ffec4f3c768: 00007fddb5fd9bc8 00007fddb7b027e0
  
Stacktrace Stage0:

  RSP 00007ffec4f3c378 RIP 000055908dc27bd2 FrameSize 112
  Fatal error: Stack walk must walk only frames of known code
  Error: printDiagnostics already in progress.
  Trace/breakpoint trap (core dumped)

@cstancu cstancu self-assigned this Mar 1, 2019

@cstancu cstancu added the native-image label Mar 1, 2019

dougxc pushed a commit that referenced this issue Mar 5, 2019

@cstancu

This comment has been minimized.

Copy link
Member

commented Mar 5, 2019

Fixed in e35ad42. Thanks for the report! It will be included in rc14.

It looks like this app needs some configuration. Now I get:

Message: Cannot read jaeger.properties
Path Taken: new ExecutorServiceInstrumenter([Tracer tracer])
io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type  [io.micronaut.tracing.instrument.scheduling.ExecutorServiceInstrumenter]

when I run the image.

@cstancu cstancu closed this Mar 5, 2019

@ilopmar

This comment has been minimized.

Copy link
Author

commented Mar 6, 2019

Thanks for the fix. I've tried locally and now I get the same error as you. We'll dig to see what's happening on our side because Micronaut doesn't use jaeger.properties.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.