Crash on closing RateSubscriber sample with random pause in sending/receiving

[strangelydim]

1. Modify RateReporter.java slightly by changing:

    public void onMessage(final long messages, final long bytes)
    {   
        totalBytes += bytes;
        totalMessages += messages;
    }

to:

    public void onMessage(final long messages, final long bytes)
    {
        if ((ThreadLocalRandom.current().nextInt() % 100) < 10)
        {
            System.out.println("Good time for a nap...");
            try
            {   
                Thread.sleep(1000);
            }
            catch (InterruptedException e)
            {
                e.printStackTrace();
            }
        }

        totalBytes += bytes;
        totalMessages += messages;
    }

and rebuild.

1. Start the driver:

$ java -cp aeron-samples/build/libs/samples.jar uk.co.real_logic.aeron.driver.MediaDriver

1. Start the RateSubscriber sample:

$ java -cp aeron-samples/build/libs/samples.jar uk.co.real_logic.aeron.samples.RateSubscriber

1. Start the StreamingPublisher sample:

$ java -cp aeron-samples/build/libs/samples.jar uk.co.real_logic.aeron.samples.StreamingPublisher

1. Wait a few seconds, then control-c the subscriber, and (at least for me) the JVM on the subscriber will crash with various interesting messages, like:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000000010414a229, pid=5354, tid=22019
#
# JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# V  [libjvm.dylib+0x54a229]  Unsafe_GetIntVolatile+0x3e
#
# Core dump written. Default location: /cores/core or core.5354
#
# An error report file with more information is saved as:
# /Users/ericb/GitHub/Aeron/hs_err_pid5354.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#
Abort trap: 6 (core dumped)

jstack output for the core referred to above:

    Attaching to core /cores/core.5354 from executable /Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/bin/java, please wait...
    Debugger attached successfully.
    Server compiler detected.
    JVM version is 25.31-b07
    Deadlock Detection:

    No deadlocks found.

    Thread 3847: (state = BLOCKED)


    Thread 22019: (state = IN_VM)
     - sun.misc.Unsafe.getIntVolatile(java.lang.Object, long) @bci=0 (Interpreted frame)
     - uk.co.real_logic.agrona.concurrent.UnsafeBuffer.getIntVolatile(int) @bci=20, line=425 (Interpreted frame)
     - uk.co.real_logic.aeron.common.concurrent.logbuffer.FrameDescriptor.frameLengthVolatile(uk.co.real_logic.agrona.concurrent.UnsafeBuffer, int) @bci=5, line=262 (Interpreted frame)
     - uk.co.real_logic.aeron.common.concurrent.logbuffer.LogReader.read(uk.co.real_logic.aeron.common.concurrent.logbuffer.DataHandler, int) @bci=42, line=90 (Interpreted frame)
     - uk.co.real_logic.aeron.Connection.poll(int) @bci=58, line=92 (Interpreted frame)
     - uk.co.real_logic.aeron.Subscription$$Lambda$14.apply(java.lang.Object, int) @bci=5 (Interpreted frame)
     - uk.co.real_logic.agrona.concurrent.AtomicArray.doLimitedAction(int, int, uk.co.real_logic.agrona.concurrent.AtomicArray$ToIntLimitedFunction) @bci=58, line=185 (Compiled frame)
     - uk.co.real_logic.aeron.Subscription.poll(int) @bci=40, line=89 (Compiled frame)
     - uk.co.real_logic.aeron.samples.SamplesUtil.lambda$subscriberLoop$4(java.util.concurrent.atomic.AtomicBoolean, int, uk.co.real_logic.aeron.common.IdleStrategy, uk.co.real_logic.aeron.Subscription) @bci=9, line=91 (Interpreted frame)
     - uk.co.real_logic.aeron.samples.SamplesUtil$$Lambda$13.accept(java.lang.Object) @bci=16 (Interpreted frame)
     - uk.co.real_logic.aeron.samples.RateSubscriber.lambda$main$12(java.util.concurrent.atomic.AtomicBoolean, uk.co.real_logic.aeron.Subscription) @bci=8, line=70 (Interpreted frame)
     - uk.co.real_logic.aeron.samples.RateSubscriber$$Lambda$12.run() @bci=8 (Interpreted frame)
     - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1142 (Interpreted frame)
     - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 (Interpreted frame)
     - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


    Thread 14351: (state = BLOCKED)


    Thread 13571: (state = BLOCKED)
     - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
     - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=142 (Interpreted frame)
     - java.lang.ref.ReferenceQueue.remove() @bci=2, line=158 (Interpreted frame)
     - java.lang.ref.Finalizer$FinalizerThread.run() @bci=36, line=209 (Interpreted frame)


    Thread 13059: (state = BLOCKED)
     - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
     - java.lang.Object.wait() @bci=2, line=502 (Interpreted frame)
     - java.lang.ref.Reference$ReferenceHandler.run() @bci=36, line=157 (Interpreted frame)

[strangelydim]

Putting the random sleep only on the subscriber is also enough to reproduce, it seems. Probably doesn't even have to be random; I would bet the bug has more to do with control-c'ing the subscriber while it is still in the middle of message delivery or with several messages waiting to be delivered.

[mjpt777]

Bug in the sample application. It should wait for the subscriber to complete before shutting down Aeron.