When a broker goes OOM it should be easily observable

[deepthidevaki]

When a broker goes OOM in direct memory, it logs the error but continue to work. But the thread in which OOM occured does not make any progress (it looks like it). This eventually causes OOM on heap. It would be better if it exits when OOM occurs in direct memory. We have enabled +XX:exitOnOutOfMemory, but this works only when OOM on heap.

2021-08-31 17:10:02.351 CEST
Exception in thread "Broker-2-zb-actors-3" java.lang.OutOfMemoryError: Direct buffer memory at java.base/java.nio.Bits.reserveMemory(Bits.java:175) at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118) at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317) at io.camunda.zeebe.util.allocation.DirectBufferAllocator.allocate(DirectBufferAllocator.java:20) at io.camunda.zeebe.util.allocation.BufferAllocators.allocateDirect(BufferAllocators.java:16) at io.camunda.zeebe.dispatcher.DispatcherBuilder.initAllocatedBuffer(DispatcherBuilder.java:147) at io.camunda.zeebe.dispatcher.DispatcherBuilder.build(DispatcherBuilder.java:93) at io.camunda.zeebe.logstreams.impl.log.LogStreamImpl.openAppender(LogStreamImpl.java:280) at io.camunda.zeebe.logstreams.impl.log.LogStreamImpl.createWriter(LogStreamImpl.java:208) at io.camunda.zeebe.logstreams.impl.log.LogStreamImpl.lambda$newLogStreamBatchWriter$1(LogStreamImpl.java:115) at io.camunda.zeebe.util.sched.ActorJob.invoke(ActorJob.java:73) at io.camunda.zeebe.util.sched.ActorJob.execute(ActorJob.java:39) at io.camunda.zeebe.util.sched.ActorTask.execute(ActorTask.java:122) at io.camunda.zeebe.util.sched.ActorThread.executeCurrentTask(ActorThread.java:94) at io.camunda.zeebe.util.sched.ActorThread.doWork(ActorThread.java:78) at io.camunda.zeebe.util.sched.ActorThread.run(ActorThread.java:191)

#7744 (comment)_

[Zelldon]

Similar things observed here #6059

[Zelldon]

@deepthidevaki and me had a closer look at the code and we think it just kills one of our actor threads here https://github.com/camunda-cloud/zeebe/blob/develop/util/src/main/java/io/camunda/zeebe/util/sched/ActorThread.java#L93-L106, if we have multiple actor threads they will still continue to work.

We could solve it via catching it and doing an exit(1) 🙈

[Zelldon]

https://github.com/airlift/jvmkill :D

[Zelldon]

We could also try -XX:OnError="<cmd args>;<cmd args>" https://www.oracle.com/java/technologies/javase/vmoptions-jsp.html

[npepinpe]

Proposal:

Instrument every thread (including, if possible, Netty's) to kill the JVM when fatal errors occur. We can use a starting point the following VirtualMachineError (includes StackOverflowError, OutOfMemoryError, etc.), which from the Javadoc:

Thrown to indicate that the Java Virtual Machine is broken or has run out of resources necessary for it to continue operating.

I think that's a good reason to just end things. We can think about adding also LinkageError in the future (for example, Scala's NonFatal does that), but I'm not sure this is correct for us.

Possibly OnError is already called for any VirtualMachineError, but I couldn't find any documentation specifying this. It just says triggered on fatal errors, which to me implies that but doesn't really confirm it 😅

EDIT: OnError seems to be triggered only by segmentation faults.

[npepinpe]

Also regarding -XX:OnError, is this only for HotSpot or most JVMs?

[npepinpe]

So -XX:OnError is not triggered by out of direct memory. See this test:

import java.nio.ByteBuffer;

public class Test {
  public static void main(final String[] args) throws InterruptedException {
    final var t =
        new Thread(
            () -> {
              final var buf = ByteBuffer.allocateDirect(128 * 1024 * 1024);
            });
    t.start();
    Thread.sleep(60_000);
  }
}

Ran with: java -XX:OnError="kill -9 %p" -Xms64M -Xmx64M -XX:MaxDirectMemorySize=64M Test.java, which produces:

Exception in thread "Thread-0" java.lang.OutOfMemoryError: Direct buffer memory
        at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
        at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)
        at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
        at io.camunda.zeebe.broker.transport.externalapi.Test.lambda$main$0(Test.java:10)
        at java.base/java.lang.Thread.run(Thread.java:829)

And then sleeps for a minute.

[npepinpe]

One quick fix could be to do this in the standalone broker/gateway:

    Thread.setDefaultUncaughtExceptionHandler(
        (thread, error) -> {
          if (error instanceof VirtualMachineError) {
            Loggers.SYSTEM_LOGGER.error(
                "An unexpected fatal error was thrown; exiting now.", error);
            System.exit(1);
          }
        });

I think only in Atomix do we overwrite the uncaught exception handler.

[npepinpe]

So this won't quite work everywhere since the thread pool executor will swallow exceptions 😅

So we would have to:

• Instrument the actor threads to catch VirtualMachineError and die
• Instrument the Atomix ThreadContext implementations to catch VirtualMachineError and die
• Instrument the Netty ChannelHandler (e.g. BasicClientChannelInitializer and BasicServerChannelInitializer) to customize the exception catching behavior, catch VirtualMachineError, and die.

We should still set the default one just in case, of course, but that should cover most places.

[lenaschoenburg]

OOM's should be handled correctly in most places now. I've created a new issue for handling Netty's OutOfDirectMemory: #8543