Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IllegalReferenceCountException with DefaultHttp2ConnectionEncoder#writeData() and PooledSlicedByteBuf #5999

Closed
rkapsi opened this issue Nov 9, 2016 · 27 comments
Closed

IllegalReferenceCountException with DefaultHttp2ConnectionEncoder#writeData() and PooledSlicedByteBuf #5999

rkapsi opened this issue Nov 9, 2016 · 27 comments
Assignees
@Scottmitch
Labels
defect
Milestone
4.1.7.Final

Comments

@rkapsi
Copy link
Member

rkapsi commented Nov 9, 2016 •
edited

I have a custom Channel class that treats each Http2Stream like a connection and it acts like a proxy to translate between H2 and HTTP/1.1. It looks something like this (very similar to Netty's own HttpToHttp2ConnectionHandler):

// HttpToHttp2ConnectionHandler
public class HttpToHttp2Handler extends ChannelDuplexHandler {
  
  private static final boolean HACK = true;
  
  private final Http2ConnectionEncoder encoder;
  
  private final Http2Stream stream;
  
  private final ChannelHandlerContext h2callback;
  
  private final Channel h2channel;
  
  public HttpToHttp2Handler(Http2ConnectionEncoder encoder, Http2Stream stream) {
    this.encoder = encoder;
    this.stream = stream;
    
    Http2RemoteFlowController flowController = encoder.flowController();
    this.h2callback = flowController.channelHandlerContext();
    this.h2channel = h2callback.channel();
  }
  
  @Override
  public void flush(ChannelHandlerContext ctx) throws Exception {
    h2channel.flush();
  }
  
  @Override
  public void read(ChannelHandlerContext ctx) throws Exception {
    h2channel.read();
  }

  @Override
  public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
    
    SimpleChannelPromiseAggregator promiseAggregator =
        new SimpleChannelPromiseAggregator(promise, ctx.channel(), ctx.executor());
    
    boolean release = true;
    boolean endStream = false;
    
    try {
      
      int streamId = stream.id();
      
      if (msg instanceof HttpMessage) {
        HttpMessage message = (HttpMessage)msg;
        
        Http2Headers http2Headers = HttpConversionUtil.toHttp2Headers(message, true);
        
        if (message instanceof FullHttpMessage) {
          endStream = !((FullHttpMessage)message).content().isReadable();
        }
        
        encoder.writeHeaders(h2callback, streamId, http2Headers, 0, endStream, promiseAggregator.newPromise());
      }
      
      if (!endStream && msg instanceof HttpContent) {
        
        Http2Headers trailers = EmptyHttp2Headers.INSTANCE;
        if (msg instanceof LastHttpContent) {
          
          LastHttpContent lastContent = (LastHttpContent) msg;
          trailers = HttpConversionUtil.toHttp2Headers(lastContent.trailingHeaders(), true);
          
          if (trailers.isEmpty()) {
            endStream = true;
          }
        }

        ByteBuf content = ((HttpContent) msg).content();
        
        if (HACK) {
          ByteBuf copy = h2callback.alloc().buffer();
          copy.writeBytes(content);
          content = copy;
          
          // content = Unpooled.copiedBuffer(content); // <- works too
          
        } else {
          // We're transferring ownership of the ByteBuf to the encoder
          // and we're not longer responsible for releasing it.
          release = false;
        }
        
        encoder.writeData(h2callback, streamId, content, 0, endStream, promiseAggregator.newPromise());
        
        if (!trailers.isEmpty()) {
          encoder.writeHeaders(h2callback, streamId, trailers, 0, true, promiseAggregator.newPromise());
        }
      }
    } catch (Throwable t) {
      promiseAggregator.setFailure(t);
      
    } finally {
      if (release) {
        ReferenceCountUtil.release(msg);
      }
      
      promiseAggregator.doneAllocatingPromises();
    }
  }
}

Please notice the if (HACK) {} condition. Everything works if I make a copy of the HttpContent's underlying ByteBuf which is a PooledSlicedByteBuf. It starts failing with an IllegalReferenceCountException if I try to passthrough the ByteBuf directly and it continues failing if I explicitly retain() it to rule out erroneous release calls on my end. All it does is to raise additional leak detector errors.

The exact exception I'm getting is this:

io.netty.util.IllegalReferenceCountException: refCnt: 0
	at io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1407) ~[netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1353) ~[netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:324) ~[netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.buffer.AbstractUnpooledSlicedByteBuf.nioBuffer(AbstractUnpooledSlicedByteBuf.java:454) ~[netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.buffer.CompositeByteBuf.nioBuffers(CompositeByteBuf.java:1498) ~[netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.buffer.AbstractUnpooledSlicedByteBuf.nioBuffers(AbstractUnpooledSlicedByteBuf.java:460) ~[netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.buffer.AbstractByteBuf.nioBuffers(AbstractByteBuf.java:1203) ~[netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:685) ~[netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:522) ~[netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.handler.codec.http2.Http2ConnectionHandler.channelReadComplete(Http2ConnectionHandler.java:464) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.handler.timeout.IdleStateHandler.channelReadComplete(IdleStateHandler.java:275) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.handler.ssl.SslHandler.channelReadComplete(SslHandler.java:928) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelReadComplete(DefaultChannelPipeline.java:1339) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.DefaultChannelPipeline.fireChannelReadComplete(DefaultChannelPipeline.java:932) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SNAPSHOT.jar:4.1.7.Final-SNAPSHOT]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]

I haven't been able to reproduce it locally/with clean traffic but it happens pretty quickly in our production environment.

I'm suspecting it's something in FlowControlledData#write(...) or maybe CoalescingBufferQueue that doesn't like PooledSlicedByteBufs.
@rkapsi
Copy link
Member Author

rkapsi commented Nov 9, 2016

The code has significantly changed on our end but I believe the IllegalReferenceCountException mentioned in #5856 is the same.

@normanmaurer
Copy link
Member

@Scottmitch @nmittler could you take a look please ?

@Scottmitch
Copy link
Member

Scottmitch commented Nov 10, 2016 •
edited

@rkapsi - Can you log the reference count of content right before the call to encoder.writeData just as a sanity check? It would also be help if we could wrap the buffer in something like AdvancedLeakAwareByteBuf that captures the stack trace of calls to release/retain so if we get a IllegalReferenceCountException we can track the reference count history for the object.

@Scottmitch Scottmitch self-assigned this Nov 10, 2016
@rkapsi
Copy link
Member Author

rkapsi commented Nov 10, 2016

@Scottmitch Shouldn't be a problem, will do it tomorrow morning.

@Scottmitch
Copy link
Member

Scottmitch commented Nov 10, 2016 •
edited

@rkapsi - Great thanks. If possible wrapping the buffer at its creation would be preferable as opposed to just wrapping in the write method here. What I'm thinking is the release methods would be something like:

private List<StackTrace> retainAndReleases = ...;
public boolean release(/*also overload the one which takes an int param*/) {
  try {
    return super.release();
  } catch (IllegalReferenceCountException e) {
     // print each element of retainAndReleases
     throw e;
  }
}

@rkapsi
Copy link
Member Author

rkapsi commented Nov 10, 2016

The creation point is HttpObjectDecoder or rather its super class ByteToMessageDecoder. I'll see how close to the creation point I can get. There was some logging I did prior to opening this ticket and the refCnt before (and after) ecoder.writeData was always >= 1. I didn't log the exact number but it was never 0.

@rkapsi
Copy link
Member Author

rkapsi commented Nov 10, 2016

@Scottmitch is it sufficient to use -Dio.netty.leakDetection.level=advanced (maybe with -Dio.netty.leakDetection.acquireAndReleaseOnly=true) or does writing a custom wrapper like above add some extra info?

@Scottmitch
Copy link
Member

Scottmitch commented Nov 11, 2016 •
edited

@rkapsi - the issue is this isn't a "leak" but instead an "over release". The exception above is not accompanied by any history in terms of what call stacks were responsible for retaining/releasing prior to the point where the IllegalReferenceCountException condition occurs. So I don't think this approach will directly give us what we want.

@rkapsi
Copy link
Member Author

rkapsi commented Nov 11, 2016 •
edited

@Scottmitch The wrapping didn't work which will be explained by the massive Stack Trace below. I recommend to Copy & Paste it into a Text Editor and look at it on a large screen.

Here's what I ended up doing. I had to augment AdvancedLeakAwareByteBuf and AdvancedLeakAwareCompositeByteBuf because the exception is happening on the "consumer side" rather than the "producer side". I hope the the code snippet explains the structure of the nested stack traces and I then ran it with -Dio.netty.leakDetection.level=PARANOID and -Dio.netty.leakDetection.acquireAndReleaseOnly=true.

class TrackingException extends RuntimeException {
  TrackingException(String message, TrackingException parent) { 
    super(message, parent); 
  }
}

class AdvancedLeakAwareByteBuf extends WrappedCompositeByteBuf {

    private final ResourceLeak leak;
    private final TrackingException tracking;

    AdvancedLeakAwareByteBuf(CompositeByteBuf wrapped, ResourceLeak leak) {
        this(wrapped, leak, null);
    }

    AdvancedLeakAwareByteBuf(CompositeByteBuf wrapped, ResourceLeak leak, TrackingException tracking) {
        super(wrapped);
        this.leak = leak;
        this.tracking = new TrackingException("hashCode=" + System.identityHashCode(this), tracking);
    }

    @Override
    public ByteBuf slice() {
        recordLeakNonRefCountingOperation(leak);
        return new AdvancedLeakAwareByteBuf(super.slice(), leak, tracking);
    }

    @Override
    public boolean release() {
        try {
            tracking.addSuppressed(new Throwable("release()"));
            boolean deallocated = super.release();
            if (deallocated) {
                leak.close();
            } else {
                leak.record();
            }
            return deallocated;
        } catch (Exception err) {
            tracking.addSuppressed(err);
            throw tracking;
        }
    }

    // Same for all the other methods
}


// Same for AdvancedLeakAwareCompositeByteBuf

The cause of an exception is the parent and the supressed exceptions are the retain/release calls. You basically need to read it backwards and inside-out. Scroll all the way to the bottom. As suspected it's happening after encoder.writeData (which I'm referring to as the "producer side"). The refCnt of the ByteBuf before the encoder.writeData call is 1.

2016-11-11 20:31:40,300 [WorkerThread-5] HttpToHttp2Handler TRACE: writeData: identityHashCode=1515679729, content=AdvancedLeakAwareByteBuf(PooledSlicedByteBuf(ridx: 0, widx: 2581, cap: 2581/2581, unwrapped: PooledUnsafeDirectByteBuf(ridx: 2588, widx: 2588, cap: 16384))), refCntBefore=1, refCntAfter=1
2016-11-11 20:31:40,320 [WorkerThread-5] ReferenceCountUtil WARN : Failed to release a message: AdvancedLeakAwareByteBuf(UnpooledSlicedByteBuf(freed))
�[m io.netty.buffer.TrackingException: buf=UnpooledSlicedByteBuf(ridx: 0, widx: 2565, cap: 2565/2565, unwrapped: CompositeByteBuf(ridx: 2565, widx: 2565, cap: 2565, components=2)), hashCode=1414100870, refCnt=1, parent=1260541213
    at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:60) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AdvancedLeakAwareCompositeByteBuf.readSlice(AdvancedLeakAwareCompositeByteBuf.java:100) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2FrameWriter.writeData(DefaultHttp2FrameWriter.java:166) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.Http2OutboundFrameLogger.writeData(Http2OutboundFrameLogger.java:44) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder$FlowControlledData.write(DefaultHttp2ConnectionEncoder.java:390) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.writeAllocatedBytes(DefaultHttp2RemoteFlowController.java:360) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor$1.write(DefaultHttp2RemoteFlowController.java:550) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.UniformStreamByteDistributor$State.write(UniformStreamByteDistributor.java:181) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.UniformStreamByteDistributor.distribute(UniformStreamByteDistributor.java:120) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor.writePendingBytes(DefaultHttp2RemoteFlowController.java:631) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController.writePendingBytes(DefaultHttp2RemoteFlowController.java:255) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:161) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:499) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:82) [echo-http-2.17.99.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
    Suppressed: java.lang.Throwable: method=release(), refCnt=1, hashCode=1414100870, this=AdvancedLeakAwareByteBuf(UnpooledSlicedByteBuf(ridx: 0, widx: 2565, cap: 2565/2565, unwrapped: CompositeByteBuf(ridx: 2565, widx: 2565, cap: 2565, components=2)))
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:970) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:493) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:499) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:82) [echo-http-2.17.99.jar:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
    Suppressed: io.netty.buffer.TrackingException: buf=UnpooledSlicedByteBuf(freed), hashCode=1741523875, refCnt=0, parent=1515679729
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:60) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AdvancedLeakAwareByteBuf.slice(AdvancedLeakAwareByteBuf.java:82) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.CompositeByteBuf.addComponent0(CompositeByteBuf.java:259) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.CompositeByteBuf.addComponent(CompositeByteBuf.java:195) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.WrappedCompositeByteBuf.addComponent(WrappedCompositeByteBuf.java:524) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AdvancedLeakAwareCompositeByteBuf.addComponent(AdvancedLeakAwareCompositeByteBuf.java:934) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.CoalescingBufferQueue.compose(CoalescingBufferQueue.java:161) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.CoalescingBufferQueue.remove(CoalescingBufferQueue.java:137) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder$FlowControlledData.write(DefaultHttp2ConnectionEncoder.java:382) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.writeAllocatedBytes(DefaultHttp2RemoteFlowController.java:360) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor$1.write(DefaultHttp2RemoteFlowController.java:550) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.UniformStreamByteDistributor$State.write(UniformStreamByteDistributor.java:181) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.UniformStreamByteDistributor.distribute(UniformStreamByteDistributor.java:120) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor.writePendingBytes(DefaultHttp2RemoteFlowController.java:631) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController.writePendingBytes(DefaultHttp2RemoteFlowController.java:255) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:161) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:499) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:82) [echo-http-2.17.99.jar:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
        Suppressed: java.lang.Throwable: method=release(), refCnt=0, hashCode=1741523875, this=AdvancedLeakAwareByteBuf(UnpooledSlicedByteBuf(freed))
            at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:970) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:971) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:493) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:499) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:82) [echo-http-2.17.99.jar:?]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
        Suppressed: io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
            at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:110) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:971) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:971) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:493) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:499) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:82) [echo-http-2.17.99.jar:?]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
    Caused by: io.netty.buffer.TrackingException: buf=PooledSlicedByteBuf(ridx: 0, widx: 2581, cap: 2581/2581, unwrapped: PooledUnsafeDirectByteBuf(ridx: 2581, widx: 2588, cap: 16384)), hashCode=1515679729, refCnt=1, parent=188424900
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:60) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AdvancedLeakAwareByteBuf.readRetainedSlice(AdvancedLeakAwareByteBuf.java:124) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:345) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:180) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        ... 2 more
    Caused by: io.netty.buffer.TrackingException: buf=PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 16384), hashCode=188424900, refCnt=1
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:60) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:54) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:44) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:271) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
        ... 2 more
        Suppressed: java.lang.Throwable: method=release(), refCnt=2, hashCode=188424900, this=AdvancedLeakAwareByteBuf(PooledUnsafeDirectByteBuf(ridx: 2588, widx: 2588, cap: 16384))
            at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:970) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:256) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
Caused by: io.netty.buffer.TrackingException: buf=CompositeByteBuf(ridx: 0, widx: 0, cap: 0, components=0), hashCode=1260541213, refCnt=1
    at io.netty.buffer.AdvancedLeakAwareCompositeByteBuf.<init>(AdvancedLeakAwareCompositeByteBuf.java:48) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AdvancedLeakAwareCompositeByteBuf.<init>(AdvancedLeakAwareCompositeByteBuf.java:42) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:66) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AbstractByteBufAllocator.compositeDirectBuffer(AbstractByteBufAllocator.java:215) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AbstractByteBufAllocator.compositeBuffer(AbstractByteBufAllocator.java:193) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.CoalescingBufferQueue.compose(CoalescingBufferQueue.java:160) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.CoalescingBufferQueue.remove(CoalescingBufferQueue.java:137) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder$FlowControlledData.write(DefaultHttp2ConnectionEncoder.java:382) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
    ... 152 more

Now, if I do something silly like this on the "producer side":

ByteBuf content = ((HttpContent) msg).content();
content.release();

encoder.writeData(..., content, ...);

It'll result in a very similar Exception (if not the same). If I however retain() the ByteBuf it'll continue to happen (which is odd).

@rkapsi
Copy link
Member Author

rkapsi commented Nov 11, 2016

@Scottmitch I think it's this...

2016-11-11 20:31:40,300 [WorkerThread-5] HttpToHttp2Handler TRACE: writeData: identityHashCode=1515679729, content=AdvancedLeakAwareByteBuf(PooledSlicedByteBuf(ridx: 0, widx: 2581, cap: 2581/2581, unwrapped: PooledUnsafeDirectByteBuf(ridx: 2588, widx: 2588, cap: 16384))), refCntBefore=1, refCntAfter=1

Suppressed: java.lang.Throwable: method=release(), refCnt=2, hashCode=188424900, this=AdvancedLeakAwareByteBuf(PooledUnsafeDirectByteBuf(ridx: 2588, widx: 2588, cap: 16384))
            at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:970) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:256) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) ~[netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP7.jar:4.1.7.Final-SNAPSHOT]
            at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]

That ByteToMessageDecoder is a HttpObjectDecoder and notice how the readerIndex==writerIndex in the log statement and the condition in ByteToMessageDecoder is to release the cumulator if it's not readable.

@rkapsi
Copy link
Member Author

rkapsi commented Nov 11, 2016

Disregard, that's not correct.

@rkapsi
Copy link
Member Author

rkapsi commented Nov 12, 2016

Ugh, this doesn't make no sense. I think I've stared too much at this but...

My HttpToHttp2Handler receives a PooledSlicedByteBuf with the identity hashCode of 1215655406. This is what gets passed into encoder.writeData(...). That in turn gets added to CoalescingBufferQueue's internal list of ByteBuf/Listener pairs and we basically lose track of it. The PooledSlicedByteBuf is a retained slice of a PooledUnsafeDirectByteBuf with the identity hashCode of 1483813062 and has a refCnt of 2. This makes sense. One refCnt is for itself and the other is for that PooledSlicedByteBuf.

On the other end there is Http2ConnectionHandler#flush() call which hits that CoalescingBufferQueue again. This time its fiddling with a CompositeByteBuf. As far as I can tell it's trying to create a slice() (with the identity hashCode of 1465116016) but that ByteBuf is already freed. WTF?! This freed ByteBuf appears to end up in the SslHandler (see first Suppressed exception). There is some error which triggers removeAndFailAll() which in turn attempt to release() our already freed ByteBuf and fails. That's essentyally the second Spressed exception.

How did that PooledSlicedByteBuf get freed? Is there something that fiddles directly with WrappedByteBuf#buf?

2016-11-11 23:38:23,096 [WorkerThread-5] HttpToHttp2Handler TRACE: content=AdvancedLeakAwareByteBuf(PooledSlicedByteBuf(ridx: 0, widx: 16384, cap: 16384/16384, unwrapped: PooledUnsafeDirectByteBuf(ridx: 16384, widx: 16384, cap: 16384))), refCnt=1, hashCode=1215655406

2016-11-11 23:38:23,203 [WorkerThread-5] AdvancedLeakAwareByteBuf ERROR: IllegalStateException
�[m java.lang.RuntimeException: AdvancedLeakAwareByteBuf(): buf=UnpooledSlicedByteBuf(freed), hashCode=1465116016
    at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:61) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AdvancedLeakAwareByteBuf.slice(AdvancedLeakAwareByteBuf.java:83) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.CompositeByteBuf.addComponent0(CompositeByteBuf.java:259) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.CompositeByteBuf.addComponent(CompositeByteBuf.java:195) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.WrappedCompositeByteBuf.addComponent(WrappedCompositeByteBuf.java:524) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AdvancedLeakAwareCompositeByteBuf.addComponent(AdvancedLeakAwareCompositeByteBuf.java:939) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.CoalescingBufferQueue.compose(CoalescingBufferQueue.java:161) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.CoalescingBufferQueue.remove(CoalescingBufferQueue.java:137) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder$FlowControlledData.write(DefaultHttp2ConnectionEncoder.java:382) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.writeAllocatedBytes(DefaultHttp2RemoteFlowController.java:360) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor$1.write(DefaultHttp2RemoteFlowController.java:550) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.UniformStreamByteDistributor$State.write(UniformStreamByteDistributor.java:181) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.UniformStreamByteDistributor.distribute(UniformStreamByteDistributor.java:120) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor.writePendingBytes(DefaultHttp2RemoteFlowController.java:631) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController.writePendingBytes(DefaultHttp2RemoteFlowController.java:255) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:161) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:499) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:82) [echo-http-2.17.107.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at com.squarespace.echo.core.channel.proxy.CallbackHandler.channelReadComplete(CallbackHandler.java:169) [echo-core-2.17.107.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.timeout.IdleStateHandler.channelReadComplete(IdleStateHandler.java:275) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelReadComplete(CombinedChannelDuplexHandler.java:441) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.ByteToMessageDecoder.channelReadComplete(ByteToMessageDecoder.java:307) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.CombinedChannelDuplexHandler.channelReadComplete(CombinedChannelDuplexHandler.java:260) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelReadComplete(DefaultChannelPipeline.java:1339) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline.fireChannelReadComplete(DefaultChannelPipeline.java:932) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
    Suppressed: java.lang.RuntimeException: release(): buf=UnpooledSlicedByteBuf(freed), refCnt=0, hashCode=1465116016
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:959) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:962) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:493) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:499) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:82) [echo-http-2.17.107.jar:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at com.squarespace.echo.core.channel.proxy.CallbackHandler.channelReadComplete(CallbackHandler.java:169) [echo-core-2.17.107.jar:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.timeout.IdleStateHandler.channelReadComplete(IdleStateHandler.java:275) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelReadComplete(CombinedChannelDuplexHandler.java:441) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.ByteToMessageDecoder.channelReadComplete(ByteToMessageDecoder.java:307) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.CombinedChannelDuplexHandler.channelReadComplete(CombinedChannelDuplexHandler.java:260) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelReadComplete(DefaultChannelPipeline.java:1339) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.fireChannelReadComplete(DefaultChannelPipeline.java:932) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
    Suppressed: io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:110) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:962) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:962) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:493) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:499) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:82) [echo-http-2.17.107.jar:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at com.squarespace.echo.core.channel.proxy.CallbackHandler.channelReadComplete(CallbackHandler.java:169) [echo-core-2.17.107.jar:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.timeout.IdleStateHandler.channelReadComplete(IdleStateHandler.java:275) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelReadComplete(CombinedChannelDuplexHandler.java:441) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.ByteToMessageDecoder.channelReadComplete(ByteToMessageDecoder.java:307) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.CombinedChannelDuplexHandler.channelReadComplete(CombinedChannelDuplexHandler.java:260) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelReadComplete(DefaultChannelPipeline.java:1339) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.fireChannelReadComplete(DefaultChannelPipeline.java:932) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
Caused by: java.lang.RuntimeException: AdvancedLeakAwareByteBuf(): buf=PooledSlicedByteBuf(ridx: 0, widx: 16384, cap: 16384/16384, unwrapped: PooledUnsafeDirectByteBuf(ridx: 16384, widx: 16384, cap: 16384)), hashCode=1215655406
    at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:61) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AdvancedLeakAwareByteBuf.readRetainedSlice(AdvancedLeakAwareByteBuf.java:125) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:345) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:180) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    ... 6 more
Caused by: java.lang.RuntimeException: AdvancedLeakAwareByteBuf(): buf=PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 16384), hashCode=1483813062
    at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:61) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:54) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:44) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:271) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
    ... 6 more
    Suppressed: java.lang.RuntimeException: release(): buf=PooledUnsafeDirectByteBuf(ridx: 16384, widx: 16384, cap: 16384), refCnt=2, hashCode=1483813062
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:959) ~[netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:256) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]

@Scottmitch
Copy link
Member

@rkapsi - Thanks for providing some more info. Its a bit tough to follow but do you have any insight as to what exception is caught here:

at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:493) [netty-all-4.1.7.Final-SQSP11.jar:4.1.7.Final-SNAPSHOT]

There is an interesting flush -> channel writability change -> flush re-entry condition here but from the quick glance nothing popped out at me ... I need to look in more detail.

@rkapsi
Copy link
Member Author

rkapsi commented Nov 12, 2016

@Scottmitch I've refined the logging a little bit and added a pseudo time stamp to each event (just a global atomic counter).

2016-11-12 16:46:48,980 [WorkerThread-2] AdvancedLeakAwareByteBuf ERROR: EXCEPTION:
--- BEGIN ---
--- BEGIN PARENT ---
  --- BEGIN PARENT ---
    168886
    java.lang.Exception: AdvancedLeakAwareByteBuf(): buf=PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 16384), hashCode=1710341163
        at io.netty.buffer.Caller.<init>(Caller.java:32)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:61)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:54)
        at io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:44)
        at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:271)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170)
        at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131)
        at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)
      168888
      java.lang.Exception: release(): buf=PooledUnsafeDirectByteBuf(ridx: 16384, widx: 16384, cap: 16384), refCnt=2, hashCode=1710341163
        at io.netty.buffer.Caller.<init>(Caller.java:32)
        at io.netty.buffer.Caller.<init>(Caller.java:26)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:967)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:256)
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

  --- END PARENT ---
    168887
    java.lang.Exception: AdvancedLeakAwareByteBuf(): buf=PooledSlicedByteBuf(ridx: 0, widx: 16384, cap: 16384/16384, unwrapped: PooledUnsafeDirectByteBuf(ridx: 16384, widx: 16384, cap: 16384)), hashCode=1345596783, parent=AdvancedLeakAwareByteBuf(PooledUnsafeDirectByteBuf(ridx: 16384, widx: 16384, cap: 16384)), parentHashCode=1710341163
        at io.netty.buffer.Caller.<init>(Caller.java:32)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:61)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.readRetainedSlice(AdvancedLeakAwareByteBuf.java:133)
        at io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:345)
        at io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:180)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)
--- END PARENT ---
  169333
  java.lang.Exception: AdvancedLeakAwareByteBuf(): buf=UnpooledSlicedByteBuf(freed), hashCode=644462234, parent=AdvancedLeakAwareByteBuf(PooledSlicedByteBuf(ridx: 16, widx: 16384, cap: 16384/16384, unwrapped: PooledUnsafeDirectByteBuf(freed))), parentHashCode=1345596783
    at io.netty.buffer.Caller.<init>(Caller.java:32)
    at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:61)
    at io.netty.buffer.AdvancedLeakAwareByteBuf.slice(AdvancedLeakAwareByteBuf.java:91)
    at io.netty.buffer.CompositeByteBuf.addComponent0(CompositeByteBuf.java:259)
    at io.netty.buffer.CompositeByteBuf.addComponent(CompositeByteBuf.java:195)
    at io.netty.buffer.WrappedCompositeByteBuf.addComponent(WrappedCompositeByteBuf.java:524)
    at io.netty.buffer.AdvancedLeakAwareCompositeByteBuf.addComponent(AdvancedLeakAwareCompositeByteBuf.java:939)
    at io.netty.channel.CoalescingBufferQueue.compose(CoalescingBufferQueue.java:161)
    at io.netty.channel.CoalescingBufferQueue.remove(CoalescingBufferQueue.java:137)
    at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder$FlowControlledData.write(DefaultHttp2ConnectionEncoder.java:382)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.writeAllocatedBytes(DefaultHttp2RemoteFlowController.java:360)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor$1.write(DefaultHttp2RemoteFlowController.java:550)
    at io.netty.handler.codec.http2.UniformStreamByteDistributor$State.write(UniformStreamByteDistributor.java:181)
    at io.netty.handler.codec.http2.UniformStreamByteDistributor.distribute(UniformStreamByteDistributor.java:120)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor.writePendingBytes(DefaultHttp2RemoteFlowController.java:631)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController.writePendingBytes(DefaultHttp2RemoteFlowController.java:255)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:161)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938)
    at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599)
    at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565)
    at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208)
    at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273)
    at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
    at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
    at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
    at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
    at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
    at com.squarespace.echo.core.channel.proxy.CallbackHandler.channelReadComplete(CallbackHandler.java:169)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
    at io.netty.handler.timeout.IdleStateHandler.channelReadComplete(IdleStateHandler.java:275)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelReadComplete(CombinedChannelDuplexHandler.java:441)
    at io.netty.handler.codec.ByteToMessageDecoder.channelReadComplete(ByteToMessageDecoder.java:307)
    at io.netty.channel.CombinedChannelDuplexHandler.channelReadComplete(CombinedChannelDuplexHandler.java:260)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelReadComplete(DefaultChannelPipeline.java:1339)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
    at io.netty.channel.DefaultChannelPipeline.fireChannelReadComplete(DefaultChannelPipeline.java:932)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
    at java.lang.Thread.run(Thread.java:745)
    169389
    java.lang.Exception: release(): buf=UnpooledSlicedByteBuf(freed), refCnt=0, hashCode=644462234
        at io.netty.buffer.Caller.<init>(Caller.java:32)
        at io.netty.buffer.Caller.<init>(Caller.java:26)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:967)
        at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675)
        at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65)
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:970)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109)
        at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:484)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938)
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599)
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565)
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208)
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273)
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
        at com.squarespace.echo.core.channel.proxy.CallbackHandler.channelReadComplete(CallbackHandler.java:169)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.handler.timeout.IdleStateHandler.channelReadComplete(IdleStateHandler.java:275)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelReadComplete(CombinedChannelDuplexHandler.java:441)
        at io.netty.handler.codec.ByteToMessageDecoder.channelReadComplete(ByteToMessageDecoder.java:307)
        at io.netty.channel.CombinedChannelDuplexHandler.channelReadComplete(CombinedChannelDuplexHandler.java:260)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelReadComplete(DefaultChannelPipeline.java:1339)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.DefaultChannelPipeline.fireChannelReadComplete(DefaultChannelPipeline.java:932)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

    169394
    java.lang.Exception: refCnt: 0, decrement: 1
        at io.netty.buffer.Caller.<init>(Caller.java:32)
        at io.netty.buffer.Caller.<init>(Caller.java:26)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:978)
        at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675)
        at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65)
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:970)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109)
        at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:484)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938)
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599)
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565)
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208)
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273)
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
        at com.squarespace.echo.core.channel.proxy.CallbackHandler.channelReadComplete(CallbackHandler.java:169)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.handler.timeout.IdleStateHandler.channelReadComplete(IdleStateHandler.java:275)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelReadComplete(CombinedChannelDuplexHandler.java:441)
        at io.netty.handler.codec.ByteToMessageDecoder.channelReadComplete(ByteToMessageDecoder.java:307)
        at io.netty.channel.CombinedChannelDuplexHandler.channelReadComplete(CombinedChannelDuplexHandler.java:260)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelReadComplete(ChannelInboundHandlerAdapter.java:97)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:384)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelReadComplete(DefaultChannelPipeline.java:1339)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:409)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:391)
        at io.netty.channel.DefaultChannelPipeline.fireChannelReadComplete(DefaultChannelPipeline.java:932)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

--- END ---

Timeline
168886: new PooledUnsafeDirectByteBuf -> refCnt should be 1
168887: PooledUnsafeDirectByteBuf.readRetainedSlice() -> PooledSlicedByteBuf -> refCnt should be 2
168888: PooledUnsafeDirectByteBuf.release() -> refCnt was 2, should be 1 now
169333: CoalescingBufferQueue.remove() tries to create a slice() of PooledSlicedByteBuf but its already freed
169389: Pre-Exception: Attempting to release the slice()
169394: Exception caught

There is no record for the 169333 event how the PooledSlicedByteBuf became freed. The only explanation would be that 168888 freed it. An alternative explanation would be if there is some code that manipulates the WrappedByteBuf#buf field directly and bypasses my wrapper which tries to capture these events.

@rkapsi
Copy link
Member Author

rkapsi commented Nov 12, 2016

Added more logging to see whether PooledUnsafeDirectByteBuf.release() transitions from refCnt 2 to 1 or 0. It doesn't.

2016-11-12 17:44:01,451 [WorkerThread-6] AdvancedLeakAwareByteBuf ERROR: EXCEPTION:
--- BEGIN ---
--- BEGIN PARENT ---
  --- BEGIN PARENT ---
    277140
    BEFORE: AdvancedLeakAwareByteBuf(): buf=PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 16384), hashCode=1723091000
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:37)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:61)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:54)
        at io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:44)
        at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:271)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170)
        at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131)
        at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)
      277142
      BEFORE: release(): buf=PooledUnsafeDirectByteBuf(ridx: 16384, widx: 16384, cap: 16384), refCnt=2, hashCode=1723091000
      AFTER: refCnt=1, deallocated=false
      java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:37)
        at io.netty.buffer.Caller.<init>(Caller.java:30)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:967)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:256)
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

  --- END PARENT ---
    277141
    BEFORE: AdvancedLeakAwareByteBuf(): buf=PooledSlicedByteBuf(ridx: 0, widx: 16384, cap: 16384/16384, unwrapped: PooledUnsafeDirectByteBuf(ridx: 16384, widx: 16384, cap: 16384)), hashCode=2087748684, parent=AdvancedLeakAwareByteBuf(PooledUnsafeDirectByteBuf(ridx: 16384, widx: 16384, cap: 16384)), parentHashCode=1723091000
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:37)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:61)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.readRetainedSlice(AdvancedLeakAwareByteBuf.java:133)
        at io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:345)
        at io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:180)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)
--- END PARENT ---
  279224
  BEFORE: AdvancedLeakAwareByteBuf(): buf=UnpooledSlicedByteBuf(freed), hashCode=1082132212, parent=AdvancedLeakAwareByteBuf(PooledSlicedByteBuf(ridx: 16, widx: 16384, cap: 16384/16384, unwrapped: PooledUnsafeDirectByteBuf(freed))), parentHashCode=2087748684
  AFTER: null
  java.lang.Exception
    at io.netty.buffer.Caller.<init>(Caller.java:37)
    at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:61)
    at io.netty.buffer.AdvancedLeakAwareByteBuf.slice(AdvancedLeakAwareByteBuf.java:91)
    at io.netty.buffer.CompositeByteBuf.addComponent0(CompositeByteBuf.java:259)
    at io.netty.buffer.CompositeByteBuf.addComponent(CompositeByteBuf.java:195)
    at io.netty.buffer.WrappedCompositeByteBuf.addComponent(WrappedCompositeByteBuf.java:524)
    at io.netty.buffer.AdvancedLeakAwareCompositeByteBuf.addComponent(AdvancedLeakAwareCompositeByteBuf.java:939)
    at io.netty.channel.CoalescingBufferQueue.compose(CoalescingBufferQueue.java:161)
    at io.netty.channel.CoalescingBufferQueue.remove(CoalescingBufferQueue.java:137)
    at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder$FlowControlledData.write(DefaultHttp2ConnectionEncoder.java:382)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.writeAllocatedBytes(DefaultHttp2RemoteFlowController.java:360)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor$1.write(DefaultHttp2RemoteFlowController.java:550)
    at io.netty.handler.codec.http2.UniformStreamByteDistributor$State.write(UniformStreamByteDistributor.java:181)
    at io.netty.handler.codec.http2.UniformStreamByteDistributor.distribute(UniformStreamByteDistributor.java:120)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor.writePendingBytes(DefaultHttp2RemoteFlowController.java:631)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController.writePendingBytes(DefaultHttp2RemoteFlowController.java:255)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:161)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938)
    at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599)
    at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565)
    at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208)
    at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273)
    at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
    at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
    at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
    at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.access$13(AbstractChannelHandlerContext.java:777)
    at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
    at java.lang.Thread.run(Thread.java:745)
    279282
    BEFORE: release(): buf=UnpooledSlicedByteBuf(freed), refCnt=0, hashCode=1082132212
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:37)
        at io.netty.buffer.Caller.<init>(Caller.java:30)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:967)
        at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675)
        at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65)
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:971)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109)
        at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:484)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938)
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599)
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565)
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208)
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273)
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.access$13(AbstractChannelHandlerContext.java:777)
        at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

    279286
    BEFORE: refCnt: 0, decrement: 1
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:37)
        at io.netty.buffer.Caller.<init>(Caller.java:30)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:982)
        at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675)
        at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65)
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:971)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109)
        at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:484)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938)
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599)
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565)
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208)
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273)
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.access$13(AbstractChannelHandlerContext.java:777)
        at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

--- END ---

277140: new PooledUnsafeDirectByteBuf, refCnt=1
277141: PooledUnsafeDirectByteBuf.readRetainedSlice() -> PooledSlicedByteBuf, refCnt=2
277142: PooledUnsafeDirectByteBuf.release(), refCnt 2 to 1
279224: CoalescingBufferQueue.remove(), PooledSlicedByteBuf is freed
279282: Pre-Exception, attempting to PooledSlicedByteBuf.release()
279286: Exception

@rkapsi
Copy link
Member Author

rkapsi commented Nov 12, 2016

I made an another experiment and everything works fine if I switch to -Dio.netty.allocator.type=unpooled. This is pointing towards PooledUnsafeDirectByteBuf (I also tried -Dio.netty.noPreferDirect=true which didn't make a difference in terms of outcome).

@rkapsi
Copy link
Member Author

rkapsi commented Nov 14, 2016 •
edited

@Scottmitch I finally found where the missing release() is (it's the last stack trace in the output). I'm not entirely sure what's happening but it seems:

  1. We pass PooledSlicedByteBuf into encoder.writeData
  2. It gets added to CoalescingBufferQueue
  3. FlowControlledData#write() or CoalescingBufferQueue#remove() takes out the whole or some of PooledSlicedByteBuf
  4. That gets passed to SslHandler where it gets released (the previously missing event at 220270)
  5. FlowControlledData/CoalescingBufferQueue tries to create a slice() of the now released PooledSlicedByteBuf (event at 220291)
  6. Kaboom.

// Via HttpObjectDecoder
214009: New PooledUnsafeDirectByteBuf@1206794823
214010: PooledUnsafeDirectByteBuf@1206794823.readRetainedSlice() -> PooledSlicedByteBuf@394798607
214012: PooledUnsafeDirectByteBuf@1206794823.release()

// Calling encoder.writeData with PooledSlicedByteBuf@394798607 which has refCnt of 1
220270: PooledUnsafeDirectByteBuf@1206794823.deallocate()
220291: PooledSlicedByteBuf@394798607.slice() -> UnpooledSlicedByteBuf@117200773
220417: Pre-Exception: UnpooledSlicedByteBuf@117200773.release()
220424: Exception

2016-11-14 14:46:16,532 [WorkerThread-7] AdvancedLeakAwareByteBuf ERROR: ---EXCEPTION:
--- BEGIN ---
--- BEGIN PARENT ---
  --- BEGIN PARENT ---
    214009
    BEFORE: AdvancedLeakAwareByteBuf(): buf=PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 16384), hashCode=1206794823
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:41)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:62)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:54)
        at io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:44)
        at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:271)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170)
        at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:131)
        at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)
      214012
      BEFORE: release(): buf=PooledUnsafeDirectByteBuf(ridx: 12491, widx: 12491, cap: 16384), refCnt=2, hashCode=1206794823
      AFTER: refCnt=1, deallocated=false
      java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:41)
        at io.netty.buffer.Caller.<init>(Caller.java:34)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:973)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:256)
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

  --- END PARENT ---
    214010
    BEFORE: AdvancedLeakAwareByteBuf(): buf=PooledSlicedByteBuf(ridx: 0, widx: 12484, cap: 12484/12484, unwrapped: PooledUnsafeDirectByteBuf(ridx: 12484, widx: 12491, cap: 16384)), hashCode=394798607, parent=AdvancedLeakAwareByteBuf(PooledUnsafeDirectByteBuf(ridx: 12484, widx: 12491, cap: 16384)), parentHashCode=1206794823
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:41)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:62)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.readRetainedSlice(AdvancedLeakAwareByteBuf.java:139)
        at io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:345)
        at io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:180)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)
--- END PARENT ---
  220291
  BEFORE: AdvancedLeakAwareByteBuf(): buf=UnpooledSlicedByteBuf(freed), hashCode=117200773, parent=AdvancedLeakAwareByteBuf(PooledSlicedByteBuf(ridx: 16, widx: 12484, cap: 12484/12484, unwrapped: PooledUnsafeDirectByteBuf(freed))), parentHashCode=394798607
  AFTER: null
  java.lang.Exception
    at io.netty.buffer.Caller.<init>(Caller.java:41)
    at io.netty.buffer.AdvancedLeakAwareByteBuf.<init>(AdvancedLeakAwareByteBuf.java:62)
    at io.netty.buffer.AdvancedLeakAwareByteBuf.slice(AdvancedLeakAwareByteBuf.java:97)
    at io.netty.buffer.CompositeByteBuf.addComponent0(CompositeByteBuf.java:259)
    at io.netty.buffer.CompositeByteBuf.addComponent(CompositeByteBuf.java:195)
    at io.netty.buffer.WrappedCompositeByteBuf.addComponent(WrappedCompositeByteBuf.java:524)
    at io.netty.buffer.AdvancedLeakAwareCompositeByteBuf.addComponent(AdvancedLeakAwareCompositeByteBuf.java:939)
    at io.netty.channel.CoalescingBufferQueue.compose(CoalescingBufferQueue.java:161)
    at io.netty.channel.CoalescingBufferQueue.remove(CoalescingBufferQueue.java:137)
    at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder$FlowControlledData.write(DefaultHttp2ConnectionEncoder.java:382)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.writeAllocatedBytes(DefaultHttp2RemoteFlowController.java:360)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor$1.write(DefaultHttp2RemoteFlowController.java:550)
    at io.netty.handler.codec.http2.UniformStreamByteDistributor$State.write(UniformStreamByteDistributor.java:181)
    at io.netty.handler.codec.http2.UniformStreamByteDistributor.distribute(UniformStreamByteDistributor.java:120)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor.writePendingBytes(DefaultHttp2RemoteFlowController.java:631)
    at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController.writePendingBytes(DefaultHttp2RemoteFlowController.java:255)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:161)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
    at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938)
    at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599)
    at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565)
    at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208)
    at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273)
    at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
    at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
    at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
    at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
    at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
    at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39)
    at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
    at java.lang.Thread.run(Thread.java:745)
    220417
    BEFORE: release(): buf=UnpooledSlicedByteBuf(freed), refCnt=0, hashCode=117200773
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:41)
        at io.netty.buffer.Caller.<init>(Caller.java:34)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:973)
        at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675)
        at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65)
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:977)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109)
        at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:484)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938)
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599)
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565)
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208)
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273)
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39)
        at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

    220424
    BEFORE: refCnt: 0, decrement: 1
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:41)
        at io.netty.buffer.Caller.<init>(Caller.java:34)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:993)
        at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675)
        at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65)
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:977)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109)
        at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:484)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.channelWritabilityChanged(Http2ConnectionHandler.java:385)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelWritabilityChanged(ChannelInboundHandlerAdapter.java:119)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelWritabilityChanged(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelWritabilityChanged(DefaultChannelPipeline.java:1357)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:445)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelWritabilityChanged(AbstractChannelHandlerContext.java:427)
        at io.netty.channel.DefaultChannelPipeline.fireChannelWritabilityChanged(DefaultChannelPipeline.java:938)
        at io.netty.channel.ChannelOutboundBuffer.fireChannelWritabilityChanged(ChannelOutboundBuffer.java:599)
        at io.netty.channel.ChannelOutboundBuffer.setWritable(ChannelOutboundBuffer.java:565)
        at io.netty.channel.ChannelOutboundBuffer.decrementPendingOutboundBytes(ChannelOutboundBuffer.java:208)
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:273)
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39)
        at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

    220270
    BEFORE: deallocate(), type=PooledUnsafeDirectByteBuf, hashCode=1206794823
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:41)
        at io.netty.buffer.Caller.<init>(Caller.java:34)
        at io.netty.buffer.PooledByteBuf.deallocate(PooledByteBuf.java:170)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65)
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:977)
        at io.netty.buffer.CompositeByteBuf$Component.freeIfNecessary(CompositeByteBuf.java:1675)
        at io.netty.buffer.CompositeByteBuf.deallocate(CompositeByteBuf.java:1940)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.AbstractDerivedByteBuf.release(AbstractDerivedByteBuf.java:65)
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:977)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109)
        at io.netty.channel.PendingWriteQueue.remove(PendingWriteQueue.java:252)
        at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:522)
        at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:481)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:163)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:974)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:244)
        at com.squarespace.echo.http2.streams.HttpToHttp2Handler.flush(HttpToHttp2Handler.java:80)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39)
        at io.netty.channel.AbstractChannelHandlerContext$16.run(AbstractChannelHandlerContext.java:767)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at java.lang.Thread.run(Thread.java:745)

--- END ---

@rkapsi
Copy link
Member Author

rkapsi commented Nov 14, 2016

@Scottmitch Here's a reproducer.

public class CoalescingBufferQueueTest {

  @Test
  public void testRemoveWithPooledByteBuf() {

    EmbeddedChannel channel = new EmbeddedChannel();

    ByteBufAllocator alloc = PooledByteBufAllocator.DEFAULT;
    ByteBuf cumulation = alloc.buffer();
    cumulation.writeBytes(new byte[64]);

    // HttpObjectDecoder
    ByteBuf content = cumulation.readRetainedSlice(64);
    cumulation.release();

    assertEquals(1, content.refCnt());
    assertEquals(1, cumulation.refCnt());

    CoalescingBufferQueue queue = new CoalescingBufferQueue(channel);

    // encoder.writeData
    queue.add(content);

    // FlowControlledData#write()
    ByteBuf toReturn1 = queue.remove(32, channel.newPromise());
    System.out.println("toReturn #1: " + toReturn1 + ", refCnt=" + toReturn1.refCnt());

    // Channel#write()
    toReturn1.release();

    // Consuming >= 32 doesn't trigger IllegalReferenceCountException
    ByteBuf toReturn2 = queue.remove(16, channel.newPromise());
    System.out.println("toReturn #2: " + toReturn2 + ", refCnt=" + toReturn2.refCnt());

    // Kaboom!
    toReturn2.release();
  }
}

@normanmaurer
Copy link
Member

Awesome! @Scottmitch can you check?

@Scottmitch
Copy link
Member

thanks @rkapsi ... yip @normanmaurer looking now

@normanmaurer
Copy link
Member

@Scottmitch <3

@Scottmitch Scottmitch added this to the 4.1.7.Final milestone Nov 14, 2016
@Scottmitch
Copy link
Member

Scottmitch commented Nov 14, 2016 •
edited

@rkapsi - I was able to isolate the root cause and I have a fix ... stand by for a PR.

@Scottmitch Scottmitch mentioned this issue Nov 14, 2016
Closed
@rkapsi
Copy link
Member Author

rkapsi commented Nov 14, 2016 •
edited

@Scottmitch this maybe relevant since you're mentioning #6000. I ran into a similar exception with a plain HTTP/1 proxy setup.

// Consume HttpResponse via HttpObjectDecoder
189841: New PooledUnsafeDirectByteBuf
189842: PooledUnsafeDirectByteBuf.readRetainedSlice() -> PooledSlicedByteBuf
189843: PooledUnsafeDirectByteBuf.release()

// HttpObjectEncoder
189872: MessageToMessageEncoder#write() releases HttpContent, retains ByteBuf
190514: NioSocketChannel#doWrite() releases ByteBuf
190528: Pre-Exception: NioSocketChannel#doWrite() tries to release ByteBuf again
190534: Exception

190514
    BEFORE: deallocate()
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:41)
        at io.netty.buffer.Caller.<init>(Caller.java:34)
        at io.netty.buffer.PooledByteBuf.deallocate(PooledByteBuf.java:170)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.AbstractPooledDerivedByteBuf.deallocate(AbstractPooledDerivedByteBuf.java:73)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
        at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
        at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:1029)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:972)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109)
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:271)
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
...

190528
    BEFORE: release(): buf=PooledSlicedByteBuf(ridx: 0, widx: 1024, cap: 1024/1024, unwrapped: PooledUnsafeDirectByteBuf(freed)), refCnt=1, hashCode=1662233377
    AFTER: null
    java.lang.Exception
        at io.netty.buffer.Caller.<init>(Caller.java:41)
        at io.netty.buffer.Caller.<init>(Caller.java:34)
        at io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:968)
        at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
        at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109)
        at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:271)
        at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:352)
        at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:442)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:856)
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:823)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1296)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760)
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:530)
        at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
...

@Scottmitch
Copy link
Member

@rkapsi - the scope of this issue is not limited to codec-http2 and applies to pooled duplicated buffers (e.g. retained[Slide|Duplicate]). So it wouldn't surprise me if the new stack traces are caused by the same issue. I'm guessing if you use the UnpooledByteBufAllocator you won't see this issue?

@rkapsi
Copy link
Member Author

rkapsi commented Nov 15, 2016

@Scottmitch I can confirm that.

#5999 (comment)

@Scottmitch
Copy link
Member

@rkapsi - Thanks. Just wanted to make sure that the behavior using Unpooled was consistent in this new scenario (Unpooled shouldn't suffer from this issue).

@Scottmitch Scottmitch mentioned this issue Nov 15, 2016
Closed
@Scottmitch
Copy link
Member

see #6024

Scottmitch added a commit to Scottmitch/netty that referenced this issue Nov 17, 2016
@Scottmitch
retained[Slice|Duplicate] buffer reference count bug
9fe094d 
Motivation:
Currently the ByteBuf created as a result of retained[Slice|Duplicate] maintains its own reference count, and when this reference count is depleated it will release the ByteBuf returned from unwrap(). The unwrap() buffer is designed to be the 'root parent' and will skip all intermediate layers of buffers. If the intermediate layers of buffers contain a retained[Slice|Duplicate] then these reference counts will be ignored during deallocation. This may lead to deallocating the 'root parent' before all derived pooled buffers are actually released. This same issue holds if a retained[Slice|Duplicate] is in the heirachy and a 'regular' slice() or duplicate() buffer is created.

Modifications:
- AbstractPooledDerivedByteBuf must maintain a reference to the direct parent (the buffer which retained[Slice|Duplicate] was called on) and release on this buffer instead of the 'root parent' returned by unwrap()
- slice() and duplicate() buffers created from AbstractPooledDerivedByteBuf must also delegate reference count operations to their immediate parent (or first ancestor which maintains an independent reference count).

Result:
Fixes netty#5999
@maseev maseev mentioned this issue May 25, 2017
Closed
liuzhengyang pushed a commit to liuzhengyang/netty that referenced this issue Sep 10, 2017
@Scottmitch
retained[Slice|Duplicate] buffer reference count bug
af2e77f 
Motivation:
Currently the ByteBuf created as a result of retained[Slice|Duplicate] maintains its own reference count, and when this reference count is depleated it will release the ByteBuf returned from unwrap(). The unwrap() buffer is designed to be the 'root parent' and will skip all intermediate layers of buffers. If the intermediate layers of buffers contain a retained[Slice|Duplicate] then these reference counts will be ignored during deallocation. This may lead to deallocating the 'root parent' before all derived pooled buffers are actually released. This same issue holds if a retained[Slice|Duplicate] is in the heirachy and a 'regular' slice() or duplicate() buffer is created.

Modifications:
- AbstractPooledDerivedByteBuf must maintain a reference to the direct parent (the buffer which retained[Slice|Duplicate] was called on) and release on this buffer instead of the 'root parent' returned by unwrap()
- slice() and duplicate() buffers created from AbstractPooledDerivedByteBuf must also delegate reference count operations to their immediate parent (or first ancestor which maintains an independent reference count).

Result:
Fixes netty#5999
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Scottmitch Scottmitch

Labels
defect
Projects
None yet
Milestone
4.1.7.Final
Development

No branches or pull requests
3 participants
@rkapsi @normanmaurer @Scottmitch