[Q/A][Java DUBBO RPC Triple (HTTP2)] How to close the zombi connection on the Provider side

[tommy-ham]

Pre-check

• I am sure that all the content I provide is in English.

Apache Dubbo Component

Java SDK (apache/dubbo)

Details

Environment: Dubbo 3.3.6, Triple protocol (HTTP/2)
Issue: After a network partition, the Consumer-side socket is closed, but the Provider-side connection remains in the ESTABLISHED state (a zombie / half-open connection).
Root Cause: Dubbo's Triple (HTTP/2) transport does not ship with any built-in connection liveness mechanism — neither TCP Keep-Alive tuning, HTTP/2 PING frames, nor an IdleStateHandler is configured by default. As a result, the Provider is unaware that the peer is gone, and the stale channel persists until the OS-level TCP keepalive (default: 2 hours on Linux) finally times out.
Question: What is the recommended approach to detect and close these zombie connections on the Provider side? Specifically, is there a Dubbo-level configuration, a Filter/Extension hook, or a Netty pipeline customization point that allows us to inject PING-based or idle-based connection reaping?

Code of Conduct

• I agree to follow this project's Code of Conduct

[ninadsutrave]

In 3.3.6 only the consumer side has this. NettyConnectionClient adds an IdleStateHandler (heartbeat, 60s by default), and TriplePingPongHandler sends an HTTP/2 PING and closes the channel if the ack doesn't come back within the close timeout. On the provider side, NettyPortUnificationServer leaves idle handling to the protocol, but TripleHttp2Protocol never adds an IdleStateHandler, so a half-open connection just stays there.

NettyServerHandler already closes the channel when it gets an IdleStateEvent, so all that's missing is the handler. You can add it by overriding the tri wire protocol:

public class IdleAwareTripleProtocol extends TripleHttp2Protocol {

    @Override
    public void configServerProtocolHandler(URL url, ChannelOperator operator) {
        int closeTimeout = UrlUtils.getCloseTimeout(url);
        super.configServerProtocolHandler(url, new ChannelOperator() {
            @Override
            public void configChannelHandler(List<ChannelHandler> handlers) {
                List<ChannelHandler> withIdle = new ArrayList<>(handlers.size() + 1);
                withIdle.add(new ChannelHandlerPretender(
                        new IdleStateHandler(closeTimeout, 0, 0, TimeUnit.MILLISECONDS)));
                withIdle.addAll(handlers);
                operator.configChannelHandler(withIdle);
            }

            @Override
            public ProtocolDetector.Result detectResult() {
                return operator.detectResult();
            }
        });
    }
}

and in META-INF/dubbo/org.apache.dubbo.remoting.api.WireProtocol:

tri=com.example.IdleAwareTripleProtocol

Extensions under META-INF/dubbo/ override the built-in ones, so this replaces the default tri.

I tried it on 3.3.6 with a raw socket that sends the HTTP/2 preface and then goes quiet. Without the override the connection never gets closed. With heartbeat=1000 on the provider (close timeout is 3x heartbeat, so 3s) it's closed after about 13s. The extra 10s is Triple's graceful shutdown waiting for a PING ack before the final GOAWAY. A normal consumer stays connected while idle, since its own PINGs keep the connection active.

One thing to watch: keep the provider's close timeout well above the consumers' heartbeat. The defaults (180s vs 60s) are fine, but if you set it lower than a consumer's heartbeat, idle consumers keep getting disconnected and can hit "No provider available" while they reconnect.

[tommy-ham]

Thanks for your answer. It looks like your answer is the only solution. I was hoping to find a way around it without using SPI, but it seems there's no alternative.