WebsocketOutbound sendClose never returns when nested

[rstoyanchev]

Originally reported for WebFlux in SPR-17306 but I was able to confirm with just Reactor Netty that when a call to out.sendClose is nested within the pipeline for the write publisher, then the close does not complete. The following times out:

DisposableServer server = HttpServer.create()
    .tcpConfiguration(s -> s.host("0.0.0.0"))
    .port(0)
    .handle((req, resp) ->
        resp.sendWebsocket(null, (in, out) -> {
          return out.sendObject(Flux.error(new Throwable())
              .onErrorResume(ex -> out.sendClose(1001, "Going Away"))
              .cast(WebSocketFrame.class));

        }))
    .bind()
    .block();


HttpClient.create().websocket()
    .uri("http://0.0.0.0:" + server.address().getPort())
    .handle((in, out) -> in.receiveFrames().then()).next().block(Duration.ofSeconds(5));

However if the write and close are in a sequence, then it works:

DisposableServer server = HttpServer.create()
    .tcpConfiguration(s -> s.host("0.0.0.0"))
    .port(0)
    .handle((req, resp) ->
        resp.sendWebsocket(null, (in, out) -> {

          Flux<WebSocketFrame> writeFlux = Flux.error(new Throwable())
              .onErrorResume(ex -> Flux.empty())
              .cast(WebSocketFrame.class);

          return out.sendObject(writeFlux)
              .then(Mono.defer(() -> out.sendClose(1001, "Going Away")));

        }))
    .bind()
    .block();

I've confirmed the above with Californium only but the issue was reported against Spring Framework 5.0.9 which is based on Bismuth-SR11.

[smaldini]

With the current implementation this is to be expected as we already have one queued sender and we will wait for this sender to terminate (here your sendObject) to run the publisher created by sendClose.
We need to change our current design for publisher handling to :

• authorize one publisher sender at a time and fast fail any nested call like yours
• make sendClose fast track to interrupt any current sending instead of using the send publisher path

I'll see with @violetagg can do a quick work around until such refactor happen to make sendClose different, but nested sendXxx will still have the same problem until we fix the overall internal design for sending.

[rstoyanchev]

FWIW a nested send is neither expected nor supported in the Spring WebSocket API. It is also not explicitly validated against, but the API provides only one send method with a Publisher.

The one I'm concerned with is sendClose. If it can't be fixed in the near future, we'll need to document it as a limitation since unlike other send methods, this one is actually expected to be used in addition to a send with publisher.

[rstoyanchev]

I can confirm the changes after updating to use Bismuth and Californium snapshots, and adding tests.

[violetagg]

Handling of nested send operations will be tracked with #478