Make Netty HTTP server configurable

[florian-stefan]

We want to implement "Progressive HTML Rendering" using Spring Webflux and Reactor. For this, we need to be able to configure the Netty HTTP server so that the underlying channels flush item by item. We are using spring-boot-starter-webflux (2.0.0.M3).

This can be achieved by setting the attribute flushOnEach of the ChannelOperationsHandler to true, e.g. by calling NettyPipeline.SendOptions.flushOnEach after initializing a channel.

Unfortunately, the class NettyReactiveWebServerFactory does not offer any configuration options. It would be awesome to be able configuring the Netty HTTP Server by setting properties, e.g. in the application.yml.

Example: By registering the following bean, we are able to create a working example (for more details see here):

@Bean
  public ReactiveWebServerFactory httpServer() {
    Consumer<Channel> channelConfigurator = channel -> Optional
      .ofNullable(channel.pipeline().get(NettyPipeline.ReactiveBridge))
      .map(NettyPipeline.SendOptions.class::cast)
      .ifPresent(NettyPipeline.SendOptions::flushOnEach);

    return httpHandler -> {
      HttpServer httpServer = HttpServer.builder()
        .options(serverOptionsBuilder -> serverOptionsBuilder
          .afterChannelInit(channelConfigurator)
          .port(8080))
        .build();

      return new NettyWebServer(httpServer, new ReactorHttpHandlerAdapter(httpHandler));
    };
  }

[philwebb]

I think additional properties would be sensible. As would some callback for programmatic control (similar to TomcatConnectorCustomizer / TomcatContextCustomizer).

@bclozel do you agree? Would there be any downsides?

[bclozel]

Programmatic callbacks could be useful indeed. Configuration properties for this key, I'm not 100% convinced.

Just to have more background on this @florian-stefan, how are you going to achieve progressive rendering? Does your templating engine supports this in any way? Or are you rendering fragments through separate REST API calls?

[florian-stefan]

@bclozel At the moment, we are using Thymeleaf for defining the layout of the page. Since Thymeleaf does not support progressive rendering, we are fetching the template directly from the template cache, e.g. String template = templateEngine.process("widgets", new Context());. This will be the first chunk. The other chunks are pagelets loaded from some backend services.

We achieve streaming by using a simple controller method which returns a Flux and is annotated with @GetMapping(produces = TEXT_HTML_VALUE). That means, the controller method looks somewhat like this:

@GetMapping(produces = TEXT_HTML_VALUE)
public Flux<String> widgets() {
  String template = templateEngine.process("widgets", new Context());

  return Flux.concat(
    Mono.just(template.replace("</body>", "").replace("</html>", "")),
    widgetService.loadPagelets().map(this::wrapAsJavaScriptFunction),
    Mono.just("</body></html>")
  );
}

Does this answer your question?

[bclozel]

Thanks for the details @florian-stefan !
We'll add a customizer to help you tweak the server, as this will be useful anyway.

The use case you're describing is quite interesting, and there are a few ways to support that.
Configuring your server to flush on each goes too far in my opinion:

• this will impact all write operations and can lead to severe performance drawbacks; your server might be more busy trying to flush and the overall service latency might suffer
• this can be interesting only if there is significant latency between html fragments; in that case, there are other ways to deal with that as well
• unless you control 100% of the network, proxies and CDN may alter that flushing behavior along the way

Thymeleaf may not support "progressive rendering" (I'm not sure what this means), but it offers quite a lot in that space already; for example, you can configure Thymeleaf not to render the whole template as a single buffer but split it with a maximum buffer size; this will limit the amount of in-flight memory, but I'm not sure it controls the flushing as well. Thymeleaf supports advanced scenarios like Server Sent Event or data-driven templates. More on that in this (very interesting) talk.

Now you can control the flushing behavior on a per endpoint basis; WebFlux provides ServerHttpResponse.writeAndFlushWith(Publisher<Publisher<DataBuffer>> for that purpose exactly. We're using that infrastructure to flush properly on each element for SSE endpoints. This is a bit more low-level, since you have to deal with DataBuffer instances yourself (beware of memory leaks!), but you'll get more control over things.

One last thing: writeAndFlushWith is indeed quite low-level, and we're looking into alternatives for that. Don't hesitate to subscribe and comment on SPR-14981. Concrete uses cases, experiments and feedback always help those issues move forward.

[florian-stefan]

Thank you very much for the input @bclozel (especially for the warning to flush after each chunk on each channel)! You are of course right that this pattern only makes sense for special use cases. But there are scenarios in which it improves the perceived page load time.

I will have a look into the (new) Thymeleaf features.

At the moment we are experimenting with adapting some patterns from the Play framework to Spring 5. "Progressive HTML Rendering" (Facebook calls it BigPipe) is one example. An implementation using Play is described here.

[sdeleuze]

@florian-stefan As suggested by @bclozel, don't hesitate to add a comment with more details about your use case and thoughts on SPR-14981, we are looking for this kind of input.