Performance: Streaming request/buffer-based response?

[frankier]

Currently the API says that if we want to handle websockets along with normal routes we have to use listen https://juliaweb.github.io/HTTP.jl/stable/guides/migration-1x/#WebSockets which means we have to adapt to streaming responses using e.g. https://juliaweb.github.io/HTTP.jl/stable/api/server/#HTTP.streamhandler .

In this case response bodies must be buffered so they can be written into a stream, even if the response body is an AbstractVector{UInt8}. I guess this means the highest performance code paths can't be reached. Is this assumption correct?

More generally there could be cases where we have something like a middleware that sometimes wants to work in a streaming fashion, modifying the responses on-the-fly, in other cases wants to work on a whole buffer at-a-time, and in other cases wants to pass them through. Would it be possible to support this use case?

[quinnj]

Yeah, that's basically how current HTTP/1 implementation works, with a couple of distinctions.
HTTP.streamhandler is intentionally a convenience utility from a Request -> Response handler to the lower-level stream API. It fully buffers the request body before invoking the request handler.
On the response side, a response with a known Content-Length—including the normal String and Vector{UInt8} response constructors—uses fixed-length HTTP/1 stream mode. That mode buffers the response bytes until closewrite, allowing HTTP.jl to verify the declared length and recover with an HTTP error if the handler fails before anything is committed. The tradeoff is an additional payload-sized copy, even when the original body is already a Vector{UInt8}.

BUT, This does not mean every stream response must be fully buffered. Unknown-length/chunked responses can be written incrementally, and live HTTP/2 responses do not use the same whole-response HTTP/1 buffer. A manual HTTP.listen! handler can also choose between buffering, streaming, passing through, or upgrading to WebSockets. What we do not currently have is a high-level adapter that switches between all those modes while retaining the direct fixed-body fast path used by HTTP.serve!.

Hopefully that clarifies/makes sense? If you describe a bit more what you're trying to do, maybe I can help direct a bit more, but essentially if you're using HTTP.listen!, you have full control and only need to use HTTP.streamhandler as as convenience if you want to delegate certain stream handling to the simpler Request->Response api.

[frankier]

Hi! Yes it does help quite a bit, although I must admit I've not yet grokked things fully. The context is this PR to Oxygen updating it OxygenFramework/Oxygen.jl#302 . My main takeaway at the moment is that better performance is possible with the API exposed by HTTP.jl but it's not currently possible to get it through Oxygen.jl since it works by always combining listen withstreamhandler always. Not a massive problem, especially since none of this is backed by benchmarks, but something I'll flag up as a caveat for possible future work on the Oxygen side.