Handling incoming request with stream body

[drekka]

Hi, I've setup my server to look at the request.body as a ByteBuffer to get the contents of the incoming request. However I'm receiving an unexpected request from the app (not under my control) that is returning a stream instead of a byte buffer. I'm trying to figure out how to read this stream and have been digging through the Hummingbird code base to try and figure it out but not having much luck so far.

For byte buffers I've been doing

let buffer = request.body.buffer
let data = buffer.getData(at: 0, length: buffer.readableBytes)

But this crashes when the body is set to a stream.

Any ides how to read the data?

[adam-fowler]

Can you paste the call stack for the crash?

By default when a request includes a stream of data, the router collates all the data into one ByteBuffer until it receives an HTTP end tag or the amount of data streamed reaches a certain limit (at which time it returns an error response). It should not crash.

You can flag a route to accept a stream of data by adding the option .streamBody. You would then process the request body using request.body.stream. Below is a route that returns the size of the request body. By streaming the body it doesn't require all of the request body to be in memory at the same time.

app.router.post("size", options: .streamBody) { request -> EventLoopFuture<String> in
    guard let stream = request.body.stream else {
        return request.failure(.badRequest)
    }
    var size = 0
    return stream.consumeAll(on: request.eventLoop) { buffer in
        size += buffer.readableBytes
        return request.success(())
    }
    .map { _ in size.description }
}

stream is of type HBByteBufferStreamer which has methods consume which returns the data stream so far, consumeAll which calls a closure on every block of data streamed and sequence which returns a AsyncSequence<ByteBuffer>.

[drekka]

Hi. Here is a screen shot.

When this happens the body is set with a .stream(...).

I think I see the problem, the code I'm running is actually in a piece of middleware. Like this:

    public func apply(to request: HBRequest, next: HBResponder) -> EventLoopFuture<HBResponse> {

        let x = request.body.buffer
        

I'll go look at the router code because it sounds like I need to do whatever it's doing to achieve a similar result.

[adam-fowler]

Ah, that would be the issue. If you want to access the body in a middleware you need to collate the streamed body segments. You could call consumeBody on request.body but that'll mean you lose the body when processing the route. I think I need to add a new method which returns a new HBRequest with a collated body.

[adam-fowler]

#147 should provide the functionality you require. Can you check it out

[drekka]

Thanks for that Adam. But after I typed up my comments above I did a stop-pause-reflect and realised that I didn't actually need to do this as middleware. I originally thought I would because GraphQL uses a single endpoint for all of its requests. but now that I understand more about it I realise I can build a "GraphQL router" and sit it behind some end points in the normal trirouter and that doing that would probably be simpler than going down the middleware approach.

So I'm going to refactor anyway.

[adam-fowler]

The Grafiti sample I have in hummingbird-examples uses a single route and that seems to work. Of course that is a simple example and might not cover everything you need.