Updated file serving example for streaming #222
Conversation
|
This was inspired by @jclulow 's work in buildomat: https://github.com/oxidecomputer/buildomat/blob/main/server/src/main.rs#L270-L281 It seems like streaming file transfer should be our default example, as it integrates pretty well with Hyper. I'm attempting to make use of this pattern in oxidecomputer/omicron#457 , but tests are failing without the accepted |
|
Added a couple streaming tests, showing a streaming server, and a client which can either "read the whole response into memory" or opt to receive it piece-by-piece. |
davepacheco
left a comment
davepacheco
left a comment
There was a problem hiding this comment.
These comments are all nitty.
| async fn api_streaming( | ||
| _rqctx: Arc<RequestContext<usize>>, | ||
| ) -> Result<Response<Body>, HttpError> { | ||
| let mut file = tempfile::tempfile() |
There was a problem hiding this comment.
Is there any particular reason to use an actual file instead of, say, a Cursor? Or maybe better: a custom Stream that just emits a fixed number of arbitrary bytes?
It'd be neat to have this stream emit some number of bytes that's likely to be larger than available memory. If the test works, that would validate that we're not at any point buffering the whole contents of the response anywhere.
There was a problem hiding this comment.
I used a file to show integration with the hyper_staticfile::FileBytesStream, since that was a use case that motivated adding this support - we have a file server example in here, but IMO streaming files is a really common use-case that I wanted to show.
Additionally, update test utils to accept
transfer-encodingheaders, which are transmitted alongside streams.